Historic preservationists sued President Donald Trump over his plans to paint an ornate office building housing his staff next to the White House, warning the structure could be “irreversibly” damaged.”

The suit, filed Friday by the DC Preservation League and Cultural Heritage Partners, asks the US District Court for the District of Columbia to stop Trump and federal officials from making any changes to the Eisenhower Executive Office Building unless they go through a standard review process.

CNN has reached out to the White House for comment.

Trump told Fox News’ Laura Ingraham in an interview last week that he was putting the wheels in motion to paint the EEOB, a sprawling, Gilded Age structure next door to the White House, a bright white.

“Gray is for funerals,” he said, referring to the building’s granite exterior, noting that he didn’t know whether he would go through with it, but that he was getting costs and bids from painters.

“We’ll see. It’d be a great addition to Washington,” he added.

Trump’s plans to paint the building signaled yet another effort to stamp his aesthetic tastes on structures on federal grounds. As part of those efforts, Trump has paved over the White House Rose Garden to create a Mar-a-Lago-esque patio, demolished the East Wing to create a massive ballroom, and ordered the golden pillars of the Kennedy Center painted white. He has also touted plans to install a triumphal arch to commemorate the 250th anniversary of the United State’s founding.

The suit argues the Trump administration cannot move forward with the president’s plans without going through “the environmental and historic preservation review that federal law requires.”

Going around those reviews could result in irreversible damage, the groups argue. Painting stone “can trap moisture within the wall system” and accelerate deterioration, the suit says. It adds that if the building was painted, “removing paint from the natural granite and slate would require immense expense” and potentially result in the loss of the original material.

“Regardless of who occupies the White House, our nation’s historic landmarks belong to the American people,” said Greg Werkheiser, founding partner of Cultural Heritage Partners. “Federal law requires a careful, public, and expert review before irreversible changes are made to a National Historic Landmark.”

Completed in 1888, the EEOB originally housed the State, War and Navy departments, but now serves as ancillary office space for the president’s staff, including the Office of the Vice President, the National Security Council and the Office of Management and Budget. The building’s interior includes 553 ornately gilded rooms, bronze stair balusters, hand-painted tiles, carved wooden fixtures, stained glass rotundas and intricate cast iron.

But the building’s unique French Second Empire style has been controversial since its inception. It took roughly 17 years to build, and by the time it was done, “the Second Empire style had fallen from favor, and (architect Alfred) Mullett’s masterpiece was perceived by capricious Victorians as only an embarrassing reminder of past whims in architectural preference,” according to the Biden White House.

Guides leading tourists around the White House complex are often heard quoting from the author Mark Twain, who reportedly described it as “the ugliest building in America.”

And President Harry Truman called it a “monstrosity.”