The building has not been open to the public since 2016 and was only partially used by Pudsey and District Civic Society.
Conservative Andrew Carter said alternative uses for the building, which dates back to the late 1800s, had not been properly considered.
He told a Civic Hall meeting the sale would be an act of “civic vandalism”, according to the Local Democracy Reporting Service.
Carter, who represents the nearby Calverley and Farsley ward, said: “I think the people of Pudsey will be bitterly disappointed if Pudsey Town Hall is sold.”
Concerns were raised that historic memorabilia stored in the building could be lost if the sale went ahead.
Trish Smith, Reform UK councillor and director of the non-profit Pudsey Town Hall Community Interest Company, which has been trying to make use of the building, said: “We have credible partners and a workable operating model.
“Our plan is practically phased.”
However, Labour-controlled Leeds City Council said the building cost too much to maintain and alternative options had been considered.
The proposed sale was debated on Monday as part of the council’s “call-in” process, which can see executive decisions referred for re-consideration.
In a split decision, councillors instead voted to release the sale decision for implementation.