BHBT has worked on several other historic sites across the city, such as the St Michael on the Mount Without church.
Speaking to BBC Radio Bristol, Mr Routledge said a lot of people had dabbled with trying to save the mansion over the years.
“It’s been quite a journey so far for us as BHBT, but we are now completely on board with the [city] council, Historic England and North Somerset Council.
“And it’s just one of those iconic buildings that deserves to be saved.
“It’s lain there empty for 50 odd years. If our trust doesn’t save it, I fear it’ll decline even more.”