Yes you gain the use of most of the buildings, but some like a Sion Temple are useless to you and can be razed.
The only way to raze a building I've found is to click on the Raze button on the city's build tab, then click on the building you want to raze, then confirm on the popup that yes you want to destroy it.
As said, you can't completely raze a city until you have researched the right tech.
Succession is something that may or may not be implemented many, many patches/updates/expansions from now.
Happy empire building