There is no trade off, and the armies + research move too fast to sit around and stall, maybe difficulty level is the problem.
That said, I would like to see more incentives to go tall by having some buildings with limits of 3 per faction or 5 per faction. I also like having more buildings that depend on settlement type, however there is one problem.
I will never build a unit outside of a fortress once I have my first fortress up, and I don't want unit enhancers on non-fortress settlements because it discourages my choice. I do like the building that gives 5% hp to all units, however that is technically an upgrade so its different. (However, the second reason I build a city is for that).
One possible restriction is "either or" buildings.
Shaman's Hut: +1 Essence, may not build Warriors Hall.
Warrior's Hall: +1 attack and defense for all trained units. May not build Shaman's Hut.
This is okay however the restriction is better moved to city/fortress/conclave mechanism which takes care of it.
Another possible restriction is having a building require 2 prerequisite buildings instead of one, creating a pyramid instead of a ladder. This is only fun when BOTH of the first buildings have synchronicity and the third ices the cake.
Warrior's Hall: +2 Attack for all trained units.
Guardian's Hall: +2 Defense for all trained units.
Honor Hall: +1 Attack and Defense, +1 Initiative per level for all trained units. Prerequisite Warrior's Hall and Guardian's Hall.
This would only actually be fun if there were 1 building in each tree of fortress/conclave/city. You don't want players to have to memorize prerequisites, you want them to jump in and play.
Another option is to increase the costs of buildings, which provides less benefit per resource investment and forces a militant war game without much else.
Finally, you can make a building dependent on something like a technology or a resource; which is already the case.