Cities conquered by the respective other faction group (i.e. Kingdom / Empire city conquered by the Empire / Kingdom) are way too strong and tend to become imbalanced. This is due to the fact, that the "one per settlement" restriction seems not to work with respect to the equivalent building of the other side. [Yeah, yeah. I know that a multicultural society is more productive in real life, but I'm talking game balance here.]
So these cities can end up with twice the number of basic buildings (e.g. a Merchant AND a Money Lender) and - far worse - double duplicator buildings. War Council & Barracks, a Fungi Storehouse & an Irrigation System ... and and and.
Altogether this makes those cross-faction cities not only ugly as hell, but also way more powerful than every single-faction city could ever be. Its basically two cities in one.
The only counterbalance for this - and by far not nearly counterbalance enough - is the fact, that e.g. on the Kingdom side conquered Shovels do not upgrade to Houses etc.