I think a big problem with military growth in the game right now is that a just-built city can build units almost as fast as a level 5 city.
They were trying to represent that it takes a certain amount of time to make swords and then teach someone to swing it, which is fair enough. But what they haven't taken into account is that any school (military or otherwise) has more than one level of student at a time. If it takes 4 years to train a swordsman, you'll always have some in their 1st year of training and some in their 4th year.
Maybe what they need is for each city to have a concurrent training capacity that increases as a city gets bigger. So a level 1 outpost might only be able to handle training 4 people at once (a party), while a city might be able to train 50 people at once (4 full squads of 12). The 4 squads may all commence training at the same time, or they might start (and finish) a dozen turns apart as you get the resources to pay for it. Your training capacity would naturally and sensibly be helped by adding things like barracks and war colleges.
I'd also link the quality of troop you can train to city improvements - how can you train veteran troops without barracks?
And training 4 guys shouldn't take anywhere near 4x the time it takes to train 1 - they're all doing the drills at the same time (plus the instructor can tell them to go practice amongst themselves, getting in his own R&R while still progressing the training
) Sure personal tuition is a good thing, but having classmates isn't as terrible as the current system makes out (though it does make sense if your faction has the egalitarian trait - I couldn't concentrate on training when half the recruits were athletic chicks in skimpy armour).
Put it all together and you'd have big cities able to crank out well trained units quite regularly, but they still wouldn't be able to instantly pop out a meaningful defence if you were unprotected when the big bad wolf showed up. The little outposts would be severly limited in what they can train - though that wouldn't kill the early game because who has the tech and cash for big squads anyway?