The "idea" to be able to upgrade existing units isn't new.
The problems with combining several few-figure units into one many-figure unit (as in the late game) are in the details, typically related to equipment.
What if one out of 8 guys has an item with a clicky buff or a horse? Do you get 1/8th of the effect or does it only work 1 out of 8 times? Do you get only half a horse's leg?
What if one has a bow and 7 have swords?
What weapons/armor do you display for the unit? A squad is multiple identical units and with all the fractions there probably won't be a combination of equipment items that can result in the exact average stats.
So I say: to hell with that.
What could work, is if you send unit's "back into the barracks" in a city, effectively disbanding the unit.
The barracks would only save the combined XP value of the so disbanded troops.
Disbanding an 8 figure unit with 20 XP puts 160 XP into the barrack's XP pool. (or maybe 2/3 of it... whatever)
Then, when you train a new 12 figure unit in that town, it gets 13 XP to begin with. (160 / 12)
That is easy enough to manage. Store exactly one integer per city.
You wouldn't completely lose the old and experienced units - only their gear which really is too much of a headache to deal with.
The old gear is worn and used. Use it or lose it.
Different / improved military buildings would give new units a larger withdrawal amount.
A Barracks with 2320 XP stored might have a withdrawal amount of 10. That means no unit trained there would ever leave training with more than 10 XP, regardless of how many points are stored.
You should not be able to combine 20 regulars into one super-elite unit. There has to be a limit.
The next better Barracks (Fighter's guild?) could then train fresh troops up to a max of 20 XP.
This can be fleshed out.
Say, the highest XP unit to ever "disband" to this town (provided it has any barracks) becomes the new drill sergeant.
This increases the barrack's withdrawal amount by Sergeant.XP / 10 and adds Sergeant.XP / 10 bonus XP to every unit trained from these barracks afterwards.
Again, that's only one number to store per city...
It's an abstract system to simulate how experienced units are retired from field service to form the training cadre of your future army.
Their "experience" should not simply get lost...
This completely avoids any messy system about upgrading weapons of possibly multiple unit types to other equipment configurations.
And again - it creates a choice. Use your experienced units in the field or use them to create the cadre of your future army.
Now we're talking strategy.
Of course, now you will say that the old units were useless anyway and getting "something" from them is simply a free goodie.
Valid point but a completely different subject. Small or outdated units would not need to be completely useless if there were a tactical combat system.
[Gameplay] Attack / Defense scaling with unit size
[Gameplay] Tactical movement restrictions, unit zone of control, flanking and position
Any "decision" to get something for free or just not getting it is no decision. It adds no depth, no choice, no gameplay.