think letting allies construct buildings within each others' cities might be going too far, though. Maybe you can make deals with other players where they agree to build something for you in one of your cities, though. For example your friend might have much better forges than you can build, but they aren't willing to teach you all of their secrets. They might, however, agree to build one or two forges for you in return for something. Maybe it would also have the side effect of giving you a bonus towards researching upgrades for your forge, too.
I agree
Another idea is letting allies temporarily lend each other their troops. This happens historically all the time, where troops of one nation are sent to battle under the command of allied officers. This would be most helpful for human-AI interactions, because of the intrinsic difficulty of humans and AIs coordinating. If I and an AI ally are at war with another civilization, and my army is much bigger/stronger than my ally's, but not strong enough on its own to conquer the target city or army, I could request that my AI ally temporarily place its forces under my army's control. There could be terms of agreement, such as "until army/city X is destroyed, or Y turns, whichever is first" or even just for a certain # of turns. I think this would make human-AI ally interaction much more meaningful without posing as high a challenge for the AI coders.
I did like the way that AoW did it, where several people that are all in close proximity (I think it was 2 hexes or something. It is hard to see the world map 'grid' in AoW) to a battle would become involved in the battle. So the units would still be under control of the owner, but they would be in somebody else's battle. If that isn't possible because of the X-com style battle system or whatever, then just temporary control is fine for me.
But basically, the most important thing is that allies should act like allies, like Darkodinplus said. If you are attacked, they should actively help you if they are able, either by directly defending your territory or taking off some of the pressure by invading your attacker. Too often are alliances more like non-agression pacts than actual alliances, which normally means that players don't have much of an incentive of actually defending AI allies because the favor will never be returned.
That is another thing I have a bone to pick. *Normally* the regular peace treaty or wizard pact is the non aggression deal and the alliance means actively helping each other.
That being said, breaking a wizard pact should still have a minor cool-down time so you don't just backstab people (which is the way it seems to work in Sins) and alliance should mean they are going to be bending a little to help you, or at least do joint attacks for you. in MoM, wizard pact meant nothing since you could break it at any time, and your guys didn't agro the way they do in most RTS games (where cease fire is important).
Things to say for allies include "I might be attacking yellow in the next 100 turns, might you prepare to help me?" and then "I'm attacking now, lets go" or even "I'm going to help you, but I need 5 more turns to complete these bear-riders and get them over to the yellow boarder. Delay your attack 5 turns if you can"