...uh, absolutely nothing

Yokes besides, i don't like the current prestige system, since as of now it only increases the rate of your city growth.

The problem with that is, that once your city have grown to size 5 the prestige buildings serve no purpose anymore.

To make it worthwhile even then i have some suggestions:

1) have a maximum population cap per map that will be drained as cities growth once this pool is empty all cities may only growth very slowly from natural growth (this is to imitate the barren world after the cataclysm where there are only a certain amount of survivors that repopulated the world so far)

(this was actually an suggestion made by an other user)

2) in combination with 1) if the pool of survivors is emptied you may attract citizens from other players cities to your ones

3) the more prestige you have (globally) the higher the chances are to get better quests / champions / diplomatic bonuses / techs

4) have unique buildings/units that require a certain amount of prestige

...