Though I burned myself out during the betas, I occasionally return to this game, amusing myself with different custom builds and oddball actions. Yet, the fun is still stymied by terrible AI and the often tedious endgame--problems that drove me back to these forums after many months away. Happy to see some clear thinkers still chipping away at the problems. I hope devs/modders have been digesting these ideas and figuring out how to convert them from theory to implemented.
Brad spoiled many of us by personally investing so much effort into the GalCiv AI, and I kept expecting to be spoiled again by challenging games in the more complex environment of FE (Elemental reboot, heh). Are there any non-superficial changes still being made to the core game? I witness the AI being controlled by what can only be described as fatal mismanagement, and grind my teeth--it's a blight on an otherwise good sandbox game (I don't play above Hard level anymore because of the ridiculous, artificial boosts that try to take the place of AI strategy).
At any rate, here's a few questions/suggestions:
--Diplomacy is a tangled knot, but I agree with Heavenfall--it's worth serious effort to make it more complex (as an offshoot of generally smarter AI). The AI, in effect, needs the largest boost to their sense of self preservation, because that is at the heart of their problems (others have mentioned their tendency to be passive against a bigger foe, or just "lay down and die")--they need to plot and squirm and kick and scream instead of bending over and taking it (unless that's a specific faction's M.O.--perhaps the Queen's, heh).
They "talk" to the player when another faction is getting too powerful, so this seems like a promising placeholder for actual scripting that makes them act upon it. Why not make this diplomatic message followed up by a war alliance (for that war only) against a common foe? This self-balancing act could be scripted to work in any fashion (for or against the player), and if implemented right, it would be more subtle than an all-or-nothing, kick-in-the-pants approach. (And of course, it could be something more subtle, like "Let's cancel our trade agreements with the leader and reform the trade with each other.")
If the AI weren't so clumsy about gold/item payments, it would open up a whole new area for negotiations. Attacking forces could sue for peace once their objectives were met. As an example, let's say that 2 factions ganged up on the player faction who was expanding too aggressively, and in this entirely fictitious instance, were able to beat him back by each capturing one of his cities. Then, just like in real life, they could send their diplomats to "punish" the player for his transgression: "For 10% of your gold or crystals or whatever, we will end this war and leave you alone for no less than 30 seasons." Or something like that. This might be a bluff, depending on the player's situation, or it might be a genuinely viable alternative to taking a more serious beating. This kind of interesting example can only be possible if the AI isn't terrible at strategy/optimizing, and if diplomacy gets the attention it deserves (both in making more options available, and making all options viable by understanding the maxim that unreasonable diplomacy isn't diplomacy at all).
Along those lines, it makes zero sense for an AI faction to sit by while their friendly neighbor is being overrun (the programmer knows it's a snowball effect, so why not put these perfectly reasonable hurdles in the game?). It makes no sense that you cannot donate units (the game equivalent to covert arms supplying) or form war alliances to address the shifting landscape. This is the type of thing that makes diplomacy viable, not just labeling a certain menu tab or function in the game "diplomacy" and thinking that will suffice.
You could even make factions intercede, like "We will give you x-amount of this item if you make peace with so-and-so for no less than 20 turns" or if they are more powerful or a trading partner, "We will cancel our agreements and consider war [implemented by placing a huge negative modifier on their attitude toward the player] if you do not stop this war." Along that same vein, I have also found it far more tactically responsible to raze my enemy's cities and establish my own when/where I desire (yes, heh, I am a war criminal, but no one takes me to court for it), mainly because there is almost no negative pressure against such an action, and many negative hits to keep a captured city--there seems to have been almost no thought put into how to prevent a snowball effect through common sense (versus artificial means). Why wouldn't razing cities lose you allies (once that is meaningful), or make permanent enemies, or spawn dangerous partisan units, or even negatively impact your own unrest (killing lots of people makes other people, even your own, very nervous, and with good reason)? Stuff like that to slow the grind-to-dust campaign that too easily happens, without making the game overly tedious or unfair (unless the player selects an unfair challenge level, of course). IMO, if you make the endgame easy to win, you severely cheapen the early game struggle/accomplishments.
Opposite to razing a city, why shouldn't returning a city be a large (if temporary) boon to relations, when the player chooses to take that path? Why would any faction in their right mind choose to form economic/research pacts with the most powerful faction, unless they were allies? Why wouldn't they form a pact with lesser factions to help boost both of them closer to the leader? The AI helping the leader to win is one of the ways this game breaks itself.
--Unrest is an aspect of the game that adds no challenge, no change, no fun. Eunomiac makes some fantastic points about rising unrest leading to possible rebellion/other kinds of instability like poor output/strikes (though I think any random, heavy-handed implementation of this would be a terrible idea that would hurt the game more than helping because it would be seen as unjust rather than a tactical opportunity to nip a problem in the bud). The starting point, nature of, and degree could all be altered randomly to help replayability without making it a game killer ("hey, you just had half your empire declare independence without warning--have fun with that" is not a good example of how to do it, heh). Part of this elegant solution could be financial/resource-oriented, which could also double as a maintenance/upkeep cost for larger cities (I agree with a previous poster that the simplistic build-everything mentality really hurts any tactical aspect of city-building--why is this even feasible?). For example, paying double maintenance costs lowers unrest (i.e., more sophisticated than a simple tax/unrest dicotemy--I wonder how many players even bother to tweak that?); donating mana or crystals eases unrest in Conclaves; iron donations to the war machine diverts unrest in Fortresses. You can see how this could be a tactical decision between paying something to help unrest, or using force (stationed troops), or ignoring and playing the dangerous game against resentment of the common citizen. Border pressure from other factions and even from wildlands could be great triggers that would all make sense to a player, rather than the senselessness of a random event--a little of randomness per game goes a long way, so I suggest having a large pool of random events, but never allowing more than a few to happen in any single game.
--Endgame-specific ideas: Events like the blood moon, the one increasing creature levels, or spawning more mobs are at least thinking about the problem, if not in the proper direction (because they are superficial additives, which in FE's current environment are always a joke). Perhaps if they were more carefully constructed to be meaningful without cheesy (scaled by faction power, as DsRaider suggested), they would be nothing but a welcomed addition to the randomness that should make a 4x game stand out.
Again, I think the idea of subtle empire problems/threatening instability is a golden idea, and hope devs look into it, because like many important suggestions, it's probably too big to simply mod in. The implementation would have to find the sweet spot in between micro-management frustration and yet another inconsequential fly to be swatted away.
I agree with GFireflyE about Wildlands--at least some of them ought to scale and grow to make them more interesting in the late game, but like more cutscenes/events, they cannot be a welcomed addition while the core problems still remain, because then those additions spawn a new host of problems to tack onto the complexity of previously unfixed bugs/poor AI. Speaking of Wildlands, why not consider a sea or large lake-based wildland that threatens the nearby coastal areas until it's dealt with?
While I don't agree with Eunomiac's attrition warfare idea (because that's just another predictability, and it's just not possible to implement well within the framework of this game), it's similar to an idea that would increase the unpredictability of the endgame and slow the snowball effect. What if certain factions (or random?) would decide at some point that if they are getting taken out, then they will sabotage the enemy with everything they have (like attacking a backfield city or cities that are not well defended and razing them--even GalCiv2 did something like this, penalizing the player for trying to cheat all their forces up to the "front" lines)? Of course, things like this will only work if the AI armies aren't a laughing stock.
--Optimization: It cannot be overstated, that nothing in this game is worth much if the AI is not optimized to present a challenge (preferably along the lines of GalCiv2 where at times it seemed player-controlled)--the AI shouldn't behave as poorly as it does now above the easiest setting. Also, many subtle ways to challenge players is far superior to one or two big "game changers" like Total War series' barbarian invasions or the realm divide (though credit is due to them for implementing something to challenge the late game). For FE to be a whole game, the late game must be designed to be challenging--it simply isn't finished until then, no matter how many new factions or widgets that get added. For it to be challenging without being too frustrating for your average 4x junkie, the AI must perform optimally (at Challenging+ levels) and be programmed cleverly enough to seem unpredictable at times.
Is this truly too much to ask? Heh.