A few items first:
(sorry for the wall o text... would be nice if there's a spoiler tag on this forum).
1) The problem isn't that there are good players that will beat the AI no matter what is done to it (shy of adding in real computational intelligence programming and/or a server farm to constantly adjust the AI based upon real games played) it's that the AI doesn't seem to be able to utilize what is in the game properly in the first place.
2) suggesting that players use Ctrl-N to get great starts is also a red herring. Kael has mentioned that he isn't looking at properly balanced starts (or at least ensuring 'ok' starts for everyone) so that has to change first. I don't actually use Ctrl-N in my streams, but I still manage to have a decent win rate on Ridiculous/Ridiculous provided that I focus on winning (it's easy to lose focus as the AI isn't trying).
3) The AI can get better, and it won't take massive effort to get it onto the right path.
There are two main lines of progress that should happen to get the AIs into a better situation - AI and balance changes.
Some points have already been made, so I won't belabor the points -
AI is bad at choosing settling locations. While you may think this isn't that important, it always is important. The base grain and materiels are multiplied with buildings/etc, so making the wrong choice of which tile to settle is multiplied into worse choices later. Mostly this has been a little fixed due to removal of the mass spam of locations from the map, but I've seen AIs settle on 0/0/0 tiles often enough to know there's a problem with the settling location code. 2ndly, the choice of where to settle should also pull from what the empire needs.
After settling, other issues arise -
The 'suggestion' for what type of city (Town/Fortress/Conclave) a village should turn into is wrong at about Vegas odds (so slightly more than randomly wrong). Conclaves being suggested when there's no essence. Fortresses suggested when there's 2-3 materiel, Towns/Fortresses suggested on 3-3-3 locations without forests or rivers. My point here is if the AI is using the same 'advice' as players are shown, then they aren't making the best choices for what to create, civilization wise. Fix this code and the AI will have better advice about its cities.
Building choices --
It's been said in other threads (and maybe this one), but the code for when/what order to add buildings to a city all seem to be gated in the wrong way. I won't get into this one, but my basic build order for new cities focuses on production and unrest, mixed with border growth (to connect back to the capital) and merchants if needed. I do tend to forget the research buildings a bit, but food is also pretty important to add (Towns focus on faction food buildings before city food buildings).
I also leave unit production to the fortresses, after I've managed to get a few up and running. The AI seems to build units anywhere, which takes time and production away from the town/conclave buildings (so not getting ahead even if a nice spot is found).
which also swings back into the Tech tree.
The point was already made that the AI is a bit too 'general' in researching the tech tree. For most I'd say that was true, but I've seen bad choices from the AI as well. Ie, the AI(s) that head straight to the towers always get rolled by anyone that chooses to go to to weaponsmithing/blacksmithing + cooperation. Even in AI vs AI wars, I watched one AI mass spam cities (8+) and head to the towers, just to see a 2-3 city empire wander in and attack/take the cities that made the towers. In this respect a more general path would be good, but what the AIs really need is 'focus' and 'awareness'.
Focus comes from not going broadly in the 3 tech trees early in the game, but rather just adding the parts that it needs from each. Ie, civilization first (and only) while expanding. A *short* trip into warfare to grab leather armour and maybe up to Drill (if big monsters nearby or an early war is coming) but rarely into weaponsmithing too early. Weaponsmithing is needed if the monsters are too think in the area, or if war is about to start. A small step into the magic tech tree for shards and possibly crystal mines is all that is needed early on. Possibly some spells, if a mage/have a mage and decent mana/turn, but rarely too early.
Otherwise the focus is mostly in the civilization tech tree up to cooperation and construction (plus the education/food techs in that area).
Just as important is knowing what is in the area, resource wise. if some need techs, then go get the techs at the appropriate time. Ie, get crystals just before you can actually use them.
After that, situational awareness is the more important thing. If your opponent (doesn't like you or are at war with you) has a better weapon or armour, then either build units that directly counter them (accuracy vs. dodge, spears vs armour, etc) or get a better tech to even it out. If your tech rate is low, grab research techs. etc etc.
Coming along to a more direct issue is the seeming lack of strategic spells on the part of the AI. Most people will point to the lack of delay/damage strategic spells being cast, which is true, as the problem, but if you dig a bit deeper, you might see that the AI doesn't seem to manage it's mana very well. If the AI has no mana, it can't cast high cost spells. This might also be a small issue for Pariden and why the AI refuses to use Arcane Monoliths. Effectively, fix the AIs mana management problem and we might see more strategic layer spells. This problem seems to be partly on the AI spending too much on heros wrt maintenance costing spells and partly due to not actually generating enough mana per turn. This part goes back to the not having conclaves build conclave type buildings often enough. Personally I always cast meditation on every 2+ essence city until I get enough shards or conclave buildings to make up for a potential loss of meditation on a city.
2ndly, there's a small matter of saving for strategic spells and saving for tactical spells. Both are needed, but if you're doing one, you can't do the other. The AI should know not to cast spells on enemy cities if it needs the mana for combat or defense.
I also agree with the sentiment that AIs DoW'ing each other/you without actually move forces nearby is a cause for concern. It's actually not the same as per Civ 4/5 given that stacks can move quite the distance and sometimes defense early war is better than offense early war. But the AI really should try to have some units ready to go when it declares war. I have seen this happen (AI having multiple stacks on my border at the beginning of a war) but that seems to be mainly for direct neighbours. I've had AIs on the far side of the map DoW me, even requiring them to cross other factions territories or wildlands to get to me. That part should stop.
While this post is definitely not 'complete' in most of the needs for AI change, this final point is also important --
The AIs don't seem to use their faction specific benefits.
AI Pariden will definitely build Scrying Pools, but I've never seen it cast an Arcane Monolith. A recent patch change nerfed that factions expansion/population wasting ways wrt outposts, but the real change should have been to have it actually use the spell for outposts rather than pioneers. It should know that pioneers = new city, spell = new outpost. It's that straight forward to fix that AI.
Juggernauts need to be moved one tech level earlier as they are basically fodder for Boar spears+warhammer units. The Yithril haven't been a threat in FE LH due to Juggernauts, ever. I rarely if ever see a Jugg in a normal game - even if the AI is past that tech.
Bows suck on heros, except for the Lightning bow. The Tarth suffer from thinking it's actually a useful weapon.
There's more for each faction, but the basic jist is that each of the faction AIs should be making the most out of their bonuses, rather than seemly using the same general AI.
Which also brings us back to balance.
This post is already too long to write much about it, but the weapons and traits need a better balance and the AI needs to choose the right units for the situation at hand. While the AI will use units that I make during a game in the next one, I can't constantly go back and remake every unit each time the game is changed. A unit maker (choose faction, then choose tech level, then create unit) would go a long way to helping it out.
Else, Spears are still OP (even with the recent change to the vs. armoured bonus) compared to everything else. I don't even need ranged units. Axes are useless and swords are 'ok' (the special ones are nice) but not sufficient enough to go without mass spears.
Here's the problem in a nutshell -
Spears have no drawback to using the impale ability. Maces get dazed when they use theirs. Cleave doesn't do enough damage to bother using though backswing is nice. Swords require you to get attacked -- except by a spear unit that never gets counter attacked.
Maces and axes are very slow, swords are faster (basically counter the later armour penalties) but spears keep going. In a typical fight my mace units get 2-3 actions, whereas the spear units take 5-6 actions -- mostly that's due to dazed maces missing a turn, but it counts as that's the reason to use maces.
Spears also rip through armoured units (the sword types) which means when a hammer or axe unit puts on chain/plate armour, it's effectively standing still waiting to get shredded by a spear unit.
What does that mean for the AI? Spears rip apart the AI units as they tend to have a lot of maces and axes, with only some spear units and a few sword units. It's about as bad for axes as for maces, but at least the mace units can use shields for the dodging/defense bonus.
I typically early build a hammer unit (upgradeable club, all leather only and upgradeable shield) with Lithe, Balance and Bloodthirsty (Dodging hammers). No matter the faction (lucky+wraith makes this more OP) I make these. Lithe and Balance should be cut down to 5 apiece. 20 dodge+Lucky/Wraith+shield = hard to hit. I have bloodthirsty on there as my maces rarely get a 3rd move, so the spear units go and impale/attack units for the maces to finish. Any spell that hits all units is also a really nice setup for this dodging hammer unit.
My spears are typically (depending on if I have the 1-h spears or not): spear (upgradeable) leather armour except for the Soldiers Gloves and boots (+accuracy and initiative) and then Fast/Finesse and either Precision for accuracy or Ironskin for defense to make up for the lack of leather boots/gloves.
On the more general topic of trait balance -- the +/level traits can't stack up to the +10 versions of similar traits. That's mainly due to rarely seeing units make it up to level 10. Either the game is over, or the unit just didn't get enough XP to level high enough before the game was over.
Balance wise, there is an obvious issue here -
Spears have a higher attack than all other weapons (at the same level) due to the armour piercing bonus + attack more often + can't get countered + can attack from the 2nd line.
So it's the high DPS build that also has the highest weapon strength and isn't weak to getting hit as nothing can hit back anyways.
My basic thought line on this hasn't changed much -
Hammers and Axes need to be at least 33% higher base attack than spears of the same level. This is to counter the initiative penalty applied to using those weapons. If they aren't higher attack than a 'faster' weapon, they aren't worth using. The simplest balance strategy that you were/are using in FE LH is that initiative penalties on weapons are inversely proportional to the weapons damage output. Ie, higher base damage (per weapon type) = more initiative penalty added. Right now, there's no balance between the weapons due to not following this simple strategy in the game. The problem wrt to the AI, is that the AI does not know that and keeps using the wrong weapons.
Cleave needs to do a bit more damage to each unit (+20% to current would make it nice I think). The first Axe and the Dagger should be moved to the 'starting tech' - basically available right away. Move all other axes/swords up to fill the slots left by those ones.
I'd also be open to Impale leaving the spear unit unable to dodge for a round and leaving them with a -Initiative penalty for a round.
When it comes to the complaint about 'uber-factions' and 'uber-units' --- Kael control this, so he can change it by making him add costs to 'unbalancing' faction choices. You could also continue to boost the other faction traits so that they are worth taking. There's only a few good choices for faction traits and no real downside to taking one of the penalties to be able to get another one. If the players can make OP custom factions, you can fix the basic factions to be more competitive by adjusting their choices as well.