In tactical games the AI continues to underperform by a huge margin (webbing units already webbed, etc.).  In the strategic game in all the many hours I've played I've only seen strategic spells cast on me 3 times (and yes I understand we don't get obvious notifications but I pay attention to this) -- and this is on ridiculous and insane difficulties.  The AI should be tornadoing/tremoring/freezing my armies just about every turn, I've never had that happen to me.  All the lovely spells in the game and many are not used by the AI...

I have to play at insane to be challenged, and the only difference between insane and easier difficulties is that enemy forces are more/bigger -- not smarter.

Given we have no MP, AI is all there is and it needs major improvement.  Civ4 doesn't have great AI but it's several orders of magnitude better.

LH/FE/WoM are good games with great potential that is currently unfulfilled.  You have talented people working on them.  My post is intended not as a negative against what your team has accomplished but instead a compliment that they are a great team and LH can and should be -- given the talent of the team -- better.  That's the problem with having talented folks -- we expect a lot from them!  In many ways it's easier to be average than exceptional lol.

Re AI, if people are beating the AI beyond challenging, then we need them to explain what basic things they're doing to beat it.

They're not beating the AI because spiders sometimes web an already webbed unit. There is some more fundamental strategy being employed that I'm not aware of.

I can only make the AI play an optimal version of how I play the game.  I have no doubt there are plenty of people who are better at the game than I am.  Therefore, I need those players to explain, in general terms, what they are doing.  

The mistakes you point out the AI makes there might make a difference in a challenging game. But in a hard or later, that wouln't be the difference.

There must be fundamental differences in play style going on.  I mean, after all, if what you say were the key thing, I could just bonus the AI with more resources and call it a day.  Clearly, there's something else going on.

I can't beat the AI behind hard. I know from the stats that very very few people can beat it above normal (not even challenging).

Derek and I have discussed this and what we usually see players doing is creating uber-custom sovereigns and uber-custom factions and then Ctrl-Ning until they get the perfect start-up spot.  Which is fine, it's a single player game. But those victories aren't due to "bad AI" that's due to the players giving themselves a huge advantage.

@Frogboy MadDjinn is a well known strategy beast (he's streaming live various strategy games and collaborated with develeopers to help improvements / bug fixes etc) and he plays only on ridiculous diff. I'll give him a poke to contact you (he's very sharp and notices AI mistakes strategy wise, short-long term etc).

I think it has more to do with AI priorities than with it's building path. For example am playing  game on expert world / AI, huge, everything normal only quests plenty, 10 players. 2 custom factions, one similar to Resoln - The Scryers but with enchanters instead of spiders (my faction) and a kingdom one The Seekers - focused on no penalty for traveling, chainmail, archers and assassin tools (amarians as race). What happened in mid game when I met everyone was something like this: Pariden and myself were bottom of faction power, with about 100 score. The Seekers were top of list with 400+ FP, followed by the normal Kulan ~300 (I have yet to play a game post 0.9 where he doesn't make it to top 3), Yithriil, Altar, Capitar with 200-300 each. I'm somewhat in the middle so everyone DoWs me at a point, and I'm pretty much just spending time running from one corner of my realm to other to defend / take cities back, wasn't too lucky with expansions until AI's locked me in only 1 stable and 1 warg den, 3 crystal mines and 3 metal, it doesn't help that tech wise I'm up there keeping up, even with trading I can't make more than 1 good unit for like 30 seasons, or upgrade 1 piece of gear to other units (126 crystal cost for a decent archer unit or defender - have to include fire cloaks or am toasted as I have no air heroes and everyone is making companies of mages left and right). Anyway I can't even imagine how much they expanded and cleared the map, but I bet at least 80% of map is covered by AI territory. They are all fighting eachother or have alliances etc, their troops design is quite good (can't really find anything I wouldn't do myself if I had the resources / production), so is their mix up etc. While they do lack in tactical fights regarding some spells usage (why would you cast earth grab when all my units are mounted hence immune to prone?), can't say they didn't make good use of buffs / heals / debuffs or damage spells (was quite surprised when I fought kulan 4 man army, I said umm okay so you send 4 heroes to fight my army? then he goes: haste on kulan + diamond skin + guardian wind and other air dodge spell vs ranged, growth etc you know the drill, was lucky Asha's infection landed on first cast to spread the blinds & slow).

I am holding my own for now and even stealing a city here and there from close neighbours (on top of that I got the event that spawns the spider queen in middle of my small empire, 4 epic armies with 2-2.5k hp when my freeze was on cooldown, beat them but this was really tense - I admit my sovereign was level 15 with -1 turn on spell casting and master death & water - as I was already fighting Altar and trying to delay Magnar on other corner). But I have no clue how this will pan out. I have a hard time trading for crystals (they either have metal and don't want mine, my money are not enough to afford to pay 1k for 50 crystals etc) which severely limits me in building decent troops that I can leave to defend the kingdom if I want to venture further. 

So there is no problem - in my game - with AI expanding - it actually does it at an insane pace, even in early game his heroes had double my level and gear I could only dream of. They don't struggle economic / territory / tech wise (unless they are at war with couple other AI's and start losing stuff fast), they build good armies, and use spells (I have magnar spamming every couple turns oppression on my cities for example, Kulan used several strategic damaging spells when I tried to enter his territory to break some stuff etc).

Maybe on smaller maps / different settings things are different.

The monsters destroying improvements / attacking cities is a bit random (it feels to me, if they don't have a unit to chase, they just wander, if the rng path generated for next move is overlapping / intersecting an improvement or town in their way, they will attack). I had a fel dragon army (yeah he joined up with 2 spiders) sitting inside a village borders wandering back and forth in Altar land (he was next to a city of mine) for over 30 seasons and never attacked it. And his lair (dragon nest resource or w/e is called) was inside his lands which he disturbed when expanded there. When I settled near an ogre lair, 2 turns later my city was gone. So it's rng when roaming monsters will attack players (AI's included) improvements / outposts / cities due to their movement behaviour.

I like this approach, but at the moment LH doesn't seem to be working like that, which my current game exemplifies. I'm currently playing a game on hard as Gilden. Yithril started a war against me when power ratings were as follows:

Yithril 227

Me 109

Resoln 65 

Altar 56

Pariden 32

One thing is that other civs don't seem to be doing very well. They've all settled only 3 cities (I've settled 6 and Yithril 8).

Anyway, with that sort of difference in power Yithril should roll over me. He will not, due to his complete military incompetence. I have only one strong stack that really counts. He first assaulted a city defended by it with an only slightly weaker stack of his, which I duly defeated.

Then he arrived in my borders with 3 strong stacks travelling together on one hex. I had to attack them and defeated the first stack without losing any troop (I used an artifact whose name I can't remember to take over one of his juggernauts, which was crucial). However, my troops took some damage and if he attacked me in his turn with remaining two stacks, then maybe I could defeat the first one, but there is no way I could defeat both. Then the war would be probably over. Instead, he decided to wander around aimlessly, letting me recover hit points and choose a convenient moment for the next battle.

In the meantime, he sent another stack into my territory, which is heading for my capital, but which I will defeat with no problem, because his other two stacks are not putting me under any pressure. At the moment he's got 180 power against my 145 and I have no doubt I will win the war. 

So, I will defeat an opponent of twice my power ranking, more settlements and at least five strong stacks with my single one, because he absolutely no clue on how to use his several armies together. I hope you can improve on this kind of AI behaviour and if I can help you in anyway, just ask.

I spent the last hour writing a detailed response, and the server apparently ate it.  Not sure I have it in me to recreate the post...  Maybe later I'll try.

Edit -- great, this short content-less post goes thru, the detailed lengthy one didn't.  lol at me

Frogboy, you don't honestly think you can develop an AI that can beat serious players straight up, do you? Most people play the game casually, but they also didn't know that you are collecting data on their games. Winning on challenging is as simple as developing a mage sovereign to cast fireball in 1 turn, building a mana reserve and some troop to buffer the sov, and using it to obliterate opposing troops. Almost any coherent, focused strategy will consistently beat the AI, which seems to pick traits in a breadth-first traversal of every tree. It is not that the AI just makes double-webbing mistakes, it doesn't do most things in a coherent, focused manner. The only advantage the AI has is that most players aren't trying to win the game, they are thinking about leveling up, questing, and taking down wildlands, instead of cutting the AI's throat at the most opportune moment.

I personally believe that it is not possible. I wrote a thread (and replies) about it here: https://forums.elementalgame.com/438446/page/1/#3300617 . If people are playing this game to win, either the game can be reined in until it is simple and uninteresting and the problem may be tractable or the AI can cheat and have the illusion of player-like play, but in a fun, commercial strategy game the human mind has too many advantages dealing with the crazy complexity of the game for a fair AI to compete (or even a fair AI with reasonable bonuses).

Maybe you meant the vast majority of players who play the game, because most everyone is not serious. That is doable, and you have done it well. But I don't think you could seriously hope to catch the people playing at ridiculuous and above, they will just find the new best strategies and adapt them in real time. Unless you develop a learning solution, your AI must play your moves that you intend (and actually work in the general way they are intended) ahead of time while you were blindfolded to the actual gameboard. The human mind will win that every time in a game this complex, in my opinion (with the caveats I mention in the thread).

A few items first:

(sorry for the wall o text... would be nice if there's a spoiler tag on this forum).

1) The problem isn't that there are good players that will beat the AI no matter what is done to it (shy of adding in real computational intelligence programming and/or a server farm to constantly adjust the AI based upon real games played) it's that the AI doesn't seem to be able to utilize what is in the game properly in the first place.

2) suggesting that players use Ctrl-N to get great starts is also a red herring.  Kael has mentioned that he isn't looking at properly balanced starts (or at least ensuring 'ok' starts for everyone) so that has to change first.  I don't actually use Ctrl-N in my streams, but I still manage to have a decent win rate on Ridiculous/Ridiculous provided that I focus on winning (it's easy to lose focus as the AI isn't trying).

3) The AI can get better, and it won't take massive effort to get it onto the right path.

There are two main lines of progress that should happen to get the AIs into a better situation - AI and balance changes.

AI

Some points have already been made, so I won't belabor the points  -

AI is bad at choosing settling locations.  While you may think this isn't that important, it always is important.  The base grain and materiels are multiplied with buildings/etc, so making the wrong choice of which tile to settle is multiplied into worse choices later.  Mostly this has been a little fixed due to removal of the mass spam of locations from the map, but I've seen AIs settle on 0/0/0 tiles often enough to know there's a problem with the settling location code.  2ndly, the choice of where to settle should also pull from what the empire needs. 

After settling, other issues arise -

The 'suggestion' for what type of city (Town/Fortress/Conclave) a village should turn into is wrong at about Vegas odds (so slightly more than randomly wrong).  Conclaves being suggested when there's no essence.  Fortresses suggested when there's 2-3 materiel, Towns/Fortresses suggested on 3-3-3 locations without forests or rivers.  My point here is if the AI is using the same 'advice' as players are shown, then they aren't making the best choices for what to create, civilization wise.  Fix this code and the AI will have better advice about its cities.

Building choices --

It's been said in other threads (and maybe this one), but the code for when/what order to add buildings to a city all seem to be gated in the wrong way.  I won't get into this one, but my basic build order for new cities focuses on production and unrest, mixed with border growth (to connect back to the capital) and merchants if needed.  I do tend to forget the research buildings a bit, but food is also pretty important to add (Towns focus on faction food buildings before city food buildings).

I also leave unit production to the fortresses, after I've managed to get a few up and running.  The AI seems to build units anywhere, which takes time and production away from the town/conclave buildings (so not getting ahead even if a nice spot is found).

which also swings back into the Tech tree. 

The point was already made that the AI is a bit too 'general' in researching the tech tree.  For most I'd say that was true, but I've seen bad choices from the AI as well.  Ie, the AI(s) that head straight to the towers always get rolled by anyone that chooses to go to to weaponsmithing/blacksmithing + cooperation.  Even in AI vs AI wars, I watched one AI mass spam cities (8+) and head to the towers, just to see a 2-3 city empire wander in and attack/take the cities that made the towers.  In this respect a more general path would be good, but what the AIs really need is 'focus' and 'awareness'.

Focus comes from not going broadly in the 3 tech trees early in the game, but rather just adding the parts that it needs from each.  Ie, civilization first (and only) while expanding.  A *short* trip into warfare to grab leather armour and maybe up to Drill (if big monsters nearby or an early war is coming) but rarely into weaponsmithing too early.  Weaponsmithing is needed if the monsters are too think in the area, or if war is about to start.  A small step into the magic tech tree for shards and possibly crystal mines is all that is needed early on.  Possibly some spells, if a mage/have a mage and decent mana/turn, but rarely too early.

Otherwise the focus is mostly in the civilization tech tree up to cooperation and construction (plus the education/food techs in that area).

Just as important is knowing what is in the area, resource wise.  if some need techs, then go get the techs at the appropriate time.  Ie, get crystals just before you can actually use them.

After that, situational awareness is the more important thing.  If your opponent (doesn't like you or are at war with you) has a better weapon or armour, then either build units that directly counter them (accuracy vs. dodge, spears vs armour, etc) or get a better tech to even it out.  If your tech rate is low, grab research techs. etc etc.

Coming along to a more direct issue is the seeming lack of strategic spells on the part of the AI.  Most people will point to the lack of delay/damage strategic spells being cast, which is true, as the problem, but if you dig a bit deeper, you might see that the AI doesn't seem to manage it's mana very well.  If the AI has no mana, it can't cast high cost spells.  This might also be a small issue for Pariden and why the AI refuses to use Arcane Monoliths.  Effectively, fix the AIs mana management problem and we might see more strategic layer spells.  This problem seems to be partly on the AI spending too much on heros wrt maintenance costing spells and partly due to not actually generating enough mana per turn.  This part goes back to the not having conclaves build conclave type buildings often enough.  Personally I always cast meditation on every 2+ essence city until I get enough shards or conclave buildings to make up for a potential loss of meditation on a city. 

2ndly, there's a small matter of saving for strategic spells and saving for tactical spells.  Both are needed, but if you're doing one, you can't do the other.  The AI should know not to cast spells on enemy cities if it needs the mana for combat or defense.

I also agree with the sentiment that AIs DoW'ing each other/you without actually move forces nearby is a cause for concern.  It's actually not the same as per Civ 4/5 given that stacks can move quite the distance and sometimes defense early war is better than offense early war.  But the AI really should try to have some units ready to go when it declares war.  I have seen this happen (AI having multiple stacks on my border at the beginning of a war) but that seems to be mainly for direct neighbours.  I've had AIs on the far side of the map DoW me, even requiring them to cross other factions territories or wildlands to get to me.  That part should stop.

While this post is definitely not 'complete' in most of the needs for AI change, this final point is also important --

The AIs don't seem to use their faction specific benefits. 

AI Pariden will definitely build Scrying Pools, but I've never seen it cast an Arcane Monolith.  A recent patch change nerfed that factions expansion/population wasting ways wrt outposts, but the real change should have been to have it actually use the spell for outposts rather than pioneers.  It should know that pioneers = new city, spell = new outpost.  It's that straight forward to fix that AI.

Juggernauts need to be moved one tech level earlier as they are basically fodder for Boar spears+warhammer units.  The Yithril haven't been a threat in FE LH due to Juggernauts, ever.  I rarely if ever see a Jugg in a normal game - even if the AI is past that tech.

Bows suck on heros, except for the Lightning bow.  The Tarth suffer from thinking it's actually a useful weapon.

There's more for each faction, but the basic jist is that each of the faction AIs should be making the most out of their bonuses, rather than seemly using the same general AI.

Which also brings us back to balance.

This post is already too long to write much about it, but the weapons and traits need a better balance and the AI needs to choose the right units for the situation at hand.  While the AI will use units that I make during a game in the next one, I can't constantly go back and remake every unit each time the game is changed.  A unit maker (choose faction, then choose tech level, then create unit) would go a long way to helping it out. 

Else, Spears are still OP (even with the recent change to the vs. armoured bonus) compared to everything else.  I don't even need ranged units.  Axes are useless and swords are 'ok' (the special ones are nice) but not sufficient enough to go without mass spears.

Here's the problem in a nutshell -

Spears have no drawback to using the impale ability.  Maces get dazed when they use theirs.  Cleave doesn't do enough damage to bother using though backswing is nice.  Swords require you to get attacked -- except by a spear unit that never gets counter attacked.

Maces and axes are very slow, swords are faster (basically counter the later armour penalties) but spears keep going.  In a typical fight my mace units get 2-3 actions, whereas the spear units take 5-6 actions -- mostly that's due to dazed maces missing a turn, but it counts as that's the reason to use maces.

Spears also rip through armoured units (the sword types) which means when a hammer or axe unit puts on chain/plate armour, it's effectively standing still waiting to get shredded by a spear unit.

What does that mean for the AI?  Spears rip apart the AI units as they tend to have a lot of maces and axes, with only some spear units and a few sword units.  It's about as bad for axes as for maces, but at least the mace units can use shields for the dodging/defense bonus.

I typically early build a hammer unit (upgradeable club, all leather only and upgradeable shield) with Lithe, Balance and Bloodthirsty (Dodging hammers).  No matter the faction (lucky+wraith makes this more OP) I make these.  Lithe and Balance should be cut down to 5 apiece.  20 dodge+Lucky/Wraith+shield = hard to hit.  I have bloodthirsty on there as my maces rarely get a 3rd move, so the spear units go and impale/attack units for the maces to finish.  Any spell that hits all units is also a really nice setup for this dodging hammer unit.

My spears are typically (depending on if I have the 1-h spears or not): spear (upgradeable) leather armour except for the Soldiers Gloves and boots (+accuracy and initiative) and then Fast/Finesse and either Precision for accuracy or Ironskin for defense to make up for the lack of leather boots/gloves.

On the more general topic of trait balance -- the +/level traits can't stack up to the +10 versions of similar traits.  That's mainly due to rarely seeing units make it up to level 10.  Either the game is over, or the unit just didn't get enough XP to level high enough before the game was over.

Balance wise, there is an obvious issue here -

Spears have a higher attack than all other weapons (at the same level) due to the armour piercing bonus + attack more often + can't get countered + can attack from the 2nd line.

So it's the high DPS build that also has the highest weapon strength and isn't weak to getting hit as nothing can hit back anyways.

My basic thought line on this hasn't changed much -

Hammers and Axes need to be at least 33% higher base attack than spears of the same level.  This is to counter the initiative penalty applied to using those weapons.  If they aren't higher attack than a 'faster' weapon, they aren't worth using.  The simplest balance strategy that you were/are using in FE LH is that initiative penalties on weapons are inversely proportional to the weapons damage output.  Ie, higher base damage (per weapon type) = more initiative penalty added.  Right now, there's no balance between the weapons due to not following this simple strategy in the game.  The problem wrt to the AI, is that the AI does not know that and keeps using the wrong weapons.

  Cleave needs to do a bit more damage to each unit (+20% to current would make it nice I think).  The first Axe and the Dagger should be moved to the 'starting tech' - basically available right away.  Move all other axes/swords up to fill the slots left by those ones.

I'd also be open to Impale leaving the spear unit unable to dodge for a round and leaving them with a -Initiative penalty for a round.

When it comes to the complaint about 'uber-factions' and 'uber-units' --- Kael control this, so he can change it by making him add costs to 'unbalancing' faction choices.  You could also continue to boost the other faction traits so that they are worth taking.  There's only a few good choices for faction traits and no real downside to taking one of the penalties to be able to get another one.  If the players can make OP custom factions, you can fix the basic factions to be more competitive by adjusting their choices as well.

Nice analysis!

Good points MadDjinn, remember that LP on Civ5 where Genghis Khan DoWed you from behind Germany with no way of attacking you without attacking germans who were bigger than them, and he kept throwing demmands at you which you could ignore because he had no way to get to you I think same happens in LH. I had Yithril DoW me on a huge map (I haven't found their borders yet) but they never had a single unit coming toward me because I was surrounded by 4 other faction borders. I believe the AI is DoWing as soon as their "we don't like you" factor reaches a threshold without considerations of map position and if it is possible to actually get to target.

Regarding weapons, I prefer to use 1h weapons and shields for my frontline (effectively making them tanks), while my ranged back line deals the real damage. Simply can't compare ranged damage, be it mages, bows or xbows survival (no counterattacks, out of harms way, very hard to get to and swarm etc) to any type of melee troop. Add in hero buffs / debuffs and damage spells. Yes spears are nice and dandy, playing wraiths with lucky and dodge armor / traits is great but.... then you get an event or enemy army well made and your spears vaporize in 1 turn. Example - queen spider event - 2-2.5k hp armies of venom 3 spiders who have high initiative and web stuff (they have 9 groups of 3+ spiders) - the white ones - your spears now can't move to impale, can't make use of their movement, and have poor defense, no matter how high dodge you have on them, when 3+ enemies attack same target, EVEN with blind on them, they will hit. You can't 1 shot a group of those spiders with 4 ranged units (maybe if I had a 4-5 essence city with fire dmg buff), and there's loads of them.

Dodge is nice, but is RNG. Had a 3 round fight with a lucky wraith spears trying to hit an Idiotic dodger from Altar, neither unit landed a hit. Yet my archers and mages had no problems killing it despite it had air buffs to increase dodge vs ranged cast on it (was showing 48-56% chance to hit depending on level of units I had).

I feel is too easy to stack on spell resist in game atm past mid game. Altar heroes had 90+ resists (with 50% cold or fire as well for good measure against my army). I had to cast 3 shadowbolts with -28 SR each to be able to land a spell on them (and I have close to 150 spell mastery on Asha my hero). This is more problematic when said spell resist is on a caster. Which makes it close to impossible to land a counterspell. If they are mounted, you can't prone them either to stop the cast. And you have only bash left which is a melee ability. In current game I have 8 death shrines and 6 water. Horrific wail (54 dmg) and Blizzard (28 dmg / unit) are all you need to deal with most AI armies (not monsters, that's a different problem, where you go with the normal curse spam and spread with infection, use summons etc). AI can equip / buff cold resist (and fire) which makes blizzard and fireball not so great anymore true. But how do you defend against death damage spells, poison and lightning? (only saw a single item in game, boots of grounding I think, chainmail, that give lightning resists). Maybe a spell in life should act as a cleanse - single target, tier 3 or 4, to remove negative effects? Because as it is now, I can cast blind, slow, graveseal, remove all armor, and beffudling from the first spider you train as Resoln, and spread them all with infection (web doesn't spread). Then I can put 2 poisons up with dirge of ceresa and the other aoe poison from the magic tree, which will critical hit every turn thanks to graveseal (which doesn't affect critical hits from other damage spells like blizzard, horrific wail, drain life, entropy etc, yet the spells can critical hit). 

It really feels like dark magic is best vs anything non magic immune, and life is better against magic immunes (as it buffs / heals your troops). Fights vs dragons as dark magic user with scholar (-1 on cast times): cast kill - if is earlier in game, graveseal and armor debuff with blind and slow. Fights vs dragons as life magic: buff troops, heal them every turn, shrink dragon etc.

Maybe life magic could use a spell to transform a random shard into a life one as well? For example if my focus is on life and air, or life and earth, I have little use for other types of shards since I don't have spells that would benefit from their powers. Death magic can grow very powerful very fast as you pretty much just need death shards to fuel it, and keep couple shards of your secondary magic school (water complements it nicely because of slow, mantle of oceans, and some good strategic spells).