Ok....I give in.

I'm going to be adventurous tonight and update my .EXE file when I get a chance.

Wish me luck!!!

Any chance the modders could hear what tags to use to change the value weighting the AI is giving to weapon / armour combination?  Would be nice to make certain factions have a preference for certain weapon combos. 

Also if you are working on AI unit design, could you make them use shields?

I should have a new EXE available today that has all this stuff in.

On the topic of the save game load times - would it not be an appropriate "quick fix" to, on load, populate the cities with empty/non-graphic improvements, and kick off the regeneration as a background thread alongside the other game threads?  This way the generation can replace the "empty" improvements as it gets to them, and load time cuts by 95%.  I know that I, personally, would put up with cities magically popping out of the mist in order to load my game that much faster ... and it shouldn't take more than a minute or two to finish back-filling the city graphics.

I know that typically, threading a system is not the way to save development time, but in this case, unless I misunderstand what you mean by regeneration, the only thread with write access to the visuals being generated is the theoretical generation thread, so it should be much easier to avoid collisions.

I know I've said this before, but I'll say it here again.

It would be great if the Ai could "cheat" at building improvements and training units faster.

HeavenFall: I'd like to see that as one of the XML parameters. Then people could create their own difficulty levels as well as modders.

I think a lot of people, myself included, want to focus in the near term on making the AI play a better game without cheating.

But yea, I do think on ridiculous level, the AI should be able to just ignore some of the resource restraints and just plain "cheat".

Will the new exe work with my save from 1.1c your old exe?

I haven't finished reading the whole post...

This annoyed me to no end in GalCiv (albeit, I only played to about crippling or whatever was after that): the AI would have massive military dedicated to defending remote (relatively inaccessible) planets.  It wasn't smart enough to relocate units when it noticed incoming threats, or launch massive attacks.  At least that's how it seemed.  I tried to explain it away that it was just the personality of that particular race, but no matter how you slice it, it was boring.

Frogboy,

Instead of "cheating", could you base the AI decisions based on the currrent state of the Player and then have the AI work to lag or lead these parameters based on the AI setting. In other words, If the players status is divided into trackable parameters like Level of Sov, Sov Attack, Sov Def, Mana, Shards Controlled, Number of cities, Best hand to hand unit (with AT/DF/MV), best ranged (same), ave number of each unit type per city, number of NPC, average level of NPC, ave stats of NPC, resource averages per city, etc....From there each AI faction could be set to try to be lower, at , or higher then the player within the capabilities each AI has on the map at the time. At a minimum this would help with the city defense levels and building queue needs. If the player is spamming low level guards then the AI just needs to be either a bit more in number or a bit higher in stats to keep parity. If the AI is falling behind in arcane/tech then is can look to bolster those buildings. Recruit more NPC's. I think you see the pattern.

The biggest issue the AI seems to have is when it can't sit in an ideal setup. This happens if it gets hemmed into a small corner and can't expand, loses a city in a fight (sudden shift of resources), or overproduces itself into a corner where it can't expand/buy (I had an issue doing this until I payed more attention to the amount of guilder per turn more closely and went after housing tech to expand back out). These are much tougher issues for the AI to reason with. Early in the game if factions to not have non-aggression treaty the way to new lands is trapped. This is a cornerstone of my main strategy. I never make those treaties. This keeps factions from crossing my boarders with pioneers and trapping me in a corner. I also use terrain chokepoints to cutoff land paths as well.  The AI is less capable at looking at the map and seeing the best way to block out terrain from the player. Once you end up cut off from more land your only option is to jump another weaker faction (usually one that is land trapped and stalled).

Enjoying the game and looking forward to taking 1.1d for a spin.

Thanks again for your hard work and dedication. And a Happy New Year too!

AI is fun, it is challenging and it is darn tough to do it well.  Brad you sound like you need help.  You are in the weeds and I agree you need to go low-level from time to time, but if you spent all your free time doing lieutenent work then you have none left to do general work.

I'd suggest have differing levels of AI threads that talk to each other through some sort of pre-determined protocol.

The lowest level might be the NCO thread that deals with all things TACTICAL here and now with whatever resources are available.  "Making the best out of the existing situation."

The highest level might be the General thread that deals with STRATEGIC; the future and planning, wish-lists, data analysis, etc.

An intermediate level might be the Lieutenent thread that blends some TACTICAL and STRATEGIC elements together.  This thread allows a single General thread to communicate with several to many NCO threads (each at a locale).  This thread collects data about the surroundings (Monsters, Enemies' troops/towns/location, chokepoints, etc) from the NCO threads and puts it into a format the General Thread can use to prioritize versus other limited resources and make high level decisions.  The LT thread makes historical calls to retreat from a fight or bring it based on best guess and recent results.  NCO's are always ready for a fight in my experience. :/

Spend some more time at the 10 and 40 thousand foot levels and look at the big picture with the General thread.  (recommendation)

And do something that changes the behavior of AI sov's so they stop suiciding solo in my territory.  Like avoid invading human players with an AI sov.

Hope you find this advice useful, if not just remember it was free.

The AI already manages things much as you describe.

Can you make it adjust to fast moving troops if it ever encounters them and be more defensive? I seem to be able to walk all over the AI with a squad of champions kitted out and moving at 9.

Will players be able to change the AI as you are doing now with the future Python tools?

Yes please. I would like to play against a at least somewhat challenging AI opponent that didn't cheat. That would be awesome.

I really love reading your journals as you continue to improve the AI. It is intriguing to learn how you investigate AI behavior and tackle the issues. Yet, I grow concerned as you seem to focus only on Strategic AI. Can you give more information on what your plans are for tactical battle AI?

Thanks for your hard work.

Thanks.

I don't actually program the tactical AI. That's done by different developers.  I will, however, be taking that part over partially starting 1Q2011.

As far as implementing an AI 'cheat, I'd suggest bonuses to production, gildar, tech, arcane, etc.  Nothing huge (except on ridiculous).  Just make the AI 10% better at stuff, as an example.

Except on Normal or easy, of course.  I want at least one level where things are 'equal'.

What would be better for the AI is if it had reasons to trade with players.  Tech Trading in  GalCivII was a big deal for me, and it'd be nice if we could do that here (with appropriate adjustments to research time to reflect this path of tech acquisition).  Same with spells.  Enemy empires stuck around a lot longer in that game because 1) they could be rather expensive to take out, and 2) I had reason to trade with them (I went for tech victories a lot in GalCivII).

Sometimes, you have the tech and they have the cash, or they need the cash and have the tech you want...

A quick note on early game defenses.  I find that it is important to get two defenders in place ASAP, so that the lone bear or whatever has a hard time taking out your city.  Usually I go for one peasant, followed by a lightly armored spearman (the peasant is for soaking damage and protecting the spearman).  Once these are in place, I work on getting better units to replace them.

Also, Janusk and my Sov are tasked with exploring as many goody huts as possible (they have gold, materials, metals, etc. in them), to help offset early game shortages.  I'd rather that we were able to produce resources more consistently, and rely less on the goody huts, but for now that's the mantra.  Bumping adventure tech early is a priority (so the level 2 and 3 goody huts become available), along with some equipment techs (so better armor and weapons are available).

As for the techs I grab about immediately.

Adventure tech 1 & 2 (to get level 3 notables).  Finding 150 Gildar in goody huts is a huge deal when equipping heroes.

Maybe Hero tech 1 (to get level 2 heroes) if I don't see many nearby.

Harvesting (if there are metal mines, etc. nearby)

Equipment (although I usually have Tools of War already).

The tech that reveals resources isn't an immediate priority these days.  Usually I find enough nearby that the only thing I'm short on are materials.  Goody huts almost always keep my metal supply well stocked in the early game.

As for Spells

Shard Harvesting from the spell tree (if there is a shard nearby).

Familiar - Provides a damage soaker/escort for my Sov.  And sometimes he wins too!  The 1 mana (re)summon cost probably should be looked at...

Alchemy - May need this if things aren't going well with the goodie huts.

A ranged attack spell - unless you have bows available.

Healing - so your Sov can stay in the field.  Salted pork also helps.

After that, it really depends on my situation.  If I'm awash in cash and heroes, I let them do the dirty work and steamroll enemy cities, using familiars, peasants, etc. as shields.

If you are lucky enough to find that neutral city with the awesome swords early on, that'll change your game about immediately for your heroes.  This might need to be looked at for balance purposes...

I'm a city spammer and I like it.  Pioneers get made as soon as I find stuff for them to exploit, and they may or may not get an escort depending on what's nearby.

Building priorities are materials, hut, food as required, tech, then arcane.  Note that I take the +1 tech and arcane in Sov creation, so I'm already researching on turn 2.

Your work on the 'strategic' AI will make things more difficult in the long run, so we don't simply uber our Sov and waltz in to the enemy cities and take them over.

I take pretty much every 'boost' in Sov creation I can (+1 Gildar, +1 Arcane, +1 Tech, +2 Mana, +2 Food, +1 or 2 Move).  This wipes out points for characteristics, but to be honest boosting levels for my Sov isn't that hard to do, with each level giving 3 points for Str, and other characteristics as needed.  As long as the Sov buys armor as soon as possible (4 points minimum), plus a spear (9 att is huge in the early game), the Sov can deal with most creatures, and maybe grab a lightly defended/just founded city early on.  Then kill every low level creature the Sov encounters (unless low hit points are an issue), using a Familiar as blocker/damage soaker to gain XP.

I find that a high ATT is much more important than a shield in the early game, with how damage works, but that's JMHO.  If  you successfully quest for the Sword of the Sovereign, then getting a shield isn't a bad idea, but the 9 ATT spear is almost always the way to go until better weapons come along.

The Twilight Axe (level 3 quest) probably should be slightly tougher than it is now, but again JMHO.

I try to send out heroes in at least two different directions (three or four if I have sufficient heroes) on goody hut detail, as in the early game the AI plays nicey nice.  If I'm not looking to steamroll a city, I'll get the non-agression ASAP, as that buys me an extra 99 turns of exploration in the AI's turf.  If the AI would just search their own goody huts...

Steamrolling AI players early is a priority right now, so they don't create those uber stacks that cause headaches later.  The only time I DON'T steamroll is if I simply can't take them out (due to small exploration stacks).  This is where hiring heroes becomes so important (so the Sov, etc. has friends/fellow attackers when they take on the cities). 

Keep in mind that you can shop at your newly conquered city on the next turn for new stuff that may have become available through tech research, which makes uber-ing heroes and Sovs that much easier.

Under Multiplayer, this strategy would be easier to counter, but as it stands now the AI is rarely prepared for this in the first 100-200 turns, i.e. when 'early game' weapons and a high STR can get the job done.  Sometimes I'll grab some units for escorts, but heroes are generally tougher with some equipment upgrades so they really are the way to go.

That's how I've been playing Elemental lately.  Hopefully this insight helps you in your AI tweaking.  I'm sure others will chime in with their own strategies to consider.

well I don't mind if the AI cheat at the highest difficulty level, as cheater (player) will find challenge that way. Remind me to my friend who cheated in Master of Orion 2. Because he found no challenge to lower difficulty level, then he just prefer the highest difficulty level for challenge. Well, cheater must stands against cheater

Great Dev diary and thanks for all the hard work. Not having read the entire thread, where do we get these experimental .EXEs?  Do I need to hang out at the channel at the exact time they are given out?

Ok... then my responses below might be useful...

1. What techs are the ones it should get.  You modders out there can help if you want to.  The tech tree has an AI General section in the XML that I’ve started making more use of to tell the game what techs the AI should value more than others on researching.

Gene: If the AI personality uses numbers for dominance then it should beeline for Squads, but also weight Harvesting(2) and Mana Shard Shrines(3) up in the top few priorities or as needed at the start.  On all Ridiculous AI settings, I experienced AI opponents that get something to the magnitude of a  10X bonus to Arcane and Tech research, so exact order is less important than if no cheating is allowed.  If the AI personality is using Peace through Strength approach of the Reagan-era then Focus on the Equipment branch and rush to Leather and War Staff or Battle Axe if metal is readily available.  

2. The AI has a hard time planning ahead on resources. This is a classic issue for any game and it’s typically solved with directives.  Essentially, the AI has a hard time saving up citizens to build things like Command Posts which in turn keeps it from building squadrons of soldiers.  This is a high priority.

Gene:  Does the AI use Fuzzy Math?  if so then weight it so that it "tries" to create a 10% buffer of unused/available resources.  This provides the inventory needed when it is REALLY needed.  On AI Ridiculous settings, I do not see this problem.

3. The AI is not yet good at specializing cities.  It does specialize but some APIs I don’t have easy access to yet.  For instance, the upgrade options when a city levels up have no AI data attached, they’re they might as well be random numbers.

Gene: Give the AI a reason to specialize.  If the city is or will be linked to a gold mine then specialize in Gold bonuses. If linked to an Arcane Temple then specialize into Arcane Research. If linked to Tech Temple then specialize into Tech Research.  In general, I specialize in Gold Production since Gold = Materials, unless I have a Tech or Arcane building to encorage my research specialization.  Sometimes an outpost will become a level 3 town that has gold bonuses with two huts and as many workshops as I could fit/afford.  If I have excess gold and shy on materials I go and trim forests to make up the shortfall.

4. General FOW issues that people tend to forget.  I miss the guard tower. The AI was much better at seeing approaching armies with that.

Gene:  The Guard Tower never did anything noticalbe for human players.  Not sure what it did for the AI...  Regardless, this is why the AI should have a lightweight disposible ranger unit (SCOUT).  Scouts or recon units should used for the data collection AI threads.  1-2 per town is plenty.  They should be high MP (movement point) high-visibility units to optimize their native ability to stay out of fights, there value is data collection, ie. find the resources, lift the FOW, locate targets and hazards(Prey and predators).

5. How many soldiers should be left to defend a city? How close should a protective army be left?

Gene: Using Fuzzy Logic right?  the answer is it depends...  Too few and you lose it, too many and you stagnate.   You should have some kind of slider scale that you can use as an arbitrary measuring stick and then scale or or mod it realtime as the game experience dictates.  You should be able to answer this better than us.  What does the Monster AI use to determine when it will attack a settlement versus not?  What is the Value of a particular settlement to the worth/value of the whole nation/faction?  if a nation/faction has 4 settlements, and one is level 3 and two are level 2 and one is level 1 then my rough fuzzy math suggests that 1/6-1/12 of the factions troops/force should be based in the level 1 settlement and 1/2 -1/4 in the level 3 settlement, but this is without any other parameters known.  If I am adjoined to another faction, is it friendly, is it adversarial? if I am still facing frontiers are they visibly empty or are the factions greatest threats roaming nearby?  This data needs to be able to modify the fuzzy logic used to determine troop strengths.  (I am Assuming the AI Faction do fight monsters like human players do, please pardon me if this is not the case.) 

6. What spells should be used when? There’s no AI data yet connected to spells (that’s a high priority to v1.2) so the AI has to pick things based on general classes. It’s gotten pretty decent at it but it could be way more optimal.

Gene: Spells should be considered when (A) available and (B ) considered against the risk of losing units.  If using a spell will or can prevent the loss of a combat unit then it should be used if it is available.  Scale the potency of the magic used to the situation and the mana inventory available versus the value of the units.  IE. pull all the stops out to save the Sov and most to save a family member, but only most to save a hero and maybe only half to save a combat troop.  Apply Fuzzy Logic Here.  NOTE: Flame DART has too high a mana cost to be effectively used and by the time Arcane Arrow is learned, Flame Dart is obselete.  Some of the issues are with the spells themselves.  That Earth Spell which reduces combat movement by 50% is an excellent spell and often more useful in battle than direct damage spells.  Look for and develop AI Personalities that focus on or prefer one element over another.  EG. maybe Tarth prefers earth magic over fire.  or maybe make it soe that the AI is adapable and can leverage the element that it holds the most shard types.  (2xFire, 0x others) 

7. Unit design is very challenging in general.

Gene: With the new combat mechanics, my observations are, in general, that the armor is 45% as important as the weapon now.  Wherein 1.09e and earlier, the weapon mattered 90-99%.  I recommend - design scout units with max MP (speed/movement points) and visibility. Design combat troops with full armor (Helm/Greaves/Chest/arms) and if you can spare the gold then add the soldiers cape/cloak +2 Def) and the best 2 hander you have tech'd out.  (Note: I agree btw with others that currently the 2h weps are a bit overpowered in present v1.1, maybe beef up the 1h weps rather than nerf the 2h, IDK)  Support Troops can be glass cannons to cut cost or just chest armor, but be prepared for them to become targets and maneuver accordingly in tactical combat.  If there was a Taunt available then it make it more like a WOW MMO, but there is not and I am not advocating one.

Oh and please recognize that the AI is not smart enough to move the AI Sov safely by itself.  So do not allow it to do so. 

=== THIS IS MY CHIEF AI GRIPE!!!===

NO! never make the AI 'earn' more, then you are stuck with the horrible realisation that when you take a city, it is ruining in the red, not the green like you though. Same with the 1.1 increasing of AI and NPC stats, you don't want to see the same units ruining around but not being seen as equal.

Modify the costs, research times  and dice rolls at the different AI levels. So the AI buys things cheaper, learns things faster and rolls better then you, these are mostly invisible to the player, and so doesn't look like cheating.

What i never see is the ai using bows with squads/teams. Sure its a pretty long procedure to train these but its working for me. Especially if you dont have iron it can save your arse. Archers normally dont need that much armor anyway so the cost dont get out of hand (except then i use a "Att"-weighted combo of trinkets).

Maybe the Ai should, if not done already, take its resource situation into account before choosing what to research. I mean heavy plate makes only minor sense if you dont have a good income in iron. On the other hand if Crystals are abundant you can leave some war techs out for a time and get trinkets instead that do virtually the same and leave you more design choices.

The problem with the watchtowers could be circumvented by the Ai building a small number low At/def units with iirc the travelers cloack and the explorer pack (cant remember the correct names) + fiting magical trinkets. If mounts/shoes/trinkets are included tha hand out mor movement they work perfect as scouts too. 

My guess, regarding archers, is that the tech for archery and the improvement for archers is not high enough priority as I never see the AI training archers either.  I'll take a look at it.