Strategic Advice for the "New Player" in TBS: things every basic AI should know

By on April 30, 2010 11:21:17 PM from Elemental Forums Elemental Forums

TCores

Join Date 02/2010
+13

In a recently posted Dev journal, the topic of AI was brought up. I'd shared some of my experiences in programming AIs in a reply.

One of the maxims I came away with is that AIs can really benefit from supplemental help where the programmer, having a really good understanding of how to win the game, tells the AI how to do certain things.

 

In a strategy game, what are the top three things you would tell a new player about? You can use specific examples from specific games, or things that are just generally true of most strategy games. Hopefully, we can make a big list of things the AI should "know."

 

My top three are:

  1. "Don't lose the fleet." - if you're going to lose control of a point, retreat your defenders when it becomes obvious you're going to lose, and won't be able to inflict decent loses on the enemy. The only thing worse than losing a city is losing a city and losing your army.
  2. "Don't overextend or overstay" - if you're on a roll, know when to stop. Just because you can take the next city doesn't mean you should, especially if you won't have/produce enough defenders to withstand the counterattack.
  3. "When they're down, keep them down" - Destroy their ability to mount a counterattack. Wipe unit producing structures/buildings, obliterate economy. Induce shortages, withdraw, harry, hit another place.

 

The idea is that each of these boils down to a simple algorithm. Calculate losses, determine how much damage you are inflicting on an enemy by continuing to defend, if it is below a certain value, pull out.

Locked Post 43 Replies
Search this post
Subscription Options


Reason for Karma (Optional)
Successfully updated karma reason!
May 1, 2010 1:39:06 AM from Elemental Forums Elemental Forums

I agree with the three points and would like to add another.

Design the AI to win. Not to come in second place.

Reason for Karma (Optional)
Successfully updated karma reason!
May 1, 2010 2:18:50 AM from Elemental Forums Elemental Forums

Here you could basically state the whole "Art of War" book

 

Mainly:

Fight only battles where you win with low losses

When you start a war, be prepared. The outcome of the war should already by decieded by your preperation

Know as much as possible about your enemy

Reason for Karma (Optional)
Successfully updated karma reason!
May 1, 2010 4:34:19 AM from Elemental Forums Elemental Forums

First and foremost, the AI needs to manage its economy efficiently - this includes city development, resource use, research, unit design and production, and a dozen other things - without needing to cheat horribly. If it can also be taught basic military strategy, in other words how/where to use the units it has most effectively, that would be amazing - but first it needs to build an empire capable of competing militarily, economy comes before all else.

Reason for Karma (Optional)
Successfully updated karma reason!
May 1, 2010 7:34:55 AM from Elemental Forums Elemental Forums

Just a couple of my ideas:

- Protect the cities  - Not only with troops but with magical spells as well.

- Protect the Sovereign [#1 priority in EWoM, because Sov dies = Game Over]

- Protect the Heroes/Generals

Use counter magic IF it's possible. [IE. If the player is using fire based spells mostly, try to research and use spells which offers protection against fire magic.] Obviously if the AI doesn't has the required spellbook it won't be able to research these spells.

- Careful planning & decent strategical maneuvers. - Attack the weak points of the enemy. Withdraw if it's needed. Surprise & Blitz attacks.

- Exploit diplomacy. - Make alliances against other AIs or players.

- Be effective in tactical combat. - Clever use of units and spells.

Reason for Karma (Optional)
Successfully updated karma reason!
May 1, 2010 11:04:32 AM from Elemental Forums Elemental Forums

It's fine to say all this, but to codify it in an AI routine is entirely another matter. A lot of this stuff involves qualitative analysis which is extremely difficult for an AI to do.

Reason for Karma (Optional)
Successfully updated karma reason!
May 1, 2010 12:43:04 PM from Elemental Forums Elemental Forums

Quoting ChongLi,
It's fine to say all this, but to codify it in an AI routine is entirely another matter. A lot of this stuff involves qualitative analysis which is extremely difficult for an AI to do.

Thanks for making this point, ChongLi. What I'm trying to get at with this post is creating codifiable ideas for AI design. "Make the AI better/cooler" is all well and good, but the most helpful suggestions are those that are something a computer could be made to understand how to do, without a lot of work. In other words if the idea can't be described quantitatively (instead of qualitatively) or by a step-by-step guide, it's not terribly helpful like you say.

Each of the three ideas I suggested are things that could be programmed into the AI. "Don't lose the fleet" is actually a relatively simple problem that just involves a few calculations, something computers are good at. Calling it periodically in battle would greatly boost the AI's "intelligence."

 

Tormy- has some good points as well.

Reason for Karma (Optional)
Successfully updated karma reason!
May 1, 2010 12:56:17 PM from Elemental Forums Elemental Forums

joasoze reminded me of the Art of War. In it, I seem to recall a line that goes:

Quoting ,
Hence, when able to attack, we must seem unable; when using our forces, we must seem inactive; when we are near, we must make the enemy believe we are far away; when far away, we must make him believe we are near. Hold out baits to entice the enemy. Feign disorder, and crush him.

Which gave me a further idea on handling wartime with the AI. Once the AI decides to go to war against a player, it should consider (perhaps randomly) some "recipes" for conducting war. Selecting one of them, it should use it during the war.

One such "recipe," would be to instruct the AI to create two forces: a small one and a large one. Send the small force toward one city, wait until the player has moved troops toward it and begun to mobilize their troops to deal with the "threat," then move the large force on a different (more important) city. Effectively this would be an example of the AI deciding to "bait" the player and "draw agro" from the PC. : )

This isn't something the AI can "figure out" on it's own. It needs a step-by-step deterministic guide to do it, with rules with what to do in certain cases. "Little force hasn't been attacked after ten turns - start pillaging countryside." "Big force just got scouted by the player and was within his fog of war, abandon plan and attack normally." etc. That's what I'm calling "recipes" in this game: nuggets of good ideas that can carried out quantitatively.

Reason for Karma (Optional)
Successfully updated karma reason!
May 1, 2010 1:53:20 PM from Elemental Forums Elemental Forums

I find something which is effective against overwhelming AI attacks and/or counterattacks is to implement Russian style scorched earth policy. Destroy any useful buildings one turn before the AI takes the city, but leaving those buildings which will drain the coffers of whoever owns it. Then when the conquer it, it hurts them more than it hurts me.

Reason for Karma (Optional)
Successfully updated karma reason!
May 1, 2010 2:08:34 PM from Elemental Forums Elemental Forums

A few to start:

-Exploit your advantages, avoid your weaknesses -- using knowledge gained by non-cheating methods, construct your forces to counter your foe's strengths and avoid her/his weaknesses.

-Hit them where they're weakest, where they don't anticipate (not just the nearest).

-Avoid their strengths (withdraw when a strong enemy force approaches, leading them around so as to neutralize the force)

-Don't attack until you have sufficient forces to accomplish a major goal (take a city or a few cities -- enough to seriously crimp your foe).  To do otherwise just tips your hand and gets your foe to mobilize, making subsequent attacks less likely to succeed.

-Identify your foe's strategic resources/production/etc. and prioritize taking it out.   Make this a goal (see above)

-Be aggressive, to keep your foe off-balance.  Try to determine where/when the attacks occurs.  Keep the initiative.

-Information is vital -- scout, spy, recon, etc.  (hopefully the AI won't cheat)

-learn from previous battles, and expect your opponent to do also.

-vary attacks so your foe doesn't fine-tune their defenses/attacks, and to keep your foe off-balance

 

EDITED IN:

-choose units to sacrifice (soak-off in grognard terms).  In Civ4 catapults can be routinely sacrificed to weaken a foe, allowing other units to survive and gain experience.

-Don't have the AI always attack with the unit that has the best odds.  Another Civ4 example -- the game will choose to have a highly-promoted but weakened player's unit defend when there are much less valuable units available.  Example from a game a few weeks ago -- I took a city and garrisoned with a dozen units, one unit was very highly-promoted and damaged, the rest unpromoted.  One enemy unit attacked.  Even tho the city couldn't have fallen the game had the best odds unit -- the damaged highly-promoted unit -- defend in that meaningless defense when there were many other relatively valueless units available.  The good unit died. 

Units should have a 'preservation value' assigned, allowed to over-ride basic attack/defense odds, dependent upon the seriousness of the battle (defense of capital city and Sov -- highly important, defense of small stack or in an attack where no goal (ie - capture city, destroy resource, etc.) is achievable then highly unimportant.

-attack order is important -- AoE units first, less-valuable units first, etc.  Again Civ4 -- how often will a stack have swordsmen attack then catapults attack, instead of the reverse?

 

Reason for Karma (Optional)
Successfully updated karma reason!
May 1, 2010 7:07:42 PM from Elemental Forums Elemental Forums

Quoting ,

My top three are:

"Don't lose the fleet." - if you're going to lose control of a point, retreat your defenders when it becomes obvious you're going to lose, and won't be able to inflict decent loses on the enemy. The only thing worse than losing a city is losing a city and losing your army.
"Don't overextend or overstay" - if you're on a roll, know when to stop. Just because you can take the next city doesn't mean you should, especially if you won't have/produce enough defenders to withstand the counterattack.
"When they're down, keep them down" - Destroy their ability to mount a counterattack. Wipe unit producing structures/buildings, obliterate economy. Induce shortages, withdraw, harry, hit another place.

The idea is that each of these boils down to a simple algorithm. Calculate losses, determine how much damage you are inflicting on an enemy by continuing to defend, if it is below a certain value, pull out.

That algorithm doesn't work. The problem is how to calculate that "threshold" value. If it's static, then your algorithm will only work in some situations and do the idiot in all the others, and if it's dynamic, then it will be really hard to calculate as it will need a deep analysis of you and the enemies (and here you will have to cheat about what you know about the enemies or use non-perfect information and guess, which is hard too). Even if the static value could work, your algorithm is assuming you are only fighting one enemy, how do you expand it to N enemies and M battle fronts?

So while the ideas for game AI are in theory pretty simple, in practice they are really hard to code.

Reason for Karma (Optional)
Successfully updated karma reason!
May 1, 2010 8:23:47 PM from Elemental Forums Elemental Forums

Not being a programmer or game designer, I may be way off on what's possible.  That won't stop me from offering ideas, in the hope of my learning.

Regarding algorithims/etc., an AI that learns, game design for combat, etc., what's wrong with this sort of idea:

Suppose we're starting with a simple, typical battle -- city siege.  And we're making simple assumptions (troops essentially equal on both sides, no special abilities to start, etc.) and using 'average' capabilities (wooden walls, moderate unit strength/armor/hits/etc.). 

Let's also make simple definitions -- each side's strength is the sum of its unit's 'hit points', the fight lasts until one side is eliminated (the loser), if more than half the winner's strength remains it's a significant victory (and a significant loss for the other side), etc.  And let's assume the goal for the attacker is to have 60% of it's strength remaining.

If the attacker loses significantly, add 50% to attacker's starting strength for next time.  If the attacker loses normally add 25%.  If the attacker wins but has less than 60% strength remaining add 10%.  If attacker has more than 60% remaining subtract 10%.

Start with defender having 100 strength.  Make an assumption on what strength attacking force is needed to start then run 1000 attacks, with the AI 'learning' from each to modify starting attacker strength for the next attack.  You might end up with a pretty good estimate for that scenario. 

Toss out the first 500 attacks (assume it's still 'calibrating') then look at the last 500.  Does the determined attack strength jump around a lot?  If so there may be a problem with randomization of combat results, or some similar problem.  Run an ANOVA (or whatever they're doing these days).  Maybe the results are too consistent.  If so, add more variability.  etc. etc. etc.

Then repeat the above with defender strength of 200, 400, 800, etc., and 50, 12.  This'll determine the effect of defender streng and allow adding weights to adjust accordingly.

Then add in more specific capabilities, and repeat.  You may be able to make correlations, such as -- adding in 50% of the defender's strength as ranged equals doubling standard defender strength.  This will help create algorithims for army composition.  Similar comparisons can be made for other abilities, such as fire spells vs units resistant to fire, etc.

While the above is from the point of view of the attacker, it can be used by the AI on defense (by holding attacking strength unchanged and modifying defender strength).

Having an AI that is able to 'learn', and being able to run thousands of simulations very quickly/easily, for a number of different situations -- won't that allow some good guesses for constructing algorithims to make a decent AI?

Reason for Karma (Optional)
Successfully updated karma reason!
May 1, 2010 9:06:26 PM from Elemental Forums Elemental Forums

Quoting VicenteC,



That algorithm doesn't work. The problem is how to calculate that "threshold" value. If it's static, then your algorithm will only work in some situations and do the idiot in all the others, and if it's dynamic, then it will be really hard to calculate as it will need a deep analysis of you and the enemies (and here you will have to cheat about what you know about the enemies or use non-perfect information and guess, which is hard too). Even if the static value could work, your algorithm is assuming you are only fighting one enemy, how do you expand it to N enemies and M battle fronts?

So while the ideas for game AI are in theory pretty simple, in practice they are really hard to code.

I'm not sure we are on the same page here. I'm talking about having the AI know when to Retreat from a tactical battle, and I've written equivalent AI before in different games to that. Strategy games with "autoresolve" actually greately improve the accuracy of the AI, because the AI could be given access to a virtual "auto-resolve" to see how badly it might lose. Determining a "threshold" value is an arbitrary task, but the programmer can make informed guesses that will hold true in most circumstances. For instance: "at the point I am about to lose my entire army and kill less than 10% in battlerank of the foe, retreat" is not a terrible statement. These aren't complex problems.

Reason for Karma (Optional)
Successfully updated karma reason!
May 2, 2010 12:03:46 PM from Elemental Forums Elemental Forums

Quoting TCores,

I'm not sure we are on the same page here. I'm talking about having the AI know when to Retreat from a tactical battle, and I've written equivalent AI before in different games to that. Strategy games with "autoresolve" actually greately improve the accuracy of the AI, because the AI could be given access to a virtual "auto-resolve" to see how badly it might lose. Determining a "threshold" value is an arbitrary task, but the programmer can make informed guesses that will hold true in most circumstances. For instance: "at the point I am about to lose my entire army and kill less than 10% in battlerank of the foe, retreat" is not a terrible statement. These aren't complex problems.

 

That makes me think of the movie Wargames. Let the AI play itself at tic-tac-toe until it realizes that no one will win in the long run, so it should try diplomacy instead. Or maybe have it run battle simulations a few times to give it a value for "what are the odds I'll win?". Then the AI personality can determine if it fights or runs. Some will run at low % chance, while others will have "more of a taste for risk".

Reason for Karma (Optional)
Successfully updated karma reason!
May 2, 2010 7:17:42 PM from Elemental Forums Elemental Forums

I would say that the Singlemost Improvement that could be made to the AI is a tendency to ally against a giant Steamrolling Behemoth.

Basically, in Civ or FFH, it was always weird how the stragling survivors didn't either form a super Alliance against me or vassalize. (A few neutral nations is okay, but I would like to see a tendency for alliances vs a Monolithic Superpower)

Reason for Karma (Optional)
Successfully updated karma reason!
May 3, 2010 1:13:23 PM from Elemental Forums Elemental Forums

Quoting TCores,

I'm not sure we are on the same page here. I'm talking about having the AI know when to Retreat from a tactical battle, and I've written equivalent AI before in different games to that. Strategy games with "autoresolve" actually greately improve the accuracy of the AI, because the AI could be given access to a virtual "auto-resolve" to see how badly it might lose. Determining a "threshold" value is an arbitrary task, but the programmer can make informed guesses that will hold true in most circumstances. For instance: "at the point I am about to lose my entire army and kill less than 10% in battlerank of the foe, retreat" is not a terrible statement. These aren't complex problems.

The three things you posted in the first post happen at empire level: know when not to engage, how and when to expand, and which objectives to hit. Those are really hard to code decisions.

About combat, that's pretty hard too. Auto-resolve most times gives horribly wrong results, so an AI that uses it won't be very good actually. Really, if you have coded a professional game AI, you can't be saying these aren't complex problems...

Reason for Karma (Optional)
Successfully updated karma reason!
May 4, 2010 10:39:34 PM from Elemental Forums Elemental Forums

autoresolve is a fine measure of troop strength.  Your demand that the measuring stick be flawless is a bit extreme vincente

Reason for Karma (Optional)
Successfully updated karma reason!
May 5, 2010 1:35:46 PM from Elemental Forums Elemental Forums

I'm not saying it has to be flawless, I'm saying it is a very hard problem to solve well.

Reason for Karma (Optional)
Successfully updated karma reason!
May 5, 2010 10:52:50 PM from Elemental Forums Elemental Forums

If the game is constantly running autoresolves behind the scenes during tactical combat it can easily use that information to determine when it's chances of victory become slim. 

Reason for Karma (Optional)
Successfully updated karma reason!
May 6, 2010 5:31:41 AM from Elemental Forums Elemental Forums

And an AI using that calculation will only be good if the auto-resolve gives good results, but all auto-resolves I remember give results that aren't very similar to the real result if the player is involved.

Reason for Karma (Optional)
Successfully updated karma reason!
May 6, 2010 4:23:57 PM from Elemental Forums Elemental Forums

Quoting VicenteC,
And an AI using that calculation will only be good if the auto-resolve gives good results, but all auto-resolves I remember give results that aren't very similar to the real result if the player is involved.

It doesn't need to, when you make the decision to withdraw you don't do it with perfect knowledge of exactly how things are going to play out, you do it on your assessment of comparative troop strengths.  Auto resolves are a perfect way to measure that.  Again you're demanding a level of accuracy for a simple system to be way higher than necessary.

Reason for Karma (Optional)
Successfully updated karma reason!
May 6, 2010 6:33:27 PM from Elemental Forums Elemental Forums

Quoting KellenDunk,

Quoting VicenteC, reply 19And an AI using that calculation will only be good if the auto-resolve gives good results, but all auto-resolves I remember give results that aren't very similar to the real result if the player is involved.
It doesn't need to, when you make the decision to withdraw you don't do it with perfect knowledge of exactly how things are going to play out, you do it on your assessment of comparative troop strengths.  Auto resolves are a perfect way to measure that.  Again you're demanding a level of accuracy for a simple system to be way higher than necessary.
If auto resolve gives more than a simple 'who wins' answer then a 'fudge factor' can be added to account for when a player actively participates.

VicenteC has a legitimate concern.  If AI is just hard-coded then it's very important the devs create a challenging AI right out-of-the-box.  Flashy graphics attract folks to try a game, great lore will help keep folks playing, but lousy AI will consign a strategy game to the bargain bin pretty quickly, even if it has good MP and dedicated modders.

Reason for Karma (Optional)
Successfully updated karma reason!
May 6, 2010 10:22:18 PM from Elemental Forums Elemental Forums

Quoting Nick-Danger,

Quoting KellenDunk, reply 20
Quoting VicenteC, reply 19And an AI using that calculation will only be good if the auto-resolve gives good results, but all auto-resolves I remember give results that aren't very similar to the real result if the player is involved.
It doesn't need to, when you make the decision to withdraw you don't do it with perfect knowledge of exactly how things are going to play out, you do it on your assessment of comparative troop strengths.  Auto resolves are a perfect way to measure that.  Again you're demanding a level of accuracy for a simple system to be way higher than necessary.If auto resolve gives more than a simple 'who wins' answer then a 'fudge factor' can be added to account for when a player actively participates.

VicenteC has a legitimate concern.  If AI is just hard-coded then it's very important the devs create a challenging AI right out-of-the-box.  Flashy graphics attract folks to try a game, great lore will help keep folks playing, but lousy AI will consign a strategy game to the bargain bin pretty quickly, even if it has good MP and dedicated modders.

Well it could possibly run the autoresolve hundreds or thousands of times per round depending on the algorithm used.  It can then aggregate this data to decide whether or not to retreat and preserve troop strength.  I would think each AI could have a different threshold for when to retreat and when not.  It could use measures of losses for the autoresolve.  

The AI isn't going to be hardcoded we all know that.  And his concern in this case is about a specific action, knowing when to retreat.  This is a simple solution, he's just looking to make it be harder than it is.

 

One algorithm could be as I decided, based on percentages of winning in the autoresolve (or similar algorithm) runs.  Another could compare losses on both sides, perhaps the AI is all but guaranteed to lose, but the algorithm shows that it could take out as many as 5men for every 1 it loses.  How it reacts to that situation could easily be a value in the AI's xml file or whatever that determines just how much damage he's willing to suicide charge for. 

Reason for Karma (Optional)
Successfully updated karma reason!
May 7, 2010 6:06:30 AM from Elemental Forums Elemental Forums

Quoting KellenDunk,

It doesn't need to, when you make the decision to withdraw you don't do it with perfect knowledge of exactly how things are going to play out, you do it on your assessment of comparative troop strengths.  Auto resolves are a perfect way to measure that.  Again you're demanding a level of accuracy for a simple system to be way higher than necessary.

Except when combat falls on the gray zone where AI thinks it is going to win with auto-resolve (or inflict important damage) and it gets massacred in normal tactical combat (which it is not a strange scenario in games with tactical combat). For extremes, auto-resolve works, but for the cases in the middle, simply assessing troop strengths = inaccurate results = you have an AI that takes bad decisions.

Reason for Karma (Optional)
Successfully updated karma reason!
May 7, 2010 1:24:32 PM from Elemental Forums Elemental Forums

Quoting VicenteC,

Quoting KellenDunk, reply 20
It doesn't need to, when you make the decision to withdraw you don't do it with perfect knowledge of exactly how things are going to play out, you do it on your assessment of comparative troop strengths.  Auto resolves are a perfect way to measure that.  Again you're demanding a level of accuracy for a simple system to be way higher than necessary.

Except when combat falls on the gray zone where AI thinks it is going to win with auto-resolve (or inflict important damage) and it gets massacred in normal tactical combat (which it is not a strange scenario in games with tactical combat). For extremes, auto-resolve works, but for the cases in the middle, simply assessing troop strengths = inaccurate results = you have an AI that takes bad decisions.

 

Why should the AI make perfect decisions in the gray areas?  Do humans make perfect decisions when it would be close?  Don't you see how ridiculously flawlessly you're asking it to be?

Reason for Karma (Optional)
Successfully updated karma reason!
May 7, 2010 3:36:27 PM from Elemental Forums Elemental Forums

Quoting KellenDunk,

The AI isn't going to be hardcoded we all know that.
Can you point me to where a dev has discussed this?

Thanks!

And his concern in this case is about a specific action, knowing when to retreat.  This is a simple solution, he's just looking to make it be harder than it is.
The legitimate concern of his I referenced is how auto resolve will handle human participation when deciding when to retreat (or any similar action).  Computer vs computer auto resolve shouldn't be that hard (your suggestions seem reasonable and I think we all agree here). 

Human participation will have some effect that may throw off C vs C results (depending on the relative skills of AI&players).   Won't your "...run the autoresolve hundreds or thousands of times per round depending on the algorithm used." be just C vs C, and if so, how will you account for human participation (which is the only point of his I'm addressing) without some 'fudge factor'?  Without some 'fudge factor' I think his concern is valid.  With it (or its equivalent) I agree with you.

Hyperbole aside, I don't think you two (errr...  us three) are in that much disagreement and it's more a matter of focusing on slightly different aspects of the same basic point.

Reason for Karma (Optional)
Successfully updated karma reason!
Stardock Forums v1.0.0.0    #108432  walnut2   Server Load Time: 00:00:00.0000890   Page Render Time: