Frogboy Frogboy

Tonight’s AI work

Tonight’s AI work

As some of you know, I’ve been working on a lot of the low level systems of Elemental during the Christmas break.  With the team on vacation, this is a good time for me to get into the bowels of the game itself to make changes without worrying about breaking other people’s changes.

Computer AI is my passion and in recent years (especially since GalCiv II) I haven’t had the pleasure to do nearly as much AI coding as I’d like (with the whole day job of running Stardock and all).  So this is how I’m spending my Christmas break.

Core weaknesses of the Elemental AI

I have a fairly endless list of criticisms with the Elemental AI so I won’t go into more than a few major ones.

The most significant one is that unlike GalCiv, Elemental isn’t able to simulate future turns.  In Galactic Civilizations II, the game would, in the background, play forward a few turns based on its best guess of what other players would do and then react to those future moves as if they were in the present. 

Unfortunately, Elemental is a much more complex game (magic and all) from an AI point of view so it’s reactive. In time, as machines get faster, I hope to eventually create a more general land-based AI that can, like GalCiv, play turns into the future in the background.  But for now, we’ll have to stick to reactive AI with future planning capabilities based on pattern recognition.

Now, before I go into the basic weaknesses here, it’s important to understand that Elemental is not a scripted AI.  That is, I don’t program in to tell it to build this or that in some sequence.  That would be easier to do to be sure but it would, in the long run, result in a game that would eventually become very repetitive.

Instead, my job is to teach the AI how to play the game and have it try its best to “win” the game based on a set of rules and conditions that it is trying to optimize.   The problem with this is that until I get really good at the game, there’s going to be glaring holes. 

In GalCiv, I was pretty good at it so I could teach that AI how to play pretty well. With Elemental, I’m still learning.  When I read on the forums what people are doing I get the distinct feeling I used to get when playing Civilization and thinking I was doing well when I built my first musketmen only to have the Mongols land with tanks. 

So here are the basic problems with the AI that I’m working on:

1. What techs are the ones it should get.  You modders out there can help if you want to.  The tech tree has an AI General section in the XML that I’ve started making more use of to tell the game what techs the AI should value more than others on researching.

2. The AI has a hard time planning ahead on resources. This is a classic issue for any game and it’s typically solved with directives.  Essentially, the AI has a hard time saving up citizens to build things like Command Posts which in turn keeps it from building squadrons of soldiers.  This is a high priority.

3. The AI is not yet good at specializing cities.  It does specialize but some APIs I don’t have easy access to yet.  For instance, the upgrade options when a city levels up have no AI data attached, they’re they might as well be random numbers.

4. General FOW issues that people tend to forget.  I miss the guard tower. The AI was much better at seeing approaching armies with that.

5. How many soldiers should be left to defend a city? How close should a protective army be left?

6. What spells should be used when? There’s no AI data yet connected to spells (that’s a high priority to v1.2) so the AI has to pick things based on general classes. It’s gotten pretty decent at it but it could be way more optimal.

7. Unit design is very challenging in general.

So after, a week and a half of working on it, here’s where it’s at:

Me vs. AI.

TURN 5

ME: Malice

image

Starting position.

In my private build of tonight I made some significant improvements to how resources are distributed. I got very lucky here that I found a lost library so close. So this should give me a significant advantage.

AI 1: Altar

image

They picked a farm to build right away.

AI 2: Umber

image

They too chose to build a farm.  They’re not hard coded to build farms, they just both decided this.

TURN 20:

ME: Malice

image

On turn 20, I’ve got my Beacon of Hope done, farm is built, I’m building a hut and I have a workshop. I’ve trained 1 defender and I have a Pioneer in training.

AI: Altar

 image

So here I have my first problem.  The AI is not constructing anything.

They’ve got a farm, a hut, a study, and a workshop. So they look pretty good.  The city is lightly defended.

AI: Umber is doing better, they’re already level 2 on their city and they’re building their second labor pit.

So let’s go look at the code…

What is Altar…thinking?

image

Altar isn’t building because it’s saving up materials to build their highest end soldier.

Now, the thing is, that’s probably not necessary this early on.  The problem is, the AI isn’t looking at threats very closely, it’s just “preparing for the worst”. A smarter strategy would be to have it scout a little bit better, find out the level of threat. If no threat, build up the economy. If only a minor threat, built a weak soldier and if the threat is significant then build up.  Right now, it’s basically prepping to get “rushed” by either people or monsters.

So I’m going to work on this…

[3 hours later]

Side note, why do saved games take a long time to load on bigger maps?

Answer: All those fancy city improvements use 95% of the load time to regenerate. Going to see what we can do about that in the future.  If you have a quad core (ahem) it’s much faster. But still, if we’re ever going to have sick sized, 64-bit maps, and we are, we’ll have to optimize this.

Now what does the AI do?

image

Better.

Let’s move forward a bit.

image

Learning spells.  I always got for alchemy. Early on, money is an issue so I tend to try to get that right away.

I do think that the amount you get should be based on your intelligence and it should get more expensive each time it is cast by someone of your faction.

Turn 50

On turn 50 let’s see how things are going.

ME: Malice

image

The bear has me a bit nervous. I have a couple of guards in the city. I’m sending out my pioneer (escorted).

AI: Altar

image

They have 3 cities right away. Impressive if they can hold them.

AI: Umber

Here’s an issue I’ve been trying to resolve, the overdefense of a city.

image

No nearby threats and they’ve got a defense of 40. Let’s see why…

It just determined that it needed those units. I could quibble, but I will keep an eye on it.

TURN ~100

image

So Umber thinks they can take me.  Let’s see if I can steam roll them still.

image

I have 3 good cities.

Umber has 4.

image

 

image

And here is when things get problematic.  First, the unit’s weapon is pretty weak.

Second, it’s building individuals.

So we have some questions here:

1. Could the AI be training better units?

2. Is this the best possible design the AI is capable of on turn 100?

3. Could the AI build the war council and thus train multiple units? If not, why not?

So let’s look at those 3 questions…

Umber’s capital..

 

To the question of whether it can build a war council, it does have the tech.

The problem: The War Council requires 35 citizens.  As players know, that’s a lot of citizens and will take several turns of doing nothing empire wide possibly to get to that many free citizens. And that’s one of the big challenges – delayed gratification.  It’s one of the things I’ll be working on that will make a big difference coming up.

If only the AI could cheat, it would be so much easier. Smile

So let’s try steam rolling them with my squads…

About to attack…

Ok, lost that battle.

They are building squads (teams?).

image

But they’re in no condition to go after me yet.

After much AI tweaking…

[5 hours of coding later]

image

Umber is closing in on me with their army but I have a pretty formidable defense.

Now, I could have taken them out earlier but I kept updating the AI. Tweaks here. New API there. Bug fix there.

I haven’t decided the best way to have the AI “save up” in order to build more powerful improvements yet when it requires several turns.  In GalCiv, the AI simulated future turns and thus could just wait. Here, I have to come up with something different. Strategically (from a military point of view) this is the AI’s most significant weakness.

As you can see in the above screenshot, the AI will build groups of units. It just doesn’t do it when it really should.

I also need to get the AI to be far more aggressive about combining up.  It has 6 armies in my territory but it would do well to combine those into 3 major army groups.

One of the big tweaks has been the AI taking a larger view of the battle field. It’s a more expensive calculation but with threading, it doesn’t’ slow the game down.

As a result, the AI is willing to send its forces much further away from home base.

image

[Another 3 hours of programming]

image

The AI is starting to get its act together on forming a more cohesive military strategy.

image

Combat rating of 155 means they’re ready to conquer.

Sadly for me…

image

Umber is fighting two wars at once as they slowly crush Altar.

So after many tweaks, the AI was victorious. But could it win without me giving it lots of second chances to get it right? Not in this game.  The question will be, can it do it better in a fresh game?

We’ll see.

 

## UPDATE ##

image

The AI has gotten better at storing mana for future…use.

178,139 views 90 replies +3 Loading…
Reply #76 Top

Quoting Corbeaubm, reply 73
Whoa, whoa, whoa.  I'd just like to ask, for a moment, why on earth the AI has to "save up population"?  I have never, outside of the first five turns of the game, had to wait on population growth.  Never.  Not once.  It's a complete and total non-issue.  How on earth is the AI having trouble with this given their immense prestige boosts on higher difficulties?  I genuinely do not understand this at all.  I've always felt that population was pretty meaningless as a restriction, as currently designed, given how easy it is to come by.
End of Corbeaubm's quote

I ran into this problem recently. I just played a game where I city spammed to push monster spawns back and in the process I spammed libraries. Well, that quickly maxed out my pop. I never ran into it with a small number of cities as the prestige gain, combined with housing tech improvements, outpaced anything I could spend the population on.

Reply #77 Top

There have been many times i hit the population cap and i have had to destroy a few buildings to make some troops. If you can't find the food resources on the map and your behind the comp in production, you have to focus more on the warfare tech tree or you will get run over by the ai.

However, with the tweaks in version c and d, i find this less of a problem for a HUMAN player.

Now for a COMPUTER player, they run into a different problem. Since they can't predict when they will need an army, they will take advantage of the short term benefits of building more resource production - the computer can't make a decision on the cost benefit of saving 30 population or so to pump military if it needs it. Since they have all the bonus gold, they can keep paying for more buildings and end up burning through their available population. That's why frogboy was saying they did not build groups much before. Then they would build a hut, free up some population and pump out more single units and spam some buildings.

Ai play style is different from a person's play style and frogboy is slowly bridging the gap.

Incidentally, did you play frogger much as a wee tadpole Brad ?

Reply #78 Top

Quoting AlLanMandragoran, reply 76

Quoting Corbeaubm, reply 73Whoa, whoa, whoa.  I'd just like to ask, for a moment, why on earth the AI has to "save up population"?  I have never, outside of the first five turns of the game, had to wait on population growth.  Never.  Not once.  It's a complete and total non-issue.  How on earth is the AI having trouble with this given their immense prestige boosts on higher difficulties?  I genuinely do not understand this at all.  I've always felt that population was pretty meaningless as a restriction, as currently designed, given how easy it is to come by.

I ran into this problem recently. I just played a game where I city spammed to push monster spawns back and in the process I spammed libraries. Well, that quickly maxed out my pop. I never ran into it with a small number of cities as the prestige gain, combined with housing tech improvements, outpaced anything I could spend the population on.
End of AlLanMandragoran's quote

 

My standard strategy is to spam infinite cities for their trade route bonuses and spam non-stop libraries in every satellite.  I've STILL never hit the pop cap.  You're doin' it wrong.

Reply #79 Top

I don't depend on those trade  routes for gold and i find it not worth the resources to defend those little cities which don't provide me very much benefit when i am swimming in food. Besides, caravan routes die easily to things and i hate micro managing the trade routes.

Reply #80 Top

Quoting Corbeaubm, reply 78

Quoting AlLanMandragoran, reply 76
Quoting Corbeaubm, reply 73Whoa, whoa, whoa.  I'd just like to ask, for a moment, why on earth the AI has to "save up population"?  I have never, outside of the first five turns of the game, had to wait on population growth.  Never.  Not once.  It's a complete and total non-issue.  How on earth is the AI having trouble with this given their immense prestige boosts on higher difficulties?  I genuinely do not understand this at all.  I've always felt that population was pretty meaningless as a restriction, as currently designed, given how easy it is to come by.

I ran into this problem recently. I just played a game where I city spammed to push monster spawns back and in the process I spammed libraries. Well, that quickly maxed out my pop. I never ran into it with a small number of cities as the prestige gain, combined with housing tech improvements, outpaced anything I could spend the population on.

 

My standard strategy is to spam infinite cities for their trade route bonuses and spam non-stop libraries in every satellite.  I've STILL never hit the pop cap.  You're doin' it wrong.
End of Corbeaubm's quote

I just hate that all these strategies sound so "gamey". Like in the real world, "Spamming" cities wouldn't work because there wouldn't be enough wealth to distribute on the "infinite trade route bonuses" and with spamming libraries you would run out of efficient researchers- making it pointless. Housing would be based on the wealth of the individuals living there, not based on some channelers random idea to upgrade all huts into houses. It's just a shame that more realistic mathematics haven't been put forward as ideas. I'm not a mathematician to say the least but I imagine equations similar to these could work

[individual citizens skill]x[need or lack of that skill in area]=[individual citizens wealth] if building pre-requisites (smith for a blacksmith) exist.

So naturally, high skilled citizens could afford the houses or manses while less skill citizens would get paid less, and have huts and thus having less to contribute to the coffers, or caravans. All this would be done in the background. Random citizens would have random skill, no need for micromanaging anything. So When tax time come around...

A caravan arrives in City A. Takes [% of all citizens income based on the tax rate you set]+[small % of prestige?] (higher living). Caravan would have to travel from City A all the way back to the capital then with the money it received from City A before that money would go into the coffers to be used by you. 

I imagine the devs could come up with better equations but I've never understood why games have such gamey equations instead of looking at how reality works and trying to make equations based off of those observations. This is supposed to be a living breathing world. If this could be a game where a natural, world like economy could flourish and cities grow and shrink based on wealth, and resources are treated as valuable as gold, you would have a game that would break barriers in the cause for immersion. No game has ever been able to make me feel like I was actually the leader of anything that mattered. Every game has me thinking up "gamey" strategies just to win. I want to have a game that actually makes you feel you are part of a real world and making a different experience every time you play. Is that really too much to ask for?

Reply #81 Top

I recommend playing some Dwarf Fortress.  Not that I don't think the Elemental mechanics need some serious revision (they're basically all fantasy/strategy tropes mixed together with very little feeling of coherent direction), but there are some serious levels of abstraction that most players want in a strategy game.

Reply #82 Top

what about TONIGHT'S AI WORK?

:)

Happy new year all!

Reply #83 Top

Quoting RisingLegend, reply 80

I just hate that all these strategies sound so "gamey". Like in the real world, "Spamming" cities wouldn't work because there wouldn't be enough wealth to distribute on the "infinite trade route bonuses" and with spamming libraries you would run out of efficient researchers- making it pointless. Housing would be based on the wealth of the individuals living there, not based on some channelers random idea to upgrade all huts into houses. It's just a shame that more realistic mathematics haven't been put forward as ideas. I'm not a mathematician to say the least but I imagine equations similar to these could work

[individual citizens skill]x[need or lack of that skill in area]=[individual citizens wealth] if building pre-requisites (smith for a blacksmith) exist.

So naturally, high skilled citizens could afford the houses or manses while less skill citizens would get paid less, and have huts and thus having less to contribute to the coffers, or caravans. All this would be done in the background. Random citizens would have random skill, no need for micromanaging anything. So When tax time come around...

A caravan arrives in City A. Takes [% of all citizens income based on the tax rate you set]+[small % of prestige?] (higher living). Caravan would have to travel from City A all the way back to the capital then with the money it received from City A before that money would go into the coffers to be used by you. 

I imagine the devs could come up with better equations but I've never understood why games have such gamey equations instead of looking at how reality works and trying to make equations based off of those observations. This is supposed to be a living breathing world. If this could be a game where a natural, world like economy could flourish and cities grow and shrink based on wealth, and resources are treated as valuable as gold, you would have a game that would break barriers in the cause for immersion. No game has ever been able to make me feel like I was actually the leader of anything that mattered. Every game has me thinking up "gamey" strategies just to win. I want to have a game that actually makes you feel you are part of a real world and making a different experience every time you play. Is that really too much to ask for?
End of RisingLegend's quote

Couldn't agree more. I was sold on this game because it was supposed to make you feel like the leader of a nation in a fantasy world, with living breathing cities and people. But what we have is a game where you have to rush towards a victory goal by building 'cities' filled with libraries or arcane laboratories, so you can be the first to start cranking out uber-units. The only thing I kind of care for are my heroes and children, because even though they are all pretty much the same, at least I invested some time in improving them...

Reply #84 Top

okay - finished reading the whole post.

I keep wondering if the AI could think of the future in terms of available present "virtual" units.  "I really need this other building or unit, so I'll start building it" where "building it" means doing the necessary research/buildings/saving for townspeople/etc.  It means more work to calculate how much a virtual unit costs.

And the crux of that problem is determining which virtual units (out of an infinite space) to choose from.  It's a fuzzy problem, balancing between: 1. economy, 2. intelligence (recon) and 3. security (military).  I'd want to use a genetic algorithm to approach this, where techs and units are given weights to determine build order.

Hope to god your underlying architecture is a clean design that lends itself to a lot of flexibility.  :)  As the architecture, so the game.

 

 

 

Reply #85 Top

Research balance continues to be a thorny problem. On the one hand, flexibility is always nice. It's nice when a design doesn't put limitations on players like "you can only build 1 archivist per city." On the other hand, the other options don't seem nearly as fun, useful or attractive compared to maximizing your research. It also under cuts the value of resource nodes for research.

Couple suggestions I've made in the past that I think still work now:

-DIMINISHING RETURNS, INCREASING COSTS.

Either or, take your pick. There is no trade off in buildings right now. 1 archivist will always be worth the same amount of research, and cost the same amount of resources.

In reality, setting up another 5 people in an archives somewhere shouldn't be as effective at the highest levels of research. It's not right now, actually, because the research level values get pretty high....but when you stack, you counter act the increasing scale of cost within the research trees. Since that's not really costing you much extra in the long run as a player (the resource costs, population and otherwise, are trivial. The space requirements are negligible), the player doesn't end up having to invest that much more in cheap research to get it to where they want it to go.

So every additional research building after the first in a city (each DUPLICATE that is), the cost at least should go up. At least the player is investing more heavily in something to get the benefits out of it, instead of mildly investing and getting the whole farm. Often research is the only thing left to build once the other essentials are covered, in any event. That really needs to change.

-RESOURCE NODES GO BACK TO % INCREASES TO RESEARCH

The current state of them isn't fun, balanced or interesting. To be fair, it wasn't any of those things under the old system either. % increases to research made for horribly overpowered research rates. But at least they were important. Now, LL and temples are really just saving you time and resources building that stuff yourself. (Which we know are both trivial things after the early game.)

Reply #86 Top

I always find it amusing when people say games are 'too gamey' and not real life enough, then load it up and blast a couple of dragons with a wand of freezing.

In every game there will always be strategies designed to exploit game mechanics. It's impossible to avoid. If you want realism, play the game with your own house rules. Otherwise go kill some dragons in real life and don't bitch.

Reply #87 Top

Just chipping in to say that I'd rather play an Artificial Intelligence (AI) then an Artificial Cheat (AC).

 

Reply #88 Top

I haven't been playing tons of elemental lately, since I'm waiting for a few more AI tweaks, so someone correct me if I'm wrong but in regards to archers in tactical battles.  The few times I have seen the computer with archers its been fairly easy to rush and take them out.  So once the computer gets in the habit of making them, maybe work on making it keep a guy or two back to defend them also.  Obviously your sovereign can just nuke them with spells, and your own archers cant take them out, but thats less time they are focusing on the foot soldiers baring down on them.  Also I don't know if its in the game, but maybe have all enemy archers focus their attacks on the weakest enemy to try to take them out quickly or some kind of focused attack for the computer.

Reply #89 Top

Quoting Nenjin, reply 85
So every additional research building after the first in a city (each DUPLICATE that is), the cost at least should go up. At least the player is investing more heavily in something to get the benefits out of it, instead of mildly investing and getting the whole farm. Often research is the only thing left to build once the other essentials are covered, in any event. That really needs to change.
End of Nenjin's quote

One way to do this is to introduce nation-wise corruption.  The total research generated by city buildings (not map-generated libraries/sanctums for thematic reasons) is charged a logarithmic rate of corruption, so that more research=higher rate of corruption.  This takes care of spamming libraries either in one city or via city spam.

Reply #90 Top


Answer: All those fancy city improvements use 95% of the load time to regenerate. Going to see what we can do about that in the future. If you have a quad core (ahem) it’s much faster. But still, if we’re ever going to have sick sized, 64-bit maps, and we are, we’ll have to optimize this.
End of quote

My family is very excited for when massive super sized maps will be possible.  I should have waited on buying my 6GB laptop, but it's been useful for other stuff.