Tonight’s AI work

Oh looks promising!

 

On a sidenote...

 

I did not realize AIs were advanced enough to to predict future turns. For example does this simply mean that they  cheat and see you teching say horsemen in a game and respond by training pikemen? Or does it mean they notice from their encounters with your army that you are very horse reliant, or see buildings that support horse tech and in turn realize they should train more pikemen in war preparations against you.

 

On another sidenote...

 

What is the most advanced AI you have heard used in a game, and what is the most advanced you have seen outside of games. I'm a bit fascinated by AI programming lately and of course I'm curious how long before machines start thinking for themselves and takeover.

Nice post, Brad.  I've recently jumped back into civ5 and have been enjoying it.  BUT - it's starting to become clear that the ai is not so great.  And I doubt it will receive the same treatment that you guys will put into things.  Civ5 made quite a bit of its money already, but know that you guys will keep enhancing the game and making it better and better and that means that Elemental will become a game I'll be playing quite a bit in the future - after the cobwebs have set in on civ5.  Thanks again for such a detailed post - I'm sure that took some time and I appreciate it!

I'm looking forward to seeing what you guys continue to develop!

I don't think he's talking about the AI cheating, as in building pikemen to counter a player building horsemen (when the AI shouldn't know that the player is building horsemen). It sounds like the GalCiv AI, at least, is programmed to recognize some common behavioral patterns based on how players often play the game, and to use that data to predict the player's actions. From what I know, the AI in these games (Stardock's GalCiv, Elemental) doesn't seem programmed to act on information it shouldn't have (Hence what he mentioned about the Fog of War impacting the AI's defensive play in Elemental).

The current most advanced form of AI is the neural network. Some games have been incorporating it (I know Black Prophecy stated they are using one. I'm in that beta and it really feels like you are fighting a human.)

 

The problem with neural networks is that like a newborn baby, the AI is ignorant. You have to teach it (sound familiar). What generally happens is that the company teaches the AI the basics of the game and those neurons have their sensitivity stored. However after interacting with the player the sensitivity may change on individual neurons.

 

Elemental Example (Assuming a neural network is used)

If the AI detects that the routinely parks their armies on a border before declaring war, the AI may learn to start to marshal their forces before hand. In this example, the neurons would react to the following sensors: A declaration of war, position of forces, power of forces. The AI could find this pattern if it was given enough sensors and deduce that if you are massing troops on a border, war is likely imminent.

 

 

This book is worth every penny. It has a great explanation of genetic programming as well as neural networks and how they combine to create a truly human feel. http://www.amazon.com/Techniques-Programming-Premier-Press-Development/dp/193184108X/ref=sr_1_1?ie=UTF8&s=books&qid=1293515386&sr=8-1

well... isn't ai cheating just giving them additional gold/mana/pop/etc? That's the general AI cheat.  On easy give them less gold, etc, but they play the same.  On normal give them the same gold, but they play the same... and so on.  The real tweak is getting each civ/empire doing things differently based on their natural strengths (eg unique) and adapting based on what other ai's or humans as doing.  I can imagine it gets QUITE tricky on higher difficulty levels to avoid the extra gold, etc cheat.  

In a magical world, an easy ai would be programmed VERY differently than an expert ai and there wouldn't be a resource bonus.

Well, "most advanced" is subjective, I suppose. It's not the newest, certainly, and I can't imagine the cost-benefit ratio makes sense. It's a nice thought, but sometimes more advanced doesn't make sense. It is very hard to tweak AI behavior with a neural network, for example, and sometimes (as in most of the time), the local maxima that the network arrives at aren't optimal, even with extensive training. Even with a routine called simulated annealing, which combats the problem of local maxima, is horribly processor and "number of games before you get a good AI" intensive. It's also very opaque to tweak.

But maybe you're talking about a limited implementation of neural networks for certain limited tasks, in which case you might be right. It's something to think about for pattern recognition, anyway. I would imagine it would require an AI not limited by the fog of war to be any good, though.

1. Techs

First tier priories (order is depending on the agressivness ect of the AI)
Armour (at lest one level)Weapons(at lest one level),Shrines,Houses, Libraries and the techs associated with mining and the spell and research temples.
Second tire would be parties and experienced along with more weapons and armour.
Before the current version, it was always one off world buildings I ran to, now they are borderline useless.
It is more imported to be moving techs then magic, I run about a 4 tech to one magic building ratio after the first hundred or so turns.

The most beneficial thing hear would be a /dev option for ruining the game, a new directory with a simple plain text details list each turn. You could script something to pull out what techs,spells and average values of attack and defence are for human players and AI, and map at what point they fall down and die or over take the AI into steam rolling though auto generating some graphs. This would help with a lot of your issues I feel.

2. Resources.

Wants and needs, depending on what mode the AI is in, it should have diffident wants and needs, It always needs to stay alive, but is it on the offence, the defence, stable or exploring. It should be looking at what its techs and army's are like compared to those about it (or the few bellow if its on top). Reserves should be kept until ones needs (or wants) have an over whelming priority rating (say 60/40) Those reserves are used to bring back the wants and needs to balance.

3. Specializing

When founding a city, the AI should calculate the maximum it can get returned on just resources alone in the area, this should drive a split between where to develop the city. At certain levels of resources, the AI should add more to the maximum level of the city, no point building a level 3-5 city if their is only some food their, but if their is 3+ food, well build up to get bonuses form higher buildings.

4. FOW

I have often said, anything in your ZOC should be in your FOW. You own that land, you have ppl out their every day, giving feed back, all these uncivilized wonders coming in have tails to tell. Large armies just don't go around unnoticed.

5. defence

This should be based on wondering monsters and near by other players, for the players, the AI should look at the average amount of attack and defence per city of all opponents it has contact with, and think of defending from that point. It should also compare this value to what it sees each turn, if it sees anything high 

6. Spells

This is a tough boat, and I cant think off too much to add.

 

7. Unit designl

. Always have the best armour. are we trying to expand or brake a stale mate/close call - Add offensive items are we on a successful offensive? If so, pick the largest attack value Are we on the deference? If so, one handed only, with defensive items
Always the best unless you are looking at being defeated, then and only then do you compromise.

 

Final addition.

The AI's cheating, should be limited to altering dice rolls,costs and the FOW. 

What a juicy long dev post. Now to read it .

Also, it's just a thought, but it would be helpful to know the general form of the AI code. Not saying that you need to post the actual API (I know that it's a trade secret), but I know that I, for one, can't comment on this post intelligently because I don't know if you are using just plain equations to relate certain game variables to others, or using error corrections, or logic trees, or have any idea how the pattern recognition that you mention is coded and/or related to the other AI algorithms.

Anyway, it might be more trouble than it's worth, but not knowing the underlying mechanics of the AI limits at least one person's (mine!) comments on the current build.

Regardless, I appreciate both the work over the holidays and the updates.

Hmm perhaps if the game could collect play statistics. For instance, if every-time I played a game it recorded the turn number I got various techs. Overtime it could weight the techs for the AI based on my game play.

 

On a large scale if we had the option to submit a our game stats after winning a game to stardock you could build a very accurate picture to how the average person plays the game. Also you'd see if certain techs are never used. Then the devs could give those techs a little buff so they are a bit more useful.

 

Blizzard does this sort of thing for in WOW to balance the game. If nobody is using certain abilties they know they need a buff because they must suck.

No personal info need be in there, just stuff like first tech researched was tools of war. Or whatever.

 

It is sooo stupid to work on AI before implementing all features and making general changes like global mana pool.

Except some players want AI improvements now, not later.

Best regards,
Steven.

Anyway, cool post. When can we get our hands on this new squad building AI?

Not really. Working on low level stuff such as the helper methods and assorted APIs is not a waste of time at all. Working on improving the AIs higher-level reasoning isn't a waste of time either: not only does it allow Brad to learn new ways to deal with the AI, but it also allows for the AI to adopt new gameplay mechanics and content more quickly rather than trying to write all the AI at once. So yeah, there is some overlap, but the benefits far outweigh the disadvantages.

My experience with the AI (from the test build .exe thread) is that it doesn't recognize that champions operate on an entirely difference resource system.  They don't exploit the item shop to get around resource restrictions, and thus they are trivially-steamrolled by a player who does (I obliterated the AI on "extreme" - set in both world and individual AI difficulty).  Early plate, without iron, is still a complete wrecking ball.  Squads don't matter when they can't damage heroes.

I still think that the item shop is the part that should change, but even if it doesn't change the AI should still be capable of using such mechanics to create stronger forces.  They certainly have gold, but they don't research and utilize it accordingly.

Yep, iterative versus sequential design.  It's generally better - unless you can't avoid the alternative - to design iteratively; that is, to implement parts of one system, then parts of another, and so on, rather than trying to finalize a system before moving on to the next one.  I would guess that Brad's work on the AI will help inform the work that the rest of the team does on core mechanics, and vice versa.

Think I found something with the AI, if you conquer one of its cities, the AI can fall asleep.  Had this happen in one of my groups, where the enemy sovereign just stayed in my territory and did nothing for 10-20 turns.  Didn't save and reload to see if that solved the problem, but that shouldn't be the solution anyways.  I know a MP game I would never have had the time to pull off what I did, it took about 50-70 turns of full focused research to get a super-sovereign.

Also super-sovereign with movement enchants and everything on them and spell immunity, can burn through any army 1 on 12 squads while suffering minimal damage.  Kinda ridiculous, didn't even use the sword of wrath.  The biggest downside to this start is that mop-up gets very annoying- some sort of surrender mechanic needs to be implemented just to reduce the tedium perhaps?

 

I can tell you right now, improving the AI is nice, but I'm wondering how you can handle all these contingencies. 

 

 

I was playing a 1.1 game the other night, and the biggest thing I noticed was lack of AI aggression and poor utilization of troops.  I had one lightly defended city in the middle of my territory that could have caused me problems if it fell, but instead of attacking it with a combined force of six units, the AI would instead send six single units in succession making them easy pickings for my archer.  It did this several times throughout the game.

I'm glad you're recognizing how poorly the AI is choosing what units to build, frogboy.

I've had several games where the Ai can - and does - build some strong units. Then, it will simply revert to building crappy units again.

 

From what you're saying, the AI has trouble "saving money". Could it be that this is why I've never seen 8/12-troops squads ? And why high-end units are so rare ?

 

Also, the AI needs to use shields sometimes. All it builds are glass canons: cheap, efficient, good at taking down heavy armor but utterly useless against large masses of cheap troop squads. It needs to use combined arms.

The AI should establish a limited goal and work to fulfilling it. Not sure how effective this is with a reactive situation. But here goes. 

 

AI considers relative power ratios and decides an invasion is a good idea. It then runs down the list of things that would help with war -- better armor, weapons and improved squads. The list is pulled from XML tagging techs as war, or tossing them in the tree. It weighs each one according to costs of utilization and acquisition (so, research costs, per unit costs, building costs) and then decides based on which option is the most effective by looking for the one with the lowest adjusted value.

 

Adjusted value = assigned weight * tech cost * per unit cost * building costs

 

It then reserves the resources it needs to research and build necessary buildings for the desired ability. It places floor value holds for all resources it judges necessary to use the resource -- so, if it needs to build a command post, it puts a floor on pop that guarantees that it has the required amount of pop for every city where it wishes to build a command post. This results in both positive and negative action. If insufficient resource is had for floor, it (negative) stops doing things that require the resource and (positive) starts doing things that generate it. After floor conditions are met, it goes back to using the resource as normal so long as use does not breech floor.

Goals themselves are weighted, with highest weight goal getting priority on floor values (so, it will use the resources adjacent to the zero value). Goals should have their weights checked every few turns -- so if the AI goes to peace mode, it can bump out war tech implementation in favor of peace tech implementation.

 

I used something similar with a robot sim I wrote, and it worked out ok. Got a little overboard with planning and weight assessment -- I had the agents doing regressions with historical data to establish probable future values -- but it was, in my opinion, a passable approximation of how humans do things.

Hey Frog,

You mentioned population in your post.  I've mentioned this several times that the first building AI/players should be building in the current game is a hut if you're a farmer (i guess you needs farms first if you're not a farmer).  AI/players are not putting enough value on population right now.  Population drives your economy and higher end buildings.  When I see players/AI stopped at a max population limit, I just shudder.  That should never happen.  Population is too important in this game right now.

Once you have your population growing like crazy, the AI won't want to have worry about saving up 35 citizens.  They'll already be there.

By the way, does this mean there is an 11D out there?

I think it is necessary for Mr. Frog to play against human players, analyst it and then use the most effective strategy from the experience to teach the AI. Without the challenge and experience of being defeated, humiliated or maybe win against other humans, no one know how to make a clever AI.

 

What about challenge every veteran players in this forum one on one battle? The information will be priceless, and no one will find no challenge from the AI who can beat the best players of the game.

 

That's why the game like Starcraft or DOTA can have a tough AI, as the creator know how to teach the AI to beat the players. The experience of being beaten or to beat other players are priceless.

Well most human players at the moment pimp out their champions and kill whatever troops AI makes. So you should teach the AI to do the same.

Unless future patches will change this again... I hope you are willing to change the AI whenever changes happen in game mechanics