That would be really interesting. Would the LLMs actually know how to play the game well though? It’s an extremely complicated game with tons of hidden information. It’s way more difficult than chess or poker or diplomacy for that matter!
They would not have to be able to play the game if the job of the LLM was mostly to handle some diplomacy and communicate with the player. The strategy and tactics could still be handled by good old game AI tricks and tree-searches. Sounds like it could be fun for single-player games in general.
My guess is, as is, they would be able to play, but not be very good.
The main issue I see is the spatial component is hard to describe in text. The new vision models make it easier, but still I imagine it's not trivial to integrate all the mechanics plus the spatial component on the limited prompt space.
I do think that 1) combining with the hand crafted AI and 2) having an "LLM advisors" system where for a given aspect (eg military) the "advisor" would present the options and tradeoffs to the "Main AI" and the role of the latter is to weigh the tradeoffs between the options presented by the advisors.
And what I do know is that it could be so much more immersive than the current hard coded AI!
Yes, but do they get a vague idea of what's onscreen or could they really see what's going on in each tile, keep track of all the stats and use those to inform their decisions?
I don't see how it's a more complicated game than poker, which has a huge tree of possibilities at every potential street of betting. Not just in future streets, but also in the past streets that influence your future decisions.
The game tree of Civ is vastly larger than any board or card game. In a single turn you make dozens of decisions about what to build, where to move units, who to declare war/make peace with, what technology to research… and on top of that, games last for hundreds and hundreds of turns on a map with thousands of tiles which must be explored, resources uncovered, terrain improved.
Humans are decent at Civ exactly because we play in a naturally heuristic fashion and don’t try to do any tree search. To get an AI to cope with the this vast amount of information, available and hidden, and make sensible decisions without tree search, is a huge challenge. On top of that, it needs to deal with up to a dozen opponents, multiple of which may be human, and avoid getting ruthlessly exploited in trade deals and diplomacy and all the rest.
Here is one basic decision among dozens made every turn: do I accept an open borders treaty with Bob of the Carthaginians or not? I can’t see any armies he may or may not have (fog of war) and he’s never betrayed me before so I guess he’s trustworthy… oops! He used the open borders to move a vast army into my territory and right up to all my cities in a single turn and then declared war!
That’s one decision, among dozens made every turn, that led to total defeat.
It's easily more complicated, civ 5 is like poker, except you have to manage the location of each card, an economy, map control, barbarians, nuclear weapons, warmonger penalties, forward settlements, citizen management, wonder rushes, map scripts, restart.
An LLM could at least create a greater variety of dialog, quickly translate, text-to-speech, decorate a base sprite to the style of a specific country with a specific player color. The final build wouldn't benefit much by including an LLM. The LLM would be huge and the existing diplomacy algorithms are not complicated or open-ended enough to need any kind of ML.
I looked at freeciv and the code was unwieldly. Maybe unciv is easier to tamper with!