> I think any NPC with dialogue important to a goal (a quest, a tutorial, etc) is going to be hard to use generative AI for. It not only needs to be coherent with the story, but it needs to correctly include certain ideas. I.e. if the NPC gives a quest to go find some item at some location, it needs to say what the item is and where it is.
That was my experience when I was experimenting with using current LLMs to generate quests. You can of course ask for both a human-readable quest description and also a JSON object (according to some schema) describing the important quest elements, but the failure rate of the results was too high. Maybe 10% of quests would have some important mismatch between the description and the JSON; the description would mention an important object but it would be left out of the JSON, or the JSON would mention an important NPC but the description wouldn't, etc.
As a player, I think it would get frustrating quickly if 10% of quests were unsolvable, especially since, as a player, you don't know when a quest is unsolvable; maybe you just haven't found the item/NPC yet.
Yeah, 10% about jives with what I would expect under the assumption that the generated text needs to be non-deterministic (I.e. no careful prompt tuning and turning the temperature down to basically 0).
An interesting flip side I was just thinking about is the AI saying too much. NPCs keeping secrets until the player gets enough reputation or does a favor or whatever is pretty common. I wonder how good they are at keeping those secrets.
Prompt injection is one thing, and vaguely equivalent to cheat codes which is fine, but what is the likelihood that a player just asking for more info ends with the AI spitting out the secret without completing the quest? Will the AI know to unlock the next area or whatever, because there's no reason for the player to do that NPCs quest?
Should be neat stuff, I'm looking forward to how this all works together when the kinks get ironed out.
That was my experience when I was experimenting with using current LLMs to generate quests. You can of course ask for both a human-readable quest description and also a JSON object (according to some schema) describing the important quest elements, but the failure rate of the results was too high. Maybe 10% of quests would have some important mismatch between the description and the JSON; the description would mention an important object but it would be left out of the JSON, or the JSON would mention an important NPC but the description wouldn't, etc.
As a player, I think it would get frustrating quickly if 10% of quests were unsolvable, especially since, as a player, you don't know when a quest is unsolvable; maybe you just haven't found the item/NPC yet.