Holding conversations with NPCs powered with generative AI

I have mixed feelings about this so far and need to see more of what is actually implemented into games.

I really don't like any idea of having completely open chat with NPCs. If I'm playing a game, I want to be told a story, largely in the way that the story writer intended it all to be. I don't want to have to make the conversations up myself. If I wanted to do that, I'd just go talk to a random stranger myself. I also feel like 100 people should all be able to have the same character and content experience playing a story-driven game, not 100 people having 100 different experiences with every single NPC in-game. And I don't want to get distracted from the story with every single NPC I come across just because I have to find the exact right word or phrase to say to move the dialogue forward. The conversation in most games is NOT what I buy or play a game for, it's the action, adventure, strategy of gameplay or the story-telling, or even just the escape from what is normal/real life - I specifically don't want games to be TOO realistic in every sense. The individual conversations just drive things like the action and story forward but they're not what pull me to a game. So I'm wary of things like this putting far too much emphasis, time, effort and focus into every single individual conversation in a way that distract from what the fun or interesting parts of games are.

I feel like there are two implementations of the AI NPCs in the videos you've shown though.

In the first NVidia ACE showcase, I think AI is used to generate the AI's text from prompts, background data, character presets and the information the player is giving it (from pre-set player dialogue options), so in that sense the main benefit of the AI is the developers not having to write individual story lines for every NPC but just supply the AI with some basics and let it do all the responses to the player. Which doesn't seem too bad, if it works well. It should still have the same outcome and more-or-less consistent results for each player but with a lot less design/story writing/voice acting required to make the game.

Then in the second video, I think the AI is also expecting and analysing open-ended player prompts instead of pre-set dialogue options. Which is a step further, like a full ChatGPT conversation experience. This one I'm more wary of and I don't like for gaming. This would be good for very specific VR games where the intention is complete and utter realistic immersion. In everything else, I want my normal dialogue options please to get through the conversations quickly and back to the action, adventure, strategy and fun.

More generally, although AI is getting better at all this by the month, I don't feel it's entirely "there" yet in terms of believability and detail with things like tone and emotion, when compared to a human doing voice-acting. Maybe it will get there soon and this tech will be really exciting when it does, but for now I don't feel it would provide the same immersion as humanly written or voiced counterparts.

Great thread though!

I think Keiadow nails it really.

Having every NPC able to hold an entirely random conversation with a stranger would be a little odd. Things tend to be scripted for a reason there. I mean imagine if an AI spoke for every person you saw in Cyberpunk? You could spend more time chatting crap to someone who meant nothing than story missions. Talkative NPC's should be few and far between, with something to add to the story really.

Totally agree!

I could imagine a scenario like a guard blocking a specific path who gives you a riddle and you have to answer a specific riddle yourself (so don't want pre-determined dialogue options), answer personalised questions, or trick them somehow - in my head it's something like on the "Bridge of Death" in Monty Python And The Holy Grail. I could see a ChatGPT kind of open-ended conversation being a unique, fun and puzzling implementation in that kind of situation if done very carefully. But I'd only want something like that once in the game or it'd lose its appeal and fun very fast. I wouldn't want that kind of interaction with every single NPC in the world.

Well it could be made that if you don't get the NPC to do/tell you what the develoeprs wanted you to do in that interaction after a certain time, that it switches back to a scripted version. Don't see a problem with it to be honest, it can only lead to more interesting (maybe sometimes akwward, but that#s ok) interactions in my eyes.

I wouldn't avoid playing games with AI-powered NPCs, but I bet there are going to be many many fails ahead of us.

Maybe I'm getting naively excited here, but I think this is a fantastic idea, and both examples are really interesting. The second one, in particular, being more open-ended, strikes me as a wonderful example of how, with a little training and some parameters, generative AI could be used to create very convincing characters.

All the issues that have been brought up (comprehension of accents, moving forward with the story, believability, etc.) would, I believe, be ironed out before any large studio would release a title using this tech. I also don't think a lot of them are problems at all. If anything, conversations like this add previously unheard of depth and replayability to games, especially RPGs with character-based stories. At the same time, no one is forcing you to talk with an NPC for longer if you don't want to, just like no one forces you to read through lines of text and lore in scripted RPGs if you don't want to. It's not like you would need to make small talk with an NPC until they feel comfortable talking to you and then finally give you the plot device you need to continue your quest - surely these NPCs would be in place first to move the story forward and then secondly provide some backstory and make the game more immersive.

Of course, things could easily get wacky, as they tend to do with generative AI stuff now, but give it a few years and some well-placed guard rails and I think you could definitely augment games with this tech, and it really wouldn't take anything away from the purists who want a more traditional experience. Rogue-lite games already provide players with varying amounts of discrepancy in their playthroughs, and they have a huge following. I think having a more personalized gaming journey would actually make things more interesting and give gamers more to talk about.

So yeah, I would definitely play games where I could hold conversations with NPCs, and although I wouldn't want that to be the entire point of the game I'm playing, it would be very interesting to know that capability is out there. Plus, it opens up the possibility for realms of new types of games that are based on dialogue.

Nailed it! That's almost exactly how I feel about all this. I am a story junkie, and I will always prefer games with well-written stories rather than complete openness in the form of a vast open world where you are completely free to do whatever you want and where all other entities can do whatever they want. I mostly see games like books and movies. They tell a story, and I will always choose playing someone's story and seeing how it unfolds, rather than diving into a full-blown life simulation where I am the protagonist. After all, we have the real world for that, right? 

I think it's an interesting concept. But the replies of the NPCs need to sound human and thats something chat gpt doesn't do good atm. But I can see that in the future it might be a cool thing to add in a game

While this does seem interesting narrative drives the story of a game. Having writers build the world and then make side quests and interaction fit into the core story is a must otherwise you end up with disjointed if unique dialog etc.

You'd need a very defined pool of information and lore to work from which means increasing the workload of the writers and team to fill the repo the LLM / AI etc pull from. On top of this you'll also need guard rails to ensure that the experience fits and doesn't deviate away from what's being portrayed.

At the very least this could be used to diversify the interactions with random NPCs. It's not realistic for the developers to write dialogue for each of them, so usually you will see them rotate between a few stock phrases. With AI they could make that way more realistic.

Then they could still have someone write the dialogue for the story line NPCs.

Note that I also have my doubts whether Nvidia used good prompt engineering. Ideally you want to provide the AI with a little story about the background of the character, because otherwise you get exactly what you saw during the showcase: very bland responses.