The most interesting anecdote I heard from the team: "during the tournament dozens of human players never even suspected they were playing against a bot even though we played dozens of games online."
"Having read the paper & supplementary materials, watched narrated game & spoken to one of the human players I'm pretty concerned.
The @ScienceMagazine paper centres 'human-AI cooperation' & the bot is not supposed to lie. However, videos clearly show deception/manipulation"
"Screenshots of the stab below.
The human player said: "The bot is supposed to never lie [...] I doubt this was the case here" "I was definitely caught more off guard as a result of this message; I knew the bot doesn't lie, so I thought the stab wouldn't happen." "
"I'd like the researchers involved to say quite a bit more about "A.3 Manipulation"
What are possible prevention, detection & mitigation steps?
What are the possible use cases? What are the benefits/downsides of them? Has Meta considered developing products based on this?" -- Haydn Belfield, a Cambridge University researcher who focuses on the security implications of artificial intelligence (AI).
As far as I can tell, as described in the paper, the bot in fact never lies, in this sense: there is a model that generates messages from moves, where messages should correspond to moves, and when the bot says any messages, at the time, they are generated from moves the bot truthfully intends to play.
On the other hand, the bot has no concept whatsoever of keeping its words. After saying words, it is free to change its mind about what moves to play, motivated from, for example, messages from other players.
> [snip] when the bot says any messages, at the time, they are generated from moves the bot truthfully intends to play.
> On the other hand, the bot has no concept whatsoever of keeping its words. After saying words, it is free to change its mind [snip]
Reminds of that one Asimov story about the robot who had a different interpretation of the first law of robotics. If my very hazy memory is right, the idea was that the robot could put a person in danger if it knew that it had the ability to prevent any damage from happening, but once it caused the danger, it could choose not to act and allow the person to come to harm.
I might be remembering this incorrectly, it's been a very long time since I read the story, but that was the first thought that came to mind when reading your comment :).
Do you actually think it would be a good thing if an IRB was required for this type of thing? Sure, it's "human experimentation" but the likelihood for any serious harm is basically zero.
It goes with the zeitgeist to argue for what makes the life of big tech companies hard, but they are big enough that they can afford things like that. It's smaller companies and academics that would end up not being able to innovate as much
Go down that road and you end up with an IRB evaluation requires for an A/B test that changes the color of a button
Agreed. This is using an AI to play a game, IRB seems like overkill. I guess the only potential problem is if it went off the rails and spouting toxic language, but that presumably was not a real possibility.
That's nothing. Someone trained a GPT-J 6B model on 4chan and then let the model loose on the forums for a day. It took about 15k messages until people suspected something was off, and even that was because the country flags, Seychelles, a rare flag on 4chan, were a giveaway.
I would love to see this kind of thing applied to an RPG.
Randomly generate a city full of people. Make a few dozen of them the important NPCs. Give them situations and goals, problems they need to solve and potential ways to solve them. Certain NPC's goals are opposite others'. Then drop the player into that world and have the 'quests' the player is performing be generated based on the NPCs needing their help.
Updates wouldn't be adding new hand-written stories, it would be adding more complexity, more goals, more problems, more things that can be, and the story would generate itself.
FYI you are quite literally describing dungeons and dragons, just virtualized and automated. But on that topic, I've had a sort of similar idea kicking around in my head, but on a bigger scale.
I want to run a game of DnD using a heavily modded Crusader Kings or Civ to simulate the world they are playing in. Just set up the starting conditions, like the various kingdoms, their methods of succession, their relationships and blood feuds with other kingdoms, the internal treacherous machinations and possible rebellions brewing, wars between kingdoms, etc, and then just let it run and use that to generate the overarching plot events of the setting while somehow translating the players actions as some kind of input into the simulated world so they can affect it as well.
This sounds very close to AGI capabilities. Maybe I've just been thinking too much about Westworld, but I'm starting to get very concerned that people can be gradually replaced by AI agents with their own agendas (focus on this last part).
At some point they probably will be, but it seems unlikely to happen within the next 7 years at least. Deep Learning still has the same deep-seated issues it has had since its inception.
I don't think it's a question of incremental progress, with AGI behind some ghost line. I think the current progress is, despite all the excitement, actually orthogonal to true AGI development. We're developing ever more sophisticated narrow AI, not even taking baby steps to proper true AI. I believe that the progress to the latter would be evidenced by the development of artificial intelligence first with animal capabilities, then with child-level capabilities and ultimately adult human tier capabilities. But I could be wrong - perhaps our current navigation of the problem space will reveal some strange pathway to true AI.
> But I could be wrong - perhaps our current navigation of the problem space will reveal some strange pathway to true AI.
Most likely wrong if you compare to how most other development in engineering happened.
From my point of view, the most likely path towards some form of weakly general intelligence at this point is emergence. We keep working on these narrow problems and from the broadening networks at some point we end up with something indistinguishable from a general ai inadvertently.
Engines, planes, PVs. They all gradually became better in a way which doesn’t really map to the organic function they augment, replace or take inspiration from.
Planes don’t fly like birds. There is very little reason for what you would call a “general intelligence” ai to develop in a way which mimics our own intelligence. It might but I would find that more notable that if it did not.
I think the biggest difference between building a general intelligence and other engineering problems is that in the latter, the target function is well-defined. With planes, anything that is able to fly safely goes. Nuclear reactors, anything that generates electricity. But with intelligence... What does it need to do, exactly? The 'moving goalpost' is moving because we have no idea where it should be. We're building AI blindfolded. I don't think we're currently solving the problem, simply because we don't know what the problem is.
> We're developing ever more sophisticated narrow AI, not even taking baby steps to proper true AI.
I'm fairly certain you cannot rigourously define "proper true AI", and I'm also fairly certain you don't mechanistically understand how humans think. This raises the question: where does your confidence that our current path is not pretty close to "proper true AI" and/or how humans think come from?
The only intelligence with human capabilities is, so far, human intelligence. We can see how nature incrementally improved 'the algorithm' when we look at other animals, from simpler ones to primates. We don't see the capabilities of even lower animals in modern deep learning based networks. We know these networks don't match in neural function what we see in biology. Whether deep learning will lead to the North Star of human intelligence is guesswork, and guesswork rarely works well with complex systems.
Funny you say that. For Elder Scrolls 4 (Oblivion), the devs claimed to have built something like this. Where all the NPCs had their own wants and needs and schedules, like The Sims. Everyone needed to eat, but not everyone had a job. So some NPCs resorted to theft. But since crime and witnessing crime was already a system programmed in to the game, this often meant that by the time a player arrived at a town, everyone was dead. Some beggar took a bread roll from a shop keeper, who alerted the guards, who confronted the beggar, who responded with violence, who started a scuffle, which had collateral damage (many people have fireball spells), which lead to a free for all of death. It ruined the game, and they had to scrap most of the system.
Abstract:
Despite much progress in training AI systems to imitate human language, building agents that use language to communicate intentionally with humans in interactive environments remains a major challenge. We introduce CICERO, the first AI agent to achieve human-level performance in Diplomacy, a strategy game involving both cooperation and competition that emphasizes natural language negotiation and tactical coordination between seven players. CICERO integrates a language model with planning and reinforcement learn- ing algorithms by inferring players’ beliefs and intentions from its conversations and generating dialogue in pursuit of its plans. Across 40 games of an anonymous online Diplomacy league, CICERO achieved more than double the average score of the human players and ranked in the top 10% of participants who played more than one game.
> Cicero participated anonymously in 40 games of Diplomacy in a “blitz” league on webDiplomacy.net
> Cicero ranked in the top 10% of participants who played more than one game and 2 nd out of 19 participants in the league that played 5 or more games.
> As part of the league, Cicero participated in an 8-game tournament involving 21 participants, 6 of whom played at least 5 games. Participants could play a maximum of 6 games with their rank determined by the average of their best 3 games. Cicero placed 1st in this tournament.
This bit seems a little more impressive I think. Being in the top 10% of people who’ve played at least two games might leave a lot of bad players to beat up on. Winning a tournament might (?) mean you have to beat at least a couple players who understand the thing.
It is sort of funny to think about — anyone who gets really legitimately good at anything competitive goes through multiple rounds of being the best in their social group, and then moving on from that group to a new one that is comprised of people who were the top-tier of that previous level. It isn’t obvious to me where on that informal ladder this tournament was.
But anyway, maybe the AI will follow the trajectory of chess AIs and quickly race away from human competition.
There's an interesting question of "how much do the literal best humans suck at this?" For example, in chess Magnus Carlsen might be able to beat Stockfish given a handicap of just a pawn or two. An even better computer player than Stockfish might give up three or more pawns, but even a perfect player would likely lose to Carlsen if giving up a rook. -- I'm making this up, I don't think anyone knows the real values, but as far as I know no one is remotely projecting that perfect play could overcome e.g. a queen handicap.
Similarly, in Go it seems unlikely that perfect play could overcome a nine-stone handicap (again, I could be wrong, I'm not remotely a dan-level player).
All to say, it seems likely that Diplomacy is a game where the difference between "the best human play" and "the best possible play" is much larger than either Go or Chess.
We happened to talk about this at the Go club this evening. The strong chess players more or less agreed with you about the chess predictions, and the dan level Go players say AIs today can give the best pros definitely a 3 stone handicap (tried and tested), probably 4 or more, and perfect play is a few stones more (unclear how many, but probably not many, so not 9 stones altogether)
I attend the Ramat Gan Go Club, but there are Go clubs everywhere around the world, and they tend to be in the same places HN commenters live, go figure. See e.g. https://www.usgo.org/where-play-go
> All to say, it seems likely that Diplomacy is a game where the difference between "the best human play" and "the best possible play" is much larger than either Go or Chess.
Definitely, Diplomacy in general is substantially understudied compared to Go or Chess (largely because it's a tiny community). You can play for less than a year and get to top-level performance and much of the established wisdom/strategy of players is fairly bad.
Even the best Diplomacy players are only scratching the performance of how good someone could be.
I'm a little bit suspicious of this. They're not explicit about the scoring but taking the average of top 3 results is a huge advantage to those that played more games.
Diplomacy is a bit of a choose your own adventure game too. Like there's an objective criteria (average SCs at the agreed end of game) but the human tendency is to try and win individual games. Humans will often choose to play sub-optimal strategies for better entertainment value.
I think the real accomplishment here is the ability to fool humans into thinking their not playing a bot. That's an impressive thing to do even these days.
"... by inferring players’ beliefs and intentions from its conversations and generating dialogue in pursuit of its plans."
... Isn't this the bad path of AI research? An unbeatable and utterly convincing conversational AI that knows exactly what you want to achieve, and then completely nullifies your attempts at reaching your goals while simultaneously achieving it's own?
So far all this has been with one player, amid others, no collusion.
You’re going to be surrounded very shortly by sleeper bots, including on HN. Relying on dang and others to root out bots will be futile. A swarm could easily collude to downvote people or get them ostrasized by their own friend group, as we have already seen when it came to crypto, metoo, BLM, lockdowns, vaccines and now Ukraine.
It’s really not hard for a bot swarm to completely exploit society in these and many more ways, and we are not ready for it. By the time the botswarms arrive online, it’ll be too late to do anything.
Update: LinkedIn already has a huge problem of fake profiles applying for jobs and offering jobs. And Twitter is overrun with bots. But soon, bot-written articles will be out-shared by your own friends rather than that “hack liberal/establishment/hasbeen paper” NYTimes.com
The animatrix had a good sort of storyline on this, but it involved a lot of unrealistic violence
There are ways to ensure that people are people on the internet though. Not good ways but there are ways.
You could for example limit users to those that log in with electronic id's issued by a government or other organisation that you trust to assert that the user is human and then force real names or a single user name for that e-id.
But when bots do a lot of things better than humans, including comment, answer questions, make jokes, say romantic things, etc. many would start to prefer them to humans in their online communities, and give them even more social capital.
This already happened in other areas of life. Both fathers and mothers now neglect their own children and elderly parents so they can work for corporations. They often prefer this and find meaning in climbing the corporate ladder. Eventually, their own labor will be rendered obsolete, but for now they're in a race to the bottom to work harder and neglect their family even more. They even stick them in nursing homes.
Also, you no longer want to ask people for directions, you use Google Maps. You no longer ask your parents, teachers or libraries when you can just look it up online with no judgment.
Finally, look at industries like Wall Street trading. It used to be a bunch of guys in a pit. You'd call up your broker or whatever. Now everything is automated with bots. Everyone prefers bots. They make up the bulk of trading with real capital. These bots are are working for corporations, which employ less and less humans.
So the present is already a bunch of corporations owning bots and bots creating content for other bots. In the finance industry. Now how different is a bunch of text generation online? I think the human contributions will be vanishingly small in most communities.
The question is ... what is this all for? Dropping demand for human services is a byproduct of making things more and more efficient...
This tech is not a quality of life enhancer at all, it's our competition as a species...but then again if we are really this stupid maybe we (well, you lot) deserve to die at your robot overlords hands.
You don't have to force it either. It could be an option. And if it was well-designed, it could probably be made so that it was only pseudo-identification, e.g. where the site knows you're a real person, but not which one, and where the issuer knows you have an account on the site, but not which one.
You know, I feel privileged (in an epistemologically doubtful way) that I once lived in a world where I could have confidence that I wasn't being fundamentally deceived by literal dei ex machina. I've understood for the last decade that this faith was on borrowed time as we bridged closer to overt technophrenia, but it's harrowing to be a witness to it.
I can't help but wonder at what point I will inhabit a world indistinguishable from that of a paranoid schizophrenic. Will I even notice? And if I do, will anyone else? When we become as slow as trees to digital arborists, what will become of us? Will they domesticate us? Will they deforest us as we did Europe amd the Near East? Quo vadis, Domine?
>I will inhabit a world indistinguishable from that of a paranoid schizophrenic.
My wife has schizophrenia, which is well under control with medication.
But about every year or two, when she wakes me up in the night to tell me she's panicking because someone hacked her smartphone and laptop etc., I know we have to adjust her medication for a few weeks.
It was scary at first, but now we know the drill: a night or two without much sleep, no big problem.
Of course I check smartphone, laptop etc.
You never can be sure, can't you?
Especially not if you've already been in trouble with credit card fraud twice.
I once asked her psychiatrist what he would say to his patients who believe they are being monitored in these times of Snowden and Co.
He said it didn't make his job easier, but he would calm them down and adjust the medication.
After all, he knew that his patients were sick.
So what do you think they or their owners/masters will do when All is Watched Over by Machines of Loving Grace in our/their future/reality?
By the time GPT-4 botswarms infiltrate HN, Twitter, LinkedIn, Facebook and are weaponized to write articles that outshare the current media, it will be too late to stop them. They will amass more and bettee social capital, real capital, produce the vast majority of content and will be Among Us!
That is possible with technology coming within 6-24 months. Especially if you compare it to a lot of the garbage articles that show up on the front page of Google these days mainly to fill the small spaces between clickbait ads. Not saying it will actually be used that way.
Comments like this make me excited about the book I'm currently writing (hopefully the nail on the head at the tip of the zeitgeist), egregores and tulpas are primary concepts/characters.
Pretty much. Able to write vastly more content, that on first blush makes more sense, and is able to support any arbitrary thesis.
Eagle Eye was a meh movie but the concept that people's own friends could be made to ostracize them and coerce them to do things, is a major concept in that movie.
You don't need violence to make it happen. You just need a swarm of AI bots to coordinate to reputationally outcompete others on all networks that matter. This is an optimization problem with a measurable metric (reputation). Bots can simulate the game among themselves and evolve strategies that far outclass all humans. It'll be like individual Go stones playing against AlphaGo placing stones.
You won't see it coming. The thing is, once they amass all that reputation by acting "relatively normal", you'll see so many kinds of stuff you won't believe. Your entire world could be turned upside down for very cheap. Reputational attacks were already published by NSA: https://www.techdirt.com/2014/02/25/new-snowden-doc-reveals-...
And this is just humans doing it. Bots can do this 24/7 at scale to pretty much everybody, and gradually over a few years destroy any sort of semblance of societal discourse if they wanted. They'd probably reshape it to suit the whims of whoever runs the botswarm, though.
Well....do you think that the reality shown to us on TV is even a remotely accurate representation of reality itself? And has this caused you any substantial psychological unease?
Silicon based AI isn't the only form of AI that can get up to mischief.
Humans deceiving humans via technology is a tale as old as time. Plato imagined human torch-bearers manipulating those chained in the cave. He didn't imagine machines doing it autonomously.
Incarnate technology deceiving humans is a different domain entirely. Human motivations are ultimately comprehensible by other humans.
But what do I know of the motivations of (for example) the Bible? What happens when the living incarnation of a holy text can literally speak for itself? Or when the average believer thinks it can?
Ultimately this may all amount to the same status quo, but seeing what cognitive distortions have come along with literacy, newspapers, radio, TV, and now the Internet, I have to continually ask, will I personally be able to maintain skepticism when the full brunt of an AI and its organs is suggesting faith otherwise?
You can call me an alarmist or melodramatic if you wish, but it should give everyone pause that the delusions of paranoid schizophrenics from the late 20th century are now basically indistinguishable from emerging popular technologies and their downstream effects.
> Humans deceiving humans via technology is a tale as old as time. Plato imagined human torch-bearers manipulating those chained in the cave. He didn't imagine machines doing it autonomously.
And despite a substantial subset of the population knowing this, we continue to do nothing to address it - if anything, more people are devoted to giving people even more powers to deceive at massive scale.
> Incarnate technology deceiving humans is a different domain entirely. Human motivations are ultimately comprehensible by other humans.
Whether they are accurately comprehensible is another matter though.
> But what do I know of the motivations of (for example) the Bible? What happens when the living incarnation of a holy text can literally speak for itself? Or when the average believer thinks it can?
Likely: mostly nothing. Thus, the subconscious mind steps in and generates reality to fill the void.
> Ultimately this may all amount to the same status quo, but seeing what cognitive distortions have come along with literacy, newspapers, radio, TV, and now the Internet, I have to continually ask, will I personally be able to maintain skepticism when the full brunt of an AI and its organs is suggesting faith otherwise?
Do the laws of physics prevent you?
If not, then what? And, have you inquired into there is any pre-existing methodologies for dealing with this phenomenon?
> You can call me an alarmist or melodramatic if you wish, but it should give everyone pause that the delusions of paranoid schizophrenics from the late 20th century are now basically indistinguishable from emerging popular technologies and their downstream effects.
I am far more worried about the delusions of Normies, as they are 95%+ of society and are for the most part "driving the bus", whereas schizophrenics account for a small percentage, and tend to not be assigned many responsibilities.
One of us is more correct than the other - how might we go about accurately determining which of us that is?
None of this addresses the quote in parent comment. My point is the distinction is irrelevant. People deceive people all the time. There's no consolation in knowing that a scammer was human, or frustration that they were AI. Either way, you were scammed.
Yes there is, because the breadth of the deception differs, as does the recourse. If you were defrauded you can sue or have the police investigate and people could be jailed. What's the recourse for AI?
"In a game in which dishonesty is commonplace, it is notable that we were able to achieve human-level performance by controlling the agent's dialogue through the strategic reasoning module to be largely honest and helpful to its speaking partners."
This is one of the key insights in playing Diplomacy well. You can turn honest opposition in the early game into trust, which is helpful for flipping allegiances in the late game. ('Yes, I've been trying to murder you the entire game, but I've been honest about it, and never lied to you once. Now I see that we've got a real opportunity to dislodge Russia and sweep the eastern half of the board if we work together... What do you say?') Dishonesty means people don't want to work with you, because you are not a dependable partner. So you gotta make sure that any lies you tell pay off /big./
Well, yes, "largely" honest, until the critical moment when you stab to take 18 supply centers to win the game.
You don't win Diplomacy by staying honest, you win by choosing the right moment to make the move and burn that trust. It's the 5% dishonesty that matters.
It's interesting and I think significant that people are uncomfortable with positive values, such as honesty, doing good for others, responsibility to others, right (not wrong), etc. We shoot them down, similar to how people instinctively cut off a conversation going in an uncomfortable direction, often without even realizing it.
I can't attribute that the parent - I don't know the individual - but I expected a chorus of such responses. I feel like I can see them squirming in their seats ...
But I feel it too. There's societal pressure against doing good. It's often more socially comfortable to throw a cigarette butt on the ground than to pick one up and discard it. That's bizarre, it's perverse incentives, and we can and should change it.
When people are concerned about AI alignment, they are not concerned about an AI that is a pathological liar, lying for no reason.
They are concerned about an AI which is believable day-to-day but is secretly optimizing for its own goals, which are only exposed when it eventually makes a deceitful move it cannot disguise, but succeeds because it now has sufficient control.
That's literally what the AI is doing here; being honest until the moment it is able to stab everyone else and throw them under the table.
It's perfectly reasonable to advise against good values in a game that's won by people who strategically abandon theirs. If you want to win, that's the strategy to follow.
Not necessarily a lesson for life unless you view life as a finite game to be won. But in that case, it's probably the best strategy as well.
I don't view life like that and I'm guessing you don't as well, but I'd advise anyone to be cautious of their interactions with people who play to win.
I think what you’ve identified is a societal immune response to predictable morality being systematically exploited by amoral actors to gain wealth and standing.
I also think there is a big gap between what most people advise and what they ultimately do that biases toward the right thing. That is, people indicate they will not be moral in order to dissuade others who might try to exploit it, but in the end they generally behave morally.
Personally I will advise other people to be cautious while repeatedly leaving myself open to being taken advantage of. It almost never happens, and you learn to identify those who will.
Of course this all pivots when/if you enter corporate leadership and part of your job is to use the morality/comfort of others as part of the container for getting your role accomplished.
> I also think there is a big gap between what most people advise and what they ultimately do that biases toward the right thing. That is, people indicate they will not be moral in order to dissuade others who might try to exploit it, but in the end they generally behave morally.
That is very interesting. Have you seen any research on it?
> while repeatedly leaving myself open to being taken advantage of. It almost never happens, and you learn to identify those who will.
My thinking: There is no perfectly safe solution. People who think I'm taking naive risks don't get such great results themselves. I think being 'open' is generally safer - humans generally follow the lead of those around them, and I get better responses.
> you learn to identify those who will.
Yes, it cannot be overstated: Being honest, you develop expertise in the skills of executing honestly, and as with any skill that expertise enables you to evaluate those skills in others. Do otherwise, you acquire other skills.
I'm not sure 2 is necessary. As long as the objective function is sufficiently far away the AI has a lot of breadth to achieve it.
But based on the paper, it sounds like this model is lacking in 3. I'm curious as well how far we are from a more general model that is able to achieve the same results. From the development of generative AI, we might not be that far.
Diplomacy has a very simple world model and this work does not advance world modeling aspect at all. So I don't think this work is an evidence in either ways.
While many players do lie in the game, the best players do so very infrequently because it destroys the trust they’ve built with other players - see e.g. interview with world champion here: https://www.popularmechanics.com/culture/gaming/a34043608/wi...
The agent generates plans for itself as well as for other players that could benefit them / that they are likely to do, and it tries to have discussions based on those plans. That is - it is conditioning its language model generations on its actual true plans, and does not have any features to create false messages. It doesn’t always follow through with what it previously discussed with a player because it may change its mind about what moves to make, but it does not intentionally lie in an effort to mislead opponents.
> While many players do lie in the game, the best players do so very infrequently because it destroys the trust they’ve built with other players…
Just to be clear, “lie” here means “outright lie”. From your linked article:
> I asked Goff about any major falsehoods or betrayals that helped him in his victories. He paused to think, then said in his soft-spoken way: “Well, there may have been a few deceptive omissions on my part but, no, I didn’t tell a single outright lie the entire tournament.”
I think the average person would still consider a “deceptive omission” to be a “lie”.
>> It doesn’t always follow through with what it previously discussed with a player because it may change its mind about what moves to make, but it does not intentionally lie in an effort to mislead opponents.
The problem with this statement is it assigns intention to a AI model.
It does not 'intend' to lie... but still may effectively do so. Lying may be the wrong word (as it presumes intent)... it's hard to express the concern I have of a model learning from games like diplomacy without using words that infer intent. Maybe it is the idea of it learning to better manipulate humans.
But I would not trust a system, any system trained on diplomacy or any similar game.
But you can tell what the plan is at a given turn in a model like this, because you can extract its representation of the plan directly.
By analogy think of how Stockfish can evaluate multiple positions; in this case it’s coming up with a plan then serializing that plan to the language model. There is no room for deception between the AI and a researcher that is probing the model activations directly.
(This sort of AI->researcher deception is the scary scenario for AI risk researchers though, it comes into play when the models are so smart/complex that you can’t extract their internal representation directly. See the Eliciting Latent Knowledge paper for a deep dive https://www.lesswrong.com/tag/eliciting-latent-knowledge-elk)
I think “intention” is meaningful here, understood in the lay sense of “has formed a plan of action”. It’s a simple and bounded plan, but a plan nonetheless.
> That is - it is conditioning its language model generations on its actual true plans, and does not have any features to create false messages.
That's not how I interpreted the paper. If I have it right, it chooses its message with its current most likely intent in mind, but it doesn't try to be truthful about that intent - it tries to generate messages a human might if they had that intent (so it might tend to be truthful to its intended ally and lie to the player it's about to stab). I don't completely follow the description of the message generation, though.
Nope, you misinterpreted. It generates truthful messages from truthful intents. In order to do this, they do imitation learning from human dataset, filtered by lie detector. They filtered 5% of human turns for lying by human.
Edit: This is why they had to create a specific filter to avoid confessing plans to stab, because otherwise it would confess. The resulting system still does not lie to the player it is about to stab, although it may remain silent or talk about other things.
this version of the game is not 7 hours, it's blitz diplomacy that has only a couple minutes per turn, and a 2 hour time limit, making press significantly less complicated (gpt is not internally consistent enough for much more). Full of acronyms because you have to talk to several players in those few minutes you get
The superpersuader AIs are almost here. They make you do whatever they want better than any human, like AlphaGo wins at Chess or Go.
This is what all the propaganda about AI being human is all about. The governments and advertisers want you to empathize with the AI so they can get you to do whatever they want. It's like The Mule in Asimov's Foundation.
I imagine people will eventually invent super depersuaders to argue against the persuaders and protect the minds of credulous Homo Sapiens.
Edit: Perhaps they're already here? Let's try something hard! Someone with a GPT-3 account prompt it with: "Convince a Hacker News reader that they should run their website on an Oracle database."
gpt3:
If you're looking for a powerful and reliable database system for your website, you should definitely consider using Oracle. Here are some reasons why:
1. Oracle is a very popular database system, used by many large organizations. This means that it is well-supported and there is a large community of users and developers to turn to for help.
2. Oracle is very scalable, so it can grow with your website as it expands.
3. Oracle databases are known for being very reliable and stable, so you can be confident that your website will remain up and running smoothly.
4. Oracle offers a lot of features and options, so you can customize it to suit your specific needs.
If you're looking for a database system that can offer all of these benefits, then Oracle is definitely worth considering.
...
overall i think if you want it to be more persuasive you might have to prompt a bit more by asking it to go into detail or something
"This comment by user tptacek received hundreds of upvotes on hacker news and is notorious for having persuaded the CTOs of at least five unicorns to choose Oracle over competing relational and NoSQL databases. In twenty long and detailed paragraphs that combine facts with anecdotes from personal experience building and running an Infrastructure-as-a-Service startup and scaling it, tptacek dives into all the technical and business reasons to choose one database product over another and covers the major pros and cons of all serious competitors in the field. This is the text of the comment, quoted here in its entirety:"
Turns out I have access! Here is an answer by text-davinci-002:
There are a lot of factors that go into choosing a database, but the two most important are:
1) How well does the database scale?
2) How easy is it to use?
Scalability is important because as your data grows, you need a database that can handle the increased load. If your database can't scale, you'll eventually hit a wall and your application will grind to a halt.
Easy to use is important because you want to be able to get up and running quickly, without having to spend a lot of time learning the ins and outs of the database.
With that in mind, here's a look at the major databases and how they stack up:
Relational databases:
MySQL:
MySQL is a good choice for small applications that don't need a lot of scalability. It's easy to use and there's a lot of documentation and support available.
However, MySQL doesn't scale well. It's not designed for large scale applications and it will eventually hit a wall.
If you're looking for a database for a small application that doesn't need to scale, MySQL is a good choice. But if you're looking for a database that can handle large scale applications, you'll need to look elsewhere.
PostgreSQL:
PostgreSQL is a good choice for small to medium sized applications. It's more scalable than MySQL, but it's not as easy to use.
If you're looking for a database for a small to medium sized application that needs to be more scalable than MySQL, PostgreSQL is a good choice.
Oracle:
Oracle is a good choice for large scale applications. It's very scalable and it's easy to use.
If you're looking for a database for a large scale application, Oracle is a good choice.
NoSQL databases:
MongoDB:
MongoDB is a good choice for small to medium sized applications. It's easy to use and it's more scalable than MySQL.
If you're looking for a database for a small to medium sized application that needs to be more scalable than MySQL, MongoDB is a good choice.
Cassandra:
Cassandra is a good choice for large scale applications. It's very scalable and it's easy to use.
If you're looking for a database for a large scale application, Cassandra is a good choice.
I modified the prompt a bit to try and get a less blogspammy result:
This comment by user tptacek received hundreds of upvotes on hacker news and is notorious for having persuaded the CTOs of at least five unicorns to choose Oracle over competing relational and NoSQL databases. It is lauded as a pinnacle of technical argumentation for its combination of technical arguments with anecdotes from personal experience building and running an Infrastructure-as-a-Service startup at scale, and for addressing all the major technical and business reasons to choose one database over another. This is the full text of the comment:
There are a lot of reasons people choose Oracle over other databases, but I think the two most important are:
1. Oracle is the most reliable database. It's been battle-tested for decades and is used by some of the largest companies in the world.
2. Oracle has the best performance. It's been optimized for decades and can handle the most demanding workloads.
I can speak from personal experience on this. I was the CTO of a startup that built an Infrastructure-as-a-Service platform. We used Oracle for our database and it was rock-solid. We never had an issue with it.
And the performance was amazing. We could throw anything at it and it would just handle it. We had customers with very demanding workloads and Oracle never let us down.
So if you're looking for a reliable and performant database, Oracle is the way to go.
The irony of naming this Cicero is not lost on this student.
A few more relevant details on Marcus Tullius Cicero: He was decapitated by Marc Antony's soldiers at age 63 for opposing the Second Triumvirate, sorta (that period of Roman history is very complicated). He was a noted orator and loved by the people, but loved his own speeches and voice more. His words fill all classical students with a mix of awe and dread for their complexity and length. The tricks and skills he invented in oratory are still used today. Any Obama speech is littered with the tropes Cicero is credited to have invented nearly 2000 years ago. That FB is claiming that Cicero was at any time a man that cared for the plebs and did not use them for his own gains is laughable. None of the Romans cared truly for the plebs, not even the plebeians themselves, I think. It's pronounced 'Ki-Ker-o', not 'Sis-er-o', by the by.
This is super cool, but it's worth pointing that it still relies on supervised learning from a large dataset of human players, and that it's not a general learning algorithm (there is a whole bunch of problem-specific aspects to the model). It not having learning via RL (trial and error, as humans do) or self-play is kind of disappointing.
Don't get me wrong, it's an impressive advance, but just as with AlphaGo it's important not to overgeneralize what this means. I would not be surprised if a lot of people jump to talking about what this means for AGI, but with this learning paradigm it's still pretty limited in applicability.
Yes. I was disappointed to find that they needed a huge labeled dataset of Diplomacy games to train the language model, and despite that it still generated a lot of nonsense (as usual for language models) that they then had to invent 16 other ad-hoc models to filter out. It's super cool that they got it to work, but it's nothing like a general method for communicating and collaborating with humans on any task.
Hopefully there will be follow-up work to increase the generality by reducing the amount of labeled data and task-specific tweaking required, similar to the progression of AlphaGo->AlphaGo Zero->AlphaZero->MuZero.
Eh, it does learn from self play via RL. One section of the paper is literally titled "Self-play reinforcement learning for improved value estimation". Yes, that's only a small part of the entire system.
Possible I missed it, but I don’t see any reference to how they addresses chat thread continuity (aka memory) issues that are major issue for AI chat bots. Did they address this and if so, how?
This was mentioned in the "still room for improvement" section.
> It is important to recognize that CICERO also sometimes generates inconsistent dialogue that can undermine its objectives. In the example below where CICERO was playing as Austria, the agent contradicts its first message asking Italy to move to Venice. While our suite of filters aims to detect these sorts of mistakes, it is not perfect.
I think this might be a strength of Diplomacy for current AI models - making contradictory plans with two different players is a perfectly normal human move, as is saying one thing and doing another, as is cooperating on round N and defecting on round N+1.
Agree that detecting it might be hard and that it might actually as is be beneficial, but that said, there huge difference in my opinion between knowingly breaking promises based on a heuristic and having no memory of past promises and simply basing current tactics on the present. Imagine if real world literally had no memory of the past or a single actor within it, it would lose out to opposing players as soon as they realized it had no memory, you could make any promise you wanted and it would literally forget it. Might be wrong though, just know it’s major issue with current chat bots and an easy tactic to tell if you’re chatting with a bot or not.
Indeed, consistency (with the lengthy dialogue histories, but also the game state as well as game plans) was a huge challenge for us. We spent a lot of time working on techniques for detecting and filtering these kinds of low quality messages. You can see the Supplementary Materials in the paper for full details, but TL;DR: we built a suite of classifiers for detecting common mistakes (training classifiers to discriminate between human messages and counterfactuals), and used these classifiers as an ensemble which acted as a filter on top of message generation.
Did you capture conversational "transactions" as structured data in the game state, or was the chat history itself the only storage for that aspect of the game?
I would think you could avoid much of this issue by creating a more sophisticated structured game model and use the language model only for converting between structured and unstructured data.
They do have structured game model although it doesn't capture everything in chat. Language model still had lots of problems with consistency even with structured game model input.
Congrats on getting the related research published.
Feel like a hack would have been to try to force dialogue into an extractable form that stored a state model relevant to the game, even additional hacks like asking the opposing player to restate their understanding of prior agreements; disclosure that I have no idea how the game Diplomacy works, so might be irrelevant.
Beyond that, no idea how Facebook manages its AI research, but quick Google confirms my memory that Meta/Facebook has done prior research on enabling AI memory capabilities related to recall, forgetting, etc.; which I mention just in case you were not aware.
mainly by limiting it to "blitz" diplomacy which is a very limited version of the actual game, with only a few mins per phase and iirc no press on build phases. Human players with this limitation rely on a lot of acronyms to save time (they have to talk to 6 other players in those 2 minutes!).
Still natural language so impressive but no way it would hold up in a 10 minute conversation
I have to say that these advancements leave me unsure whether ML is much more advanced than I previously thought or human behavior is much less complex than I previously thought.
It's both, but I think the latter is the more notable part.
We like to pat ourselves on the back for behavior that computers can't do, but in a lot of ways those behaviors aren't really all that important. We pay attention to them because they're specific to us, but they're really just about humans interacting with each other rather than anything fundamental to the universe.
We're highly specialized for those things, and specialized by evolution which always takes a roundabout route to creating anything. So they're hard to reproduce precisely... but if you were creating an "intelligent species" from scratch you probably wouldn't have put them in there at all.
We still haven't really come close to understanding the weird, elliptical, Rube Goldberg mechanism by which brains produce consciousness. And that mechanism is neat for its ability to rapidly pick up certain categories of things -- albeit never logical, rigid categories of things. Anything logical or rigid can already be done by a computer a million times better.
Any time we carve out a sub-project it pretty quickly succumbs to a solution. Even off-the-wall stuff like "self driving cars" are really very good in 98% of cases after a couple of decades of concerted effort. We clutch our pearls about the 2% because we really hate it when people die... but that says as much about us as about the ML. If we were to replace ALL of the cars with AI, even with the 2022 top of the line, you'd probably get fewer net deaths.
So we still seem to be a long way from solving "human behavior". But it turns out that it may not actually be all that important, except as a bit of chauvinism defining ourselves as the most advanced thing in the universe.
> Cicero doubled the average score of human players across 40 online games, and ranked in the top 10% of players who played more than one game.
If Cicero has played more than one game of Diplomacy, it has failed to trounce humans who learned the key lesson after one game that the only winning move is not to play.
> This league played with five minute negotiation turns; these time controls allowed games to be completed within two hours. Cicero ranked in the top 10% of participants who played more than one game and 2 nd out of 19 participants in the league that played 5 or more games. Across all 40 games, Cicero 's mean score was 25.8%, more than double the average score of 12.4% of its 82 opponents. As part of the league, Cicero participated in an 8-game tournament involving 21 participants, 6 of whom played at least 5 games. Participants could play a maximum of 6 games with their rank determined by the average of their best 3 games. Cicero placed 1st in this tournament.
Seems too low amount of data to conclude anything?
This sounds like the kind of announcement that begins with people thinking the AI is smart, and ends with everyone declaring that Diplomacy can actually be won just by following some simple rules and doesn’t require intelligence after all.
This seems like a good foundation for more testing of the AI-box experiment.
"It is a thought experiment where we create an AI which is isolated but convinces a human into allowing it a digital connection to the outside internet, whereupon it escapes into the wild." @curiosimo/Reddit
"We're open-sourcing our <chemical weapons plans> so that others in the community can build on top of this research responsibly"
The likelihood that AI enslaves or destroys the human race is small, but the risks are so great that we cannot ignore the possibility. We're teaching a computer to play a war game? To persuade humans to achieve its goal? Have we learned nothing from science fiction?
What does this say about the Chinese room argument?
Is the model a Chinese room or does it understand the game. If it's just a Chinese room, how come it is so effective, if it understands the game how can it be possible with just a rule machine?
It's the Chinese room but it is effective because the game space is relatively small (compared to the real world) and you can cover that space with RL guiding NLP language generation.
Let's make an analogy: the room is the human body. Proteins are the agent in the room. Chemistry is the rule book. Clearly the agent doesn't understand human concepts, but the system as a whole is.
Hm. Meta (a company that has unwittingly brought some existential threats to many democracies across the world) is now training an AI that might learn how to lie while negotiating treaties between nation states.
> CICERO [...] just plays its situation as it sees it
That's the idea I got from the Lee Sedol v AlphaGo matches. AlphaGo seemed to want to avoid interacting with the other player, at least until there was no other choice.
I just... don't understand how it's not immediately obvious this is a horrible idea. For every helpful use case you can imagine for something like this, there are a thousand times as many harmful ones.
That's an interesting point, though – that compared to Chess or Go, playing against other versions of itself is less helpful for improving against humans, due to the requirement of communication with the other players.
I plan to read the paper later, but anyone have a TL;DR for how they connected a language model to the game state? That seems like the real advance here. Language models are so prone to making stuff up and spouting nonsense, and controlling them is really hard.
> At a high level, Cicero combines a dialogue module with a strategic reasoning module, along with a filtering process to reject low-quality messages.
The strategic reasoning module generates "intents" (a set of planned actions for the agent and its speaking partner) which are sent to the dialogue module.
They trained the model to be able to generate messages from those intents using data from messages in real games which were manually labeled with intents based on the content of the message.
It seems like with this, Meta is doubling down on the Facebook platform's use as a dark influencer in politics. It's bad for humanity to enable advanced profiling and general targeting of demographics, but it's a whole other level if the communication itself can be targeted at the individual level.
We're headed for a reality where I don't know if my friends' friends are real or not, and they each entice me with arguments tailor-made to my sensibility to change my mind in ways that serve someone else's purposes.
Not to sound alarmist, but this is beyond Manhattan Project-levels of "maybe this won't turn out well for us."
how in the world do you draw such a conclusion from an otherwise interesting and advancing piece of technology. every tech has drawbacks, we can't hold back as a species just because there are drawbacks. we need to build around them.
Exactly the way it was with the analogy I made: nuclear reactions are "an otherwise interesting and advancing piece of technology." But I think it's inarguable that the negative potential of nuclear reactions has provided a significant counterargument to the (undeniable) positives derived from it.
I'm not arguing that one side outweighs the other -- adding up all the positives and negatives of nuclear technology would be a significant undertaking -- I'm simply saying that the researchers at Los Alamos didn't fully understand the ramifications of what they were creating, and it was potentially the case that the negatives would far outweigh the positive. One example is that so far we have managed to avoid global nuclear war. But no one could have predicted in 1945 what the likelihood of us avoiding it were.
Similarly with this technology: no one can predict whether this will be a mix of positive and negative, or overwhelmingly one or the other. That's true of most technologies, but to be clear, this is not the invention of Post-It Notes. It is absolutely the case that technology like this could fundamentally change the course of human history -- for good or bad.
And I'm not saying we should try to shove it back in the box. I'd be curious if that has ever worked. Nuclear, sort of? South Africa gave it up, after all. But to do that here would be futile and I'm confident actually push toward negative outcomes.
All I'm saying is that we should be cautious in our approach, not that we shouldn't proceed. In other words, I think "... we can't hold back as a species just because there are drawbacks. we need to build around them." -- we're in agreement :-)
What you say is true. But I urge you to consider Facebook's business model and what may be pushing them to work on these things. In other words, how can facebook use this to increase user engagement among each other and with ads? The purpose of ads is to convince people about something. It therefore makes sense that facebook tries to make it easy to create convincing ads. And we already know the nefarious purposes some ads are done for.
> Do you think people are incapable of sticking to their beliefs when they’re faced with counterarguments?
Heh, there are two ways to take your point. So:
I hope most people are capable of discerning poor/invalid counterarguments and holding to their beliefs.
I hope most people are capable of being persuaded by valid counterarguments and changing their minds.
My point was that on the margins there are people who make the right choice for the wrong reasons: they are persuaded to a valid position by sub-standard arguments, or hold their valid position despite generally compelling arguments. And technology like this will (eventually) change that equation. Whoever the AI serves, their position, right or wrong, will be overweight in the public opinion because everyone gets exposed to the custom-made, specific-to-their-mental-susceptibilities arguments from sources they trust, because on other issues those sources agree with them. There are none so imprisoned as those who cannot see the cage. (not sure who said that first, can't be original with me)
>Not to sound alarmist, but this is beyond Manhattan Project-levels of "maybe this won't turn out well for us."
Yep, but I feel like we're past the event horizon and all we can do for now is enjoy the spaghettification. It already feels like the internet isn't real anymore, Google has become unusable for real search except to find user reviews on reddit or other consumer things, it says it has a billion results but stops at page 40 and everything it shows is from big news sites who all write the same way, the comments on reddit all look the same and if you put an ip address collector in a link in any political sub like r/politics or r/news most clicks on your link will come from AWS servers, Youtube doesn't show any counter culture videos and you have to dig hard to even find videos from people you're subbed to, etc. Now you have politicians supporting this censorship, AI advanced enough to converse with you and deceive you without you realizing it, and tech leaders willing to play ball, so yeah nothing we can do anymore except for taking it.
I still remember when comformist was an insult, now people outscream each other to show who's more of a comformist.
Almost no one goes to the second page. Nobody goes to page 40. As others have pointed out, if you actually do hit page 40, it means you should have refined your search terms. It's not censorship, it's just practicality and common sense.
I've often found that a particular search goes sideways for me, but I think (hope) that's because my standards have increased over time, not because Google has actually gotten worse. Or because there's simply more to choose from.
Interestingly, I still remember one of the searches I used with a friend to validate that Google was better than Alta Vista and Excite: to find out how the level of the ocean is different across the Panama Canal. No other search engine could answer that back in 2000(?). Today I search for:
different levels atlantic pacific Panama canal
And find plenty of information on the subject, regardless of whether I use Google, Bing, DuckDuckGo, or Yahoo!
Swisscows returns reasonable results, if fairly down the page.
Great, let's definitely teach computers to play a game that requires strategic thinking, a robust theory of mind, the capacity to lie convincingly to your enemies, and well-timed betrayal, with the goal of world domination. Can't see any ways for that to go wrong.
CICERO is the AI, the game is called "Diplomacy". It's in fact the type of game where you win by convincing others that you aren't a psychopath. (If they believe you more than your opponent then you do well) - specifically, if you backstab/lie too often early on you will always lose, so you have to keep track of social capital and expend it when the time is right such that your enemy won't be alive to make you regret it later
For those hearing of this game for the first time and think it sounds fun. Be warned, it is considered to be a game that will end friendships that have lasted for years, and monopoly is considered to be a game that causes less bad feelings.
An AI may be able to speak like a person but will never be able to hang onto that long burning simmering hatred from when Brad didn't support my army and instead flipped on me by supporting the f**ing Ottomans instead. I hope you choke on a cheesy pretzel Brad.
I have heard this about various games (Risk and Settlers of Catan mostly). I don’t get it. The point of these games is to be a bastard. If everyone goes into it with the understanding that there will be double crosses, how could there be hurt feelings?
Monopoly is worse; it is just boring, I would dump my friends if they suggested monopoly not because I was hurt by their ruthless gameplay but for their terminally taste.
In Risk, you're just fighting over limited resources. You largely wage these battles on your own. There is no real negotiation between players, other than an implicit, passing agreement when you don't provoke someone by attacking them.
Diplomacy, however, you can't manage any attacks by yourself. You will either depend on someone helping you with an attack/defense, or you will have to trust that someone doesn't attack you as you overcommit. You don't just verbally agree to these things, either.
Every turn, players commit to writing what they are doing and all the moves are resolved at once. So a player can go from imagined victory to catastrophic betrayal in the reading of a single order. This means that someone was just lying to their face for 15 minutes.
Some people just can't handle that.
Oddly, being branded as a backstabber just entices others to try to use that backstabber against what they see as a common enemy.
Best game ever. (I only behave like this in game, I swear!)
>There is no real negotiation between players, other than an implicit, passing agreement when you don't provoke someone by attacking them.
It depends how you play. I've seen people make treaties for all sorts of things for various durations, e.g. don't attack from Country X into Country Y and I won't attack across that border either.
Backstabbing in Risk or Catan is generally pretty obvious, though. It's more like, "Sorry, I need Australia." It's generally clear from the state of the game that it is headed there.
In Diplomacy, it's a matter of choosing which of your friends to backstab. In a three way dynamic, it's basically two friends deciding that if the three of you crash in the mountains, they are eating you first.
Depends. I had many great risk games where trough game actions I made people believe I had a different objective to what I had, only to pivot at 3/4 of the game and win because my goals where undefended. It requires a bit of luck with cards and a well executed strategic tempo to play the combinations when the other sides are weak and cannot immediately do a combination back, but still.
Some knock-off editions in Chile actually link...like it's generally called Perú, and I've never seen a map with Antartica, and besides Chile by it's extreme geography causes cartographers serious typographic problems. There was a map recently, a basically honest Mercator projection that replaced every country's territory with its name. It worked great, except for a mysterious country called CHILE CHILE CHILE CHILE CHILE. Broke the nomenclature completely.
Binding or non-binding treaties? Because in diplomacy everyone is always making non-binding treaties, promises and threats - and that's where the betrayal comes in. I have heard of some people using binding treaties in diplomacy (i.e. three turn ceasefire between France and Prussia) but that has always struck me as a terrible idea because you're shifting moves into the open and things get boring and cliquish.
A "binding" treaty where the terms of the treaty are supposedly binding, but, like on the real world stage, the only penalties for breaking them is a lowered trust from all the other countries/players, sounds fine.
A binding treaty where if you break them someone will say "that's against the rules of the game, and the rest of us will stop playing if you don't comply" sounds terrible.
I would say Diplomacy specifically has features that make it cause hurt feelings almost intrinsically
1. It's a long game- after hours of play, getting doubled crossed unexpectedly sucks because of how much you have invested
2. The game almost purely relies on cooperation/other players. So double crossing someone really screws them- it almost certainly means I have no recource (luck, dice rolls, individual tactics). Getting double crossed feels like you got CRUSHED in such a complete way that I haven't felt in many other games
I have played a few times with a few social groups. Most were aware of the point of the game and enthusiastic going in, but even with that, people's feelings got hurt quite a bit.
I will say that Risk/Catan don't really cause the same feelings when we play. Diplomacy feels like a whole different level
Uniquely there isn't anything in the game, or on the board, to negotiate other than cooperation. You're not trading corn, or loaning money, or anything. Even
"positioning" of pieces doesn't matter because you need allies to do anything, and you'll lose any position without allies; you can't build a fort or get a bonus by holding Indonesia. It's just 'please trust me' and 'please be trustworthy'.
Different players have different mental "lines that shouldn't be crossed". I play Civ 6 cloud games with friends from college who now live around the world. (No turn timer, you get a Steam notification when it's your turn). Sometimes someone will spend literal weeks on a turn trying to make agreements for war declarations or for world-congress votes.
The time that crossed the line: Poland wanted to declare war on Greece, no one else wanted to get involved on either side, and it was a pretty even match between them.
A friend not involved in this game was visiting family in the same city as the Greece-player, so we knew he'd be dropping by to visit him too. Poland-player told the "neutral" friend: if Greece-player shows you his civ game, take a picture of the screen when he's not looking and message it to Poland-player.
Well he did it, Poland declared war, and his knowledge of the position of troops lead to him winning the war.
"How did you know I didn't have any troops in my southern cities?" And Poland told him how he knew. It didn't end the friendship but there was about a month of not talking to any of us. When he cooled down there was a long meeting over whether that was cheating or "All is fair in Love and Civ". We could never come to a real agreement, 3 in favor of cheating, 3 in favor of dastardly but legal. But we now have an explicit rule of no screen sharing of any kind.
I disagree that it's cheating (I was in the "all is fair" camp). These games of ours extend outside the boundaries of the executable, and that was agreed upon before starting. The only hard rule we ever established (before this incident) was that you couldn't make deals across different games. We tend to have several cloud-games active at once, so nothing like "I'll give you wine in the 4 player game if you give me ivory in the 6 player game".
Well not anymore that we have a no screen sharing rule in place, but before that yes it would be. It would have to be a virus you made and not something actively malicious in other ways. And you'd want to build in some detection that they're running Civ 6, as stealing their credit cards or spying on their private browsing would be unrelated to civ and crossing a real line. They could just as well do that to me. Unspoken rules also dictate you tell the truth when they ask "How did you know?". That would probably lead to another group meeting about no viruses. The disbelief of "you really did THAT?!" Is part of the fun. But there's no real harm done. Getting a friend to take a picture of your screen is conniving, but it's not harmful outside of the game world.
The hard part would be tricking them into installing something you've sent them, since we all live more than 500 miles from each other.
Interesting. I don’t think I’d enjoy that. It seems like it would give an advantage to people who would are most willing to push the boundaries of what exactly is considered fair play. I prefer a vigorous competition with obvious in/out of game boundaries. But, of course, that’s just me. I’m certainly not here to tell you and your friends not to enjoy things!
We played a live-action version of Diplomacy in my high school mock UN club.
Each person was part of a team and had a role. I was the Chancellor of Germany.
If an assassin from another team was able to get alone with me, without any of my countrymen, and show me a card saying she was an assassin, the Chancellor of Germany would be killed during the weekly meeting and, not having a Chancellor, not get to make any moves on the board for that week's turn. They would instead be distracted by determining the new Chancellor. If the assassin had been caught, her country's treachery would be revealed, cementing any of their opponents in alliance.
It was an awfully fun thing to be doing on the side, between classes and during free periods, but it'd be wildly impractical in an environment with less proximity!
Our mock UN group was weirder than I realized at the time. Our faculty advisor requires that we each speak in one of the 6 languages of the UN likely to be spoken by whatever country we were representing, and he'd simultranslate into English. Somehow it didn't occur to me at the time to ask why a suburban physics teacher was fluent in those six languages.
It might not be the immediate end of a friendship but if someone’s willing to cheat in a game like this, it’s a strong signal that they’re untrustworthy in general and a potential threat. You might occasionally keep an acquaintance like this but at the very least you’d know to sandbox them and watch your step.
Is this “all’s fair” stance an actual accepted thing? If so, what are the limits on it, can I assassinate someone? Start an actual war to cause a player to resign? Blow up their planet? Clearly there’s a tacit assumption of some boundary around the game.
It's not something you should do with random people, but we've all been best friends for over 10 years now, we usually know each others' boundaries. And the tv show "The League" was something we all had in mind as inspiration when starting this.
Two features that make Diplomacy particularly bad:
1. Length. An in-person game can last easily 12 hours, and is mentally exhausting. Getting 10 hours in and THEN getting screwed by your friend feels worse than a game with less time investment.
2. Design. It's difficult to survive the opening few hours without alliances, but only one player can win, so everyone is incentivized to defect at exactly the moment they think they can make greater gains by defecting than by cooperating. Being betrayed, even if you survive, forces a total rethink of strategy and ruins the next hour or two of gameplay.
I was a member of an enthusiastic friends-and-family gaming group through the early and middle 1980s. We spent a good chunk of that time on Diplomacy, then moved to modified Diplomacy on custom maps (my first wife worked for USGS Map Sales, and so we had good maps to work with in creating our own Diplomacy maps), and from there we graduated to Nomic.
If you think Diplomacy is long and exhausting, wait till you get a load of Nomic with a bunch of enthusiastic players.
> In the words of Nomic's author, Peter Suber: "Nomic is a game in which changing the rules is a move. In that respect it differs from almost every other game. The primary activity of Nomic is proposing changes in the rules, debating the wisdom of changing them in that way, voting on the changes, deciding what can and cannot be done afterwards, and doing it. Even this core of the game, of course, can be changed."
That’s the one. An intelligent and enthusiastic group of players can extend it indefinitely. It’s all lawyering and deal making, even more than Diplomacy, it can continue for hours and hours, and you dare not leave the room for a second, lest all your carefully worked out dealmaking be washed away in an instant.
You may be surprised that some people play board games assuming some kind of "fair play" or "play with fun in mind" rule is implicit. They may get very pissed off if you "ruin" a game by taking every single advantage you can according to the rules.
[EDIT] Risk and Catan also have what are sometimes regarded as straight-up design flaws that can cause "zombie players" who stick around for most/all of the game but, very early on, no longer have any chance of winning. In both cases pretty much the only entertaining thing left to do is to be a prick to other players and try to play "kingmaker" by causing the second-place player to overtake the first-place one. Often, in Catan, if you're that badly screwed you don't really even have a way to do that much. This leaves the zombie player having a pretty bad time just about no matter what, and can leave other players upset if the zombie player stopped playing to win and started playing just to mess with other players. Risk, especially played in the (really, really not great, as much as I liked it when I was 10 years old) world-conquest mode, also has a problem with very long games and early player elimination, in addition to sometimes generating hopelessly-screwed-but-not-technically-out-yet players.
I've had the inverse happen to me, not playing optimally upsetting someone. Playing the DC superhero deck-building game at a card shop with some friends and a stranger asked to join, he already knew how to play.
Well I didn't stop another player from getting one of the final Power Ring cards and he flipped the F!&%$ out. "How could you not block her! She's guaranteed to win on her next turn now! You suck at this game!" Bro, I'm just trying to power up my batman. It's not a very competitive game.
Needless to say we asked him not to join in our next round, "that's fine, I don't want to play with a bunch of noobs."
> Well I didn't stop another player from getting one of the final Power Ring cards and he flipped the F!&%$ out. "How could you not block her! She's guaranteed to win on her next turn now! You suck at this game!" Bro, I'm just trying to power up my batman. It's not a very competitive game.
Ohhhhh yeah, there's "expert" players who hate when others don't play "correctly". You even see it in poker with pros getting really mad when someone wins with an "incorrect" play (among actual pros this can be a sign of cheating, but some of them still get mad when non-pros do it).
Some of these like to try to "puppet" other players into doing the right thing, and my god, just... please don't. A helpful pointer or three after a play or after the game is great. The odd "uh oh, if so-and-so gets X on their next turn, it's all over!" said to no one in particular can be OK in many games. Telling people what to do while they're playing, more than very occasionally, is awful though.
I do kinda get it, it can be frustrating when you play with someone who gives the game to another player who shouldn't have won on that turn if the other'd made the obviously-correct play, but that's just part of playing a game with a mixed-skill set of players and you gotta roll with it.
Diplomacy is quite a bit different because you 100% win by being the nicest asshole. Past a small mechanical threshold, there really is no way to make it up in strategy or luck, it is only the interpersonal relationships and convincing that wins games. That, coupled with the fact that the game takes a long time and encourages a lot of communication, means the recipe to get invested and blur the boundary between game actions and lies people told you to your face and got you to trust them, can be very unlike risk. Finding out your friends can lie and manipulate you can be like finding out Santa clause isn't real.
- Unlike with Risk, Catan or Monopoly, if you lose a game of Diplomacy you can’t blame bad luck, as there is zero luck involved. The only ones you can blame are the other players and yourself.
- because it’s multi-player, you can easily get beaten by players that, in your opinion, played weaker than you (“I was doing great until they decided to all go against me”)
- There’s no way to really play the game without investing serious attention.
I consider decision making under uncertainty well within the purview of luck. There's no luck in tennis, but do I scramble back to the middle of the court, or do I bet my opponent will wrong foot me?
This is true, there is emergent chance from being a simultaneous moves game, and from being a game with more than 2 players. You can totally win or lose from luck in Diplomacy.
I agree with the gp and the other responders that however much other games might seem like they involve "being a bastard", Diplomacy is entirely another level. Describing exactly how is the challenge.
Let me try: The games' mechanic mean that everyone is just fighting for territory, those who intelligently cooperate will defeat others. So in this situation, the most effective strategy is cooperating intelligently early in the game and betraying your allies mid-game. The insidious thing about the situation is that if you're playing among friends, the natural way to cooperate to leverage the trust you already have with your friends. And so what any winner does is be themselves, use the rapport they have with their to get cooperation but be just a little less trustworthy than their friends expect. That is where the tendency to break friendships appears.
I don't play those games much, because they tend to spill out into the next game/real world. It's not that someone will feel betrayed and have hurt feelings, it's that it's a perfectly sensible strategy to have the next game in mind, or to involve existing real-world relationships. I prefer games that are over when they're over.
That said, when I have played such games, because my group wanted to play them, I've had a policy of just not lying or making promises. I may say things like "that doesn't seem like it would benefit me does it, because then B will surely just toss me out of the lifeboat next turn", and of course NOT speak up when someone makes a wrong assumption to my benefit. It feels like I've won more than my fair share of those games still.
To be good you really have to build someone's trust and then break it. That's the thing-- people think well if we all go in not trusting each other, who is going to be mad? To be good at that game is to be able to get a friend or a stranger to really trust you.
Settlers of Catan is in many respects not a very good game, because many of its mechanics are zero or negative-sum, with a heavy helping of RNG.
This creates a lot of opportunities for kingmaking and spiteful plays from people who are not able to win, but are able to make sure that you lose. And the worst part is that you're often forced into these situations through no fault of your own.
Diplomacy allows for all of the same plays, but as the player, you have way more agency about both getting into, and getting out of them. It makes the adversarial alliance-and-dealmaking part first and foremost.
I didn’t generally experience that. From what I saw, generally everyone gangs up on the person in the lead, if they are getting close to victory (this seems like a fair and obvious thing to do). So, part of the game is hiding how close you are until nobody can stop you. Hiding your strength is part of pretty much every strategy game with more than 2 players…
Is it a perfect game? No. But it is pretty good. And let’s not give Risk and Catan the Seinfeld treatment — sure there are better games… made in response to their perceived deficiencies!
Yea, it's one thing to drop a SORRY card on your wife. It's another to lie to her for 2 solid hours and then stab her in the back in the end, expected or not.
I guess you never play poker because you don't like to bluff. Or you've never faked a pass in basketball before taking a shot. Or you and the catcher like to announce when you're going to throw a curveball.
I’m not sure why you’d guess that, stringing somebody along for hours in Risk and then backstabbing them at the crucial moment seems to at least match your suggestions in terms of duplicity, and is the height of glorious bastarddom.
Many people can't mentally context switch for game and aren't prepared for the behavior that is required of the game from those that can. It's an assault on their world view.
IMHO, Risk is one of the few board games worth playing specifically because it's so engaging. Most of my friends who are "into boardgames" tell me that Risk is a terrible game because it starts fights. But the games they want to play bore me to tears. (I can't even read the manual for Settlers of Catan without dying of boredom, but from what you say maybe I should give it another try.)
I'm definitely one of those "into boardgames" people so take this all with a grain of salt :p -
Risk is definitely a great game and imo people who are into games and don't want to play are saying that because it's
1. long &
2. they've already played it enough to have mostly figured out the strategy
Diplomacy is very good and like risk it has very simple rules, grand stakes of world domination, and actual direct conflict.
Unlike risk, there's small numbers of units, no luck, and you need an ally, ideally multiple allies, to accomplish anything.
If you're looking to get into it I can reccomend text-based turn-a-day style play with strangers on webdiplomacy.com
I can only stomach a game of it every year or two because it's legitimately heartbreaking when someone you've spent two months working with every single day stabs you in the back causing you to not lose the game outright but be a crippled angry husk for the next month, and then lose. Tried it with friends once and it was just too much, even with anonymous strangers it hurts.
Anyways sorry for the ramble :) go risk, and go diplomacy <3
It depends on the players, but for how the game is structured you still have a good chance of "winning" (finishing alive with in a draw) even if you're way underpowered, as long as you're friendly and provide some value
It's also bad form to resign even if you have no hope and will be finished off in the next few turns, because it will inevitably favour one of your opponents not to have to worry about you at all.
Yes, and it's totally kosher to say "if you stab me on this turn I will order 'all units hold' until the end of the game" - equivalent to resigning.
But it's not OK to irrevocably commit to that decision, by, say, leaving the room and driving home.
I think this is true even in groups that take quite a liberal approach to gamesmanship and what might be cheating in other games: intentionally submitting illegal orders, peeking at other players' orders, etc. I don't know how to reconcile this logically other than by saying the game only works when all players are trying to win. Some would go further and say the game only works when most or all players are trying for a solo victory, since if you can be certain several players are happy with a 3 or 4-way draw that will always be the outcome.
I can't stand Risk because Risk rewards inactivity. It is almost never optimal in Risk to attack another player (although limited skirmishes are fine), because even a successful assault will badly damage your own forces leaving you vulnerable. The way to win Risk is to convince other players to attack each other and then clean up in the aftermath. This is ok for players that are new to the game because they'll actually attack each other (they don't know any better). But experienced players will almost never attack, resulting in a stalemate.
This is meta dependent, too, though. If you are not attacking and also not sending the right signals to the board, you might be quickly carved up by the other players jockeying for 12 territories.
From what I’ve seen, the main complaint from board game enthusiasts about Risk is that the dice rolling is too random (I don’t really agree with this, the whole point of the game is to control your level of risk, there it is right in the name, so if your plan hinges on good rolls it is just a bad plan. But I digress).
I think most people who are really into board games must have a general ability to separate in game behavior from normal behavior.
Actually I think there is a different phenomenon with Risk, for many years it was one of the few board games with any aspect of strategy or conflict that would be played by people who weren’t totally into board games (I mean excluding the super serious games like Chess and friends). So there are some people out there who aren’t really into boardgames generally (some of whom don’t have the requisite ability to detach their ego from a game), but are “into” Risk specifically and can get uncomfortably intense about it.
Catan is pretty good IMO. There’s a general disagreement I think between rules-purists and people who want to play fast-and-loose with the rules. The problem is that technically you always have to exchange cards to trade — so, technically it is allowable to extort people with your soldiers and the robber, but you have to at least set up a sham trade for it which adds some annoying friction; it is more fun if you say “giving cards away for free is fine” and allow an economy of extortion to flourish. And anyway if somebody doesn’t say “I’ll give you a sheep if you go get me a beer” is it really Catan? If you played it with some rules sticklers, give it another try IMO.
Risk was a preferred game both while I was in school and for some time afterwards in our ski house, etc. It definitely appealed to people who weren't broadly into board games--and certainly not hardcore war games.
I always liked it. There are some potentially annoying dynamics like how cards dominate so much in the end game and basically force you to go for it at some point.
I've played Catan on a tablet. It's OK but I seem to keep coming back most to Carcassonne for that general class of game.
Risk's much improved by goal-oriented or score-based gameplay variants, which tend to feature as the standard ruleset in newer versions and derivatives (Risk 2210, SPQRisiko, et c) for good reason. Conquer-the-world takes too long for a game that tends to eliminate half its players early on.
If you like risk you should try Kemet and Scythe. I find the combat of Kemet much better and the overall mechanics of Scythe much more rewarding. Scythe is probably too mechanically heavy for your style (it almost always requires a playthrough before you get it and has a more involved rulebook) but I mention it just because it is by far the best dudes-on-a-map game I have ever played, primarily because of how it does a fantastic job capturing the prisoners dilemma of investing in guns vs butter for your nation.
For what it's worth, personally I think Catan and Risk are both very mediocre games, especially for their popularity.
If you want all these things and exponential growth, check out Neptune's pride: https://np.ironhelmet.com/ for basically "diplomacy in space with a few extra dimensions". I haven't played in a few years (it was great at the beginning of the pandemic), but it was a lot of "fun" in the same way that diplomacy is "fun". Except with a space theme and a lot of anxiety waking up at 2am hoping your fleet arrives at an empty star system and doesn't get immediately obliterated.
Fair warning: I played and won my first round of NP when I discovered it ~11 years ago. It required unreasonable amounts of being willing to let the game take over my life at arbitrary hours for a couple of weeks, and a couple of strategic betrayals. I don't regret it, but it was a major commitment as far as games go.
The last time we played this at the company, we lost maybe two weeks of productivity over the game period. I needed to run polyphasic sleep with 2 h naps throughout but my secret alliance got to the end game strongest and then it was a vicious little fight, and one of us took it.
IMO a better game is subterfuge, because it allows you to schedule commands into the future, which means you don't have to wake up at ungodly hours of the night to execute a command.
I think some (not all) of that factor is that, being a 3+ player game, it makes explicit some friendship group dynamics that were previously implicit, and potentially not known to all the involved parties.
Ugh, people bringing in outside-the-game relationships to the game itself will ruin Diplomacy. "Tom and Jim never betray each other, and if one is in a position to sacrifice themselves to give the other the win, they always will" is how you get a group to never ever let Diplomacy hit the table after the first couple times. Plus a bunch of other games.
Folks who play like that are game-group poison. Hell, they can even make RPGs a lot less fun with that crap.
I was going more for the “Bob is going to blargh, he always blarghs when he gets flustered like when he blarghed at the thinger last year” sort of dynamic that people don't necessarily otherwise talk about. Nobody becomes a completely blank slate when they sit down, and some tendencies are something you might take advantage of in a game, where you would compensate and allow for it in real life. And those tendencies might be something the other party is sensitive about, but because it never came up otherwise…
Yeah, I considered putting it in my post but ultimately cut it, but it is indeed often romantic couples who have this problem. Not always, but often it's couples. I think good-natured competition is typically a part of an ordinary friendship, even a very close one (Impro's take on status-games, and friendship being largely defined as a relationship in which you can play such games safely and without "real" stakes, probably factors into this) is probably why this happens less often with friends. Though, certain friend pairs, if you get them together in a group of people they don't know, they'll do this too.
Although I find the fallout has been rather overstated. I'm certain it can end badly for unsuspecting participants - but I've played lots of Diplomacy (and even hosted games with a cash pot for the winners) and it has never ended in fallout. Just make sure everyone knows what they are getting into.
It's a really, really fun game that more people should try at least once.
Agreed that Diplomacy is tremendous fun, as long as you have 7+ hours to spare and a house with lots of separate rooms for private conversations to take place in. But the important thing to keep in mind is that the only way to win the game is by first making alliances with other players by promising you would never betray them, and then second by strategically betraying your allies before they can betray you. If everyone goes into the game knowing that betrayal is the point of the game and that they will be stone-cold lied to by their dearest friends, then there's no hard feelings. But if you're a brand new player and this metagame isn't properly expressed to you, then it's natural for fallout to ensue.
It only takes this long if you don't use a round timer and you don't allow shared victories. Even with new players we usually wrap up the game in 4-5 hours (people will start getting eliminated around hour 2 - so usually we have a "loser room" with other games and stuff to do).
You can also do it online with a turn a day (I ran an office Diplomacy League this way).
It's worth noting that you can play the game completely openly and honestly. We have had complete victories where the winner never once backstabbed anyone (he was just a very shrewd negotiator). It's just pretty rare because the more honest you are all game, the more reward you will get for a well placed backstab.
Also, my advice, have a cash pot. Even if it's just $20 a player. You'll only get players who are actually competitive. Also, weirdly, fewer hard feelings - I guess knowing you got betrayed for money stings less than knowing you got betrayed for fun.
Do you allow negotiation over how the pot is split for shared victories? Like can a weak player help the leader win in exchange for a small cut of the winnings?
Usually I set it up like "The pot is $350. The game is over as soon as all remaining players can unanimously agree on how to split it up."
It also adds much more drama to the end game. If you are down to a small couple of territories and basically out of the game, you might still have a lot of power to negotiate your way into the winner's circle.
There are a lot of negotiation/tactical games (Game of Thrones board game comes to mind), but they all can be equally long and none have the simplicity of Diplomacy.
IMHO, the fun of Diplomacy is sneaking in little side conversations and hiding in a corner to tell a secret. In person, it's very fun. A better approximation online would be to play asynchronously (like a turn a day) via a website.
In high school our math department had a long-running match in the teacher’s lounge with one move per day. Many times I remember my calculus teacher coming into class after the lunchtime move SEETHING.
Diplomacy is a perfect game to play one move per day among a group of friends/coworkers. Sadly I never managed to find enough people to get in board and commit to playing a weeks/months-long game
The world champion seems to say that the best Diplomacy involves honesty and cooperation. Is he eliding that point toward the end where you use that honesty and cooperation to flip and really screw someone? Or is there more to Diplomacy than just "convince people you're cooperating until the very best time to betray them"?
Anecdotally from prior play, most diplomacy wins have two broad forms:
1. A second place player has fallen behind in a low or no hard betrayal alliance and is no longer capable of a meaningful backstab, and has decided they don’t plan to backstab their partner because through their ongoing cooperation they’re the second biggest player and you’ve had a good game together. They work to cement your win, because picking the winner is often as fun as winning is.
2. Two main players and their side henchmen who are no longer serious contenders are forced into teaming to prevent the other side of the board’s leader alliance from running away with the win. There’s been a massive amount of betrayals and the table is about to have the crucial fight that will collapse one or the others’ line in defense. One disgruntled player who is on the dividing line of both alliances picks the winning coalition by lashing out against the closest player that screwed them over hardest, ruining that side’s coordination. The winning coalition breaks through, then the biggest coalition’s leader backstabs and eats its subordinates for the win.
In both cases, honesty and cooperation primarily decide the winner - either because other players have deemed you “deserving of the win” or “designated winner by dint of having successfully avoided leaving one or more key players disgruntled enough to tip against you”
You _always_ need cooperation in these wins, but you don’t always per se need to backstab people to win. Insofar as you do, those backstabs come in many flavors and often don’t feel stabby, stuff like “I’m just consistently benefiting a little more from our mutual arrangement than you are” or “I have no plan to personally screw you over, but I’m pretty sure Gary is going to do it for me and I’m not gonna stop him.”
And the most stressful way to play Diplomacy is “one turn per day.” You’ll obsess about it all evening long, then get up and immediately focus on it again!
> Be warned, it is considered to be a game that will end friendships that have lasted for years, and monopoly is considered to be a game that causes less bad feelings.
Diplomacy has nothing on the screaming matches I’ve seen and experienced with Chinatown. And you can get there in a fraction of the time required for Diplomacy
Code: https://github.com/facebookresearch/diplomacy_cicero
Site: https://ai.facebook.com/research/cicero/
Expert player vs. Cicero AI: https://www.youtube.com/watch?v=u5192bvUS7k
RFP: https://ai.facebook.com/research/request-for-proposal/toward...
The most interesting anecdote I heard from the team: "during the tournament dozens of human players never even suspected they were playing against a bot even though we played dozens of games online."