"Game theory optimal" (as poker players like to call it) is not really the optimal strategy. It's the nash equilibrium strategy assuming the other players are also playing a nash equilibrium strategy. As soon as one player deviates from the nash equilibrium it's not optimal anymore :).
It's interesting how this affects play. If a player is bluffing slightly less than they should be, the adjustment is drastic, you should actually never call with hands that do not beat their value hands. If they are bluffing optimally, you are supposed to call using what is known as "minimum defense frequency". What's interesting about this is the minimum defense frequency is based on what the strongest hands you can possibly have in that situation are and the opponents possible hands do not even factor into it. It's required to prevent your opponent from bluffing with any two cards profitably.
To do the math, if you are on the river and the opponent bets 100 into 100, for this to be profitable for them they need to win 50% of the time or more. If your opponent is bluffing optimally, you need to call with 50% of the strongest hands you have in that specific situation (if you don't know what hands you have in a specific situation that's a problem) and sometimes they can be dogshit like King high.
But, very important to note, very few players actually bluff enough and if they bluff less than they should you should only ever call when your hand actually beats their range of possible value hands. (value vs bluff is kind of a difficult thing to communicate, generally it's value if you want your opponent to call)
Most players don't bluff enough as a result of most players calling too much! When they call too much you should obviously not bluff! This leads to very boring games of poker.
>”Game theory optimal" (as poker players like to call it) is not really the optimal strategy. It's the nash equilibrium strategy assuming the other players are also playing a nash equilibrium strategy. As soon as one player deviates from the nash equilibrium it's not optimal anymore :).
This is not really what most people mean when they say “optimal strategy”. It’s true that exploitative play will make more money if you know exactly what your opponent is doing and if they keep doing it despite it not working. Neither of these will generally hold in an actual game.
The reason it’s called “optimal strategy” is because it works no matter what your opponent is doing. It will not make as much as a strategy tailored perfectly to your opponent but it will never lose to anyone under any circumstances, assuming infinite games (assuming infinite games just so we can ignore variance). The worse case the strategy has is break even to anyone else using it.
I'm kind of curious how accidental collusion could work out. Like imagine multiple players playing in such a way that they help each other - but purely out of ignorance on how best to play the game!
It sounds like you are maybe describing a dominant strategy? Oh, wait, no, you are saying that if you play the Nash equilibrium, then no other strategy among opponents will do better against it than the Nash equilibrium would, and therefore (as the game is zero sum) the worst their choice of strategy can make you get on average, is breaking even?
Ok, now that I (I think) understand your comment: I don't think you have to know exactly what non-Nash strategy someone is playing in order to exploit it. I don't think trying to estimate how someone is likely deviating from the Nash equilibrium, in order to try to exploit it, is necessarily a mistake. I think it could be feasible for someone to get better returns on average by noticing off-Nash patterns of play in other players, than playing Nash regardless would? (Not that I could win this way. I couldn't.)
A Nash equilibrium is a pair of strategies (or, one strategy for each player) such that no player can get a better result on average from deviating from it.
If one player isn’t playing a strategy that is part of any Nash equilibrium, then the best response might also not be part of any Nash equilibrium.
If all other players are playing a strategy from a given Nash equilibrium, then you can’t do better (in expectation) than you would if you were to play the strategy for you in that Nash equilibrium.
(A game may have multiple Nash equilibria. Possibly one such equilibrium could be better for you (or for everyone) than another.)
> What's interesting about this is the minimum defense frequency is based on what the strongest hands you can possibly have in that situation are and the opponents possible hands do not even factor into it.
This is actually not quite true. MDF is purely a formula based on pot size and the bet size (pot size / pot size + bet size). The fact that it doesn't consider various ranges is why it's not really useful - it was a simplified formula used to try to understand the game before solvers existed.
There are situations where your opponent can bet any two cards profitably and you do have to fold - imagine they bet the size of the pot, but have the better hand 99% of the time, you're simply forced to let them bluff the 1% of the time they're bluffing. MDF is a pre-solver concept and not an especially useful concept in the modern game.
I'm pretty sure mdf applies to rivers when you are last to act. I'd be interested in being proven wrong however if you have solver output that shows it. I remember studying solver output and seeing it in action.
I know that before the river there are range advantages that make defending mdf a losing play.
What's true is that equilibrium strategies typically converge to solutions where the better makes the caller indifferent between calling and folding. In the toy example I've given where the betters range is so strong, the caller should always fold, the better now has an incentive to add more bluffs to the range to take advantage of the folds. Then the caller will want to call more. This might converge to the MDF which might be what you're suggesting, assuming we started with ranges that could have enough bluffs given the runouts.
If you open up the solver, and give one player only Ace-Ace as their starting range, and the other player a pair of twos, and the board Ace-Ace-Three-Three-Three, then the pair of twos will fold 100% on river and will not call at MDF.
I think another way to say this is that MDF works only if you're in a spot where you have hands that are strong enough to call. If you play every hand, and you see every river in that 100into100 situation, you shouldn't call with 50% of your hands because your hand range is too wide for that to be profitable.
So you can't make a ton of mistakes say "MDF" and call off, you have to have done the right things in previous streets to end up with a range that can call at MDF. That range (and those street actions) require an understanding of GTO (and the adjustments needed when someone isn't playing GTO).
You're writing something that is at best accidentally misleading and at worst confidentially incorrect.
What is your formal definition of "optimal strategy"? A Nash equilibrium is considered optimal in the sense that it's a state where no one can gain an advantage by deviating from the equilibrium.
Sure, if your opponents don't play the Nash equilibrium, there is room to exploit that deviation and potentially gain more than what you would get from playing the Nash equilibrium. However, you also make yourself exploitable in return, so I don't think you're presenting the whole picture here.
Poker players typically optimize for making the most money in expectation per hand. Either way, I'm certain that the exploits I've described for players that bluff too much or call too much are correct so I'm not too worried about being slightly off. Poker strategy is about heuristics.
For practical poker, I'd formally define "optimal strategy" as the strategy that maximizes profit per time (or per game) for a set of opponents, including also the actions needed to "explore" and discover any bias before exploiting it.
Assuming at least one of opponents is not playing Nash equilibrium (which is a very solid assumption), playing the Nash equilibrium becomes suboptimal as it doesn't exploit the exploitable as much.
In the narrow range of poker variants (all heads up, ie only two players, not full ring like all but the last few hands of the Main Event) where it's meaningful to talk about a truly optimal game, any theoretical optimal play will still take money from all humans it's just slower (but with zero risk) compared to exploitative play.
In live cash games, speed matters, you want to take all the available chips before the fish realise they're out-matched, but to protect yourself the optimal play, if you could memorise it, would be safer because it can't be exploited yet it does take the opponent's chips.
Poker players are gamblers. So "safer" isn't really what they were going for anyway.
Trying to understand what you are getting at made me realize why I do not gamble I am just not smart enough or lucky enough. I had a friend try and convince me he knew a sure fire way to beat roulette but in the end the house always wins. He eventually had to admit he had a gambling addiction and quit doing it all together.
Like anything else, Poker is a skill that you can learn with time and practice.
It is not really related to your smartness or luck (doesn't apply to _everyone_ of course but I'd wager that the average HN reader is already smart enough for poker)
> When they call too much you should obviously not bluff! This leads to very boring games of poker.
I don't know, poker theory is all about optimal ranges and Nash equilibrium, but there's something satisfying (and very practically important, since if all your opponents even understand the phrase Nash equilibrium you should find a different game) about trying to make the most money against an opponent who calls or bluffs way too much.
Modern Poker Theory by Acevedo was the premier book on this, but I've been out of the game a few years. Idk if I'd fully trust his charts given modern sizing theory, but it's going to improve your game if you understand the concepts. If you're really serious, you want to get a solver: GTO+, PIOSolver, or GTOWizard (online version).
Chess manages in practice to be a game of imperfect information, like poker. Obviously it is explicitly a game of perfect information on the board, but the hidden information is all psychological. For instance: "do they actually know this opening/endgame? do they see a tactic? did they take a long time because they calculated that it's a good move, or are they bluffing by making it seem like they calculated something? are they actually better than me or just acting like they are?". etc.
It's true that "bluffing has minimal value against best play", but no human is in that situation. Even super-GMs will play "bluffs" if they are behind (or playing a lower-rated player and sure they can recover later. or just for fun if the stakes are low).
And that's not even mentioning optimal strategy under time pressure. For instance some of the Lichess tournaments are structured such that winning fast is more valuable than winning slowly because the resulting score comes from how many wins you can get in (or in other cases, how big of a streak you can get). So people will play in a way that optimizes for winning quickly by taking big bets / bluffing / creating chaos with un-calculated gambits, especially if they have a good reason to believe they're better than their opponents.
But every serious chess game is a matter of time allocation. The best players in the world are unable to calculate as much as they want in every position: Ultimately there is bluffing and risk taking. See the last game of the last candidates, where both Fabi and Ian have to win to get into a playoff, and they get themselves into extremely complicated positions, where accurate play just takes too long for a human. At an 8 hour time limit, the game is very different than at the time the players actually had, as ultimately Fabi just couldn't calculate to the end on every position he knew was key.
It wasn't the most accurate game of the tournament, but the most instructive as far as the psychology of chess goes
That being said, when Super GMs play a bad move against a lower rated GM, they quite often gets a pass. They don't capitalize, simply because they assume the move is excellent.
One thing I haven't seen anyone mention yet is that Nash equilibria do not actually exist when you move beyond heads-up into multi-way play. There's strong empirical evidence that solving abstracted games using MCCFR and using real-time depth-limited tree search dominates humans and outperforms all other AI strategies, but these results aren't theoretically sound, unlike in heads-up play.
I've actually been working slowly on https://github.com/krukah/robopoker, an open-source Rust implementation of Pluribus, the SOTA poker AI. What I've found interesting is the difference in how I approach actually playing poker versus how I approach building a solver. Playing the game naturally consists of reasoning about narratives and incorporating information like hand history, play style, live tells. Whereas solving the game is about evaluating tradeoffs between the guarantees of imperfect-information game theory and the constraints of Texas Hold'em, finding a balance between abstract and concrete reasoning.
Essentially all AI work I've seen in games aims for game theory optimal play, but I think it could be really interesting to consider AI for exploitative play. Does this exist? Poker with imperfect information, human pressure and fallibility means that players will inevitably stray from Nash equilibrium. The decision on how to exploit without getting exploited back oneself seems really fascinating to consider from an AI perspective. At a glance it seems to require considering how others view you..
Within solvers, you can do something called "node locking", which means you "lock" a tree in the game node to play a fixed strategy. You would typically lock it to play as you suspect your opponent plays. This lets the solver calculate the optimal exploitative solution against your specific oppoents.
Piosolver, the first public solver and the one mentioned in the article, has this feature.
However, what often happens is if you lock one node, then several other nodes in the game tree over-adjust in drastic ways, forcing you to lock all of the, which may be infeasiable. As a result, Piosolver recently introduced "incentives", which gives a player in the game an additional incentive to take a certain action . For example, you may suspect your opponent calls too much and doesn't raise enough, so you can just set that incentive and it will include that in its math equations and give you something similar to an exploitative solution with a much simpler UX.
This feature was literally just introduced a few months ago so it's still very much an active area of research, both for game theory nerds, and people trying to use the game theory nerd research to make money !
I want to see strong AI used in video games, especially strategy games. People often retort that strong AI is not fun; it's too challenging and that's not what players want. But once we have a strong AI we can adjust its goal function in fun ways. What you're describing is effectively the same, and it's the first time I've seen this used in a strong AI.
An AI that plays a fixed exploitative strategy will end up getting figured out relatively quickly and counter exploited pretty hard. This actually happens in real life sometimes when people attempt to deploy poker bots online.
Any exploitative AI also needs the ability to adjust in real time to a different exploitative strategy, which also needs to be not easily predictable, etc.
Yes, this exists! Look up models based on counterfactual regret minimisation - they learn to exploit regularities in their opponents play, and often stray from the GTO play when it makes sense. I believe they have beaten poker professionals in thousands-of-hands playoffs but I may be misremembering.
no, CFR is mainly just a way of computing Nash equilibria and (although in some sense it is an online, iterative algorithm) would typically be used to precompute Nash strategies, not update them in real time. real poker playing systems augment the CFR strategies with some real-time solving, but just to get even closer to Nash at the end of a hand.
on top of this, you could think about augmenting these systems to exploit weaknesses in opponent strategies. there is some work on this, but I don't think it's done much. The famous systems that played against professionals don't use it, they just try to get as close to GTO as possible and wait for opponents to screw up.
Hmm, I see, thanks for the reply. My mistake - I watched an interview (that I can't find now, ugh) with a poker player who played against one of the top CFRM bots and claimed that it felt like it was adapting to his playstyle.
But it sounds like that must have been either misunderstanding or some other part of the bot's algorithm I guess.
...in case you would be willing to share some knowledge - what exactly is a GTO play in poker? Does it mean a Nash equilibrium strategy? Something else entirely?
Whenever I search this stuff I get practical poker strategy guides, but none of them seem to define the term haha
Two player poker is a zero sum game, where GTO play is very well-defined as just playing a Nash equilibrium strategy. The solvers try to get as close as they can to that.
Life is a lot more complicated in multiplayer poker. There are Nash equilibria, but potentially many with different payoffs, and you can't force your opponents to choose the one you're aiming for. So in that case, it's not so obvious what "optimal" means.
As for CFR adapting to opponent play: CFR could bias its compute resources towards really finely optimizing strategies for the most likely scenarios facing certain players, and it seems like this has been done during poker tournaments.
But within those situations, it would still be trying to more perfectly approximate the Nash strategy, vs. more experimental approaches which actually choose a different strategy to exploit opponent weaknesses.
it took me a while to track this down last month:
https://codegolf.stackexchange.com/questions/tagged/king-of-...
there's also cops and robbers and at least one other "all AI compete against eachother" with the submitter usually making the first couple of "naive" bots.
I think it'd be interesting to see if an AI with visual input playing exploitatively can out perform AI playing GTO. In doing so, we can measure the effect of visual tells.
I think you have to be careful with saying stuff like "optimal strategies are unexploitable", because it usually means "unexploitable in a particular game theory sense".
Whether the assumptions of the Nash equilibrium (or any of the others) make sense for your situation in a game of poker is an empirical question, right? It's not a given that playing a NE means you'll be "perfect" in the human sense of the word, or that you'll get the best possible outcome.
The best superhuman poker AIs at the moment do not play equilibriums either, for instance.
I agree that because of, for example rake or table fees for cash games or competition structure for tournament in practice a game theoretically optimal choice may not be the right choice in practical play.
However the situation with an AI powered competitor which uses exploitative play is identical to a human, the GTO play will gradually take their chips at no risk.
It's not that they're optimal but that they've chosen not to be optimal and so that's why they lose money against GTO.
The AI is at least unemotional about this, humans with a "system" easily get tilted by GTO play and throw tantrums. How can it get there with KToff? What kind of idiot bluffs here with no clubs? Well the answer will usually be the one that's taking all your chips, be better. Humans used to seeing exploitable patterns in the play of other humans may mistake ordinary noise in the game for exploitable play in a GTO strategy and then get really angry when it's a mirage.
Right, I see what you're saying, but this is what I'm disputing - in two player games, what you wrote is true, but those properties of Nash equilibria don't generalise.
When there are more players, there can be multiple Nash equilibria, and (unlike the two player case) combinations of equilibrium strategies may no longer be an equilibrium strategy. So it's no longer true that you cannot be exploited, because that depends on other player's strategies too, and you cannot control those.
There is a huge difference between game theoretic optimal strategy and actual profitable strategy due to the human nature of the players. I imagine a professional poker player as someone who certainly knows the odds (math of the poker game in general), but is also very good in interpreting his opponents behavior (which would minimize the information revealed by them, but could they do it completely?). There are so many biases which even professional players have to overcome, that in my opinion poker is strongly psychological game. Also because of that I think that online poker and live poker are slightly different games.
It’s funny they mentioned Magic: the Gathering, because the first thing I thought of when reading the headline is all the conversation I see in r/EDH, and less frequently in other related subreddits, about a reluctance or even disdain for “optimal” play, which would be trying to win as efficiently as possible. To preserve the excitement and surprise of gameplay, people discourage building decks out of “staple” cards that would quickly homogenize play. “Broken” or “busted” cards that turn out to be more powerful than anticipated by the game’s designers wind up banned in sanctioned tournament play.
EDH is especially prone to it as a format, because it's a format which allows almost every card ever printed (with a pretty... inconsistent banlist). This makes for some pretty wild variations in power levels, and the top end tends to be both very samey and very expensive. It's very popular as a format but it only really works because there's a culture of trying to match power levels between decks (to be fair, there's also the fact that 100 card singleton does tend to mean it's hard to be completely consistent, so there's a lot more 'bleedover' between power levels in terms of chance to win in a given match, as well as the >2 player aspect levelling things out as a player obviously in the lead will tend to be ganged up on)
EDH is the variant of the game for people who aren't that competitive. They spend a lot of time trying not to win too hard. The format is fine but I don't like it beacuse of this culture around it, it basically has this social metagame where you can get better players removed out of play by complaining about them.
Draft, Standard and Modern are for people who want a real game without having to worry about playing too well.
If you’re going to play MTG as a simple game theory optimal affair, might as well switch to poker… a Nash equilibrium surely exists all the same, but charm of the game is in the variety and checks & balances.
Well, from the perspective of optimal play the more complex and changing nature of MTG means that optimal play is harder to find and doesn't stay still, so there's a lot more metagame of figuring out what that play is.
As a person who enjoys poker recreationally, whenever I visit these submissions I soon realize that this is a very different game from the one I’m playing.
Sort of related: I was at a restaurant last night where a sports channel was playing a poker tournament. Paint drying would have been more interesting; it was hard to tell if one of the players was even alive.
This is analogous to what top chess players do. They try to take the opponent into suboptimal positions hoping that they are not familiar with the best lines, even though technically the position isn’t as good as the main line.
I do personally dislike that GTO became the nomenclature , as I prefer "theory-based", since it causes this confusion, but trying to fight it at this point is hopeless because GTO is the search term people are using. And when people say they "play GTO" they usually mean "equilibrium" rather than "optimal against my specific opponents" which is "exploitative".
If you actually watch what the top players advocate for, everyone suggests you want to play exploitatively. However, there's one equilibrium solution and effectively infinite exploitative solutions, so equilibrium is a reasonble starting point to develop a baseline understanding of the mechanics of the game. It's tough to know how much "too much" bluffing is unless you know a baseline.
Furthermore, if you "exploit" people by definition you are opening yourself up to being exploited so you need to be very careful your assumptions are true.
Also, with solvers like piosolver, you can "node lock" (tell a node in the game tree to play like your opponent, rather than an equilibrium way plays), but there's many pitfalls, such as the solver adjusting in very unnatural ways on other nodes to adjust, and it being impractical to "lock" a strategy every node in the tree. There's new ideas called "incentives" which gives the solver an "incentive" to play more like a human would (e.g. calling too much) but these are new ideas still being actively explored.
Rock paper scissors is frequently used to explain GTO but it's not the best example because equilibrium in rock paper scissors will break even against all opponents, but equilibrium poker strategy will actually beat most human poker players, albeit not as much as a maximally exploitative one.
There's two other huge pieces this article glosses over:
1) It's as impossible for a human to play like a computer in poker as in chess - in fact far more impossible, because in poker you need to implement mixed strategies. In chess there's usually a best move, but in poker the optimal solution often involves doing something 30% of the time and something else 70% of the time. The problem is that, not only are there too many situations to memorize all the solutions, but actually implementing the correct frequencies is impossible for a human. Some players like to use "randomizers" like dice at the table, or looking at a clock, but I find that somewhat silly since it still so unlikely you are anywhere near equilbrium.
2) Reading someone's "tells" live is still a thing. While solvers have led to online poker to decline due to widespread "real time assistance", live poker is booming (the 2024 World Series of Poker Main Event just broke the record yet again) , and in person in live poker, people still give off various information about their hand via body language. From the 70s to the early 2000s, people were somewhat obsessed with "tells" as a way to win at poker. Since computers have advanced so much, it's fallen out of favor, but the truth is, both are useful. It's totally mistaken to think that advancement in poker AI , GTO , and solvers have rendered live reads obsolete. In fact, in 2023, Tom Dwan won the biggest pot in televised poker history (3.1 million) and credited a live read to his decision, in a spot where the solver would randomize between a call and a fold.
Yes indeed glad you noticed! Been too addicted to that godforsaken game at points so figured borrow some of its qualities for my studying apps...in general I'm interested in gamification + studying.
I vaguely recall a poker tournament where a player employed this exact strategy and it worked out for him. I think it was the circumstances and just a bit of luck that allowed him to advance to later stages.
It's interesting how this affects play. If a player is bluffing slightly less than they should be, the adjustment is drastic, you should actually never call with hands that do not beat their value hands. If they are bluffing optimally, you are supposed to call using what is known as "minimum defense frequency". What's interesting about this is the minimum defense frequency is based on what the strongest hands you can possibly have in that situation are and the opponents possible hands do not even factor into it. It's required to prevent your opponent from bluffing with any two cards profitably.
To do the math, if you are on the river and the opponent bets 100 into 100, for this to be profitable for them they need to win 50% of the time or more. If your opponent is bluffing optimally, you need to call with 50% of the strongest hands you have in that specific situation (if you don't know what hands you have in a specific situation that's a problem) and sometimes they can be dogshit like King high.
But, very important to note, very few players actually bluff enough and if they bluff less than they should you should only ever call when your hand actually beats their range of possible value hands. (value vs bluff is kind of a difficult thing to communicate, generally it's value if you want your opponent to call)
Most players don't bluff enough as a result of most players calling too much! When they call too much you should obviously not bluff! This leads to very boring games of poker.