You might be thinking about the Kelly criterion [0]. It does go the other way around: even if the expected value of a lottery were positive, you should only bet insignificant amounts.
If I have a lottery, and you have a 1 in a billion chance to gain 10 billion utility, tickets cost 1 utility. How much of your current utility wealth should you put in?
Classical expected value reasoning would pour in everything, even though you are almost guaranteed bankrupt at the end of that transaction.
The Kelly criterion recommends an exact (small) percentage for this style of lotteries, and is therefore probably more sensible than decision making based on expected values.
Notable: Kelly only works if you're playing a multi-round game.
Also careful; I believe money-to-utility is already logarithmic for most people. We don't have a good intuition for what "billions of utility vs 1 utility" represents.
Kelly criterion assumes you will increase your bet when you win, to increase overall final value, and the EV is positive. It doesn't work when you expect to only win at most once.
If I have a lottery, and you have a 1 in a billion chance to gain 10 billion utility, tickets cost 1 utility. How much of your current utility wealth should you put in?
Classical expected value reasoning would pour in everything, even though you are almost guaranteed bankrupt at the end of that transaction.
The Kelly criterion recommends an exact (small) percentage for this style of lotteries, and is therefore probably more sensible than decision making based on expected values.
[0] https://en.m.wikipedia.org/wiki/Kelly_criterion