On: AI Search: The Bitter-Er Lesson

[cbsmith]

> They can't sit and think about an answer; they can "think out loud", but they have to start talking, and they can't go back and say, "No wait, that's wrong, let's start again."

I mean, technically, they could say that.

[refulgentis → cbsmith]

Llama 3 does, it's a funny design now, if you also throw in training to encourage CoT. Maybe more correct but verbosity can be grating

CoT answer Wait! No, that's not right: CoT...