I find it interesting that geohot says it is what you do “when you are out of ideas,” I can’t help but think that having multiple blended models is what makes GPT-4 seem like it has more “emergent” behavior than earlier models.
OpenAI has always stuck to “simpler” approaches that scale. It’s the best bet to make and it’s paid off time and time again for them. I don’t think it’s so much “they’re out of ideas”, it’s just an idea that scales well
Yeah, and who says that this is all that they're currently working on? They likely found this to work better, launched, and did other things in the meantime.
