On: Learning to Reason with LLMs

[logicchains]

I guess they'd also charge for the chain of thought tokens, of which there may be many, even if users can't see them.

[fraboniface → logicchains]

That would be very bad product design. My understanding is that the model itself is similar to GPT4o in architecture but trained and used differently. So the 5x relative increase in output token cost likely already accounts for hidden tokens and additional compute.

[natrys → fraboniface]

> While reasoning tokens are not visible via the API, they still occupy space in the model's context window and are billed as output tokens.

https://platform.openai.com/docs/guides/reasoning

So yeah, it is in fact very bad product design. I hope Llama catches up in a couple of months.

[AkelaWolf → natrys]

Most likely the model has similar size compared to the original gpt4, which also has similar price.