On: DeepSeekMath 7B achieved 51.7% on MATH benchmark

[riku_iki]

It's actually interesting results in a sense we see the limitation of LLM to memorize complicated information correctly. Gemini ultra also reported around 50% accuracy

[Davidzheng → riku_iki]

I think the SOTA is GPT4+tool use? I heard near 80%

[riku_iki → Davidzheng]

Yes, tools help to advance over LLM limitations. GPT4 without tools is about 50% too.