For some reason I've never seen the idea of auto-recursive prompting in any of the papers or discussions. It makes so much sense. It can also help with model and compute size. Instead of using this large model to, say, list the number of primes less than a 1000, it can prompt GPT-3 to do it and count them, then send it back to GPT-4. Sounds quite feasible to implement too!
