It's a fundamentally different problem. AlphaZero (DeepMind) was able to be trained this way because it was setup with an explicit reward function and end condition. Competitive self-play needs a reward function.
It can't just "self-improve towards general intelligence".
Actually you are stating 2 different problems at the same time.
A lot of intelligence is based around "Don't die, also, have babies". Well AI doesn't have that issue as long as it produces a good enough answer we'll keep the power flowing to it.
The bigger issue, and one that is likely far more dangerous, is "Ok, what if it could learn like this". You've created super intelligence with access to huge amounts of global information and is pretty much immortal unless humans decide to pull the plug. The expected alignment of your superintelligent machine is to design a scenario where we cannot unplug the AI without suffering some loss (monetary is always a good one, rich people never like to lose money and will let an AI burn the earth first).
The alignment issue on a potential superintelligence is, I don't believe, a solvable problem. Getting people not to be betraying bastards is hard enough at standard intelligence levels, having a thing that could potentially be way smarter and connected to way more data is not going to be controllable in any form or fashion.
Can ChatGPT evaluate how good ChatGPT-generated output is? This seems prone to exaggerating blind-spots, but OTOH, creating and criticising are different skills, and criticising is usually easier.
Not General, but there's IQ tests. Undergraduate examinations. Can also involve humans in the loop (though not iterate as fast), through usage of ChatGPT, CAPTCHA's, votes on reddit/hackernews/stackexchanges, even pay people to evaluate it.
Going back to the moat, even ordinary technology tends to improve, and a headstart can be maintained - provided its possible to improve it. So a question is whether ChatGPT is a kind of plateau, that can't be improved very much so others catch up while it stands still, or it's on a curve.
A significant factor is whether being ahead helps you keep ahead - a crucial thing is you gather usage data that is unavailable to followers. This data is more significant for this type of technology than any other - see above.
It surely will have huge blindspots (and people do too), but perhaps it will be good enough for self-improvement... or will be soon.