The real magic of recursive self improvement happens only after you have human-level AI that is able to match and surpass human ability in designing AI architectures. Escape-velocity-breaking recursive self improvement doesn't look like a human-made architecture being trained further, it looks like an AI understanding why transformers/etc were successful and coming up with an advancement over transformers.
