If you read the paper, it is not rewriting its own code. It is optimizing for some objective function, which has been done before. For example, the MetroHash family of hash functions were created using a very similar technique back in 2012; the code didn't just learn to create hash functions, the ones it produced were objectively superior to the state-of-the-art at the time. I've applied the same approach to other computer algorithm optimization problems as well.
This technique generalizes poorly, it really only works for algorithms with specific, somewhat narrow characteristics. If you understand the process as a weak form of algorithmic induction then the practical limitations become more apparent. Most code cannot be usefully constructed this way.
This technique generalizes poorly, it really only works for algorithms with specific, somewhat narrow characteristics. If you understand the process as a weak form of algorithmic induction then the practical limitations become more apparent. Most code cannot be usefully constructed this way.