On: RWKV: Reinventing RNNs for the Transformer Era

[cypress66]

> Transformers et al would have been figured out in no time had we had the sort of data tools we have today.

I disagree. Even things that seem obvious in retrospect, take some time to be figured out.

Resnets, batch norm, dropout, etc are examples of this.

And I don't think transformers are obvious.