On: The Google employees who created transformers

[cjbprime]

> It's just not true that everything not transformer is outdated.

Am not expert, do you have some links about this? i.e. a neural net construction that outperforms a transformer model of the same size.

[pedrosorio → cjbprime]

ConvNets Match Vision Transformers at Scale (Oct 2023)

https://arxiv.org/abs/2310.16764

[golol → cjbprime]

Well transformers still have some plausible competition in NLP but besides that, there are other fields of AI where convnets or RNNs still make a lot of sense.