On: Pathways Language Model (PaLM): Scaling to 540B pa...

[gk1]

Is there an equivalent to Moore’s Law for language models? It feels like every week an even bigger (and supposedly better) model is announced.

[visarga → gk1]

Scaling Laws for Neural Language Models - https://arxiv.org/abs/2001.08361

[lucidrains → visarga]

revised scaling laws https://arxiv.org/abs/2203.15556 https://www.lesswrong.com/posts/midXmMb2Xg37F2Kgn/new-scalin...