> doesn’t it then mean that whatever process is encoded in the NN, it should both be possible to represent in some more efficient representation...?
Not if NNs are complex systems[1] whose useful behavior is emergent[2] and therefore non-reductive[3]. In fact, my belief is that if NNs and therefore also LLMs aren't these things, they can never be the basis for true AI.[4]
Not if NNs are complex systems[1] whose useful behavior is emergent[2] and therefore non-reductive[3]. In fact, my belief is that if NNs and therefore also LLMs aren't these things, they can never be the basis for true AI.[4]
---
[1] https://en.wikipedia.org/wiki/Complex_system
[2] https://en.wikipedia.org/wiki/Emergence
[3] https://en.wikipedia.org/wiki/Reductionism, https://www.encyclopedia.com/humanities/encyclopedias-almana..., https://academic.oup.com/edited-volume/34519/chapter-abstrac...
[4] Though being these things doesn't guarantee that they can be the basis for true AI either. It's a minimum requirement.