The thing is, GPT-4 isn't useful to me because it understands language. It's useful to me because it understands a wide variety of advanced, complex phenomena and is able to synthesize these insights into novel solutions to all sorts of problems.
A lot of knowledge could be taken out and put in a vector search system which dynamically loads context into the input, but some knowledge seems worth embedding into the model. The question becomes what knowledge is critical in order to create a useful, large-context language model which can run on the edge.
A lot of knowledge could be taken out and put in a vector search system which dynamically loads context into the input, but some knowledge seems worth embedding into the model. The question becomes what knowledge is critical in order to create a useful, large-context language model which can run on the edge.