AFAIK the training works by just selecting random words in sentences to have the...

sam0x17 on July 19, 2023 | parent | context | favorite | on: Wikipedia-grounded chatbot “outperforms all baseli...

AFAIK the training works by just selecting random words in sentences to have the LLM "fill in" until it generalizes to being able to write entire sentences. The training data can be completely unstructured.