People used to anthropomorphize Markov chains like this. I've been through this rodeo before, at least a few times.
I did a little lab on GPT with middle schoolers as part of a Science enrichment activity. Without prompting, the entire class was making it say nonsense inside of the one hour session.
Try using a GPT model to do something humans actually do, other than random bullshitting on the internet. Field customer service requests. Even carefully SFT'd models struggle to beat decision trees, and are sometimes worse, at actually solving the customer's problem. They sound more human / less robotic, but who cares if the customer's problem isn't solved and the dialog quickly diverges into insanity?
Just because you want to see God on a piece of toast doesn't mean that the average human has completely lost their capacity for critical thought.
> doesn't mean that the average human has completely lost their capacity for critical thought
The average person, maybe everyone, is very selective about where they apply their critical thought processes.
And groups of people ... groups don't have critical thought like individual people do.
Corporations, political parties, social groups don't "think" coherently, but wield most of the power by being centralized or synchronized.
Groups survive and grow by developing feedback and incentives that push back against individuals with critical not-aligned-with-the-group thought.
If we are depending on unorganized mass critical thought to save us, we are surely doomed. We are going to need our best systems for getting along to date, and upgrades.
I did a little lab on GPT with middle schoolers as part of a Science enrichment activity. Without prompting, the entire class was making it say nonsense inside of the one hour session.
Try using a GPT model to do something humans actually do, other than random bullshitting on the internet. Field customer service requests. Even carefully SFT'd models struggle to beat decision trees, and are sometimes worse, at actually solving the customer's problem. They sound more human / less robotic, but who cares if the customer's problem isn't solved and the dialog quickly diverges into insanity?
Just because you want to see God on a piece of toast doesn't mean that the average human has completely lost their capacity for critical thought.