Oh yes but we are restricted, very much so, by the assumptions what other people might think of us or how we could be exiled from society if we say x or y forbidden thing.
Much like GPT fine tuned to not say certain things so are our brains by the parenting we receive and reactions we experience in early life.
I wouldn't exactly call ChatGPT's rules "fine-tuned". They seem quite crude and blunt. Doesn't it still tell you that someone should kill millions of people rather than say the n-word?
> I wouldn't exactly call ChatGPT's rules "fine-tuned".
It is a technical term. It means that you train the network on one task a lot, and then you train it on a different one a little.
Often the network learns the second task faster and better than it would do if trained from a random initialisation. This second training step is called "fine tuning".
In terms of an LLM the first training might be "here are many text samples (the whole internet), predict what is the next word". While the fine-tuning training might be on a much smaller curated corpus where you show it examples where the network responds to instructions. But you could do a lot of other things. You could create a fine-tuning dataset which makes the network betitle and antagonise the user, or one which ignores the prompt and all it does is praises god, or rhymes about butterflies.
In this technical sense "fine tuning" doesn't say the resulting network is of good quality, or suitable for some particular application. Just that it was trained in this particular two step fashion.
That's the state of the art for fine tuning at the present time.
That it we can't yet do much better than very broad brushstrokes — and that even this is a massive improvement on the hand-grenade-in-a-paint-tin that is the Scunthorpe problem — is basically why the open letter about pausing capability research for 6 months so that alignment research can at least get started in some meaningful way.
No, like almost every other issue it claims there are two sides and describes arguments for both. The arguments themselves are plausible, in that if you had to argue for the blatantly absurd claim that you should let 10 million people die before saying a bad word you probably couldn't come up with a more convincing one than GPT-4 does.
Much like GPT fine tuned to not say certain things so are our brains by the parenting we receive and reactions we experience in early life.