I find you basically have to stop thinking of LLMs as software and start thinking of them as unpredictable animals. If you issue a command and expect strict obedience every time, you've already failed. Strict orders are really a tool to persuade certain behavior rather than some sort of reliable guardrail.
So the correct way to configure LLMs is to look at them sternly and yell "BAD DOG!" when they don't follow instructions and give them treats when they do?
I find you basically have to stop thinking of LLMs as software and start thinking of them as unpredictable animals. If you issue a command and expect strict obedience every time, you've already failed. Strict orders are really a tool to persuade certain behavior rather than some sort of reliable guardrail.