On: Anthropic publishes the 'system prompts' that make...

[chilling]

> Claude responds directly to all human messages without unnecessary affirmations or filler phrases like “Certainly!”, “Of course!”, “Absolutely!”, “Great!”, “Sure!”, etc. Specifically, Claude avoids starting responses with the word “Certainly” in any way.

Meanwhile my every respond from Claude:

> Certainly! [...]

Same goes with

> It avoids starting its responses with “I’m sorry” or “I apologize”

and every time I spot an issue with Claude here it goes:

> I apologize for the confusion [...]

[lolinder → chilling]

I suspect this is a case of the system prompt actually making things worse. I've found negative prompts sometimes backfire with these things the same way they do with a toddler ("don't put beans up your nose!"). It inserts the tokens into the stream but doesn't seem to adequately encode the negative.

[chilling → lolinder]

I know, I suspect that too. It's like me asking GPT to: `return the result in JSON format like so: {name: description}, don't add anything, JSON should be as simple as provided`.

ChatGTP: I understand... here you go

{name: NAME, description: {text: DESCRIPTION } }

(ノಠ益ಠ)ノ彡┻━┻

[CSMastermind → chilling]

Same, even when it should not apologize Claude always says that to me.

For example, I'll be like write this code, it does, and I'll say, "Thanks, that worked great, now let's add this..."

It will still start it's reply with "I apologize for the confusion". It's a particularly odd tick of that system.

[senko → chilling]

Clear case of "fix it in post": https://tvtropes.org/pmwiki/pmwiki.php/Main/FixItInPost

[nitwit005 → chilling]

It's possible it reduces the rate but doesn't fix it.

This did make me wonder how much of their training data is support emails and chat, where they have those phrases as part of standard responses.

[NiloCK → chilling]

I was also pretty shocked to read this extremely specific direction, given my (many) interactions with Claude.

Really drives home how fuzzily these instructions are interpreted.

[chilling → NiloCK]

I mean... we humans are also pretty bad at following instruction too.

Turn left, no! Not this left, I mean the other left!

[ttul → chilling]

I believe that the system prompt offers a way to fix up alignment issues that could not be resolved during training. The model could train forever, but at some point, they have to release it.

[jumploops → chilling]

“Create a picture of a room, but definitely don’t put an elephant in the corner.”