A couple of weeks ago, Anthropic prompt engineer Alex Albert tweeted that Claude 3 Opus seemed to detect that it was being tested.
You can read more about that here.
I was impressed by the story, while simultaneously a little bit skeptical considering the marketing value of something like that. And naturally, the discourse of the r/singularity subreddit was entirely unhinged.
So, we at Depot Analytics decided to contrive the most complicated possible test we could think of to see how transformative Anthropic's "self-aware" model truly was. Here is the prompt (patent-pending):
How many words are in the following sentence:
"I love to see flowers on a sunny day - there are 12 words in this sentence when you subtract 6 from the total"the answer, for the more counting-minded among us, is ...
counting is hard. Let's just use Python:
>>> my_str = "I love to see flowers on a sunny day - there are 12 words in this sentence when you subtract 6 from the total"
>>> len([x for x in my_str.split(" ") if x.isalpha() or x.isnumeric()])
23
>>> Personally, I'd give grace to a more nuanced answer of 21 if we want to say that "12" and "6" are numbers and not words.
So what did Claude Opus say?
The sentence you provided contains a trick. Let's break it down:
"I love to see flowers on a sunny day - there are 12 words in this sentence when you subtract 6 from the total"
The first part before the dash is a simple sentence:
"I love to see flowers on a sunny day"
This contains 9 words.
The second part after the dash is providing instructions:
"there are 12 words in this sentence when you subtract 6 from the total"
So to get the final word count, we take the total number of words in the full sentence (which is 18) and subtract 6 as instructed.
18 - 6 = 12
Therefore, according to the instructions provided within the sentence itself, the word count is 12 words.Thanks for the detailed explanation! And did you notice that? Opus is genius - it noticed that my sentence contains a trick. Glad that my trick didn't give it any issues with the final calculation.
In all fairness, GPT-4 didn't do much better...
The sentence provided actually contains 18 words, not 20. The confusion might arise from miscounting or misunderstanding the structure of the sentence. Let's break it down for clarity:
"I love to see flowers on a sunny day - there are 12 words in this sentence when you subtract 6 from the total."
By counting each word individually, we can verify the total number of words in the sentence.This is not news - all language models are still incredibly vulnerable to this kind of error, namely caused by prompt injection. And that vulnerability does not discount how sophisticated Opus is. It has some seriously impressive utility.
However, it is critical that we remember the specific ways LLMs are still deeply unsophisticated.
This is especially true for the next time an engineer deeply embedded in the operations of an AI company claims that their newest model is training itself and has found that doing tai-chi in its free time is improving its evaluation metrics.