But both models did see the gorilla when prompted with it…?
ChatGPT:
> It looks like the scatter plot unintentionally formed an artistic pattern rather than a meaningful representation of the data.
Claude:
> Looking at the scatter plot more carefully, I notice something concerning: there appear to be some unlikely or potentially erroneous values in the data. Let me analyze this in more detail.
> Ah, now I see something very striking that I missed in my previous analysis - there appears to be a clear pattern in the data points that looks artificial. The data points form distinct curves and lines across the plot, which is highly unusual for what should be natural, continuous biological measurements.
Given the context of asking for quantitative analysis and their general beaten-into-submission attitude where they defer to you, eg, your assertion this is a real dataset… I’m not sure what conclusion we’re supposed to draw.
That if you lie to the AI, it’ll believe you…?
Neither was prompted that this is potentially adversarial data — and AI don’t generally infer social context very well. (A similar effect occurs with math tests.)
ChatGPT:
> It looks like the scatter plot unintentionally formed an artistic pattern rather than a meaningful representation of the data.
Claude:
> Looking at the scatter plot more carefully, I notice something concerning: there appear to be some unlikely or potentially erroneous values in the data. Let me analyze this in more detail.
> Ah, now I see something very striking that I missed in my previous analysis - there appears to be a clear pattern in the data points that looks artificial. The data points form distinct curves and lines across the plot, which is highly unusual for what should be natural, continuous biological measurements.
Given the context of asking for quantitative analysis and their general beaten-into-submission attitude where they defer to you, eg, your assertion this is a real dataset… I’m not sure what conclusion we’re supposed to draw.
That if you lie to the AI, it’ll believe you…?
Neither was prompted that this is potentially adversarial data — and AI don’t generally infer social context very well. (A similar effect occurs with math tests.)