> There's no way to verify you're getting the original prompt.
(author here) I do suggest a verification method for readers to pursue https://lspace.swyx.io/i/93381455/prompt-leaks-are-harmless . If the sources are correct, you should be able to come to exactly equal output given the same inputs for obviously low-temperature features. (some features, like "Poem", are probably high-temp on purpose)
In fact I almost did it myself before deciding I should probably just publish first and see if people even found this interesting before sinking more time into it.
The other hint of course is that the wording of the prompts i found much more closely match how I already knew (without revealing) the GPT community words their prompts in these products, including templating and goalsetting (also discussed in the article) - not present in this naive Jasper attempt.
I guess it depends what the goal of the reverse engineering is.
If it's to get a prompt that produces similar output, then this seems like a reasonable result.
If it's to get the original prompt, I don't think that similar output is sufficient to conclude you've succeeded.
This type of reverse engineering feels more like a learning tool (What do these prompts look like?) as opposed to truly reverse engineering the original prompt.
It also depends what you hope to accomplish with what you’ve reverse engineered. For example, the spectrum of acceptably usable reverse engineered gaming consoles ranges from some baseline of targets known to work, all the way to obsessive dedication to feature and bug parity. Most (not all!) emulators opt for high compatibility, rather than exhaustive. I don’t know where that high bar is for AI prompts, but I’d bet it’s more forgiving than this exacting standard. And it’s more thorough than the learning tool characterization too.
>> There's no way to verify you're getting the original prompt.
> I do suggest a verification method for readers to pursue … you should be able to come to exactly equal output given the same inputs for obviously low-temperature inputs 90ish% of the time.
This sounds like “correct, there’s no way to verify,” but with more words.
(author here) I do suggest a verification method for readers to pursue https://lspace.swyx.io/i/93381455/prompt-leaks-are-harmless . If the sources are correct, you should be able to come to exactly equal output given the same inputs for obviously low-temperature features. (some features, like "Poem", are probably high-temp on purpose)
In fact I almost did it myself before deciding I should probably just publish first and see if people even found this interesting before sinking more time into it.
The other hint of course is that the wording of the prompts i found much more closely match how I already knew (without revealing) the GPT community words their prompts in these products, including templating and goalsetting (also discussed in the article) - not present in this naive Jasper attempt.