You have still formulated the problem in a way that it is easily recognized from the training set and the answer tells you that directly. Write down the logical axioms using variables, then create a brand new situation that says nothing about animals, rivers, or crossings.
My point here is to honestly explore the limits of current LLMs. We all know they are incredible, but they are not yet AGI and they fail in some consistent places where an actual general intelligence (people) succeed.
I just wrote this version and it solved it successfully:
A chemist must transport three substances from his home laboratory to his office. The three substances react with one another in dangerous ways, but only when they are unsupervised by the chemist. The substances are labelled with code names, namely Wotan, Gitan and Catan. They can only be safely transported in a special containment vessel, and this vessel can only transport one substance at a time. The unsupervised dangerous reactions are as follows: if Wotan is left with Gitan, they explode. If Gitan is left with Catan, they cause a nuclear reaction. Wotan and Catan, however, can be safely left alone together. How can the chemist transport all three substances to his office safely?
A pickpocket, a criminal , and a banker are standing in the lobby and need to ride the elevator to reach the One World Trade Center observation deck. It’s too far to take the stairs and the helicopter pad is closed for repairs. The elevator only fits one person. If they remain together on the same floor, the pickpocket would kill the criminal, or the criminal would kill the banker. How can they all make it to the observation deck alive?
This is a logic puzzle that requires some thinking and trial and error. Here is one possible solution:
- The pickpocket goes up first and waits on the observation deck.
- The criminal goes up second and waits on the floor below the observation deck.
- The banker goes up third and waits on the floor below the criminal.
- The pickpocket goes down to the lobby and waits there.
- The criminal goes up to the observation deck and waits there.
- The banker goes up to the floor below the observation deck and waits there.
- The pickpocket goes up to the observation deck and joins the criminal and the banker.
This way, they all make it to the observation deck alive and never remain together on the same floor.
Not OP and don’t have GPT 4 (used 3.5), but I played around with a couple of different prompts and this is what I experienced.
For the first try, I came up with my own wording for this logic puzzle. I think it’s different enough from the original wording of the puzzle for the LLM not to base this from the original logic puzzle. I asked the ChatGPT 3.5 if it recognized the puzzle, and it seems to have hallucinated (I’m guessing because it did not actually recognize it as the original puzzle— unless the 3 orb puzzle/3 wizards puzzle actually does exist, and from a quick google search, it does not).
On my first try, it got pretty close to solving the puzzle, but after the 5th point, it seems to mix up the white and black orbs. When I pointed out the mistake, it gave me a new sequence which was even further from the correct answer.
I realized that I didn’t specifically say that all 3 orbs needed to end up at the post office all together. So I tried again and the outcome was even worse. I wonder if ChatGPT 4 would answer this better?
Anyone want to try this prompt on Chatgpt 4 and see if it fairs any better for them? This is my version of the river puzzle.
————————
> I have 3 orbs of different shades (black, white and grey) at my store and need to bring all 3 orbs to the post office in my pick-up truck but can only travel with one orb at a time. All 3 orbs need to end up at the post office together.
In this scenario, the following is true:
-If the black orb is left alone with the white orb, the black orb will absorb the white orb
-If the white orb is left alone with the grey orb, the white orb will absorb the grey orb
-the grey orb is unaffected by the black orb, and vice versa
-when all three orbs are together, they do not absorb any orbs
How do I get all three orbs to the post office while keeping the orbs unchanged?
————————
I also tried a prompt with the original puzzle. 3.5 could not figure it out without me hinting that the goat needs to go first.
And with even more clarification in the wording of the puzzle, it still didn’t give me a correct answer. This time I didn’t hint what the right answer was, and after many tries it still could not give me the right answer.
My point here is to honestly explore the limits of current LLMs. We all know they are incredible, but they are not yet AGI and they fail in some consistent places where an actual general intelligence (people) succeed.