> just keep asking it questions and maybe its reasoning would check out for most of them.
But the AI isn't reasoning... is it? Perhaps it could give an explanation, but you couldn't (currently) conflate that with any actual understanding of why it did what it did?
It's not reasoning like a human, but if it's using code it memorized from the past it might be able to string along comments it memorized from the past into something that is relevant for the context it's being asked for.
But the AI isn't reasoning... is it? Perhaps it could give an explanation, but you couldn't (currently) conflate that with any actual understanding of why it did what it did?