"Indeed, the assumption that consciousness will just come along for the ride as AI gets smarter echoes a kind of human exceptionalism that we’d do well to see the back of. We think we’re intelligent, and we know we’re conscious, so we assume the two go together."
Isn't this assumption actually arguing AGAINST human exceptionalism? I feel like there are only two options to what consciousness can be explained by (the idea coming from another HN comment sometime/somewhere that I have forgotten, so I can't claim originality)
1. Consciousness ultimately arises out of a physical process, albeit incredibly complicated, but could be computed given a sufficiently powerful computer, or
2. The concept of a human soul is true, or
3. There is no third way, pick one of the above.
For human consciousness to be "exceptional"... we would have to have some undefiniable "other" quality that could never be explained by science and is therefore necessarily supernatural?
> For human consciousness to be "exceptional"... we would have to have some undefiniable "other" quality that could never be explained by science and is therefore necessarily supernatural?
No, it only means that science has limits on explaining the natural world. See Colin McGinn's argument for cognitive closure. He thinks we can't solve certain problems in philosophy because we lack the correct cognitive machinery. Consciousness being a primary example since it proposes a subjective/objective split where science only explains the objective. Nagel argued along similar lines in his paper, "What Is It Like to Be a Bat?".
There's no need to invoke the supernatural. It just means some natural things may be beyond our capability to understand. Why would we have evolved to understand everything?
You're talking about the hard problem of consciousness, the post you're replying to is talking about the "easy" problem.
Science can't and won't ever be able explain why being conscious feels like this, but it should in principle be possible to figure out what is physically going on in the brain while it's happening.
Sure, but the tricky part is when we apply that to non-biological intelligences. Or even non-human ones, and thus questions about bat consciousness. At any rate, there's lots of room for different positions on consciousness that don't have to invoke the supernatural.
3. Consciousness is mediated by a noncomputable-in-principle physical process.
A lot of assumptions we make about the world, baked in so far down that we can barely see them, are completely trashed by Bell’s Theorem. So I’m not prepared to assume that all physical processes are computable, either. Plato’s Cave is how it is and there’s only so much we can reasonably infer from the shadows we see on the wall.
Isn't this assumption actually arguing AGAINST human exceptionalism? I feel like there are only two options to what consciousness can be explained by (the idea coming from another HN comment sometime/somewhere that I have forgotten, so I can't claim originality)
1. Consciousness ultimately arises out of a physical process, albeit incredibly complicated, but could be computed given a sufficiently powerful computer, or
2. The concept of a human soul is true, or
3. There is no third way, pick one of the above.
For human consciousness to be "exceptional"... we would have to have some undefiniable "other" quality that could never be explained by science and is therefore necessarily supernatural?