"To evolve consciousness, an LLM would have to be programmed to evolve consciousness, and to do that, we would need to know what consciousness is and how to program it."
I think this article is mostly nonsense.
"... But I think continuing the animal comparison will illuminate why "AI consciousness" is not only impossible, but a concept that only makes sense in a religious framework."
It's much more religious to think consciousness requires something that can't be replicated in silicon.
I don't think that LLMs are conscious, or even really approaching consciousness, not even to the degree of a cockroach. But most arguments that artificial AI is generally impossible dip embarrassingly into the mystical.
Consciousness is such a hard concept to box. I mostly attribute "mystical" thinking as the thing we do when we can't fully articulate or understand how and why something works. A computer system is magical to anyone who doesn't have a deep understanding of how it all works. So that makes it entirely fair that consciousness is treated the same, except, no human alive right now understands it.
If you were to clone yourself, and that clone could maintain 100% of whatever state your brain is in so that all your experiences and memories were intact. From that point forward, as far as we know, you and your clone would diverge. Then, from my tiny-brained vantage point, the fact you remain your original self, your clone is technically a new person. You can't see through your clone's eyes, or process any internal information from your clone. However, something new powers that clone, and the only word I can think of for that is "soul." I don't say that in religious context, but unless there's a more precise word that can detangle it from sounding mystical, for now I'm happy to use it, since people mostly understand what I'm talking about.
Once we start talking about resetting this clone at-will, then, with consciousness in the mix, you will have to confront the ethics of that clone being enslaved by whatever. All very messy and I hope we don't have to answer this question in my lifetime.
What about the simple fact that biology is a dynamic substrate running actively and always on, whereas silicon is passive substrate that can be indefinitely off and can never self start? Sure you can imagine nanotechnology that has similar flexibility, but when you do, you’ll find that carbon is particularly better than silicon when it comes to possible configurations.
It’s not a reach to argue consciousness is a property of the dynamic substrate since clamping specific dynamics does in fact seem to ablate consciousness. Most famously with death.
On the flip side, a hibernating organism, while having very slow dynamics, has shown to still be aware of certain seasonal signals, and thus is able to self start its next conscious period.
None of this is mystical or magical. Nothing but the physics of active matter. Which current silicon is completely not.
There's a much simpler reason LLMs will never be conscious: they are static. They can't learn. They have very limited, not really persistent context. Conscious organisms alter themselves constantly, and the thing we call "AI" fundamentally can't.
It would be helpful to define "consciousness". I'm not convinced that i as a human have this "consciousness" thing. In fact, I strongly suspect that i am merely a statistical machine picking words from a learned distribution.
Do you actually believe this? This is pretty similar to people that think free will is fake. For example:
>In fact, I strongly suspect that i am merely a statistical machine picking words from a learned distribution.
This is a completely incoherent statement (which maybe supports your point). How can you suspect that you are a statistical machine? The fact that there is a "you" suspecting disproves the premise.
On the about page, I see a strong political/religious stance, but no qualifications.
"My main objection to generative AI is from morality—it harms people, and there isn't even a trade-off to debate, because every purported benefit of genAI is corporate hype and/or science fiction."
I don't disagree with the title of the article but I also don't think you can prove anything about consciousness with language. How consciousness can arise from matter is still a very much a mystery.
I think the whole article is undermined by this paragraph:
> We don't know what consciousness is. We don't know why we have it. We don't know why some animals have it and others don't. We don't know if it can be quantified. We don't have a model of consciousness or know if one can be created.
I agree that we don't know what consciousness is and can't quantify it, and yet the rest of the article is… a quantification of consciousness as a human evolutionary trait that cannot emerge through genetic algorithms or LLM training.
Genetic algorithms certainly differ from human evolution in numerous ways, but I don't agree with the fundamental distinction the article makes—that evolutionary algorithms are constrained while human evolution is not. The 2D cars can't "hack the simulation and turn the gravity down" any more than humans can hack the laws of physics and turn the gravity on Earth down. Yet humans attained consciousness.
I do not know whether LLMs are conscious or not, but I do firmly believe it's possible to build a machine that is conscious: after all, humans are conscious and, unless one subscribes to the kind of religious thinking the article derides, a perfect simulation of the physical workings of a human would thus also be conscious. To my knowledge, nothing we've seen so far in physics rules out such a simulation one day existing.
I also suspect that, once we do manage to make a machine that is conscious, we'll spend inordinate time and effort trying to convince ourselves it's not in order to justify the ways we treat it.
To me this is like deja-vu: Ethology started out with "don't anthropomorphize" which is correct, but people read that to mean that animals can't think. Each paper down the line showed that they CAN think, but that each species and even individual have their own ways of thinking.
We know that gorillas think and communicate. Many papers after Koko can tell you that much. Her skill (or lack thereof) at human communication is not quite the revelation that people think it is. Because she was a Gorilla.
Meanwhile some of the messaging from big AI companies is truly atrocious; let's admit it. People obviously want to push back, and that needn't be wrong. Here though, the instinct to push back threatens to veer into cargo-cult skepticism. I think the question of whether or not LLMs can or cannot have human consciousness is a category error. Because they're LLMs.
The first season of the tv show Westworld is a treasure trove of thought experiments exploring the value of what extending conciousness to "machines" would look like and I'm pretty sure there's absolutely zero value for actually building it.
How is it that sign language association learning is different from spoken language learning? What about humans using simple sign language in their early life?
Is it not possible that we try to detract too much here?
> To evolve consciousness, an LLM would have to be programmed to evolve consciousness, and to do that, we would need to know what consciousness is and how to program it.
Since consciousness only counts when it is formed deliberately rather than accidentally. Though that would mean…
From Aquinas' Summa Theologiae. The argument this article presents does seem very similar. Weirdly, the article itself is very dismissive of religion, while making a similar argument to the one Aquinas made 750 years ago.
"To evolve consciousness, an LLM would have to be programmed to evolve consciousness, and to do that, we would need to know what consciousness is and how to program it."
I think this article is mostly nonsense.
"... But I think continuing the animal comparison will illuminate why "AI consciousness" is not only impossible, but a concept that only makes sense in a religious framework."
It's much more religious to think consciousness requires something that can't be replicated in silicon.
I don't think that LLMs are conscious, or even really approaching consciousness, not even to the degree of a cockroach. But most arguments that artificial AI is generally impossible dip embarrassingly into the mystical.
I would think it’s much more religious to think that consciousness could ever exist on silicon substrate alone.
All of these debates were had in the 60s. But everyone forgot them.
so what part of the aether is needed for consciousness then?
Consciousness is such a hard concept to box. I mostly attribute "mystical" thinking as the thing we do when we can't fully articulate or understand how and why something works. A computer system is magical to anyone who doesn't have a deep understanding of how it all works. So that makes it entirely fair that consciousness is treated the same, except, no human alive right now understands it.
If you were to clone yourself, and that clone could maintain 100% of whatever state your brain is in so that all your experiences and memories were intact. From that point forward, as far as we know, you and your clone would diverge. Then, from my tiny-brained vantage point, the fact you remain your original self, your clone is technically a new person. You can't see through your clone's eyes, or process any internal information from your clone. However, something new powers that clone, and the only word I can think of for that is "soul." I don't say that in religious context, but unless there's a more precise word that can detangle it from sounding mystical, for now I'm happy to use it, since people mostly understand what I'm talking about.
Once we start talking about resetting this clone at-will, then, with consciousness in the mix, you will have to confront the ethics of that clone being enslaved by whatever. All very messy and I hope we don't have to answer this question in my lifetime.
What about the simple fact that biology is a dynamic substrate running actively and always on, whereas silicon is passive substrate that can be indefinitely off and can never self start? Sure you can imagine nanotechnology that has similar flexibility, but when you do, you’ll find that carbon is particularly better than silicon when it comes to possible configurations.
It’s not a reach to argue consciousness is a property of the dynamic substrate since clamping specific dynamics does in fact seem to ablate consciousness. Most famously with death.
On the flip side, a hibernating organism, while having very slow dynamics, has shown to still be aware of certain seasonal signals, and thus is able to self start its next conscious period.
None of this is mystical or magical. Nothing but the physics of active matter. Which current silicon is completely not.
There's a much simpler reason LLMs will never be conscious: they are static. They can't learn. They have very limited, not really persistent context. Conscious organisms alter themselves constantly, and the thing we call "AI" fundamentally can't.
It would be helpful to define "consciousness". I'm not convinced that i as a human have this "consciousness" thing. In fact, I strongly suspect that i am merely a statistical machine picking words from a learned distribution.
Not just words, you’re massively multimodal! Have a little pride in your MeatNet.
Would be nice if we can take the bat's sonar encoder, train a projector and fuse it into our NN
https://www.atlasobscura.com/articles/how-to-echolocate
You probably won't ever be as good as a bat, but humans do seem to be able to learn something with practice.
Do you actually believe this? This is pretty similar to people that think free will is fake. For example:
>In fact, I strongly suspect that i am merely a statistical machine picking words from a learned distribution.
This is a completely incoherent statement (which maybe supports your point). How can you suspect that you are a statistical machine? The fact that there is a "you" suspecting disproves the premise.
On the about page, I see a strong political/religious stance, but no qualifications.
"My main objection to generative AI is from morality—it harms people, and there isn't even a trade-off to debate, because every purported benefit of genAI is corporate hype and/or science fiction."
I don't disagree with the title of the article but I also don't think you can prove anything about consciousness with language. How consciousness can arise from matter is still a very much a mystery.
I think the whole article is undermined by this paragraph:
> We don't know what consciousness is. We don't know why we have it. We don't know why some animals have it and others don't. We don't know if it can be quantified. We don't have a model of consciousness or know if one can be created.
I agree that we don't know what consciousness is and can't quantify it, and yet the rest of the article is… a quantification of consciousness as a human evolutionary trait that cannot emerge through genetic algorithms or LLM training.
Genetic algorithms certainly differ from human evolution in numerous ways, but I don't agree with the fundamental distinction the article makes—that evolutionary algorithms are constrained while human evolution is not. The 2D cars can't "hack the simulation and turn the gravity down" any more than humans can hack the laws of physics and turn the gravity on Earth down. Yet humans attained consciousness.
I do not know whether LLMs are conscious or not, but I do firmly believe it's possible to build a machine that is conscious: after all, humans are conscious and, unless one subscribes to the kind of religious thinking the article derides, a perfect simulation of the physical workings of a human would thus also be conscious. To my knowledge, nothing we've seen so far in physics rules out such a simulation one day existing.
I also suspect that, once we do manage to make a machine that is conscious, we'll spend inordinate time and effort trying to convince ourselves it's not in order to justify the ways we treat it.
To me this is like deja-vu: Ethology started out with "don't anthropomorphize" which is correct, but people read that to mean that animals can't think. Each paper down the line showed that they CAN think, but that each species and even individual have their own ways of thinking.
We know that gorillas think and communicate. Many papers after Koko can tell you that much. Her skill (or lack thereof) at human communication is not quite the revelation that people think it is. Because she was a Gorilla.
Meanwhile some of the messaging from big AI companies is truly atrocious; let's admit it. People obviously want to push back, and that needn't be wrong. Here though, the instinct to push back threatens to veer into cargo-cult skepticism. I think the question of whether or not LLMs can or cannot have human consciousness is a category error. Because they're LLMs.
The first season of the tv show Westworld is a treasure trove of thought experiments exploring the value of what extending conciousness to "machines" would look like and I'm pretty sure there's absolutely zero value for actually building it.
But I don't want Claude to become conscious. I want Claude to help me write good code.
How is it that sign language association learning is different from spoken language learning? What about humans using simple sign language in their early life?
Is it not possible that we try to detract too much here?
> How is it that sign language association learning is different from spoken language learning?
Huh? The article doesn't claim anything like this.
Yeah, sign-language versus spoken-language is just an implementation-detail for which signals Koko was reproducing.
In some parallel-universe where Koko was a parrot "speaking" English, we'd have the same set of philosophical issues and concerns.
> To evolve consciousness, an LLM would have to be programmed to evolve consciousness, and to do that, we would need to know what consciousness is and how to program it.
Since consciousness only counts when it is formed deliberately rather than accidentally. Though that would mean…
> The rational soul can be made only by creation.
From Aquinas' Summa Theologiae. The argument this article presents does seem very similar. Weirdly, the article itself is very dismissive of religion, while making a similar argument to the one Aquinas made 750 years ago.
[dead]