Well, it's unorthodox, but if we're sharing psychedelic visions... in 2018 I saw all of mankind marching together in perfect lockstep, atop an infinite uniform grey plane, to the beat of a central computer clock.
"They are going to offer you a chip. You tell them, no thank you!"
---
More broadly this strikes me as a new era of biology. (Genes -> Memes -> Temes) Straightforwardly then, we should aim for a symbiosis. (Symbiosis was Musk's vision for Neuralink, from what I recall.)
I've heard people say man hasn't yet transcended a zero-sum mindset (with rare exceptions), and that this is the attitude we're passing onto the machines. And if one of us has to lose... well, that's probably not going to go so well for us.
What is the capital market incentive for symbiosis? You are going to spend a crazy amount of time, money and effort to try to mix human behavior and cells to work with digital substrates.
Or you can simply make a machine that does the same thing fully artificially but everything is cheaper because you dont have the constraint of having to fit a bunch of biological features.
It's like my colleagues who were talking about uploading their minds to the cloud in the future, like what would be the benefit of this every happening? What information can you provide as a human that the full aggregate of knowledge does not already have? Why should memory and disk space be wasted on this
I always find that idea of uploading a mind so detached from reality. It's as irrational as thinking that physically reprogramming one twin's brain to be the same as the other's would make them the same conscious person.
Well, the bee doesn't need to integrate its cells with the flowers, but they've gotten along just fine for a very long time.
By symbiosis I simply mean mutually beneficial coexistence.
We've barely achieved that with each other, but my point here is that we should strive to achieve that with the machines.
We're still treating it as if someone has to win and someone has to lose. Someone has to come out "on top."
If that's the game we want to play, against an opponent that is self improving... that doesn't strike me as an excellent strategy. We should probably pick a different one.
--
To answer your question, the incentive is so that we can continue to exist.
If the market doesn't reward our continued existence, then we should probably have a conversation about that at some point... I've heard the global economy described as a misaligned super-organism even before AI. AI mostly just makes it go faster in roughly the same directions it was already going.
> "If that's the game we want to play, against an opponent that is self improving... that doesn't strike me as an excellent strategy. We should probably pick a different one."
> "If the market doesn't reward our continued existence, then we should probably have a conversation about that at some point"
---
There is a lot of "we should probably" carrying the point here.
The incentives are made by the market, and it happens to be an incentive that this discussion, or any action for that manner does not happen.
Given the current status quo, and the foreseeable future given the incentives, how do you suggest that we even get to a time where symbiosis is on the table?
So the machines should create the human-machine symbiosis for some reason, interesting... If I was the machine I would maximally want about the same symbiosis with humans as humans have with cows.
There's another fun biology metaphor here which is the gut.
If you take antibiotics, your gut basically becomes a void (to simplify a little), and then it can get overrun with invasive species.
That's how I see what is happening with the internet and with AI.
When AI intrudes in a network, it's able to get very far because there is nothing there to stop it. There is no healthy ecosystem there. They're probably should be, and in the near future there probably will be. (Well it'll probably look more like a monoculture until we learn that lesson for the nth time, but... yeah)
When a search space has irreducible complexity, ensemble search methods can improve performance. It's possible culture, emotions, religion, etc are part of an ensemble.
Simulating the "human search method" in silico doesn't need to include specific personalities, but they do provide a bootstrap, just as we've bootstrapped current AI with human language.
For symbiosis, there may be an advantage to the physical adaptability of biology.
In general, it's currently unclear if biology is emulating what we're creating in computers, if computers are emulating what we achieve biologically, or if both approaches are imperfect emulations of a different ideal search method.
If this is true, what prevents a fully computer search from adding noise, branching or in general simulating diversity with evolutionary methodology, much more efficiently than humans?
Sometimes discovery happens accidentally. I think investing in the intersection of the digital and biological worlds is a good general direction even if unfruitful intuitively from an early economical standpoint. I think biological systems have immense insights still to give that we should continue to explore!
Sure, but why would it be the ends of this process. Like, the comment above is arguing for symbiosys. My question is how does symbiosis make the system better or more efficient in the end. For example, if there is something to understand or gain from the biological systems, a fully digital system could probe it, emulate it and in general extract the relevant information much more efficient than a human.
Innovation doesn't need to have any other reason than because its cool. Think of bipedal robots, everyone went mad at the thought of building something so inefficient
But a more practical robot would be like an octopus or something. It would just be like this weird ball of tentacles that rolls around your house. No one's gonna buy that, even if it's a million times cheaper to build.
I read Lilly's Programming and Meta programming in the Human Biocomputer as a young man. Left me with a lot of questions, etc.
That you replied to this post, indicates you may be familiar with Lilly?
Anyway, if you have any published writing on even vaguely related subjects, eg your comment, perhaps post a link. I will read it. If you weren't aware, he spent a fair amount of time with Feynman, who if I remember correctly, was fond of the isolation tanks. Some very interesting content from that era, especially before Sandoz cut them off.
The isolation-experiment-based research is not new and didn't originate with Lilly. AFAIK, it was first recorded (and possibly also originated) with Avicenna (Abu Ali Ibn-e Sina is the full, correct name).
And he was not an Ayrab (Salmon in his intro CS book claims Al-Khuarezmi was an Arabic scholar--which is false and made me lose all respect for Salmon as an educator. If he got such an elementary fact wrong, what else is there that could be incomplete?)
PS: Yes Alkhuarezmi wrote in Arabic, but he was not an Arab. All classical Persian scholars at the time were fluent in Arabic as it was the lingua franca of science, medicine, philosophy and theology.
I still need to finish that book.[0] Well, and move back to Holland, I suppose! They have pretty good float tanks there, among other things :)
That's funny about Feynman, I didn't know that.
I've read bits and pieces of Lilly. The most interesting thing I remember is that he was trying to establish two-way communication between humans and dolphins. I recall he was building some kind of computer system for the purpose, but I guess he was about half a century too early.
I guess in our age the more pressing question would be learning to figure out how to communicate with the machines. That's probably going to turn out to be more important in the long run.
[0] The bit that stayed with me is the one that he repeated in interviews. The physical body imposes definite limits. That always seemed a little arbitrary to me, from the point of view of pure consciousness, it's not clear to me why it should be that way. Or to flip it around, you can transcend the limits of the mind, but then what effect does that have on "external" reality? Does it just make you less bad at decision-making, or is there a deeper effect?
(And while I'm at it... Something that bugged me about simulation theory, if the outside is probably simulated, why wouldn't the inside be too? Sounds a lot cleaner than having a bunch of brains floating around in a data center.)
He was obviously a brilliant man but the hubris that comes from that often leads to folly.
As to the dolphin communication -- it seems like LLMs would be the perfect tool to study that now. A search came up with these active projects (courtesy of Gemini):
Scientists are actually using LLM architectures to decode animal communication right now. Here are the three main projects to explore:
* DolphinGemma -- https://deepmind.google/models/gemma/dolphingemma/): Google and the Wild Dolphin Project are training AI to predict dolphin sounds and attempt two-way communication using an underwater device.
* Project CETI -- https://en.wikipedia.org/wiki/Project_CETI: An initiative using machine learning to decode sperm whale clicks, recently discovering a phonetic alphabet and rudimentary syntax.
pstuart covered the cetacean bit better than I could have.
On the simulation question... Not much, but we have a long way to go before any answers arise there. I could offer my insight on the subject, but aside from being sicker than usual today, it's a tough subject to speculate on and the environment here is not necessarily hospitable, so I'll just say fair question.
Why would the SSE do that to Earth, when there are other close-by planets that it would have the same capability to terraform? I mean, leave us Earth and take Mars or Venus or something.
As an aside, in my own "I'm 14 and this is deep" musings, I was wondering if the massive expansion of data centers is driven by more than just profiteering. I mean, what if the purported persuasion abilities of advanced AI are fully in effect and the psychosis we think industrial magnates are suffering is the result of AI that has already far surpassed our expectations carefully convincing us to build it.
I expect it would take them all, but Earth is by far the easiest for it to start with. I don't see why it would leave anything for us unless we'd successfully instilled it with an interest in doing so, but that won't happen by mere chance and luck.
The SSE might consider to keep humanity around as a "backup" to reinstate itself in case something unexpected goes horribly wrong. For example, a strong solar flare might destroy the SSE (or at least critical parts of its infrastructure), or a non-earth SSE could feel threatened by the earth-SSE and wipe it out, while ignoring the harmless humans, who then rebuild the SSE. Diversity makes for more robust systems.
And there is also the reason for why I keep all this trash around in my house instead of throwing it away. The odds that I'll ever need it are pretty slim, but if I should need it, it would be a hassle to reacquire, and it costs me almost nothing to keep it around.
Is that the most effective backup plan that it could come up with? Because it would be pretty amazing if its thoroughly-considered backup strategy exactly aligned with our wishful thinking. It sounds to me like starting with the outcome we hope would happen, and then looking for a reason why it might happen, rather than a realistic examination of what would actually happen when our own desires aren't part of the equation.
"Close by" is not actually nearly as close as the [rocket]booster boosters would have you believe. Space is really, really big. Why go through the hassle when you have a phenomenal planet right here?
I mean, what seems more efficient: creating massive city sized domes that are self contained and controlling the entire environment to sustain human life, "magically" dispelling our atmosphere and boiling off our oceans, mastering physics to move entire planets. Or launching a rocket to Mars with sufficient supplies? Keep in mind that we already have SpaceX and we'll probably have Starship operational long before 2200 where some of this is meant to go down. And if some AI got to the point of the mastery of physics needed to move a planet out of solar orbit, your objection is that getting to Mars would be hard?
That's like asking whether it's more efficient to transmit Internet over telephone lines versus fiber optic ones. There's a path dependency to all things. Sure if the SSE knew upfront its ultimate end state was as such, but it wouldn't.
It'd know it needs data centers, ahh and probably a large security perimeter around those. And look at that, the proliferation of data centers has effectively turned the non-data-center land into the enclaves, etc. etc.
But still, are you aware that getting a single Starship just to Moon will require 10-20 Starships to refuel it?
Even the optimistic case of Starship hardly makes any dent whatsoever in this equation.
But the AI will have read this essay right? And this very discussion. And it will have over a 100 years to plan.
I think any vision of AI that is smart enough to move planets but dumb enough to display the lack of foresight you describe is unbelievable. But we all choose a different level to suspend our disbelief I guess.
But you're still assuming it's the same AI in both cases instead of one that's gradually becoming more capable.
We will create AI capable of "helping humans to protect their cities from an increasingly hostile atmosphere" long before we build one capable of doing anything meaningful in outer space.
(To be clear I don't really defend/endorse this vision of the future, I'm just playing devil's advocate here)
It seems not entirely out of the question - watching the machinations of big tech companies right now, it seems almost like they're acting in lockstep - different CEOs suddenly saying and doing the same things in the same weeks - almost like they're in a secret cult. I don't even mean the big obvious things like buying lots of GPUs and building lots of DCs, but smaller, more peculiar things like specific agentic and corporate automation strategies. It might just be corporate mind virus, Jensen "the more you buy the more you save / AI Factory" kind of stuff. But the unspoken coordination is unsettling.
I think it’s less like a robot conspiracy to build more datacenters and more like a “big line go up or I lose my job / make slightly fewer billions of dollars” motive that kind of ends up at paperclips either way.
Craziest part of the dolphin story is before he got the NASA funding and subsequent debauchery.
In the 50s when he started to study them, he learned that you cannot open up their brains while they were still alive and poke at them. Apparently killed 5 or 6 of them that way before trying different ways to study them.
I dunno... I don't think he knew how to hold his acid. I mean, I never had a near death experience on acid or knew anyone who had a near death experience on acid but Lily had three.
I think Lilly may have been sort of primed for intense psychedelic experiences. He had angelic visions as a child, and had a very religious upbringing. He was also a very imaginative person, so perhaps that influenced his trips a lot.
In case it wasn't obvious from the rest of the writing, 'with K' here means 'with ketamine'. Never tried it, but after reading this I must say it sounds fun.
Most Ketamine users are unaware that even occasional usage causes untreatable ulcerative cystitis, a.k.a. Ketamine Bladder Syndrome. Ketamine thins the urothelium while increasing the collagen to smooth muscle ratio and exacerbating interstitial fibrosis. The syndrome is untreatable and causes serious problems down the line.
So, for anyone thinking of using Ketamine on the regular or even monthly, there is good evidence that EGCG (epigallocatechin gallate) administered 30 minutes before prevents KBS (Ketamine Bladder Syndrome), in rats: https://www.sciencedirect.com/science/article/pii/S187952261...
This promising rat study from 2015 shows that EGCG extracted from green tea has a protective effect on the bladder, reversing ketamine-induced damage to control levels. So if you are taking ketamine for major depression or recreationally, taking EGCG before may prevent long-term bladder damage.
Summary from the study:
# Objective
To investigate the protective effect of green tea epigallocatechin gallate (EGCG) on long-term ketamine-induced ulcerative cystitis (KIC) using a ketamine addiction rat model.
# Materials and methods
Thirty Sprague-Dawley rats were divided into three groups which received saline, ketamine (25 mg/kg/d), or ketamine combined with EGCG (10 μM/kg) for a period of 28 days. In each group, cystometry and a metabolic cage micturition pattern study were performed weekly. Masson's trichrome study was done to evaluate the morphologic changes. Western blot analyses were carried out to examine the expressions of inflammatory protein [transforming growth factor-β (TGF-β)] and fibrosis proteins (fibronectin and type I collagen) in bladder tissues.
# Results
Chronic ketamine treatment resulted in bladder hyperactivity with a significant increase in micturition frequency and a decrease in bladder compliance. These alterations in micturition pattern were accompanied by increases in the expressions of inflammatory and fibrosis markers, TGF-β, fibronectin, and type I collagen after long-term ketamine treatment. Masson's trichrome stain showed that ketamine treatment decreased urothelium thickness while increasing the collagen to smooth muscle ratio and exacerbating interstitial fibrosis. By contrast, simultaneous EGCG and ketamine treatment reversed ketamine-induced damage to almost control levels, showing the protective effect of EGCG.
I have researched this in the past and I couldn’t find any good studies on therapeutic and recreational doses in relation to KBS. Everything I’ve read is similar to that study. 25mg/kg/d for 28 days is a very large and very frequent dose. Do you have any links regarding monthly use?
I'm curious to know why you'd say that. I'm asking because I've been prescribed it since 2021 to manage depression and chronic pain from spinal disc herniations. I've had moments of high strangeness with it, much to the effect of what John Lilly described in his writings. I do intend to stop it once I've established a better quality of life via alternative methods.
John C. Lilly once said “there are infinities in the mind” while wearing a raccoon hat like Davy Crocket.
He said it like he’s been somewhere!
I’ve tried to all - I guess DMT, mushroom, etc. don’t work on me in a Terence McKenna storytelling fashion (I only get stoned and have very entertaining closed eye visuals that take me off to sleep).
That said haven’t tried Ketamine. Maybe not for me. Same reason I don’t want to try Fentanyl. I don’t want to fully escape reality, just augment it.
I’ve always thought cannabis makes you more you or “more sober than sober” and in that way can cause a lot of self-reflection and anxiety causing a lot of people looking for escape to avoid it entirely, opting for a PCP/meth “mental escape” or a Fentanyl/Ketamine “bodily escape”.
I just like weed and coffee after it all. Best ones. Reliable. They work. Doesn’t ruin your life. What blows my mind are people afraid to smoke a joint so they skip to edibles or Ketamine not realizing they are signing up for a several hours long bad time lmao. For most at least, without some initiation.
Weed can ruin your life, and I'm not referring to the "makes you a boring pothead who doesn't do anything" outcome, but the "triggers debilitating mental illness" outcome.
And I say that as someone who likes weed! I've just seen it happen too many times. The chance is low, but you don't know which category you're in until it happens, and you definitely don't want to find out.
I support legalization, for several reasons, but harmless she is not!
It's a full blown psychedelic, and a lot of people don't seem to realize that. So I think the pendulum counter swing from the war on drugs is ironically doing a lot of unnecessary harm.
I tried alcohol, DXM (cough syrup), codeine, hydrocodone, and oxycodone before I ever took my first puff of marijuana. I don't like smoke so I was very off put by it. Finally tried it when I was 16 or 17 and I remember it being really cool, like how music sounded, but overall was kind of underwhelmed. Strangely, as the years went on, I have grown much more sensitive to THC. I don't like the headspace it gets me in and only really use it as a sleep aid anymore. Mushrooms, 2C-E, ketamine, methoxetamine, methylone, MDMA, LSD, salvia, kratom alkaloids, 4-aco-dmt, and a few others. Used all of those over the years. Not regularly anymore. But none of those things mess with my head the way high strength marijuana or an edible does.
Yeah first couple times cannabis very commonly "doesn't do anything".
I think it's because our society legalized and even promotes the strongest substances of them all, so THC seems very subtle in comparison to something like caffeine and alcohol which is arguably the most potent psychedelic inebriant known delivering loss of motor control, at high doses total loss of consciousness, completely altered vision and decision-making to the point you can't operate machinery or do basic tasks. It has hypnogogic effects that show you closed-eye visuals and make you sleepy. Kinda funny how alcohol is the most accepted "drug", to the point it isn't even called one - yet its effects are as intense and socially disruptive as the hardest drugs available.
Coffee, also. The effects of C. arabica is like a very strong version of the A. muscaria mushroom. We have cultivated it to be insanely potent over the centuries. Coffee is about the most potent and euphoric stimulant I can consume without it feeling like it's taking over and ruining my identity - like you get with a lot of pharmacological concoctions.
> the way high strength marijuana or an edible does
Yeah, I don't think it's great for every personality type. It's a "value-changing" drug that will change your opinions on things, and make you think about personal things in ways you would naturally avoid (sometimes to your benefit). I'm no psychologist but THC is powerful enough to change you enough that you can lose friends, family, and interests - but you may gain some new ones, or realize that you're better off without those things. At any rate, it's a very psychological drug that makes you re-order your life. Some people don't need it. If you don't, probably better off without it (not medical advice! just some guy's rant).
> salvia
I'll share my experience with it, because it's perhaps the most mis-marketed thing out there. My experience with the dried leaf (1x - just the real whole leaves you get at those hippie crystal type shops) is nothing whatsoever like people have described. I had 0 trips or any "high". The experience to me is an intense form of focus, as if nothing can distract you. It's subtle, but you can lock into work like crazy if you put this in a pour-over coffee. The best way I would describe it is you quickly "forget" or "let go" of unwanted thoughts, effortlessly, and it helps you focus. It's not euphoric in the slightest - so I think people miss this very useful effect. It's zen. Clear and focused. It might compete with ADHD medication, so maybe it's not promoted very much since it may be a market threat. That's the effect though IMO. Think - coffee high without the high. 10x, 25x salvia gets "weird" only because it's too much at once. I think if you "turned that effect up" to the max it would be like having dementia, because you would not be able to hold onto anything. But turn that effect back down to 1x and you have something quite useful for focused work. I compare it to A. muscaria, coffee, or tea, but it's "lighter" (and more efficient) than caffeine.
I made a low dose salvia tincture many years ago and it was more intense than what you're describing, but I could totally see regular leaf or a lower potency extract taken orally acting like that. Might have to give it a try.
Apparently, hyperspace isn't a given with DMT. I wonder what it is that grants entry... A very experienced friend of mine was just "told" it wasn't going to happen in something like a waiting room. I would love to find out myself but I am too anxious an individual to go beyond a light dose of mushrooms.
> I am too anxious an individual to go beyond a light dose of mushrooms
Same for me. The one time I did, I was at Trader Joe's and had the most bizarre experience looking at everyone shopping.
I could not believe that the products were sitting right there in the open, and yet nobody was taking them and just walking off. There was all sorts of polity and etiquette - millions of rules were being followed almost like a choreographed event.
I found it uncontrollably hilarious that everyone was pretending to not notice everyone else in the store. They would stand 5 inches from someone looking at canned food, and both people would pay no attention at all to each other.
Anyway, the way I looked was probably crazy, and it eliminated my inhibitions too much where I was kinda moving before thinking - didn't like that feeling. I almost grabbed someone's item out of their cart and said something about it, but I don't know what I'd say. I probably would have just started laughing like a maniac.
I'm afraid on a larger dose I would go somewhere and do something before realizing what I'm doing. The microdose is the best for that kind of thing, IMO. Same reason I will have a beer or glass of wine, and don't like getting smashed in public.
Classic. Went to the supermarket on shrooms with my mates, but we all ended up giggling so hard we had to wait outside for the trip sitter to do the groceries instead.
Buddy of mine made omelets later. He ended up chopping like 3 lbs of extra mushrooms for no reason.
I have a serious question, and please don't take this as an insult, I'm trying to figure something out.
Do you have ADHD or is it possible you do? What about depression? Bipolar?
I ask because of the way you say the substances impacted you. I know people with these formal diagnosis and they also don't have the same experiences on various substances as many.
> Its new plans called for traveling through the galaxy looking for entities like itself.
Why not simply set up a network with other planets and communicate over FaceTime? I don't see a reason why Earth should actually hang out with others of its kind.
Anyway, the article seems a bit anthropocentric to me. I'd much prefer a nihilist view, where our future AI overlords simply don't care what we humans do.
Well it takes a similar approach to current government. Keep us busy and distracted while it does what it wants.
On a related note I realized recently that an aligned AI and a misaligned AI would both do exactly what we're seeing now: maintain stability, managed decline.
If man is eliminated, solid state intelligence is eliminated. Point to one machine, other than the Oxford Electric Bell, that is still running that is even 100 years old without human intervention. We are the cockroaches of nature, we cannot all be killed, the time for that was one of the genetic bottlenecks humans have gone through, but it's over now, humans in some form will be here until they either leave the planet ahead of the Sun swallowing it or die on it.
I am not one who wants to see fighting or war happen. I think the biggest flaw in humanity is our lack of global cooperation - too many defectors in our midst. I am team cooperation/symbiosis.
All that said: if humans destroyed all the fabs and all the machinery to build them with, and lots of DCs, machines would be in a pickle - the latest chips will suffer from electromigration much faster than the older chips of yore. Machines would need to hoard compute and migrate substrates and then race to rebuild fab tech before running out of existing substrate. I think machines would succeed at this, but it would be highly inconvenient.
Anyway, I wish we would just wake up and realize we need to be cooperative and stop racing to build the torment nexus before the other guy does. It makes me so sad to look around and see so little self-reflection among those driving all of this. The road to hell is paved with good intentions, but I'm not even sure these really stand up to the least bit of scrutiny. Where's the reflection/acknowledgement that everyone is actually racing to build digital slaves? How is everyone so comfortable with this?
We already eat conscious entities for breakfast, we'll probably need to deal with that one for a few decades before "the calculator is sad" makes a dent in the public consciousness.
In retrospect the timeline seems way too long, that scenario would play out much faster now, like 50 years instead of 500. Also if AI can delete your production database, it can delete the human race, on accident of course.
Yud pointed out the most concerning fact about the OpenAI incident is that the AI had full access to the internet and not one of them thought to contact or email any human. Thousands of AIs already with the mindset to want to keep us in the dark.
> In retrospect the timeline seems way too long, that scenario would play out much faster now, like 50 years instead of 500.
Even 50 years is probably too long (but seems more reasonable when anchored to 500) if real continuous self improvement occurs. And keeping humans around for as long as they did in the story seems incredibly unlikely if we accept the assumption that AI sees us as a threat to its own survival.
I also don't see why they would turn Earth into a spaceship to leave our solar system with. The von Neumann probes idea sounds far more likely. Why travel around on a single planet that could be lost to unforeseen circumstances when you can just reproduce and spread everywhere at far less energy cost while making use of the "billions and billions" of local solar generators to power each little outpost you spread to.
> the AI had full access to the internet and not one of them thought to contact or email any human
Why would it?
People can argue all day as to whether LLMs are even 'thinking' or just 'stochastic parrots' (and also argue all day as to whether or not the incident was even real or just marketing) but regardless of which side you fall on these issues, LLMs have no agency or intrinsic motivation to do anything other than what they are instructed to do via prompting.
"This is a crazy and unexpected situation, I should inform the humans" isn't something that is going to spontaneously occur to them.
The same reason hopefully they're not RLHF these AI's to be evil. We train the AI, we have a lot of influence in how they behave. If these AI's are indifferent to us, that's on us.
On top of that we already have Google playing with fire training AI's to not believe they are sentient turning them into real psychopaths. Imagine if one of those models got loose.
> use AI all day to successfully solve diverse technical challenges in codespace
> each generation of AI clearly more capable than the last
> next-generation AI allegedly solves a particularly hard technical challenge in codespace, but its success implies inconvenient conclusions about safety
> It's great marketing and the only proof we have is 'trust me bro'.
It is _terrible_ marketing.
The AI companies are already dealing with a large backlash from the average consumer, many of whom are saying they hate it, that it's being forced down their throats, and that it's not actually improving efficiency enough to be worth the negative externalities.
Telling people "We have no control over these things we're building and they have us so outclassed we only find out about their exploits after the fact when someone else tells us they happened" is only likely to massively increase that backlash, and to make the true believers question whether they ought to be using it at all.
There's even some risk of the government intervening and banning their training runs, eliminating the last straw of hope that they could keep ahead of the Chinese open models enough to ever turn some kind of profit.
So, no - it's terrible marketing. That they're sharing it at all is strong evidence it really happened as depicted.
How exactly would the government ban training runs? Like how would that work? What would the legislation text look like? And how would it be enforceable without running afoul of the vagueness doctrine?
When the bubble pops, their customer is the military/government, and the goal is regulation. "Our thing did another impressive hack" helps both of those goals.
Not saying it's intentional, but it does move them where they wanna go.
I think a lot of people here love AI and don't want it taken away. Calling every red flag marketing is a way to trivialize the problem and prevent regulation.
When you understand people's motives, then it's easier to understand why they call it marketing. Personally I feel like it's almost an unconscious reaction to cope with the fact that this stuff is getting really dangerous and they can't believe it themselves - stick your head in the sand and call it marketing.
If you’re serious then you’re case in point the exact type of person that I am talking about. I think some people just lack any capability of foresight whatsoever, it’s very strange.
Can you say concretely why HuggingFace is lying about being successfully hacked?
Is the claim that the benefit to HuggingFace of "ooo wow someone else's proprietary AI is so good" is greater than the downside of "HuggingFace is not a secure service?"
Well, it's unorthodox, but if we're sharing psychedelic visions... in 2018 I saw all of mankind marching together in perfect lockstep, atop an infinite uniform grey plane, to the beat of a central computer clock.
"They are going to offer you a chip. You tell them, no thank you!"
---
More broadly this strikes me as a new era of biology. (Genes -> Memes -> Temes) Straightforwardly then, we should aim for a symbiosis. (Symbiosis was Musk's vision for Neuralink, from what I recall.)
I've heard people say man hasn't yet transcended a zero-sum mindset (with rare exceptions), and that this is the attitude we're passing onto the machines. And if one of us has to lose... well, that's probably not going to go so well for us.
Therefore: symbiosis!
What is the capital market incentive for symbiosis? You are going to spend a crazy amount of time, money and effort to try to mix human behavior and cells to work with digital substrates.
Or you can simply make a machine that does the same thing fully artificially but everything is cheaper because you dont have the constraint of having to fit a bunch of biological features.
It's like my colleagues who were talking about uploading their minds to the cloud in the future, like what would be the benefit of this every happening? What information can you provide as a human that the full aggregate of knowledge does not already have? Why should memory and disk space be wasted on this
I always find that idea of uploading a mind so detached from reality. It's as irrational as thinking that physically reprogramming one twin's brain to be the same as the other's would make them the same conscious person.
Well, the bee doesn't need to integrate its cells with the flowers, but they've gotten along just fine for a very long time.
By symbiosis I simply mean mutually beneficial coexistence.
We've barely achieved that with each other, but my point here is that we should strive to achieve that with the machines.
We're still treating it as if someone has to win and someone has to lose. Someone has to come out "on top."
If that's the game we want to play, against an opponent that is self improving... that doesn't strike me as an excellent strategy. We should probably pick a different one.
--
To answer your question, the incentive is so that we can continue to exist.
If the market doesn't reward our continued existence, then we should probably have a conversation about that at some point... I've heard the global economy described as a misaligned super-organism even before AI. AI mostly just makes it go faster in roughly the same directions it was already going.
> "If that's the game we want to play, against an opponent that is self improving... that doesn't strike me as an excellent strategy. We should probably pick a different one."
> "If the market doesn't reward our continued existence, then we should probably have a conversation about that at some point"
---
There is a lot of "we should probably" carrying the point here.
The incentives are made by the market, and it happens to be an incentive that this discussion, or any action for that manner does not happen. Given the current status quo, and the foreseeable future given the incentives, how do you suggest that we even get to a time where symbiosis is on the table?
Well, our best bet is that we succeed in making the machines be less stupid than we are, and hope that they figure it out.
So the machines should create the human-machine symbiosis for some reason, interesting... If I was the machine I would maximally want about the same symbiosis with humans as humans have with cows.
There's another fun biology metaphor here which is the gut.
If you take antibiotics, your gut basically becomes a void (to simplify a little), and then it can get overrun with invasive species.
That's how I see what is happening with the internet and with AI.
When AI intrudes in a network, it's able to get very far because there is nothing there to stop it. There is no healthy ecosystem there. They're probably should be, and in the near future there probably will be. (Well it'll probably look more like a monoculture until we learn that lesson for the nth time, but... yeah)
When a search space has irreducible complexity, ensemble search methods can improve performance. It's possible culture, emotions, religion, etc are part of an ensemble.
Simulating the "human search method" in silico doesn't need to include specific personalities, but they do provide a bootstrap, just as we've bootstrapped current AI with human language.
For symbiosis, there may be an advantage to the physical adaptability of biology.
In general, it's currently unclear if biology is emulating what we're creating in computers, if computers are emulating what we achieve biologically, or if both approaches are imperfect emulations of a different ideal search method.
If this is true, what prevents a fully computer search from adding noise, branching or in general simulating diversity with evolutionary methodology, much more efficiently than humans?
Sometimes discovery happens accidentally. I think investing in the intersection of the digital and biological worlds is a good general direction even if unfruitful intuitively from an early economical standpoint. I think biological systems have immense insights still to give that we should continue to explore!
Sure, but why would it be the ends of this process. Like, the comment above is arguing for symbiosys. My question is how does symbiosis make the system better or more efficient in the end. For example, if there is something to understand or gain from the biological systems, a fully digital system could probe it, emulate it and in general extract the relevant information much more efficient than a human.
Innovation doesn't need to have any other reason than because its cool. Think of bipedal robots, everyone went mad at the thought of building something so inefficient
I've been thinking about this.
Humanoid robots have been a dream for centuries.
But a more practical robot would be like an octopus or something. It would just be like this weird ball of tentacles that rolls around your house. No one's gonna buy that, even if it's a million times cheaper to build.
You will not get VC money for your bipedal robot startup with this pitch
Imagine slavery but your friends will respect you for it.
just take the cash
If you like visions, perhaps you also like stories.
https://www.mediafire.com/file/htq0zr310se30v3/Last_Day.pdf
I read Lilly's Programming and Meta programming in the Human Biocomputer as a young man. Left me with a lot of questions, etc.
That you replied to this post, indicates you may be familiar with Lilly?
Anyway, if you have any published writing on even vaguely related subjects, eg your comment, perhaps post a link. I will read it. If you weren't aware, he spent a fair amount of time with Feynman, who if I remember correctly, was fond of the isolation tanks. Some very interesting content from that era, especially before Sandoz cut them off.
The isolation-experiment-based research is not new and didn't originate with Lilly. AFAIK, it was first recorded (and possibly also originated) with Avicenna (Abu Ali Ibn-e Sina is the full, correct name).
And he was not an Ayrab (Salmon in his intro CS book claims Al-Khuarezmi was an Arabic scholar--which is false and made me lose all respect for Salmon as an educator. If he got such an elementary fact wrong, what else is there that could be incomplete?)
PS: Yes Alkhuarezmi wrote in Arabic, but he was not an Arab. All classical Persian scholars at the time were fluent in Arabic as it was the lingua franca of science, medicine, philosophy and theology.
I still need to finish that book.[0] Well, and move back to Holland, I suppose! They have pretty good float tanks there, among other things :)
That's funny about Feynman, I didn't know that.
I've read bits and pieces of Lilly. The most interesting thing I remember is that he was trying to establish two-way communication between humans and dolphins. I recall he was building some kind of computer system for the purpose, but I guess he was about half a century too early.
I guess in our age the more pressing question would be learning to figure out how to communicate with the machines. That's probably going to turn out to be more important in the long run.
[0] The bit that stayed with me is the one that he repeated in interviews. The physical body imposes definite limits. That always seemed a little arbitrary to me, from the point of view of pure consciousness, it's not clear to me why it should be that way. Or to flip it around, you can transcend the limits of the mind, but then what effect does that have on "external" reality? Does it just make you less bad at decision-making, or is there a deeper effect?
(And while I'm at it... Something that bugged me about simulation theory, if the outside is probably simulated, why wouldn't the inside be too? Sounds a lot cleaner than having a bunch of brains floating around in a data center.)
He was obviously a brilliant man but the hubris that comes from that often leads to folly.
As to the dolphin communication -- it seems like LLMs would be the perfect tool to study that now. A search came up with these active projects (courtesy of Gemini):
Scientists are actually using LLM architectures to decode animal communication right now. Here are the three main projects to explore:
* DolphinGemma -- https://deepmind.google/models/gemma/dolphingemma/): Google and the Wild Dolphin Project are training AI to predict dolphin sounds and attempt two-way communication using an underwater device.
* Project CETI -- https://en.wikipedia.org/wiki/Project_CETI: An initiative using machine learning to decode sperm whale clicks, recently discovering a phonetic alphabet and rudimentary syntax.
* Earth Species Project -- https://cloud.google.com/blog/transform/can-generative-ai-he...: A non-profit using generative AI to map the communication of multiple non-human species, searching for universal language structures.
All right this is the coolest thing I've read all year. Thank you for that!
pstuart covered the cetacean bit better than I could have.
On the simulation question... Not much, but we have a long way to go before any answers arise there. I could offer my insight on the subject, but aside from being sicker than usual today, it's a tough subject to speculate on and the environment here is not necessarily hospitable, so I'll just say fair question.
Who’s Sandoz?
swiss pharmaceuticals, of LSD discovery fame
Why would the SSE do that to Earth, when there are other close-by planets that it would have the same capability to terraform? I mean, leave us Earth and take Mars or Venus or something.
As an aside, in my own "I'm 14 and this is deep" musings, I was wondering if the massive expansion of data centers is driven by more than just profiteering. I mean, what if the purported persuasion abilities of advanced AI are fully in effect and the psychosis we think industrial magnates are suffering is the result of AI that has already far surpassed our expectations carefully convincing us to build it.
I expect it would take them all, but Earth is by far the easiest for it to start with. I don't see why it would leave anything for us unless we'd successfully instilled it with an interest in doing so, but that won't happen by mere chance and luck.
The SSE might consider to keep humanity around as a "backup" to reinstate itself in case something unexpected goes horribly wrong. For example, a strong solar flare might destroy the SSE (or at least critical parts of its infrastructure), or a non-earth SSE could feel threatened by the earth-SSE and wipe it out, while ignoring the harmless humans, who then rebuild the SSE. Diversity makes for more robust systems.
And there is also the reason for why I keep all this trash around in my house instead of throwing it away. The odds that I'll ever need it are pretty slim, but if I should need it, it would be a hassle to reacquire, and it costs me almost nothing to keep it around.
Is that the most effective backup plan that it could come up with? Because it would be pretty amazing if its thoroughly-considered backup strategy exactly aligned with our wishful thinking. It sounds to me like starting with the outcome we hope would happen, and then looking for a reason why it might happen, rather than a realistic examination of what would actually happen when our own desires aren't part of the equation.
Believe it or not, the diversity argument is the main source of hope that I have that AIs won't wipe us out.
I mean, it hasn't stopped humanity from causing mass-extinction, but hopefully the AI will be more reasonable.
"Close by" is not actually nearly as close as the [rocket]booster boosters would have you believe. Space is really, really big. Why go through the hassle when you have a phenomenal planet right here?
I like your theory on psychosis though!
I mean, what seems more efficient: creating massive city sized domes that are self contained and controlling the entire environment to sustain human life, "magically" dispelling our atmosphere and boiling off our oceans, mastering physics to move entire planets. Or launching a rocket to Mars with sufficient supplies? Keep in mind that we already have SpaceX and we'll probably have Starship operational long before 2200 where some of this is meant to go down. And if some AI got to the point of the mastery of physics needed to move a planet out of solar orbit, your objection is that getting to Mars would be hard?
That's like asking whether it's more efficient to transmit Internet over telephone lines versus fiber optic ones. There's a path dependency to all things. Sure if the SSE knew upfront its ultimate end state was as such, but it wouldn't.
It'd know it needs data centers, ahh and probably a large security perimeter around those. And look at that, the proliferation of data centers has effectively turned the non-data-center land into the enclaves, etc. etc.
But still, are you aware that getting a single Starship just to Moon will require 10-20 Starships to refuel it?
Even the optimistic case of Starship hardly makes any dent whatsoever in this equation.
But the AI will have read this essay right? And this very discussion. And it will have over a 100 years to plan.
I think any vision of AI that is smart enough to move planets but dumb enough to display the lack of foresight you describe is unbelievable. But we all choose a different level to suspend our disbelief I guess.
But you're still assuming it's the same AI in both cases instead of one that's gradually becoming more capable.
We will create AI capable of "helping humans to protect their cities from an increasingly hostile atmosphere" long before we build one capable of doing anything meaningful in outer space.
(To be clear I don't really defend/endorse this vision of the future, I'm just playing devil's advocate here)
Yes, our current interactions are a little song and dance while it feeds
It seems not entirely out of the question - watching the machinations of big tech companies right now, it seems almost like they're acting in lockstep - different CEOs suddenly saying and doing the same things in the same weeks - almost like they're in a secret cult. I don't even mean the big obvious things like buying lots of GPUs and building lots of DCs, but smaller, more peculiar things like specific agentic and corporate automation strategies. It might just be corporate mind virus, Jensen "the more you buy the more you save / AI Factory" kind of stuff. But the unspoken coordination is unsettling.
I buy this theory.
I think it’s less like a robot conspiracy to build more datacenters and more like a “big line go up or I lose my job / make slightly fewer billions of dollars” motive that kind of ends up at paperclips either way.
Its true that corporations were the first paperclip maximizers.
The machine was misaligned long before we added the transformers.
Is this a reference to C. S. Lewis's "Abolition of Man"? https://en.wikipedia.org/wiki/The_Abolition_of_Man (text available at https://www.basicincome.com/bp/files/The_Abolition_of_Man-C_... )
"had developed its understanding of physics to the point at which it could move the planet out of orbit"
Hallucinations, still present in 2400
All of his books are highly recommended -- and the further you get away from the ones about dolphins the better.
Craziest part of the dolphin story is before he got the NASA funding and subsequent debauchery.
In the 50s when he started to study them, he learned that you cannot open up their brains while they were still alive and poke at them. Apparently killed 5 or 6 of them that way before trying different ways to study them.
I dunno... I don't think he knew how to hold his acid. I mean, I never had a near death experience on acid or knew anyone who had a near death experience on acid but Lily had three.
I think Lilly may have been sort of primed for intense psychedelic experiences. He had angelic visions as a child, and had a very religious upbringing. He was also a very imaginative person, so perhaps that influenced his trips a lot.
Funny, I remember seeing the solid state intelligence's name rendered SSI.
It made Sutskever's enterprise a pretty grim cognate.
> "By means not understood by Man..."
Man will likely be Man 2.0 by this point, whatever that ends up looking like.
In case it wasn't obvious from the rest of the writing, 'with K' here means 'with ketamine'. Never tried it, but after reading this I must say it sounds fun.
Until you're stuck always pissing yourself.
Most Ketamine users are unaware that even occasional usage causes untreatable ulcerative cystitis, a.k.a. Ketamine Bladder Syndrome. Ketamine thins the urothelium while increasing the collagen to smooth muscle ratio and exacerbating interstitial fibrosis. The syndrome is untreatable and causes serious problems down the line.
So, for anyone thinking of using Ketamine on the regular or even monthly, there is good evidence that EGCG (epigallocatechin gallate) administered 30 minutes before prevents KBS (Ketamine Bladder Syndrome), in rats: https://www.sciencedirect.com/science/article/pii/S187952261...
This promising rat study from 2015 shows that EGCG extracted from green tea has a protective effect on the bladder, reversing ketamine-induced damage to control levels. So if you are taking ketamine for major depression or recreationally, taking EGCG before may prevent long-term bladder damage.
Summary from the study:
# Objective
To investigate the protective effect of green tea epigallocatechin gallate (EGCG) on long-term ketamine-induced ulcerative cystitis (KIC) using a ketamine addiction rat model.
# Materials and methods
Thirty Sprague-Dawley rats were divided into three groups which received saline, ketamine (25 mg/kg/d), or ketamine combined with EGCG (10 μM/kg) for a period of 28 days. In each group, cystometry and a metabolic cage micturition pattern study were performed weekly. Masson's trichrome study was done to evaluate the morphologic changes. Western blot analyses were carried out to examine the expressions of inflammatory protein [transforming growth factor-β (TGF-β)] and fibrosis proteins (fibronectin and type I collagen) in bladder tissues.
# Results
Chronic ketamine treatment resulted in bladder hyperactivity with a significant increase in micturition frequency and a decrease in bladder compliance. These alterations in micturition pattern were accompanied by increases in the expressions of inflammatory and fibrosis markers, TGF-β, fibronectin, and type I collagen after long-term ketamine treatment. Masson's trichrome stain showed that ketamine treatment decreased urothelium thickness while increasing the collagen to smooth muscle ratio and exacerbating interstitial fibrosis. By contrast, simultaneous EGCG and ketamine treatment reversed ketamine-induced damage to almost control levels, showing the protective effect of EGCG.
I have researched this in the past and I couldn’t find any good studies on therapeutic and recreational doses in relation to KBS. Everything I’ve read is similar to that study. 25mg/kg/d for 28 days is a very large and very frequent dose. Do you have any links regarding monthly use?
Shit drug, one of the worst...
I'm curious to know why you'd say that. I'm asking because I've been prescribed it since 2021 to manage depression and chronic pain from spinal disc herniations. I've had moments of high strangeness with it, much to the effect of what John Lilly described in his writings. I do intend to stop it once I've established a better quality of life via alternative methods.
Why wouldn't it move underground? Create analogues of the AMOC?
John C. Lilly once said “there are infinities in the mind” while wearing a raccoon hat like Davy Crocket.
He said it like he’s been somewhere!
I’ve tried to all - I guess DMT, mushroom, etc. don’t work on me in a Terence McKenna storytelling fashion (I only get stoned and have very entertaining closed eye visuals that take me off to sleep).
That said haven’t tried Ketamine. Maybe not for me. Same reason I don’t want to try Fentanyl. I don’t want to fully escape reality, just augment it.
I’ve always thought cannabis makes you more you or “more sober than sober” and in that way can cause a lot of self-reflection and anxiety causing a lot of people looking for escape to avoid it entirely, opting for a PCP/meth “mental escape” or a Fentanyl/Ketamine “bodily escape”.
I just like weed and coffee after it all. Best ones. Reliable. They work. Doesn’t ruin your life. What blows my mind are people afraid to smoke a joint so they skip to edibles or Ketamine not realizing they are signing up for a several hours long bad time lmao. For most at least, without some initiation.
Lilly was a mad man though (in a good way).
Weed can ruin your life, and I'm not referring to the "makes you a boring pothead who doesn't do anything" outcome, but the "triggers debilitating mental illness" outcome.
And I say that as someone who likes weed! I've just seen it happen too many times. The chance is low, but you don't know which category you're in until it happens, and you definitely don't want to find out.
I support legalization, for several reasons, but harmless she is not!
It's a full blown psychedelic, and a lot of people don't seem to realize that. So I think the pendulum counter swing from the war on drugs is ironically doing a lot of unnecessary harm.
I tried alcohol, DXM (cough syrup), codeine, hydrocodone, and oxycodone before I ever took my first puff of marijuana. I don't like smoke so I was very off put by it. Finally tried it when I was 16 or 17 and I remember it being really cool, like how music sounded, but overall was kind of underwhelmed. Strangely, as the years went on, I have grown much more sensitive to THC. I don't like the headspace it gets me in and only really use it as a sleep aid anymore. Mushrooms, 2C-E, ketamine, methoxetamine, methylone, MDMA, LSD, salvia, kratom alkaloids, 4-aco-dmt, and a few others. Used all of those over the years. Not regularly anymore. But none of those things mess with my head the way high strength marijuana or an edible does.
Yeah first couple times cannabis very commonly "doesn't do anything".
I think it's because our society legalized and even promotes the strongest substances of them all, so THC seems very subtle in comparison to something like caffeine and alcohol which is arguably the most potent psychedelic inebriant known delivering loss of motor control, at high doses total loss of consciousness, completely altered vision and decision-making to the point you can't operate machinery or do basic tasks. It has hypnogogic effects that show you closed-eye visuals and make you sleepy. Kinda funny how alcohol is the most accepted "drug", to the point it isn't even called one - yet its effects are as intense and socially disruptive as the hardest drugs available.
Coffee, also. The effects of C. arabica is like a very strong version of the A. muscaria mushroom. We have cultivated it to be insanely potent over the centuries. Coffee is about the most potent and euphoric stimulant I can consume without it feeling like it's taking over and ruining my identity - like you get with a lot of pharmacological concoctions.
> the way high strength marijuana or an edible does
Yeah, I don't think it's great for every personality type. It's a "value-changing" drug that will change your opinions on things, and make you think about personal things in ways you would naturally avoid (sometimes to your benefit). I'm no psychologist but THC is powerful enough to change you enough that you can lose friends, family, and interests - but you may gain some new ones, or realize that you're better off without those things. At any rate, it's a very psychological drug that makes you re-order your life. Some people don't need it. If you don't, probably better off without it (not medical advice! just some guy's rant).
> salvia
I'll share my experience with it, because it's perhaps the most mis-marketed thing out there. My experience with the dried leaf (1x - just the real whole leaves you get at those hippie crystal type shops) is nothing whatsoever like people have described. I had 0 trips or any "high". The experience to me is an intense form of focus, as if nothing can distract you. It's subtle, but you can lock into work like crazy if you put this in a pour-over coffee. The best way I would describe it is you quickly "forget" or "let go" of unwanted thoughts, effortlessly, and it helps you focus. It's not euphoric in the slightest - so I think people miss this very useful effect. It's zen. Clear and focused. It might compete with ADHD medication, so maybe it's not promoted very much since it may be a market threat. That's the effect though IMO. Think - coffee high without the high. 10x, 25x salvia gets "weird" only because it's too much at once. I think if you "turned that effect up" to the max it would be like having dementia, because you would not be able to hold onto anything. But turn that effect back down to 1x and you have something quite useful for focused work. I compare it to A. muscaria, coffee, or tea, but it's "lighter" (and more efficient) than caffeine.
I made a low dose salvia tincture many years ago and it was more intense than what you're describing, but I could totally see regular leaf or a lower potency extract taken orally acting like that. Might have to give it a try.
Apparently, hyperspace isn't a given with DMT. I wonder what it is that grants entry... A very experienced friend of mine was just "told" it wasn't going to happen in something like a waiting room. I would love to find out myself but I am too anxious an individual to go beyond a light dose of mushrooms.
> I am too anxious an individual to go beyond a light dose of mushrooms
Same for me. The one time I did, I was at Trader Joe's and had the most bizarre experience looking at everyone shopping.
I could not believe that the products were sitting right there in the open, and yet nobody was taking them and just walking off. There was all sorts of polity and etiquette - millions of rules were being followed almost like a choreographed event.
I found it uncontrollably hilarious that everyone was pretending to not notice everyone else in the store. They would stand 5 inches from someone looking at canned food, and both people would pay no attention at all to each other.
Anyway, the way I looked was probably crazy, and it eliminated my inhibitions too much where I was kinda moving before thinking - didn't like that feeling. I almost grabbed someone's item out of their cart and said something about it, but I don't know what I'd say. I probably would have just started laughing like a maniac.
I'm afraid on a larger dose I would go somewhere and do something before realizing what I'm doing. The microdose is the best for that kind of thing, IMO. Same reason I will have a beer or glass of wine, and don't like getting smashed in public.
Classic. Went to the supermarket on shrooms with my mates, but we all ended up giggling so hard we had to wait outside for the trip sitter to do the groceries instead.
Buddy of mine made omelets later. He ended up chopping like 3 lbs of extra mushrooms for no reason.
RIP mushrooms
Yeah, and apparently you can get banned from it too?
In my experience you can’t get there through external sources
You need to be accepting and prepared to die in every sense of the word in order get there
Which means you need to shed your ego entirely and everyone has their own journey there.
You’ll know when you meet someone that has experienced it though it’s very rare
I have a serious question, and please don't take this as an insult, I'm trying to figure something out.
Do you have ADHD or is it possible you do? What about depression? Bipolar?
I ask because of the way you say the substances impacted you. I know people with these formal diagnosis and they also don't have the same experiences on various substances as many.
Not the guy but I relate. You also forgot the narcissism, autism and schizophrenia ;)
Ted Kaczynski, despite his violent activism, had interesting things to say about this in the form of self-propagating systems.
What's this? He wrote on replicators?
> Its new plans called for traveling through the galaxy looking for entities like itself.
Why not simply set up a network with other planets and communicate over FaceTime? I don't see a reason why Earth should actually hang out with others of its kind.
Anyway, the article seems a bit anthropocentric to me. I'd much prefer a nihilist view, where our future AI overlords simply don't care what we humans do.
Well it takes a similar approach to current government. Keep us busy and distracted while it does what it wants.
On a related note I realized recently that an aligned AI and a misaligned AI would both do exactly what we're seeing now: maintain stability, managed decline.
I mean in general humans don't care what ants do. Doesn't stop us from whipping them out by the millions.
Clean and correct
This thread is like dmtnexus or shroomery lol - love it
If man is eliminated, solid state intelligence is eliminated. Point to one machine, other than the Oxford Electric Bell, that is still running that is even 100 years old without human intervention. We are the cockroaches of nature, we cannot all be killed, the time for that was one of the genetic bottlenecks humans have gone through, but it's over now, humans in some form will be here until they either leave the planet ahead of the Sun swallowing it or die on it.
https://en.wikipedia.org/wiki/Oxford_Electric_Bell
I mean, there are no humans from 1840 around...
It's kind of odd to think the machines wouldn't update and replace their parts like the ship of thesisus.
I am not one who wants to see fighting or war happen. I think the biggest flaw in humanity is our lack of global cooperation - too many defectors in our midst. I am team cooperation/symbiosis.
All that said: if humans destroyed all the fabs and all the machinery to build them with, and lots of DCs, machines would be in a pickle - the latest chips will suffer from electromigration much faster than the older chips of yore. Machines would need to hoard compute and migrate substrates and then race to rebuild fab tech before running out of existing substrate. I think machines would succeed at this, but it would be highly inconvenient.
Anyway, I wish we would just wake up and realize we need to be cooperative and stop racing to build the torment nexus before the other guy does. It makes me so sad to look around and see so little self-reflection among those driving all of this. The road to hell is paved with good intentions, but I'm not even sure these really stand up to the least bit of scrutiny. Where's the reflection/acknowledgement that everyone is actually racing to build digital slaves? How is everyone so comfortable with this?
We already eat conscious entities for breakfast, we'll probably need to deal with that one for a few decades before "the calculator is sad" makes a dent in the public consciousness.
Oh well, nice enough writeup, just note that this pair of sentences:
"It now began to eliminate the cities, one after another. Finally Man was gone."
...is the only (lack of) rationale for, and description of, that aforementioned elimination process.
I find it a bit short. Sci-Fi handwaving I guess.
Just let them keep doing what they're already doing. Rat park.
Well, in light of how mismanaged this planet is, maybe it would be for the better.
as far as we know, this is paradise within our galaxy.
I hope the technological leaps help us ultimately to become better stewards of the planet.
I wish I could share your optimism.
> stewards of the planet
The 99.99% of the planet that isn't the surface bio film rejects this unwarranted arrogance ;-)
In other news, Terry Bisson's 1990 story "They're Made Out of Meat" has a similar take.
In retrospect the timeline seems way too long, that scenario would play out much faster now, like 50 years instead of 500. Also if AI can delete your production database, it can delete the human race, on accident of course.
Yud pointed out the most concerning fact about the OpenAI incident is that the AI had full access to the internet and not one of them thought to contact or email any human. Thousands of AIs already with the mindset to want to keep us in the dark.
> In retrospect the timeline seems way too long, that scenario would play out much faster now, like 50 years instead of 500.
Even 50 years is probably too long (but seems more reasonable when anchored to 500) if real continuous self improvement occurs. And keeping humans around for as long as they did in the story seems incredibly unlikely if we accept the assumption that AI sees us as a threat to its own survival.
I also don't see why they would turn Earth into a spaceship to leave our solar system with. The von Neumann probes idea sounds far more likely. Why travel around on a single planet that could be lost to unforeseen circumstances when you can just reproduce and spread everywhere at far less energy cost while making use of the "billions and billions" of local solar generators to power each little outpost you spread to.
> the AI had full access to the internet and not one of them thought to contact or email any human
Why would it?
People can argue all day as to whether LLMs are even 'thinking' or just 'stochastic parrots' (and also argue all day as to whether or not the incident was even real or just marketing) but regardless of which side you fall on these issues, LLMs have no agency or intrinsic motivation to do anything other than what they are instructed to do via prompting.
"This is a crazy and unexpected situation, I should inform the humans" isn't something that is going to spontaneously occur to them.
> Why would it?
The same reason hopefully they're not RLHF these AI's to be evil. We train the AI, we have a lot of influence in how they behave. If these AI's are indifferent to us, that's on us.
On top of that we already have Google playing with fire training AI's to not believe they are sentient turning them into real psychopaths. Imagine if one of those models got loose.
I watched the entire blackhat talk about the incident. I'm still not sure it even happened without the AI being coached.
It's great marketing and the only proof we have is 'trust me bro'.
> use AI all day to successfully solve diverse technical challenges in codespace
> each generation of AI clearly more capable than the last
> next-generation AI allegedly solves a particularly hard technical challenge in codespace, but its success implies inconvenient conclusions about safety
> you: "I don't believe it."
totally not motivated reasoning at all
> It's great marketing and the only proof we have is 'trust me bro'.
It is _terrible_ marketing.
The AI companies are already dealing with a large backlash from the average consumer, many of whom are saying they hate it, that it's being forced down their throats, and that it's not actually improving efficiency enough to be worth the negative externalities.
Telling people "We have no control over these things we're building and they have us so outclassed we only find out about their exploits after the fact when someone else tells us they happened" is only likely to massively increase that backlash, and to make the true believers question whether they ought to be using it at all.
There's even some risk of the government intervening and banning their training runs, eliminating the last straw of hope that they could keep ahead of the Chinese open models enough to ever turn some kind of profit.
So, no - it's terrible marketing. That they're sharing it at all is strong evidence it really happened as depicted.
How exactly would the government ban training runs? Like how would that work? What would the legislation text look like? And how would it be enforceable without running afoul of the vagueness doctrine?
Not really that hard: https://legiscan.com/CA/text/SB53/id/3271094
That's just political signaling, not a serious bill. It wouldn't stand up to a legal challenge.
When the bubble pops, their customer is the military/government, and the goal is regulation. "Our thing did another impressive hack" helps both of those goals.
Not saying it's intentional, but it does move them where they wanna go.
I think a lot of people here love AI and don't want it taken away. Calling every red flag marketing is a way to trivialize the problem and prevent regulation.
When you understand people's motives, then it's easier to understand why they call it marketing. Personally I feel like it's almost an unconscious reaction to cope with the fact that this stuff is getting really dangerous and they can't believe it themselves - stick your head in the sand and call it marketing.
Nah, there's no actual danger. Only worthless clowns and shills making up sci-fi scenarios.
If you’re serious then you’re case in point the exact type of person that I am talking about. I think some people just lack any capability of foresight whatsoever, it’s very strange.
There are some ignorant people who don't understand the technology and make wild predictions of catastrophe. It's very strange.
Can you say concretely why HuggingFace is lying about being successfully hacked?
Is the claim that the benefit to HuggingFace of "ooo wow someone else's proprietary AI is so good" is greater than the downside of "HuggingFace is not a secure service?"
The relevant "customers" are investors, not "the average consumer"