Speculated to be responsible for the "AI restaurant-menu gore" that made the rounds on social media. Restaurants AI-generate the entire menu sheet from whole cloth, and while it looks "good" at first with modern image models, repeated edits to the menu text/layout transform the passable food "photos" into trypophobia-inducing horrors. (And somehow they don't notice or care)
Notably, the recent ChatGPT Images 2.5 announcement claimed to address this issue specifically ("multi-turn editing consistency")
Depends. With or without quality control? Not any more than in a society that rewrites all of its knowledge over and over using using humans. We even have a term for doing so without quality control: A game of telephone.
Though you might well argue that the risk of us dropping the quality control gets higher the cheaper it gets to churn things out with AI.
Definitely already happening with many c-suites.
We unleashed Nyarlatothep…
“I do not recall distinctly when it began, but it was months ago. The general tension was horrible. To a season of political and social upheaval was added a strange and brooding apprehension of hideous physical danger; a danger widespread and all-embracing, such a danger as may be imagined only in the most terrible phantasms of the night. I recall that the people went about with pale and worried faces, and whispered warnings and prophecies which no one dared consciously repeat or acknowledge to himself that he had heard. A sense of monstrous guilt was upon the land, and out of the abysses between the stars swept chill currents that made men shiver in dark and lonely places. There was a daemoniac alteration in the sequence of the seasons—the autumn heat lingered fearsomely, and everyone felt that the world and perhaps the universe had passed from the control of known gods or forces to that of gods or forces which were unknown.
And it was then that Nyarlathotep came out of Egypt”
It would have been interesting to try something closer to the movie poster example; say have the AI rewrite the passage with the room toggling between the initial light-mode decor and a dark-mode decor, then back, repeatedly. Or even change the woman's name to "Mrs. Smythe" and back.
As the article explains: “[…] In other words, if you instruct a model to change a single word in a paragraph of text, it can almost always handle the task with no collateral damage.“
> PS. For obvious reasons, this post will fail Pangram tests.
Actual pangram to me says it's 100% human. Current pangram can flag different passages differently, but these are all flagged human by it, even the LLM written ones.
This reminded me of a story/vignette which my fallible memory tags as by someone like Twain or Poe, about an automatic rewriting machine. It was put to work on the same passage with an ever higher intensity setting, becoming more and more gothic to the point of parody -- iirc the final output was verse including the line "to atoms dash the doubly damned".
No memory, but searching says it was Lewis Carroll in Photography Extraordinary, published in 1885:
The Milk-and-Water School
Alas! she would not hear my prayer!
Yet it were rash to tear my hair;
Disfigured, I should be less fair.
She was unwise, I may say blind;
Once she was lovingly inclined;
Some circumstance has changed her mind.
The Strong-Minded or Matter-of-Fact School
Well! so my offer was no go!
She might do worse, I told her so;
She was a fool to answer "No".
However, things are as they stood;
Nor would I have her if I could,
For there are plenty more as good.
The Spasmodic or German School
Firebrands and Daggers! hope hath fled!
To atoms dash the doubly dead!
My brain is fire--my heart is lead!
Her soul is flint, and what am I?
Scorch'd by her fierce, relentless eye,
Nothingness is my destiny!
The existence of attractors (e.g. "stable at iteration 31") for AI models is one of the weirdest things in AI land. They are a wonderful space to explore, fascinating and full of horrors.
Meanwhile, my hat's off to the (currently) last comment on the page: "I like how the shear-sheathed legs still earned their keep — probably load-bearing for Mrs. Regan"
Speculated to be responsible for the "AI restaurant-menu gore" that made the rounds on social media. Restaurants AI-generate the entire menu sheet from whole cloth, and while it looks "good" at first with modern image models, repeated edits to the menu text/layout transform the passable food "photos" into trypophobia-inducing horrors. (And somehow they don't notice or care)
Notably, the recent ChatGPT Images 2.5 announcement claimed to address this issue specifically ("multi-turn editing consistency")
Does the same effect happen in a society which rewrites all of its knowledge over and over using AI?
Depends. With or without quality control? Not any more than in a society that rewrites all of its knowledge over and over using using humans. We even have a term for doing so without quality control: A game of telephone.
Though you might well argue that the risk of us dropping the quality control gets higher the cheaper it gets to churn things out with AI.
Definitely already happening with many c-suites. We unleashed Nyarlatothep…
“I do not recall distinctly when it began, but it was months ago. The general tension was horrible. To a season of political and social upheaval was added a strange and brooding apprehension of hideous physical danger; a danger widespread and all-embracing, such a danger as may be imagined only in the most terrible phantasms of the night. I recall that the people went about with pale and worried faces, and whispered warnings and prophecies which no one dared consciously repeat or acknowledge to himself that he had heard. A sense of monstrous guilt was upon the land, and out of the abysses between the stars swept chill currents that made men shiver in dark and lonely places. There was a daemoniac alteration in the sequence of the seasons—the autumn heat lingered fearsomely, and everyone felt that the world and perhaps the universe had passed from the control of known gods or forces to that of gods or forces which were unknown. And it was then that Nyarlathotep came out of Egypt”
It would have been interesting to try something closer to the movie poster example; say have the AI rewrite the passage with the room toggling between the initial light-mode decor and a dark-mode decor, then back, repeatedly. Or even change the woman's name to "Mrs. Smythe" and back.
As the article explains: “[…] In other words, if you instruct a model to change a single word in a paragraph of text, it can almost always handle the task with no collateral damage.“
It doesn't merely stabilise, it almost converges on itself
> PS. For obvious reasons, this post will fail Pangram tests.
Actual pangram to me says it's 100% human. Current pangram can flag different passages differently, but these are all flagged human by it, even the LLM written ones.
This reminded me of a story/vignette which my fallible memory tags as by someone like Twain or Poe, about an automatic rewriting machine. It was put to work on the same passage with an ever higher intensity setting, becoming more and more gothic to the point of parody -- iirc the final output was verse including the line "to atoms dash the doubly damned".
Does anyone remember this?
No memory, but searching says it was Lewis Carroll in Photography Extraordinary, published in 1885:
https://pinkmonkey.com/dl/library1/carol47.pdfThanks! You're better at search than me. It was part of a story, described here (last 3 pages): https://www.researchgate.net/publication/403044354_Photograp...
It would be interesting how many iterations of LLM-based PR reviews and fixes it takes to turn the code into horror. It’s already started happening.
The existence of attractors (e.g. "stable at iteration 31") for AI models is one of the weirdest things in AI land. They are a wonderful space to explore, fascinating and full of horrors.
Meanwhile, my hat's off to the (currently) last comment on the page: "I like how the shear-sheathed legs still earned their keep — probably load-bearing for Mrs. Regan"
It’s not that bad though. I expected way worse. If Robin Hobbs wrote it, I’d barely lift a brow.
Sing it with me now, "these things aren't inteeeeeeeelligeeeeeeeent"
The same happens with humans, in e.g. chain stories, cumulative stories, etc.
Not unique to AI.
>I don’t write about generative AI much. Everyone else does; not much of consequence ever gets said.
I see he chose not to break with tradition.