I thought the article was interesting from a visualization perspective, even if it's not perfectly solid on the actual mechanics in a model. It helps describe the scale of the connections.
> There’s no pre-formed thought “behind” the words that then gets translated into language. The words are the thinking.
The recent paper on "J-space" [0] seems to contradict this, models can "think" of things without emitting as text.
That's a mischaracterization. Latent space is simply a (multidimensionally) sorted collection, it's only a piece of the pie. A massive amount of structure is held in the unembedding layer. Generative AI models are a very specific ordering, LLMs a very specific subset of that, and they're hardly the only users of the concept.
I get what the author is going for, and they're on the right track. There is something interesting going on with embedding spaces: When used as the substrate for a neural network, you can effectively treat them as a kind of continuous form of computation. That is, given two functions, you can trivially derive a function which sits exactly between those two, and do so ad infinitum, for any arbitrary program (in theory. Obviously everything materially accessible is finite.) This is only one such manipulation. You can deform a function in an unenumerable amount of ways. Think like a bezier curve path tool in something like Krita or Photoshop, but for a function. You can keep adding points and twist it to your heart's content.
It's wrong to focus on LLMs specifically, as well. This is a much, much broader topic than you realize. Most of the interesting stuff has nothing to do with language models at all. I get a huge chunk of the industry is currently having a stroke over LLMs being able to brute-force problem solving, but if we're to talk philosophy, theory, and so on, we have to get past the surface level misuse of Machine Translation's holy grail. That's like having a conversation about the potential of computation itself, but all you talk about is web browsers, using them interchangeably with "computer".
This article and your comment reminded me of something Stephen Wolfram was saying about "mining" the latent space of, in his case, cellular automata as a computational medium. A quick search yields this somewhat older talk: Mining the Computational Universe https://www.stephenwolfram.com/publications/mining-computati...
That phrasing and analogy stuck in my mind, of looking at the space of all possible programs as a resource to be explored for valuable nuggets of algorithms. Your description of interpolating between two functions gives me a similar perspective, of seeing algorithms not only as discrete and separate objects/processes, but "slices" of a larger space, the continuum of computation.
What the article is describing seems to me like "slices of semantic space", not just similar on the surface, but it's actually talking about the same space explored using different tools and lenses.
I'm a fan of Michael Levin's work, a biologist and philosopher riding that edge between genius and mad science. In the interview he mentions how behavioral psychology may be a useful tool in studying the behavior of algorithms and other phenomena typically not associated with having a mind of its own.
>Latent space is simply a (multidimensionally) sorted collection, it's only a piece of the pie.
I agree about the small piece of the pie, but I can't really see it as a sorted collection.
The essence of many dimensions is that you can head off in another direction that doesn't impact the relationship of other dimensions. It seems common to consider a latent space as a encoding of meaning. It is certainly a mapping of relationships, and I think there's some pretty good philosophy arguing that a set of relationships is synonymous with meaning.
A long distance view of LLMs is embeddings encode meaning, Attention finds relevant meanings, and the perceptron does the thinking on things that have meaning and are relevant to each other. The transformer is those things stacked to turn input latents into output latent with a different meaning relative to the input. Stack enough transformer layers to get lots of thinking about lots of meanings.
I'm not entirely convinced that embeddings are doing exactly the things that the ideas like the King - man + woman = Queen examples suggest. It seems to me that the number of dimensions are too low for that to allow a good combination of ideas.
I have wondered how things would look if you considered, instead of cosine difference have something like min(cosine_difference(dimension_mask_0)... cosine_difference(dimension_mask_n)).
The idea would be instead of dimensions being pure encodings of some group of meaning some dimensions are expendable.
Like if you had W,X,Y,Z dimensions and you wanted to encode the relationship between items if they all had identical circular dials with a thousand concepts written around the rim. and the dial faces were around WX,WY,WZ, XY, XZ, YZ, you could link any two concepts with any two dials.
In higher dimensions the combination of relationships possible become astronomical. and to me it seems intuitively more expressive for relationships than the weighting of some dimensions representing a vector that encodes a magnitude of a single concept.
I think you're running into difficulty because you're conflating 'sorting' with what we would call a total or linear ordering. Partial orders are non-linear, and allow you to form posets, which are isomorphic with DAGs. Then you have the broader family of orderings containing weak orders, preorders, strictness, etc. which make matters more complicated. Cyclic orders drop transitivity, for example, allowing you to describe directed graphs with cycles. The thing is that sorting also isn't strictly about orderings. It also encapsulates classification, which are a family of operations entirely distinct from order. There's overlap in the structural implications of some orderings and classifications, but they're also distinct categories (and both are important in ML.)
There's a lot of very strong claims made at the start of this article.
(emphasis mine)
> A Large Language Model (LLM) is like a small zip file that contains all human knowledge.
> In a strange but real way the resulting tiny file contains all the information that is on the internet and in our libraries.
> Likewise, the LLM could recognize the face of almost any person, and it could generate any possible human face,
All writings? All of human knowledge in general? Any person??
The example he gives for writing is Shakespeare, which might be one of the most overrepresented writers in the entire training dataset. So yes, of course LLMs can replicate his writings with high accuracy. That doesn't mean that the same applies to literally all of human writings and knowledge.
> We've never had a system to integrate everything we know and everything we can imagine.
Yeah, we still don't.
At first I thought he was just being intentionally hyperbolic for effect, but the rest of the article is even worse. From the closing paragraph:
> We’ll soon depend on this oracle to such an extent that we’ll wonder how we lived without it.
No! AI is not an oracle! That is honestly an extremely dangerous way to think about this technology.
This is the type of baseless hype that OpenAI and Anthropic have been exploiting for years, and I really wish it would stop.
>In other words, correctness, truth, cohesiveness, completeness, comprehension, etc are all essentially patterns that are mapped in this space.
The caveat is that truth in latent space is just a reflection of the consensus from the corpus, and you find truth by comparing the answer in latent space to what is in reality.
But I just hate this idea that truth and facts are no longer real, they're just "directions". The more we rely on these models for our lives, the more we lose touch with reality and are pushed and pulled in all these different directions. Feels like the future is just ai psychosis and there's no way out. Is that what complete agi victory looks like?
I think virtual reality and AI are a prelude to humanity living in Hong Kong style coffin apartments, and things will get progressively worse until we're in Matrix pods.
Dear Kevin: AI models don't contain all of human knowledge. They don't even contain, for example, the complete curricula of the least comprehensive K-12 program.
I thought so too, but the results are too consistent. It never combines foo and bar to make "Certainly! It's foobar" or "I can't do that" or "foobar. Would you like anything else?"
This article is written like someone sat around and wrote out as many potential buzzwords as possible and then came up with definitions after the fact.
I thought the article was interesting from a visualization perspective, even if it's not perfectly solid on the actual mechanics in a model. It helps describe the scale of the connections.
> There’s no pre-formed thought “behind” the words that then gets translated into language. The words are the thinking.
The recent paper on "J-space" [0] seems to contradict this, models can "think" of things without emitting as text.
[0] https://www.anthropic.com/research/global-workspace
That's a mischaracterization. Latent space is simply a (multidimensionally) sorted collection, it's only a piece of the pie. A massive amount of structure is held in the unembedding layer. Generative AI models are a very specific ordering, LLMs a very specific subset of that, and they're hardly the only users of the concept.
I get what the author is going for, and they're on the right track. There is something interesting going on with embedding spaces: When used as the substrate for a neural network, you can effectively treat them as a kind of continuous form of computation. That is, given two functions, you can trivially derive a function which sits exactly between those two, and do so ad infinitum, for any arbitrary program (in theory. Obviously everything materially accessible is finite.) This is only one such manipulation. You can deform a function in an unenumerable amount of ways. Think like a bezier curve path tool in something like Krita or Photoshop, but for a function. You can keep adding points and twist it to your heart's content.
It's wrong to focus on LLMs specifically, as well. This is a much, much broader topic than you realize. Most of the interesting stuff has nothing to do with language models at all. I get a huge chunk of the industry is currently having a stroke over LLMs being able to brute-force problem solving, but if we're to talk philosophy, theory, and so on, we have to get past the surface level misuse of Machine Translation's holy grail. That's like having a conversation about the potential of computation itself, but all you talk about is web browsers, using them interchangeably with "computer".
This article and your comment reminded me of something Stephen Wolfram was saying about "mining" the latent space of, in his case, cellular automata as a computational medium. A quick search yields this somewhat older talk: Mining the Computational Universe https://www.stephenwolfram.com/publications/mining-computati...
That phrasing and analogy stuck in my mind, of looking at the space of all possible programs as a resource to be explored for valuable nuggets of algorithms. Your description of interpolating between two functions gives me a similar perspective, of seeing algorithms not only as discrete and separate objects/processes, but "slices" of a larger space, the continuum of computation.
What the article is describing seems to me like "slices of semantic space", not just similar on the surface, but it's actually talking about the same space explored using different tools and lenses.
I recently came across a video of the author discussing this paper: https://arxiv.org/abs/2401.05375
I’ve been mulling it over the past couple of weeks. I’m not sure what to make of it.
Here's a Lex Fridman interview discussing it: https://www.youtube.com/watch?v=jr1sNYY2t9A
I'm a fan of Michael Levin's work, a biologist and philosopher riding that edge between genius and mad science. In the interview he mentions how behavioral psychology may be a useful tool in studying the behavior of algorithms and other phenomena typically not associated with having a mind of its own.
>Latent space is simply a (multidimensionally) sorted collection, it's only a piece of the pie.
I agree about the small piece of the pie, but I can't really see it as a sorted collection.
The essence of many dimensions is that you can head off in another direction that doesn't impact the relationship of other dimensions. It seems common to consider a latent space as a encoding of meaning. It is certainly a mapping of relationships, and I think there's some pretty good philosophy arguing that a set of relationships is synonymous with meaning.
A long distance view of LLMs is embeddings encode meaning, Attention finds relevant meanings, and the perceptron does the thinking on things that have meaning and are relevant to each other. The transformer is those things stacked to turn input latents into output latent with a different meaning relative to the input. Stack enough transformer layers to get lots of thinking about lots of meanings.
I'm not entirely convinced that embeddings are doing exactly the things that the ideas like the King - man + woman = Queen examples suggest. It seems to me that the number of dimensions are too low for that to allow a good combination of ideas.
I have wondered how things would look if you considered, instead of cosine difference have something like min(cosine_difference(dimension_mask_0)... cosine_difference(dimension_mask_n)).
The idea would be instead of dimensions being pure encodings of some group of meaning some dimensions are expendable.
Like if you had W,X,Y,Z dimensions and you wanted to encode the relationship between items if they all had identical circular dials with a thousand concepts written around the rim. and the dial faces were around WX,WY,WZ, XY, XZ, YZ, you could link any two concepts with any two dials.
In higher dimensions the combination of relationships possible become astronomical. and to me it seems intuitively more expressive for relationships than the weighting of some dimensions representing a vector that encodes a magnitude of a single concept.
I think you're running into difficulty because you're conflating 'sorting' with what we would call a total or linear ordering. Partial orders are non-linear, and allow you to form posets, which are isomorphic with DAGs. Then you have the broader family of orderings containing weak orders, preorders, strictness, etc. which make matters more complicated. Cyclic orders drop transitivity, for example, allowing you to describe directed graphs with cycles. The thing is that sorting also isn't strictly about orderings. It also encapsulates classification, which are a family of operations entirely distinct from order. There's overlap in the structural implications of some orderings and classifications, but they're also distinct categories (and both are important in ML.)
There's a lot of very strong claims made at the start of this article.
(emphasis mine)
> A Large Language Model (LLM) is like a small zip file that contains all human knowledge.
> In a strange but real way the resulting tiny file contains all the information that is on the internet and in our libraries.
> Likewise, the LLM could recognize the face of almost any person, and it could generate any possible human face,
All writings? All of human knowledge in general? Any person??
The example he gives for writing is Shakespeare, which might be one of the most overrepresented writers in the entire training dataset. So yes, of course LLMs can replicate his writings with high accuracy. That doesn't mean that the same applies to literally all of human writings and knowledge.
> We've never had a system to integrate everything we know and everything we can imagine.
Yeah, we still don't.
At first I thought he was just being intentionally hyperbolic for effect, but the rest of the article is even worse. From the closing paragraph:
> We’ll soon depend on this oracle to such an extent that we’ll wonder how we lived without it.
No! AI is not an oracle! That is honestly an extremely dangerous way to think about this technology.
This is the type of baseless hype that OpenAI and Anthropic have been exploiting for years, and I really wish it would stop.
>In other words, correctness, truth, cohesiveness, completeness, comprehension, etc are all essentially patterns that are mapped in this space.
The caveat is that truth in latent space is just a reflection of the consensus from the corpus, and you find truth by comparing the answer in latent space to what is in reality.
But I just hate this idea that truth and facts are no longer real, they're just "directions". The more we rely on these models for our lives, the more we lose touch with reality and are pushed and pulled in all these different directions. Feels like the future is just ai psychosis and there's no way out. Is that what complete agi victory looks like?
I think virtual reality and AI are a prelude to humanity living in Hong Kong style coffin apartments, and things will get progressively worse until we're in Matrix pods.
Dear Kevin: AI models don't contain all of human knowledge. They don't even contain, for example, the complete curricula of the least comprehensive K-12 program.
That's a really interesting claim, can you give an example of a question that could reasonably be on a K-12 exam that a frontier model can't answer?
I learned to drive in a drivers ed course at school in tenth grade.
There's also home ec, wood shop, metal shop, auto shop, sculpture, choir, theater arts, marching band, phys ed.
Probably the most important thing I've read this month. The key concepts are valuable.
This post made me want to watch Everything Everywhere All at Once again
Some math related to training on latents, not tokens:
https://arxiv.org/abs/2605.27734
In future, to provide guard rails to models, we will build token flow pipes in mathematical sense. Sort of like Mathematical Engineering.
Infinite Craft is a web game based on combining two things using LLMs, but it's not clear how it works, whether latent space or something else.
I believe it's just giving a prompt to an LLM saying "combine these things ..."
I thought so too, but the results are too consistent. It never combines foo and bar to make "Certainly! It's foobar" or "I can't do that" or "foobar. Would you like anything else?"
what an awful, nonsensical article
This is nonsensical AI slop with so many technical mistakes.
This article is written like someone sat around and wrote out as many potential buzzwords as possible and then came up with definitions after the fact.