I don't see what we gain for the mention of category theory here, and I find the categorical content in the book to be pretty buried.
If we're talking about categories, then we should be able to provide definitions of objects and morphisms clearly and independently. I guess I expected this to give denotational semantics of machine learning in an appropriately structured category, and then afterwards we can provide an implementation of these abstractions in Rust; however, this doesn't seem to be the case in this book. Rather, the category theory does seem somewhat stapled on. I wished that this would talk about things like Markov categories (or some other appropriate appropriate semantic domain) and then characterized machine learning algorithms via adjunctions between certain categories, such as in
https://link.springer.com/article/10.1007/s44163-025-00707-w
As it's written, I don't see much of an opportunity for deriving theorems about the implementation from abstract nonsense, which, to me, would be the biggest strength of such a categorical description. This seems to be a simultaneous high-level introduction to Rust, machine learning, and category theory. The writing suffers from this, as the reader doesn't have much of an opportunity here to detangle these ideas from each other or see how one aids in understanding the others. Instead, they are all provide at once, and to an insufficient level of detail (for the amount of skimming that I did).
This is not a critque of the work, but I have now been often encountering category theory in many of the research topics I'm working on, and even as somebody who majored in math, I sort of feel like it doesn't add much. I know ML frameworks intimately, and you really don't need category theory to describe them. But this is maybe (probably) a failure of mine, because I have not yet groked what category theory is really bringing to the table.
Category theory isn't usually intended to add things. Often, what category theory brings to the table is connecting different branches of math. (It's a bit similar to abstract algebra in that way, but at a different level.)
For example, the lambda calculus is the base for many useful programming languages. But the lambda calculus maps to a "closed Cartesian category". And many, many other interesting things in math can be mapped to a closed Cartesian category.
So now you can ask, "What if affine or linear logic were a programming language?" And the answer is, "You'd get a language with safe resource management, like Rust." Or you might ask, "What if probability were a programming language?"
Or on a smaller scale, a parameterized collection type with "map" is a functor. Add a single-element constructor and a "flatten" operation, and you have a monad. Functions with parameterized types are often natural transformations. And so on. This can then be directly analogized to constructs in different areas of math. Which might sometimes produce an interesting idea or two.
So category theory isn't always used to add something new and profound. Sometimes it's just a handy way to see something already there.
It helps to not make a cluttered mess of an API. Breaking up the problem space with a few composable functions that can express all that you want to express is a definite win.
Category theory is all about relationships and structural patterns. so its useful when you want to interoperability and composition between systems i.e. invariants under transformations, etc.
Without reading too much into what this framework does, I'd say category theory could be useful for some ML problems (i.e. layer composition, gradient propagation, etc.) - but I'd think it would be more useful as an analytical tool than as actual lib/code structures.
> as somebody who majored in math, I sort of feel like it doesn't add much. I know ML frameworks intimately, and you really don't need category theory to describe them.
Not that the work linked here is doing this, but.. using category theory to describe the approach could make a lot of sense even if it's not required. The idea would be that instead of inventing the next architecture from scratch maybe you aim to correct problems in the current generation of systems by showing some generalization/transformation that doesn't have the problems.
Now that "more is different" is something that everyone believes implicitly, the alternative to an abstract existence proof is literally millions/billions spent on a proof of concept that may not work. Doesn't mean we need to use category theory, I mean there's reasons that might be a good idea, but if this were doable without some big change in perspective then it seems likely it would be done already
Category theory is rarely useful by itself, but it can be a mental scaffold when designing things like query languages. Microsoft's LINQ dsl within C# used category theory ideas to ensure consistency. That said, the applicability surface area in practice is typically quite limited in my experience. It's like formal methods -- elegant in practice, but a good problem fit is often rare. It's like writing a LEAN proof for your web app -- rarely needed, but if your web app needs a high degree of correctness, then indispensable.
This is John D Cook's take:
Category theory can be very useful, but you don’t use it the same way you use other kinds of math. You can apply optimization theory, for example, by noticing that a problem has a certain form, and therefore a certain algorithm will converge to a solution. Applications of category theory are usually more subtle. You’re not likely to quote some theorem from category theory that finishes off a problem the way the selecting an optimization algorithm does.
I had been skeptical of applications of category theory, and to some extent I still am. Many reported applications of category theory aren’t that applied, and they’re not so much applications as post hoc glosses. At the same time, I’ve seen real applications of categories, such as the design of LINQ mentioned above. I’ve been a part of projects where we used category theory to guide mathematical modeling and software development. Category theory can spot inconsistencies and errors similar to the way dimensional analysis does in engineering, or type checking in software development. It can help you ask the right questions. It can guide you to including the right things, and leaving the right things out. [1]
You don't need category theory to describe the Result type. But the people who first introduced it to programming languages, were thing about category theory.
Eh. I’ve been writing a small HoTT language over the past two years based on schemes, stacks, and sites and anything short of Agda or Lean falls short in terms of Real™ Category Theory.
So much has to be pragmatised, carefully considered and made concrete categories anyway that much of the category theory exists in the documentation rather in the actual code.
Edit: The more I read the more I think that this has at least been edited using AI. The number of instances of "this is not X, it is Y" is not super high, but it's higher than I'd expect from a human.
I don't understand, this looks to me like regular Rust, or regular programming for that matter.
You use types to represent domain objects, and the program is composed of functions that transform domain objects into other domain objects.
Sure types are used a bit more aggressively than usual to restrict domains, particularly the newtype pattern (`struct TokenId(usize)` instead of just `usize`). But it doesn't look too exotic to me, or Category Theory influenced, other than in the sense that Category Theory terminology can be used to describe the structure of a regular typed program.
It's possible that I'm wrong and I'm missing the point. Frankly I really struggled reading this because of the AI generated vibe of the language, more than usual. I generally hate when content is criticised for just being AI generated, you can write very good and valuable things with AI by guiding it properly with authorial intent, but this one does really reek of bloated slop.
I don't see what we gain for the mention of category theory here, and I find the categorical content in the book to be pretty buried.
If we're talking about categories, then we should be able to provide definitions of objects and morphisms clearly and independently. I guess I expected this to give denotational semantics of machine learning in an appropriately structured category, and then afterwards we can provide an implementation of these abstractions in Rust; however, this doesn't seem to be the case in this book. Rather, the category theory does seem somewhat stapled on. I wished that this would talk about things like Markov categories (or some other appropriate appropriate semantic domain) and then characterized machine learning algorithms via adjunctions between certain categories, such as in https://link.springer.com/article/10.1007/s44163-025-00707-w
As it's written, I don't see much of an opportunity for deriving theorems about the implementation from abstract nonsense, which, to me, would be the biggest strength of such a categorical description. This seems to be a simultaneous high-level introduction to Rust, machine learning, and category theory. The writing suffers from this, as the reader doesn't have much of an opportunity here to detangle these ideas from each other or see how one aids in understanding the others. Instead, they are all provide at once, and to an insufficient level of detail (for the amount of skimming that I did).
This is not a critque of the work, but I have now been often encountering category theory in many of the research topics I'm working on, and even as somebody who majored in math, I sort of feel like it doesn't add much. I know ML frameworks intimately, and you really don't need category theory to describe them. But this is maybe (probably) a failure of mine, because I have not yet groked what category theory is really bringing to the table.
Category theory isn't usually intended to add things. Often, what category theory brings to the table is connecting different branches of math. (It's a bit similar to abstract algebra in that way, but at a different level.)
For example, the lambda calculus is the base for many useful programming languages. But the lambda calculus maps to a "closed Cartesian category". And many, many other interesting things in math can be mapped to a closed Cartesian category.
So now you can ask, "What if affine or linear logic were a programming language?" And the answer is, "You'd get a language with safe resource management, like Rust." Or you might ask, "What if probability were a programming language?"
Or on a smaller scale, a parameterized collection type with "map" is a functor. Add a single-element constructor and a "flatten" operation, and you have a monad. Functions with parameterized types are often natural transformations. And so on. This can then be directly analogized to constructs in different areas of math. Which might sometimes produce an interesting idea or two.
So category theory isn't always used to add something new and profound. Sometimes it's just a handy way to see something already there.
It helps to not make a cluttered mess of an API. Breaking up the problem space with a few composable functions that can express all that you want to express is a definite win.
Category theory is all about relationships and structural patterns. so its useful when you want to interoperability and composition between systems i.e. invariants under transformations, etc.
Without reading too much into what this framework does, I'd say category theory could be useful for some ML problems (i.e. layer composition, gradient propagation, etc.) - but I'd think it would be more useful as an analytical tool than as actual lib/code structures.
> as somebody who majored in math, I sort of feel like it doesn't add much. I know ML frameworks intimately, and you really don't need category theory to describe them.
Not that the work linked here is doing this, but.. using category theory to describe the approach could make a lot of sense even if it's not required. The idea would be that instead of inventing the next architecture from scratch maybe you aim to correct problems in the current generation of systems by showing some generalization/transformation that doesn't have the problems.
Now that "more is different" is something that everyone believes implicitly, the alternative to an abstract existence proof is literally millions/billions spent on a proof of concept that may not work. Doesn't mean we need to use category theory, I mean there's reasons that might be a good idea, but if this were doable without some big change in perspective then it seems likely it would be done already
Category theory is rarely useful by itself, but it can be a mental scaffold when designing things like query languages. Microsoft's LINQ dsl within C# used category theory ideas to ensure consistency. That said, the applicability surface area in practice is typically quite limited in my experience. It's like formal methods -- elegant in practice, but a good problem fit is often rare. It's like writing a LEAN proof for your web app -- rarely needed, but if your web app needs a high degree of correctness, then indispensable.
This is John D Cook's take:
Category theory can be very useful, but you don’t use it the same way you use other kinds of math. You can apply optimization theory, for example, by noticing that a problem has a certain form, and therefore a certain algorithm will converge to a solution. Applications of category theory are usually more subtle. You’re not likely to quote some theorem from category theory that finishes off a problem the way the selecting an optimization algorithm does.
I had been skeptical of applications of category theory, and to some extent I still am. Many reported applications of category theory aren’t that applied, and they’re not so much applications as post hoc glosses. At the same time, I’ve seen real applications of categories, such as the design of LINQ mentioned above. I’ve been a part of projects where we used category theory to guide mathematical modeling and software development. Category theory can spot inconsistencies and errors similar to the way dimensional analysis does in engineering, or type checking in software development. It can help you ask the right questions. It can guide you to including the right things, and leaving the right things out. [1]
[1] https://www.johndcook.com/blog/applied-category-theory/
You don't need category theory to describe the Result type. But the people who first introduced it to programming languages, were thing about category theory.
https://en.wikipedia.org/wiki/Result_type
Maybe expand ML to Machine Learning to avoid confusing with the programming language
> Rust
> No HKTs
> Category Theory
Eh. I’ve been writing a small HoTT language over the past two years based on schemes, stacks, and sites and anything short of Agda or Lean falls short in terms of Real™ Category Theory.
So much has to be pragmatised, carefully considered and made concrete categories anyway that much of the category theory exists in the documentation rather in the actual code.
That sounds really cool, do you mind dropping a link if the work is public?
The style of writing here feels very Claude...
Edit: The more I read the more I think that this has at least been edited using AI. The number of instances of "this is not X, it is Y" is not super high, but it's higher than I'd expect from a human.
It's nice to see that.
I don't understand, this looks to me like regular Rust, or regular programming for that matter.
You use types to represent domain objects, and the program is composed of functions that transform domain objects into other domain objects.
Sure types are used a bit more aggressively than usual to restrict domains, particularly the newtype pattern (`struct TokenId(usize)` instead of just `usize`). But it doesn't look too exotic to me, or Category Theory influenced, other than in the sense that Category Theory terminology can be used to describe the structure of a regular typed program.
It's possible that I'm wrong and I'm missing the point. Frankly I really struggled reading this because of the AI generated vibe of the language, more than usual. I generally hate when content is criticised for just being AI generated, you can write very good and valuable things with AI by guiding it properly with authorial intent, but this one does really reek of bloated slop.
nice work
Off topic, removed
I think this would be considered off topic.
I'll remove the comment then