It might be kind of cool to implement just a basic loop and let the agent itself to implement memory and all other aspects on the fly, fully embracing self-modifying code.
But it's also extremely brittle and unstable. It's really the opposite of what people look in agents.
I like Lisp, I’ve used Common Lisp with a passion, but this doesn’t seem like a valid argument for Lisp.
Homoiconicity, as I understand, is that the code is structured data that is easy to programmatically modify, hence allowing Lisp macros. While some might disagree, I see Rust macros as the closest thing that demonstrates homoiconicity in mainstream Algol-based languages, as Rust macros modify the loosely structured token stream to produce new Rust code.
Eval, on the other hand, that’s more of a capability that comes from Lisp’s runtime, which used to be unique when Lisp was thriving, but not anymore — JS, Python, Ruby, all of the runtime-based languages have an eval function. The fact that they are not used as much is more of a security issue, not a capability issue, and I am not sure how having eval can be argued as Lisp being the language of agents.
> I see Rust macros as the closest thing that demonstrates homoiconicity in mainstream Algol-based languages
I don't know if you consider Elixir mainstream, but IMO their macro system is much closer to lisp's ideal.
Elixir is basically Lisp, but with better syntax, a modern ecosystem, and running on the Beam. Unlike languages like Rust, Elixir's conditionals and function definitions are just calls in the AST, even though the syntax looks mainstream and not like paren soup.
> Elixir is basically Lisp, but with better syntax
Which so helpfully removes many of the benefits of lisps :P I don't understand this argument at all, if it's not s-expressions nor quacks like a lisp, in what way is it lisp at all? Making it algol/C-like already makes it like the rest of the 99% of the languages, without any of the easy benefits of the neat and simple syntax of lisps.
Agreed. You can tell when someone “gets it” or not by whether they see sexprs as a problem to be overcome or a benefit to be embraced. Count me firmly in the benefit camp.
A Lisp program that writes a Lisp program really just needs to produce a list of (nested lists) of tokens. A JavaScript program that writes a javascript program needs to generate a string that is syntactically valid JavaScript code. That is a much bigger task than just constructing a (nested) list.
Because Lisp syntax is so much simpler than that of JavaScript etc. it is much easier to avoid errors when generating code. In JavaScript you can use JSON to generate data, but JSON can not carry functions around.
I think this idea makes a lot sense. Instead of making the LLM generate JSON or XML, why not make it generate Lisp, which can carry both programs and data?
Well, then ad absurdum it would be easier to just output a bunch of 0s and 1s, right?
The same goes for "just a list of nested lists", sure it is easy to produce it, and for trivial examples it may actually be easy, but for more complex and realistic problems lack of "higher order" structure is a negative!
For something like JavaScript, you can just have a language-native AST object with a few helper functions and then can just call "addFunction" on it with proper arguments so that the API shape validates plenty of properties of your output.
Except you can express the higher order structure in s-expressions too, and using an AST usually means expressing it as a different syntax than the other parts of your program.
Or maybe I've just been sleeping on the power of Javascript AST.
And that's more of a library ergonomics question how nice it is. There are plenty of other languages with nice ASTs and `eval`, and that's the only ingredient you need.
I haven’t seen an LLM generate syntactically invalid code in any language in … a year? So I don’t know if “more likely to be syntactically valid” is a good enough selling point.
I take it you never run models locally then? Local models that can run on reasonable consumer hardware have gotten to the point of being very useful, but they still occasionally are tripped up with syntax errors. Especially if you quantize the model and KV.
> A Lisp program that writes a Lisp program really just needs to produce a list of (nested lists) of tokens. A JavaScript program that writes a javascript program needs to generate a string that is syntactically valid JavaScript code. That is a much bigger task than just constructing a (nested) list.
> Because Lisp syntax is so much simpler than that of JavaScript etc. it is much easier to avoid errors when generating code. In JavaScript you can use JSON to generate data, but JSON can not carry functions around.
First of all, the LLM does not produce structured tokens, it produces (tokens of) text. It does not have a concept of nested or structured tokens. Which means that producing a Lisp program and a JavaScript program is basically the same difficulty, i.e. LLMs producing function foo () {} is about the same task as producing (defun foo () ()).
In fact, because most Lisps uses the same token ( and ) for almost all delimiters, the LLM in fact gets confused a lot more than Algol-family languages that uses various different tokens for different purposes. It generates thinking traces that are a few screens long while trying to count the closing parenthesis and the depth, something that I have not found to be the case for other languages, even with languages much more obscure than Lisp. (And no, it is not a training data issue, because the Lisp family as a whole is pretty well represented in the data set, due to Emacs Lisp.)
> I think this idea makes a lot sense. Instead of making the LLM generate JSON or XML, why not make it generate Lisp, which can carry both programs and data?
You do realize that all programming languages contains both programs and data, right? i.e. JSON is literally a subset of the JavaScript language, all JSON documents are valid JavaScript code, can be embedded in JavaScript programs, and so on. This isn't even a JavaScript-specific thing, almost all recent programming languages have data structure literals.
The thing that makes Lisp unique is that it can modify programs as data in the language easily, and why would that be a unique capability that would be beneficial for LLMs, when it can just sed/awk or tree-sitter-parse programs with more conventional languages and modify it as easily?
Yeah, unfortunately "LLMs are bad at counting" seems to apply to counting parentheses.
With modern tool calling I wonder if a better way to go about it is for the LLM to express the program changes as a function or otherwise use an editor that auto-balances parens. There's a lot of relatively simple tooling that makes it easier to write in a Lisp. The languages tend to lend themselves to being straightforward to check like that.
What's special about Lisp's repl is that it's perfectly possible to construct your entire program in the repl, testing each addition live as you write it. (Many Lisp-focused editors assume you'll want to do this, such as Emacs making it easy to run the interpreter on a single function in a file.) That tooling is lost if you just try to one-shot the file, and before 2026 the majority of LLMs originally just tried to one-shot every file.
But, just like a lot of early LLMs had huge problems with whitespace and numbers because the tokenization was taking efficiency shortcuts that made sense for text but absolutely wrecked code syntaxe, I wonder if the current optimizations are badly formatted for Lisp.
At the very least, using a varient like Clojure that also uses [] and {} in addition to () might help.
I believe the problem stems from ) and )) both being one token wide. I wonder if repeated parentheses being removed from the token vocabulary would improve accuracy for Lisp code generation.
> Eval, on the other hand, that’s more of a capability that comes from Lisp’s runtime, which used to be unique when Lisp was thriving, but not anymore — JS, Python, Ruby, all of the runtime-based languages have an eval function.
And it's good we have those for troubleshooting but those eval still offer nowhere near the power of a Lisp REPL.
> and I am not sure how having eval can be argued as Lisp being the language of agents
I've seen several Lisp programmers saying that it's really the REPL (and the 'E' in REPL is for "Eval") that is the godsend when working with LLMs.
With LLMs we've seen terminals/ssh/tmux and CLI tools calling making a huge comeback (not that they ever went very far).
Now I wouldn't be surprised if we were soon to see a Lisp AI harness also using extensively the REPL becoming succesful.
It's too early to tell it's not a powerful combo (LLMs + Lisp REPL).
How is that different from other languages that have proper repls? Lisps don't really give you much here, if the language can parse itself and can eval it, you have the sufficient primitives.
What helps is how "small" the hot reloaded parts can get - lisps are good here due to a lot of functions and not much shared state. But that is again, not something specific to lisps, neither are they the best at it - there are far purer FP languages for example
> Homoiconicity, as I understand, is that the code is structured data that is easy to programmatically modify
I think a more accurate description is that lisp code is just cons cells and cons cells is both how we write the code and how the runtime itself implements lists. So there’s basically no distinction between a lisp list and the text representation of the source. Rust and its macros is a different situation because the text representation of the code and the syntax tree have completely different shapes
> I think a more accurate description is that lisp code is just cons cells and cons cells is both how we write the code and how the runtime itself implements lists. So there’s basically no distinction between a lisp list and the text representation of the source.
Heresy I suppose, but doesn't feel (to me) like it "specifically" has to be cons cells, as long as it's a "list" of some sort, regardless of the specific type of list, while I'd say "cons cells" is a specific implementation of a list, famously implemented by Common Lisp and similar lisps.
Besides that I agree with you, the code you write and the code the compiler uses is the same, that's why it gets easy to edit with code itself. Rust and similar don't have this feature at all, it's very different experience writing macros with lots of special syntax, compared to just writing "normal" code doing the same sort of operations. "Doing a loop of X" isn't the same inside of a normal runtime function body, as when you want to do a loop of arguments for the function signature in a macro, as just one simple example.
Not to mention in Rust you have two different types of macros, declarative vs procedural macros, already speaking to that the entire concept of homoiconicity isn't there. You want it to be easy to implement a read, evaluate, print and loop back workflow by passing native data through the entire chain, except parsing the initial user-input string into your AST (whatever shape that ends up being), which pretty-printed (basically) looks the same as the user-input.
But this is just not true. An utf8 string is most definitely not a bunch of cons cells. Having a one-to-one correspondence is not the same thing, and one might argue that a rust code snippet and the corresponding AST object is just as arbitrarily far apart with a clean one-to-one correspondence.
So like, what's the material difference between the two? You have a parse and an eval in both cases
So, writing an agent in Lisp is interesting but not particularly novel. If there’s a big idea here it’s giving Lisp’s eval function to the AI as a tool. Yes, AIs write Python and Bash all day long already, but those scripts are run out of process. In this case, the AI can modify the process running the harness, extending it directly and potentially changing it (evolving it). That’s powerful. And obviously quite dangerous. Definitely only for a sandbox. But I wonder how far that can go…
There is no good reason for running this in-process. The maximally inefficient Fibonacci function in the article is a good demonstration why: Call it with a slightly larger number, and your agent slows to a crawl, with no way to enforce a timeout. Call it with a slightly larger number yet, and you might bring your agent down entirely, with no way to recover.
There is a case to be made for a dynamically evolving "tool server", but it should be a separate process. That would be more flexible for other use cases too. For example, multiple independent agent processes could talk to one shared tool server. Like a blackboard system, more classic AI!
And if you really do want to evolve the agent itself: As the article observes, its entire state can be serialized. Nothing is gained from hanging on to one particular agent process. Serialize its state, ask the tool server to kill it, rewrite its code, then start the new version and replay the state.
Lately I've been adding a repl service to my docker compose configs so the agent can easily send fragments of code for execution in the project context without incurring the clojure startup costs each time.
So cool to watch the AI get into a tight learning loop when it has access to all the internal data structures.
Clearly, you’re not going to do it for just Fibonacci. And yes, you can always rewrite the agent code, quit, and restart to modify the agent. This is just like using an LLM to modify Pi and then restarting, which is done today. But being able to rewrite code in a live, running system is useful for all the reasons that Lispers have done it that way for 50+ years. In some cases, you can’t just serialize your state and reload it easily. Sometimes, for instance, you might not even have access to the whole state because some of it resides on a remote server.
> Sometimes, for instance, you might not even have access to the whole state because some of it resides on a remote server.
Not sure what you mean. The system I outlined is one where some "state" resides outside the process in a separate server. You don't need to serialize that, you just need to serialize the information to need to reconnect.
And my first point is even more relevant the more complex/distributed/brittle you make the whole thing: The more important it is for some specific process to stay alive no matter what, the less you want to live-slop code into it.
(Edit: Yes I'm aware of the live-patched space probe story. Human live-patching is not the same as letting an LLM try to one-shot the correct patch.)
Maybe this is a reductive comment, but how does this differ from just letting your agent bash tool a `python -c` command (or anything of that class)? I'm not really getting where this is a "wow" moment?
It is always nice to appreciate how much power you get out of (Model + the absolute bare minimum of control flow). There is just so much baked into the models now that given an inch they will take a mile.
If you give your agent a `bash` or `python -c` tool, it starts a separate process that produces some output and then exits. After that, only the output and the exit code are available.
In contrast, `eval` runs the code in the same execution context as the agent loop. When `eval` finishes, that execution context still exists. For example, any functions defined during an `eval` call remain available for later use.
That’s a distinction in search of a reason though.
Being able to run code in the same unix process or a new one doesn’t really matter all that much in the context of self modifying code. But even if we cared about that, this isn’t a LISP specific feature. All dynamic languages support eval.
And having the agent cache the tool for reuse is a really trivial problem to solve. Though I do agree that LISP makes this much easier than in many other languages.
This is certainly a cool tech demo. But the claims of its novelty are overstated
That's technically true but practically an agent can just save the script file/rerun it/write a tool that lets it call a reply with memory etc.This aspect is a bit more elegant when it's in the execution context, but the core of "you don't need to predefine tools, the agent can dynamically write its own code" is not exactly new - that's pretty much the basis of why Claude code and codex and all the other agent harnesses are so much more effective than old ways if working with llms - they can just write giant incomprehensible bash scripts on the fly to accomplish random things without having pre-defined tools, write state to files, etc
The blog is about writing an agent when you dont already have an agent, but only a plain LLM. It stitches the minimal pieces together. Agents dont need lots of supporting infra, so it is good to keep the code concise. Not a wow moment for sure, though some people think that agents and harnesses are complicated.
Right but given that the agent only outputs text, and agents are perfectly happy writing python, the supposed benefit of Lisp is completely irrelevant here.
> the supposed benefit of Lisp is completely irrelevant here
Yeah, it's a small example (it's in the title, "100 lines") so obviously doesn't highlight the best benefits once you reach larger codebase size.
Still think ~8 lines for the core loop is probably more elegant, readable and concise than you can achieve in other algol/C-like languages, but happy to be shown that I'm wrong :)
Most of the code in agents is to handle unhappy paths and human friendly interfaces. I’m sure you could trim most languages core loop too if you discarded all the stuff that actually make agents peasant to use.
Please do show the equivalent of those ~8 lines in the other languages you think of so we can explicitly compare :) I gave it a try in the other languages I know, the Clojure one I have still ends up the smallest and fastest to comprehend.
But that’s not just 8 lines. If I were to try and run those 8 lines on their own then the compiler would fail because half of those functions are undefined. Even the article itself doesn’t claim the agent is only 8 lines long.
And once you start including the boilerplate code you end up with something that’s a lot more equivalent to the other languages you tried.
I love functional languages, and LISP specifically too. But the point of that article wasn’t even to say “LISP is better at code golfing than other languages”. so this doubling down on the SLOC that you’re doing isn’t even a relevant tangent.
You're not wrong. I did a similar agent in lisp back with Sonnet 3.5 and had a wow moment, but the wow was mostly for seeing an agent working effectively at all at that point in time.
The part that killed it for me was losing everything if the lisp crashed (sonnet 3.5 was prone to doing that) and solving persistence had too many edge cases and confused the model.
Later realized that writing the agent as 20 lines of bash was equivalently powerful to the lisp agent, but made persistence trivial from the easy file system interop.
you can use LFE (Lisp flavored erlang) = lisp on BEAM runtime.
you get a snippet from LLM, compile it to module, and hot-load it into the running node. the module lives in the node's code table, so it persists and every other agent can call it. not just the one that wrote it.
the agents themselves are seaprate supervised processes, so if one crashes - e.g. because the snippet was crap, it doesn't take down whole system.
of course you can do that in just elixir too, the lisp is just cosmetics really.
By my eye it's not that different, it's riffing in it from a Lisp perspective.
It's pretty amazing to write your own agent BTW. I've got a zero-dependency all-in-one-file agent harness I wrote myself. I use it all the time now because I can get it from anywhere and I can know EXACTLY what it'll do (as much as you can with any model), what it's been told vs not. Using it as a harness for models I'm hosting myself makes me feel like some kind of LLM homesteader: it's a set of tools I'll always have that will only change as much as I want it to change.
I never realized that when chatting with the bot, the entirety of the session was sent each time. I guess I just figured there was some state maintained Out There, rather than simply resubmitted each time.
This is a thing of beauty, no matter what the general feeling in the comments seems to be against LISP. And yeah, you can do it in JS/TS/Python/etc., but somehow it doesn’t feel as elegant.
A symbolic AI solution to a problem requires vastly less energy. And is deterministic; you can cover it with expected input/output pair regression test cases.
import json,sys,uuid;from subprocess import getoutput as sh;from urllib.request import Request as R,urlopen
b={"model":"gpt-5.6","prompt_cache_key":uuid.uuid4().hex,"input":[],"tools":[{"type":"custom","name":"shell"}]}
while prompt:=input("> "):
b["input"]+=[{"role":"user","content":prompt}]
while True:
o=(r:=json.load(urlopen(R(sys.argv[1],json.dumps(b).encode(),{"Content-Type":"application/json"}))))["output"]
b["input"]+=o;calls=[i for i in o if i["type"]=="custom_tool_call"];used=r["usage"]["total_tokens"]/10500
if not calls: print(o[-1]["content"][0]["text"],f'\n[{used:06.3f}%]'); break
b["input"]+=[{"type":"custom_tool_call_output","call_id":i["call_id"],"output":sh(i["input"])} for i in calls]
Which realistically, is the only way to be productive with agents. If you aren't running them inside of something that limits their scope/impact/potential damage, you're already doing it wrong.
Most of those "permission"-systems are built on the idea that an LLM can decide what to ask for approval to run or not anyways, which obviously don't work out great in practice. Might as well give them blanket permission to do whatever, then put them in an isolated environment.
This is beautiful, I can't deny that. But the claim that Claude Code is a fixed set of tools is quite wrong. Claude Code regularly generates new code to carry out tasks, and you can choose to promote those tasks to skills, workflows, etc.
is that wrong though? you write all your code with AI already, but you still need to supervise and steer it. same with blog posts - as long as you steer it towards converying your point, it doesn't matter the text itself was written by AI rather than typed by you, does it?
It doesn't matter if a human or a machine wrote it, it's just unpleasant to read. I don't know what the OP's motivation for their post was, but I think it's reasonable to point out that something is harder to read than it should be.
Similarly, why should I be bothered reading this kind of comment in each and every discussion thread?
What does it contribute?
I can read and discern this for myself, I can then stop reading or decide I don’t care.
Seriously, at some point all you “ai writing sleuths” should just get your own discussion thread together. It’s been months of this , we get it already.
(Not directly just at you, but anyone who feels the need to drop these comments in every thread)
I'm not saying you should, I'm saying you shouldn't clutter up every thread with this type of useless comment. It just detracts from the people that do want to engage with it.
It might be kind of cool to implement just a basic loop and let the agent itself to implement memory and all other aspects on the fly, fully embracing self-modifying code.
But it's also extremely brittle and unstable. It's really the opposite of what people look in agents.
I like Lisp, I’ve used Common Lisp with a passion, but this doesn’t seem like a valid argument for Lisp.
Homoiconicity, as I understand, is that the code is structured data that is easy to programmatically modify, hence allowing Lisp macros. While some might disagree, I see Rust macros as the closest thing that demonstrates homoiconicity in mainstream Algol-based languages, as Rust macros modify the loosely structured token stream to produce new Rust code.
Eval, on the other hand, that’s more of a capability that comes from Lisp’s runtime, which used to be unique when Lisp was thriving, but not anymore — JS, Python, Ruby, all of the runtime-based languages have an eval function. The fact that they are not used as much is more of a security issue, not a capability issue, and I am not sure how having eval can be argued as Lisp being the language of agents.
> I see Rust macros as the closest thing that demonstrates homoiconicity in mainstream Algol-based languages
I don't know if you consider Elixir mainstream, but IMO their macro system is much closer to lisp's ideal.
Elixir is basically Lisp, but with better syntax, a modern ecosystem, and running on the Beam. Unlike languages like Rust, Elixir's conditionals and function definitions are just calls in the AST, even though the syntax looks mainstream and not like paren soup.
> Elixir is basically Lisp, but with better syntax
Which so helpfully removes many of the benefits of lisps :P I don't understand this argument at all, if it's not s-expressions nor quacks like a lisp, in what way is it lisp at all? Making it algol/C-like already makes it like the rest of the 99% of the languages, without any of the easy benefits of the neat and simple syntax of lisps.
Agreed. You can tell when someone “gets it” or not by whether they see sexprs as a problem to be overcome or a benefit to be embraced. Count me firmly in the benefit camp.
A Lisp program that writes a Lisp program really just needs to produce a list of (nested lists) of tokens. A JavaScript program that writes a javascript program needs to generate a string that is syntactically valid JavaScript code. That is a much bigger task than just constructing a (nested) list.
Because Lisp syntax is so much simpler than that of JavaScript etc. it is much easier to avoid errors when generating code. In JavaScript you can use JSON to generate data, but JSON can not carry functions around.
I think this idea makes a lot sense. Instead of making the LLM generate JSON or XML, why not make it generate Lisp, which can carry both programs and data?
Well, then ad absurdum it would be easier to just output a bunch of 0s and 1s, right?
The same goes for "just a list of nested lists", sure it is easy to produce it, and for trivial examples it may actually be easy, but for more complex and realistic problems lack of "higher order" structure is a negative!
For something like JavaScript, you can just have a language-native AST object with a few helper functions and then can just call "addFunction" on it with proper arguments so that the API shape validates plenty of properties of your output.
Except you can express the higher order structure in s-expressions too, and using an AST usually means expressing it as a different syntax than the other parts of your program.
Or maybe I've just been sleeping on the power of Javascript AST.
So can you in binary!
And that's more of a library ergonomics question how nice it is. There are plenty of other languages with nice ASTs and `eval`, and that's the only ingredient you need.
I haven’t seen an LLM generate syntactically invalid code in any language in … a year? So I don’t know if “more likely to be syntactically valid” is a good enough selling point.
But maybe I’m interpreting you too narrowly?
I take it you never run models locally then? Local models that can run on reasonable consumer hardware have gotten to the point of being very useful, but they still occasionally are tripped up with syntax errors. Especially if you quantize the model and KV.
> A Lisp program that writes a Lisp program really just needs to produce a list of (nested lists) of tokens. A JavaScript program that writes a javascript program needs to generate a string that is syntactically valid JavaScript code. That is a much bigger task than just constructing a (nested) list.
> Because Lisp syntax is so much simpler than that of JavaScript etc. it is much easier to avoid errors when generating code. In JavaScript you can use JSON to generate data, but JSON can not carry functions around.
First of all, the LLM does not produce structured tokens, it produces (tokens of) text. It does not have a concept of nested or structured tokens. Which means that producing a Lisp program and a JavaScript program is basically the same difficulty, i.e. LLMs producing function foo () {} is about the same task as producing (defun foo () ()).
In fact, because most Lisps uses the same token ( and ) for almost all delimiters, the LLM in fact gets confused a lot more than Algol-family languages that uses various different tokens for different purposes. It generates thinking traces that are a few screens long while trying to count the closing parenthesis and the depth, something that I have not found to be the case for other languages, even with languages much more obscure than Lisp. (And no, it is not a training data issue, because the Lisp family as a whole is pretty well represented in the data set, due to Emacs Lisp.)
> I think this idea makes a lot sense. Instead of making the LLM generate JSON or XML, why not make it generate Lisp, which can carry both programs and data?
You do realize that all programming languages contains both programs and data, right? i.e. JSON is literally a subset of the JavaScript language, all JSON documents are valid JavaScript code, can be embedded in JavaScript programs, and so on. This isn't even a JavaScript-specific thing, almost all recent programming languages have data structure literals.
The thing that makes Lisp unique is that it can modify programs as data in the language easily, and why would that be a unique capability that would be beneficial for LLMs, when it can just sed/awk or tree-sitter-parse programs with more conventional languages and modify it as easily?
Now that you mention it, Claude has had trouble at times with balancing brackets in the elisp I have it write.
Yet I still had the idea that LLMs should be better at lisp than other languages.
A fascinating contradiction, thanks for pointing this out.
Yeah, unfortunately "LLMs are bad at counting" seems to apply to counting parentheses.
With modern tool calling I wonder if a better way to go about it is for the LLM to express the program changes as a function or otherwise use an editor that auto-balances parens. There's a lot of relatively simple tooling that makes it easier to write in a Lisp. The languages tend to lend themselves to being straightforward to check like that.
What's special about Lisp's repl is that it's perfectly possible to construct your entire program in the repl, testing each addition live as you write it. (Many Lisp-focused editors assume you'll want to do this, such as Emacs making it easy to run the interpreter on a single function in a file.) That tooling is lost if you just try to one-shot the file, and before 2026 the majority of LLMs originally just tried to one-shot every file.
But, just like a lot of early LLMs had huge problems with whitespace and numbers because the tokenization was taking efficiency shortcuts that made sense for text but absolutely wrecked code syntaxe, I wonder if the current optimizations are badly formatted for Lisp.
At the very least, using a varient like Clojure that also uses [] and {} in addition to () might help.
I believe the problem stems from ) and )) both being one token wide. I wonder if repeated parentheses being removed from the token vocabulary would improve accuracy for Lisp code generation.
This deepmind paper showed big limitations on nested structures with transformers, LSTMs actually did better:
Neural Networks and the Chomsky Hierarchy
https://arxiv.org/abs/2207.02098
Does eval in these other languages evaluate ASTs or strings? It makes a difference.
eval(parse(yourString))
No it doesn't
It's not just eval. It's LISP macros what makes it an interesting powerful language for agents.
> Eval, on the other hand, that’s more of a capability that comes from Lisp’s runtime, which used to be unique when Lisp was thriving, but not anymore — JS, Python, Ruby, all of the runtime-based languages have an eval function.
And it's good we have those for troubleshooting but those eval still offer nowhere near the power of a Lisp REPL.
> and I am not sure how having eval can be argued as Lisp being the language of agents
I've seen several Lisp programmers saying that it's really the REPL (and the 'E' in REPL is for "Eval") that is the godsend when working with LLMs.
With LLMs we've seen terminals/ssh/tmux and CLI tools calling making a huge comeback (not that they ever went very far).
Now I wouldn't be surprised if we were soon to see a Lisp AI harness also using extensively the REPL becoming succesful.
It's too early to tell it's not a powerful combo (LLMs + Lisp REPL).
How is that different from other languages that have proper repls? Lisps don't really give you much here, if the language can parse itself and can eval it, you have the sufficient primitives.
What helps is how "small" the hot reloaded parts can get - lisps are good here due to a lot of functions and not much shared state. But that is again, not something specific to lisps, neither are they the best at it - there are far purer FP languages for example
> Homoiconicity, as I understand, is that the code is structured data that is easy to programmatically modify
I think a more accurate description is that lisp code is just cons cells and cons cells is both how we write the code and how the runtime itself implements lists. So there’s basically no distinction between a lisp list and the text representation of the source. Rust and its macros is a different situation because the text representation of the code and the syntax tree have completely different shapes
> I think a more accurate description is that lisp code is just cons cells and cons cells is both how we write the code and how the runtime itself implements lists. So there’s basically no distinction between a lisp list and the text representation of the source.
Heresy I suppose, but doesn't feel (to me) like it "specifically" has to be cons cells, as long as it's a "list" of some sort, regardless of the specific type of list, while I'd say "cons cells" is a specific implementation of a list, famously implemented by Common Lisp and similar lisps.
Besides that I agree with you, the code you write and the code the compiler uses is the same, that's why it gets easy to edit with code itself. Rust and similar don't have this feature at all, it's very different experience writing macros with lots of special syntax, compared to just writing "normal" code doing the same sort of operations. "Doing a loop of X" isn't the same inside of a normal runtime function body, as when you want to do a loop of arguments for the function signature in a macro, as just one simple example.
Not to mention in Rust you have two different types of macros, declarative vs procedural macros, already speaking to that the entire concept of homoiconicity isn't there. You want it to be easy to implement a read, evaluate, print and loop back workflow by passing native data through the entire chain, except parsing the initial user-input string into your AST (whatever shape that ends up being), which pretty-printed (basically) looks the same as the user-input.
> Heresy I suppose, but doesn't feel (to me) like it "specifically" has to be cons cells, as long as it's a "list" of some sort
Yeah, "cons" is definitely a implementation detail. Maybe the central ideia is just "parenthesized s-expr".
But this is just not true. An utf8 string is most definitely not a bunch of cons cells. Having a one-to-one correspondence is not the same thing, and one might argue that a rust code snippet and the corresponding AST object is just as arbitrarily far apart with a clean one-to-one correspondence.
So like, what's the material difference between the two? You have a parse and an eval in both cases
So, writing an agent in Lisp is interesting but not particularly novel. If there’s a big idea here it’s giving Lisp’s eval function to the AI as a tool. Yes, AIs write Python and Bash all day long already, but those scripts are run out of process. In this case, the AI can modify the process running the harness, extending it directly and potentially changing it (evolving it). That’s powerful. And obviously quite dangerous. Definitely only for a sandbox. But I wonder how far that can go…
There is no good reason for running this in-process. The maximally inefficient Fibonacci function in the article is a good demonstration why: Call it with a slightly larger number, and your agent slows to a crawl, with no way to enforce a timeout. Call it with a slightly larger number yet, and you might bring your agent down entirely, with no way to recover.
There is a case to be made for a dynamically evolving "tool server", but it should be a separate process. That would be more flexible for other use cases too. For example, multiple independent agent processes could talk to one shared tool server. Like a blackboard system, more classic AI!
And if you really do want to evolve the agent itself: As the article observes, its entire state can be serialized. Nothing is gained from hanging on to one particular agent process. Serialize its state, ask the tool server to kill it, rewrite its code, then start the new version and replay the state.
Lately I've been adding a repl service to my docker compose configs so the agent can easily send fragments of code for execution in the project context without incurring the clojure startup costs each time.
So cool to watch the AI get into a tight learning loop when it has access to all the internal data structures.
Clearly, you’re not going to do it for just Fibonacci. And yes, you can always rewrite the agent code, quit, and restart to modify the agent. This is just like using an LLM to modify Pi and then restarting, which is done today. But being able to rewrite code in a live, running system is useful for all the reasons that Lispers have done it that way for 50+ years. In some cases, you can’t just serialize your state and reload it easily. Sometimes, for instance, you might not even have access to the whole state because some of it resides on a remote server.
> Sometimes, for instance, you might not even have access to the whole state because some of it resides on a remote server.
Not sure what you mean. The system I outlined is one where some "state" resides outside the process in a separate server. You don't need to serialize that, you just need to serialize the information to need to reconnect.
And my first point is even more relevant the more complex/distributed/brittle you make the whole thing: The more important it is for some specific process to stay alive no matter what, the less you want to live-slop code into it.
(Edit: Yes I'm aware of the live-patched space probe story. Human live-patching is not the same as letting an LLM try to one-shot the correct patch.)
Now this is AI going full circle, back to their roots. Yeah, I want to see that system working.
Maybe this is a reductive comment, but how does this differ from just letting your agent bash tool a `python -c` command (or anything of that class)? I'm not really getting where this is a "wow" moment?
It is always nice to appreciate how much power you get out of (Model + the absolute bare minimum of control flow). There is just so much baked into the models now that given an inch they will take a mile.
If you give your agent a `bash` or `python -c` tool, it starts a separate process that produces some output and then exits. After that, only the output and the exit code are available.
In contrast, `eval` runs the code in the same execution context as the agent loop. When `eval` finishes, that execution context still exists. For example, any functions defined during an `eval` call remain available for later use.
That’s a distinction in search of a reason though.
Being able to run code in the same unix process or a new one doesn’t really matter all that much in the context of self modifying code. But even if we cared about that, this isn’t a LISP specific feature. All dynamic languages support eval.
And having the agent cache the tool for reuse is a really trivial problem to solve. Though I do agree that LISP makes this much easier than in many other languages.
This is certainly a cool tech demo. But the claims of its novelty are overstated
That's technically true but practically an agent can just save the script file/rerun it/write a tool that lets it call a reply with memory etc.This aspect is a bit more elegant when it's in the execution context, but the core of "you don't need to predefine tools, the agent can dynamically write its own code" is not exactly new - that's pretty much the basis of why Claude code and codex and all the other agent harnesses are so much more effective than old ways if working with llms - they can just write giant incomprehensible bash scripts on the fly to accomplish random things without having pre-defined tools, write state to files, etc
The blog is about writing an agent when you dont already have an agent, but only a plain LLM. It stitches the minimal pieces together. Agents dont need lots of supporting infra, so it is good to keep the code concise. Not a wow moment for sure, though some people think that agents and harnesses are complicated.
Right but given that the agent only outputs text, and agents are perfectly happy writing python, the supposed benefit of Lisp is completely irrelevant here.
> the supposed benefit of Lisp is completely irrelevant here
Yeah, it's a small example (it's in the title, "100 lines") so obviously doesn't highlight the best benefits once you reach larger codebase size.
Still think ~8 lines for the core loop is probably more elegant, readable and concise than you can achieve in other algol/C-like languages, but happy to be shown that I'm wrong :)
Most of the code in agents is to handle unhappy paths and human friendly interfaces. I’m sure you could trim most languages core loop too if you discarded all the stuff that actually make agents peasant to use.
Please do show the equivalent of those ~8 lines in the other languages you think of so we can explicitly compare :) I gave it a try in the other languages I know, the Clojure one I have still ends up the smallest and fastest to comprehend.
But that’s not just 8 lines. If I were to try and run those 8 lines on their own then the compiler would fail because half of those functions are undefined. Even the article itself doesn’t claim the agent is only 8 lines long.
And once you start including the boilerplate code you end up with something that’s a lot more equivalent to the other languages you tried.
I love functional languages, and LISP specifically too. But the point of that article wasn’t even to say “LISP is better at code golfing than other languages”. so this doubling down on the SLOC that you’re doing isn’t even a relevant tangent.
You're not wrong. I did a similar agent in lisp back with Sonnet 3.5 and had a wow moment, but the wow was mostly for seeing an agent working effectively at all at that point in time.
The part that killed it for me was losing everything if the lisp crashed (sonnet 3.5 was prone to doing that) and solving persistence had too many edge cases and confused the model.
Later realized that writing the agent as 20 lines of bash was equivalently powerful to the lisp agent, but made persistence trivial from the easy file system interop.
you can use LFE (Lisp flavored erlang) = lisp on BEAM runtime.
you get a snippet from LLM, compile it to module, and hot-load it into the running node. the module lives in the node's code table, so it persists and every other agent can call it. not just the one that wrote it.
the agents themselves are seaprate supervised processes, so if one crashes - e.g. because the snippet was crap, it doesn't take down whole system.
of course you can do that in just elixir too, the lisp is just cosmetics really.
By my eye it's not that different, it's riffing in it from a Lisp perspective.
It's pretty amazing to write your own agent BTW. I've got a zero-dependency all-in-one-file agent harness I wrote myself. I use it all the time now because I can get it from anywhere and I can know EXACTLY what it'll do (as much as you can with any model), what it's been told vs not. Using it as a harness for models I'm hosting myself makes me feel like some kind of LLM homesteader: it's a set of tools I'll always have that will only change as much as I want it to change.
this is as if python was written in python, and you added support for llm in python so that it can write it's own code
I never realized that when chatting with the bot, the entirety of the session was sent each time. I guess I just figured there was some state maintained Out There, rather than simply resubmitted each time.
This is a thing of beauty, no matter what the general feeling in the comments seems to be against LISP. And yeah, you can do it in JS/TS/Python/etc., but somehow it doesn’t feel as elegant.
> Symbolic AI lost.
Symbolic AI lost? At what, surely not chess?
A symbolic AI solution to a problem requires vastly less energy. And is deterministic; you can cover it with expected input/output pair regression test cases.
Technically the current best chess engine is a neurosymbolic hybrid.
But I have found that sometimes the best use of an LLM is to write code for symbolic AI.
9 lines of python:
nb:
- stdlib only, 0 external dependencies
- works with openai compatible api (including local models)
- shows context usage in % when turn goes back to user
- cache friendly (keeps stable prefix and provides uuid v4 as session cache key)
- uses 'shell' as open ended tool
'shell' as tool name is sufficient context for gpt 5.6 sol
Nice, but this is the equivalent of always running with `--dangerously-skip-permissions` with all the security implications of that.
Which realistically, is the only way to be productive with agents. If you aren't running them inside of something that limits their scope/impact/potential damage, you're already doing it wrong.
Most of those "permission"-systems are built on the idea that an LLM can decide what to ask for approval to run or not anyways, which obviously don't work out great in practice. Might as well give them blanket permission to do whatever, then put them in an isolated environment.
Pi does this by default. Since the frontier lab agents are vibe coded you really should sandbox any agent you run period.
This is beautiful, I can't deny that. But the claim that Claude Code is a fixed set of tools is quite wrong. Claude Code regularly generates new code to carry out tasks, and you can choose to promote those tasks to skills, workflows, etc.
I made an interpreted lisp and stored the AST in postgres and exposed the functions with htmx, and honestly it works pretty great.
Prolog would have been a better choice
why?
it is also possible to sandbox elisp at least. I allow pure functions through by default: https://github.com/elij/macher-agent/blob/main/macher-agent-... -- more an interpreter then straight eval but works as author states
Littered with AI writing tells.
is that wrong though? you write all your code with AI already, but you still need to supervise and steer it. same with blog posts - as long as you steer it towards converying your point, it doesn't matter the text itself was written by AI rather than typed by you, does it?
It doesn't matter if a human or a machine wrote it, it's just unpleasant to read. I don't know what the OP's motivation for their post was, but I think it's reasonable to point out that something is harder to read than it should be.
This doesn't need to be posted on every post. Everyone that cares is well aware. It's like saying "this was written in english" on every post.
Why should I bother reading something that the “author” couldn’t even be bothered to write themselves?
Similarly, why should I be bothered reading this kind of comment in each and every discussion thread?
What does it contribute? I can read and discern this for myself, I can then stop reading or decide I don’t care.
Seriously, at some point all you “ai writing sleuths” should just get your own discussion thread together. It’s been months of this , we get it already.
(Not directly just at you, but anyone who feels the need to drop these comments in every thread)
> why should I be bothered reading this kind of comment in each and every discussion thread?
To help other people who don't care about it so they can skip it. A subject tag would be nicer.
> It’s been months of this , we get it already.
And I am tired of the constant barrage of AI proselytizing.
I'm not saying you should, I'm saying you shouldn't clutter up every thread with this type of useless comment. It just detracts from the people that do want to engage with it.
he's sharing an idea, not "text". text is just a medium.
> No framework. No state machine. The agent’s state is just ...
Indeed. The new language, llmish, has definitely been used.
Still a good TFA IMO.
P.S: this particular construct "No X. No Y. ... just ..." is half my LinkedIn feed.
Actually, not really. The vocabulary is sophisticated, but there isn’t a single “the honest value” or A bunch of itemized bold fluff.