LLM is just data, there's no code there in the input. It's recursive, but there's no real manipulation that happens at the higher level. The separation between code and data in lisp is intentional. The programmer already knows that something is code, but for now we're manipulating it as data. You don't have the same distinction with LLMs.
Metaprogramming stems from the same thought. You think of an algorithm, discern some part that are contextual, and create an higher level algorithm that takes the context is account. It's not merely creating programs that write programs, it's about solving class of problems instead of a specific instance. For LLMs to fit this pattern, it would be like going from "A prompt that write a specific ffmpeg invocation" to "A prompt that create a prompt to write a specific ffmpeg invocation". I believe classifying problems is more difficult that solving one.
I'm not that familiar with CL's condition system, but from what I know, it works well because the language is evaluation/reduction based instead of stack/heap/counter based like C. This allows discarding branches more easily because there's no mutable memory to alter. Can you discard context that easily in LLMs? I think agents and subagents with IPC would fit that definition more easily.
LLM is just data, there's no code there in the input. It's recursive, but there's no real manipulation that happens at the higher level. The separation between code and data in lisp is intentional. The programmer already knows that something is code, but for now we're manipulating it as data. You don't have the same distinction with LLMs.
Metaprogramming stems from the same thought. You think of an algorithm, discern some part that are contextual, and create an higher level algorithm that takes the context is account. It's not merely creating programs that write programs, it's about solving class of problems instead of a specific instance. For LLMs to fit this pattern, it would be like going from "A prompt that write a specific ffmpeg invocation" to "A prompt that create a prompt to write a specific ffmpeg invocation". I believe classifying problems is more difficult that solving one.
I'm not that familiar with CL's condition system, but from what I know, it works well because the language is evaluation/reduction based instead of stack/heap/counter based like C. This allows discarding branches more easily because there's no mutable memory to alter. Can you discard context that easily in LLMs? I think agents and subagents with IPC would fit that definition more easily.
Thank you for boiling it down to essentials.
Lisp is all about the read-eval-print loop (REPL). John McCarthy's first lisp paper focused on implementing function EVAL in terms of "itself".
Maybe with LLM "attention" this is a new and perhaps lossy take on eval via predicting text tokens.
Cool thought. Thank you.