Can someone ELI5 or point me to an idiots guide as to why macros are desirable? As a non professional programmer who work on small side projects I'm struggling to understand what benefit they bring.
Generally meta features like macros are _far_ more useful to library authors that need to do some sort of "reflection" on your application code.
Serialization libraries are frequently the largest beneficiary.
In my ideal world I don't need to write macros or use language meta/reflection features while (only) writing code to solve business problems, but to get to that ideal often the libs you use do need reflection capabilities
I'd argue that it's entirely reasonable for the average application/service developer to not appreciate the usefulness of macros. Not everyone needs to be a foundational library author, honestly we probably need fewer (looking at you npm/cargo micropackages)
There's also the Lisp philosophy which emphasizes that you are your own libraries' author, too! Third-party libraries can only help you with common business code - anything specific to your domain, or your business, you have to model yourself. Using macros is fundamentally not different than using classes or functions to create an environment[0] where expressing your business logic is straightforward, and invalid states are not representable. Metaprogramming just lets you go further in this direction than "traditional" tools languages offer.
--
[0] - The correct term here is Domain-Specific Language, but by now it's badly overloaded and people have knee-jerk reactions to it...
Your language doesn't have pattern matching? It doesn't have Python's "with" or do...while? You want to generate an enum from a CSV file? Want to add a useful abstraction specific to your domain? How do you implement short-circuiting yourself (i.e. `A && B` only evaluating B when A succeeds) without thunking?
One of the main use-cases for compile-time metaprogramming (like macros) has been to be able to write performant code that does not type-check correctly in a typed language. Library writers encounter this issue frequently, e.g. the C++ standard library is heavily based on template metaprogramming. One example of code we want to write in a generic way are map and flatMap operations that you can define on lists, binary trees, hashmaps, rose trees and many other container-like data structures. But many typing system do not let you write the map and flatMap abstraction in a type-safe way once and for all. In dynamically typed languages, there is no such issue.
Some modern languages (Haskell, Scala) overcome the lacking expressivity for library writers with higher-kinded types and principled support for ad-hoc polymorphism (e.g. typeclasses), thus reducing the need for meta-programming. Notably, Haskell and Scala have unusually principled support for metaprogramming.
As a heuristic, I would suggest that using metaprogramming for small or medium sized normal ("business") code is a sign that something maybe be suboptimal, and it might be worth considering a different approach (either to the choice of implementing business logic or the chosen programming language.)
Macros are capable of much the same thing as reflection, but without the reflection part. Anything you would use an annotation/decorator/attribute/etc for, can be done as a macro to eliminate the runtime performance impact and the vector for action-at-a-distance. In addition to removing internal copy-pasting/templating when you just can't quite replace it with an interface instead.
lexing -> parsing/macro expansion -> compilation of core language
The macro expansion phase removes all uses of macros and produces a
program in the core language (which has no macros).
The idea is that a user can extend the language with new features.
As long as a program that uses the new feature can be transformed
into an equivalent program that doesn't use the feature, it can
be implemented with a macro transformation.
Macros are needed when the user wants a construct that:
- uses a non-standard evaluation order
- introduces new bindings
- remove boiler-plate (not covered by functions)
- analyzes program elements at compile time
Non-standard evaluation order
What the programmer can use macros depend on how powerful the macro system is.
Reasonable macro systems can be used to implement pattern matching (non-standard evaluation order) and object systems. Implementing, say, pattern matching as a macro
has the advantage that it can be done within the language without changing the compiler.
General constructs such as pattern matching are usually provided by the standard library
of a language - but users are free to experiment with their own versions if they have special needs.
Removing boiler plate
In my 6502 emulator I have two macros `define-register` and `define-flags`.
This allows me to define the registers of the CPU as:
(define-register A) ; accumulator ( 8 bit)
(define-register X) ; index register ( 8 bit)
(define-register Y) ; index register ( 8 bit)
(define-register SP) ; stack pointer ( 8 bit)
(define-register PC) ; program counter (16 bit)
And the individual flags of the status register are defined as:
(define-flags
(C 0 carry) ; contains the result affecting the flag (set if result < #0x100 )
(Z 1 zero) ; contains the last byte affecting the flag
(I 2 interrupt) ; boolean
(D 3 decimal-mode) ; boolean
(B 4 break) ; boolean
(U 5 unused) ; true
(V 6 overflow) ; 0 or 1
(S 7 sign)) ; contains the last byte affecting the flag
Note that macro expansion happens on compile time. Thus there is no overhead at runtime.
Notes
The convention in languages with macros is to use them only if functions can't do the job.
Since macros follow non-standard evaluation order, macros need to be documented carefully.
Over time one finds the balance of when to use them and when not to.
[In the beginning everything looks like a nail.]
Language exploration
An often overlooked positive effect of macros is that the entire community can experiment with new language features.
It's no longer just the "compiler team" that have the ability to add new features. This means that iteration of language design happen faster.
> An often overlooked positive effect of macros is that the entire community can experiment with new language features.
Sure, that sounds positive - but with enough macros, you can turn "your" version of the language into something completely unrecognizable to people that are only familiar with the "basic" version (or, otherwise said: congratulations, you've got yourself a DSL!), and I would say that's a rather negative effect...
TeXInfo is a plain-TeX system that rewrites the parser to a completely new syntax, whilst maintaining compatibility with importing and linking to other libraries. It is extensively used, especially in emacs.
Google announced 2 years ago they would be adding "macros" support to the Dart language. They stopped that work yesterday. As former Eng Director of Dart (and co-founder of Flutter) I've offered my hot-take.
Can someone ELI5 or point me to an idiots guide as to why macros are desirable? As a non professional programmer who work on small side projects I'm struggling to understand what benefit they bring.
Generally meta features like macros are _far_ more useful to library authors that need to do some sort of "reflection" on your application code.
Serialization libraries are frequently the largest beneficiary.
In my ideal world I don't need to write macros or use language meta/reflection features while (only) writing code to solve business problems, but to get to that ideal often the libs you use do need reflection capabilities
I'd argue that it's entirely reasonable for the average application/service developer to not appreciate the usefulness of macros. Not everyone needs to be a foundational library author, honestly we probably need fewer (looking at you npm/cargo micropackages)
There's also the Lisp philosophy which emphasizes that you are your own libraries' author, too! Third-party libraries can only help you with common business code - anything specific to your domain, or your business, you have to model yourself. Using macros is fundamentally not different than using classes or functions to create an environment[0] where expressing your business logic is straightforward, and invalid states are not representable. Metaprogramming just lets you go further in this direction than "traditional" tools languages offer.
--
[0] - The correct term here is Domain-Specific Language, but by now it's badly overloaded and people have knee-jerk reactions to it...
Your language doesn't have pattern matching? It doesn't have Python's "with" or do...while? You want to generate an enum from a CSV file? Want to add a useful abstraction specific to your domain? How do you implement short-circuiting yourself (i.e. `A && B` only evaluating B when A succeeds) without thunking?
Read https://gigamonkeys.com/book/macros-standard-control-constru...
One of the main use-cases for compile-time metaprogramming (like macros) has been to be able to write performant code that does not type-check correctly in a typed language. Library writers encounter this issue frequently, e.g. the C++ standard library is heavily based on template metaprogramming. One example of code we want to write in a generic way are map and flatMap operations that you can define on lists, binary trees, hashmaps, rose trees and many other container-like data structures. But many typing system do not let you write the map and flatMap abstraction in a type-safe way once and for all. In dynamically typed languages, there is no such issue.
Some modern languages (Haskell, Scala) overcome the lacking expressivity for library writers with higher-kinded types and principled support for ad-hoc polymorphism (e.g. typeclasses), thus reducing the need for meta-programming. Notably, Haskell and Scala have unusually principled support for metaprogramming.
As a heuristic, I would suggest that using metaprogramming for small or medium sized normal ("business") code is a sign that something maybe be suboptimal, and it might be worth considering a different approach (either to the choice of implementing business logic or the chosen programming language.)
Macros are capable of much the same thing as reflection, but without the reflection part. Anything you would use an annotation/decorator/attribute/etc for, can be done as a macro to eliminate the runtime performance impact and the vector for action-at-a-distance. In addition to removing internal copy-pasting/templating when you just can't quite replace it with an interface instead.
You can think of macros as a compiler frontend.
The compilation process is more or less:
The macro expansion phase removes all uses of macros and produces a program in the core language (which has no macros).The idea is that a user can extend the language with new features. As long as a program that uses the new feature can be transformed into an equivalent program that doesn't use the feature, it can be implemented with a macro transformation.
Macros are needed when the user wants a construct that:
Non-standard evaluation orderWhat the programmer can use macros depend on how powerful the macro system is. Reasonable macro systems can be used to implement pattern matching (non-standard evaluation order) and object systems. Implementing, say, pattern matching as a macro has the advantage that it can be done within the language without changing the compiler. General constructs such as pattern matching are usually provided by the standard library of a language - but users are free to experiment with their own versions if they have special needs.
Removing boiler plate
In my 6502 emulator I have two macros `define-register` and `define-flags`.
This allows me to define the registers of the CPU as:
And the individual flags of the status register are defined as: Note that macro expansion happens on compile time. Thus there is no overhead at runtime.Notes
The convention in languages with macros is to use them only if functions can't do the job. Since macros follow non-standard evaluation order, macros need to be documented carefully. Over time one finds the balance of when to use them and when not to. [In the beginning everything looks like a nail.]
Language exploration
An often overlooked positive effect of macros is that the entire community can experiment with new language features. It's no longer just the "compiler team" that have the ability to add new features. This means that iteration of language design happen faster.
> An often overlooked positive effect of macros is that the entire community can experiment with new language features.
Sure, that sounds positive - but with enough macros, you can turn "your" version of the language into something completely unrecognizable to people that are only familiar with the "basic" version (or, otherwise said: congratulations, you've got yourself a DSL!), and I would say that's a rather negative effect...
TeXInfo is a plain-TeX system that rewrites the parser to a completely new syntax, whilst maintaining compatibility with importing and linking to other libraries. It is extensively used, especially in emacs.
Having a DSL, isn't necessarily a bad thing.
This is no different from having competing libraries for dates, regular expressions, html parsing etc.
That's the reason why many prefer "batteries-included" languages that provide a standard implementation for all the basic stuff...
Related:
Google discontinuing work on Dart macros
https://news.ycombinator.com/item?id=42871867
Google announced 2 years ago they would be adding "macros" support to the Dart language. They stopped that work yesterday. As former Eng Director of Dart (and co-founder of Flutter) I've offered my hot-take.