It would be nice to have a measure of how much a function changed. A change can be meaningful yet leave the overall shape of a function unchanged. Fixing a slightly off calculation or a corner case, adding a case for a new feature...
At a glance the analysis seems like it could be interesting, but digging in, I'm struggling with the article.
First of all, I can only take so many "the hard case", "the fun part" and "the real test" before you've exhausted my attention to what these phrasings suggest are exceptionally interesting things.
Second,
> Scramble the Go program's wiring, keeping how many calls each function makes, and it drops to 39%: the wiring itself carries much of the signal.
What wiring? What signal? The rest of the article refers frequently to these completely useless analogies without ever clarifying what they mean. After scratching my head over it for a while I'm only guessing that the "wiring" refers to function names and "signal" to their bodies? Why not refer to these very basic, simple concepts as they are? I could understand coming up with analogies (good ones though, not awful ones like these) if you were writing for a non-technical audience, but you're not.
> Across languages only the wiring carries a signal, and only for the closer translations.
WTF does this even mean?
> he Scheme and Go translations keep the original's wiring as far above chance as Chocolate Doom does. The Dart translation keeps some. The Fortran, Java, Kotlin and C# translations keep none, and programs written from scratch, or not related to DOOM at all, sit at chance, as they should. Don't take our word for it: draw the maps yourself.
Sit at chance? Why shouldn't I trust your word for it if I can trust your diagrams?
Please, if you want me to think something is interesting, the least convincing thing you can do is to offload writing about it to whatever trite generator shat this out.
It's funny how the quality of LLM writing is regressing (yes, regressing!) - I don't mean to say that it was very good at any point. Who signed off on that shit?
Authors need to realize that readers aren't simply evaluating the information in your writing. They are evaluating your authority as a narrator, your expertise in the subject, and many other attributes.
.claude/skills/popsci-interactive-article/SKILL.md isn't always enough to get you there.
DOOM draws very little. An entire level is 3000-5000 polygons and on the screen, it rarely draws more than 1500 (polygon based ports)
Whatever it's architecture is, porting in is trivial because it's not a hard problem.
It's a great game and its geneious is undeniable. But porting a 1993 game is not really going to tell you anything interesting about "How code structure survives across 44 ports".
And, I like the article, once I got through the 'jolt'. (Not gonna call it slop, I can tell it a good amount of effort on your part to make it present the data well.)
I love Doom, and I love the genealogy of it; dealing with doom modding and dehacked was probably my first foray into any sort of concept of 'programming' (after all, you could do some curious things with dehacked, as far as 'next frame' and how that sometimes could change things in curious ways)
I think, to give some constructive feedback;
1. Consider taking a markdown/output from analysis and writing your own starting scaffold.
2. Make sure to review the output and tweak it.
3. Hand refine what the end result is.
I'm not saying you shouldn't use AI to help with some of this, I'm just saying, that will help so much more. Cause this is really cool, but yeah it would be nice to have more polish around it.
If you were to make one edit to Claude's output here, I'd recommend rephrasing "of DOOM's shape shows up in Quake, Quake II and Lua" near the top; though it's explained slightly better later on, the wording seems to imply that Lua is derived from the id Tech family tree, which it obviously isn't. An explicit mention that these are negative controls would be good. Maybe rewrite it to "chance resemblance to three unrelated codebases (Quake, Quake II, and Lua)." Personally I find it fascinating that there's a tangible lineage between Keen 1 and Doom 3. (id used the same tile map editor and file format until Doom.)
I actually worked really hard on this. I'm feeling kinda low about the response. I know your some random person but if you have constructive criticism, i'm all ears.
I'm sure you did work hard on this, and I also find it very interesting from a technical perspective.
One constructive point for you:
don't even write the first draft of a document you want humans to read with AI. It's much harder to get the AI voice out than it seems. Stuart Halloway talked about this at the last Clojure Conj, and I couldn't agree more. It also makes it harder to follow the sage writing advice to minimize what you write.
"How code keeps its family resemblance" - AI would say this, a person would not. It's just noise on the page.
"Measure only their shape and you can still tell who copied whom" - this is a sentence structure Claude loves, but no human would talk about "measuring" shapes. The verbs and the nouns don't feel right.
Then you start a paragraph: "DOOM is the perfect place to test it." But you don't actually say what "it" is specifically. It doesn't seem like AI would write that though... Maybe you did in a later draft? It's hard to switch voices, like switching authors.
If you want people to read something, write it yourself.
I'd rather see shitty copy from someone who knows what they're doing because article writing isnt their stength, than an AI write up. I don't think the latter will ever suit your audience.
Just use less LLMs for presentation and writing. The text and graphical elements just scream "Opus 5.5" to me, perhaps because I use it a lot these days, I assume sol and astra are not that different.
At this point I am not sure I wouldn't get a very similar output if I just prompt Claude for the genealogy of Doom ports as an interactive dashboard. Maybe try to deviate from that, even if that would rank lower on a model benchmark: you don't have to provide an interactive data visualization for everything. You can narrate a story from your own opinionated pov.
I don't necessarily agree with the parent, but I don't fully understand your response. Was this largely written via LLM, or no? (I don't believe LLMs preclude the notion of working hard)
Edit: And if so, was it for help with English? Or another reason?
All the obvious AI generated headings is a good place to start. It may seem clever at first but it's a big giveaway it's AI generated and just turns off a large group of people.
Sometimes less is more. This page reads more like a DevOps dashboard than it does an entertaining article about DOOM. It's just overbearing right from the jump and it's a turn off. LLMs are notoriously bad at this, so this just comes off as someone prompting an LLM and posting the output as-is (even if that's not the case).
I concur with the other comment that it's an interesting idea, but you should just write the article yourself. You can even link to the technical details so that those who are interested in exploring that data can do so. But as-is it's really a good example of AI slop.
I appreciate that you liked the premise. The repo has my full methodology and data for scrutiny. I thought the voice was alright, just a straight description of the data.
Author here. Fair hits on the writing; I leaned on an LLM too heavily for the prose. The data and method are mine, and they're all in the repo. Happy to answer questions about how the extraction works.
Thanks for being candid about it. LLM text usually is too verbose and unfocused to make it an entertaining reading. For me it would be better to listen to your opinions and ideas that to get a longer more refined prose by an LLM.
It’s truly unbearable to read. I wouldn’t even know what to ask. Keep in mind the end result of a project like this is an entertaining and informative article about code history.
It would be nice to have a measure of how much a function changed. A change can be meaningful yet leave the overall shape of a function unchanged. Fixing a slightly off calculation or a corner case, adding a case for a new feature...
Copy is written by Claude. No thanks
At a glance the analysis seems like it could be interesting, but digging in, I'm struggling with the article.
First of all, I can only take so many "the hard case", "the fun part" and "the real test" before you've exhausted my attention to what these phrasings suggest are exceptionally interesting things.
Second,
> Scramble the Go program's wiring, keeping how many calls each function makes, and it drops to 39%: the wiring itself carries much of the signal.
What wiring? What signal? The rest of the article refers frequently to these completely useless analogies without ever clarifying what they mean. After scratching my head over it for a while I'm only guessing that the "wiring" refers to function names and "signal" to their bodies? Why not refer to these very basic, simple concepts as they are? I could understand coming up with analogies (good ones though, not awful ones like these) if you were writing for a non-technical audience, but you're not.
> Across languages only the wiring carries a signal, and only for the closer translations.
WTF does this even mean?
> he Scheme and Go translations keep the original's wiring as far above chance as Chocolate Doom does. The Dart translation keeps some. The Fortran, Java, Kotlin and C# translations keep none, and programs written from scratch, or not related to DOOM at all, sit at chance, as they should. Don't take our word for it: draw the maps yourself.
Sit at chance? Why shouldn't I trust your word for it if I can trust your diagrams?
Please, if you want me to think something is interesting, the least convincing thing you can do is to offload writing about it to whatever trite generator shat this out.
It's funny how the quality of LLM writing is regressing (yes, regressing!) - I don't mean to say that it was very good at any point. Who signed off on that shit?
You're absolutely right!
Authors need to realize that readers aren't simply evaluating the information in your writing. They are evaluating your authority as a narrator, your expertise in the subject, and many other attributes.
.claude/skills/popsci-interactive-article/SKILL.md isn't always enough to get you there.
DOOM draws very little. An entire level is 3000-5000 polygons and on the screen, it rarely draws more than 1500 (polygon based ports)
Whatever it's architecture is, porting in is trivial because it's not a hard problem.
It's a great game and its geneious is undeniable. But porting a 1993 game is not really going to tell you anything interesting about "How code structure survives across 44 ports".
... I think this is a fun experiment.
And, I like the article, once I got through the 'jolt'. (Not gonna call it slop, I can tell it a good amount of effort on your part to make it present the data well.)
I love Doom, and I love the genealogy of it; dealing with doom modding and dehacked was probably my first foray into any sort of concept of 'programming' (after all, you could do some curious things with dehacked, as far as 'next frame' and how that sometimes could change things in curious ways)
I think, to give some constructive feedback;
1. Consider taking a markdown/output from analysis and writing your own starting scaffold.
2. Make sure to review the output and tweak it.
3. Hand refine what the end result is.
I'm not saying you shouldn't use AI to help with some of this, I'm just saying, that will help so much more. Cause this is really cool, but yeah it would be nice to have more polish around it.
That makes sense, thanks for the input.
I fondly remember getting the DOOM hackers guide from barnes and noble in HS. Those were the days.
If you were to make one edit to Claude's output here, I'd recommend rephrasing "of DOOM's shape shows up in Quake, Quake II and Lua" near the top; though it's explained slightly better later on, the wording seems to imply that Lua is derived from the id Tech family tree, which it obviously isn't. An explicit mention that these are negative controls would be good. Maybe rewrite it to "chance resemblance to three unrelated codebases (Quake, Quake II, and Lua)." Personally I find it fascinating that there's a tangible lineage between Keen 1 and Doom 3. (id used the same tile map editor and file format until Doom.)
I remember burning a ream of paper printing the DooM FAQ
Interesting project. Out of curiosity, have you done more work studying the engine, eg, an architecture breakdown or programming pattern analysis?
Thanks for the interest! I've done some work comparing code signatures between Claude and codex on the same tasks. This was also written heavily by llm (I haven't had a chance yet to try some rewrites on my own based on this feedback yet)...https://github.com/squid-protocol/agent-habits: comparing results across languages, https://squid-protocol.github.io/gitgalaxy/03-04-claim-4-com... and k means clustering of files by scaffold metrics https://squid-protocol.github.io/gitgalaxy/03-05-claim-5-fil....
> …survives…
Slop.
[flagged]
I actually worked really hard on this. I'm feeling kinda low about the response. I know your some random person but if you have constructive criticism, i'm all ears.
I'm sure you did work hard on this, and I also find it very interesting from a technical perspective.
One constructive point for you: don't even write the first draft of a document you want humans to read with AI. It's much harder to get the AI voice out than it seems. Stuart Halloway talked about this at the last Clojure Conj, and I couldn't agree more. It also makes it harder to follow the sage writing advice to minimize what you write.
"How code keeps its family resemblance" - AI would say this, a person would not. It's just noise on the page.
"Measure only their shape and you can still tell who copied whom" - this is a sentence structure Claude loves, but no human would talk about "measuring" shapes. The verbs and the nouns don't feel right.
Then you start a paragraph: "DOOM is the perfect place to test it." But you don't actually say what "it" is specifically. It doesn't seem like AI would write that though... Maybe you did in a later draft? It's hard to switch voices, like switching authors.
If you want people to read something, write it yourself.
Thanks for your time, yeah, I can totally see it now.
I'd rather see shitty copy from someone who knows what they're doing because article writing isnt their stength, than an AI write up. I don't think the latter will ever suit your audience.
Just use less LLMs for presentation and writing. The text and graphical elements just scream "Opus 5.5" to me, perhaps because I use it a lot these days, I assume sol and astra are not that different.
At this point I am not sure I wouldn't get a very similar output if I just prompt Claude for the genealogy of Doom ports as an interactive dashboard. Maybe try to deviate from that, even if that would rank lower on a model benchmark: you don't have to provide an interactive data visualization for everything. You can narrate a story from your own opinionated pov.
I don't necessarily agree with the parent, but I don't fully understand your response. Was this largely written via LLM, or no? (I don't believe LLMs preclude the notion of working hard)
Edit: And if so, was it for help with English? Or another reason?
All the obvious AI generated headings is a good place to start. It may seem clever at first but it's a big giveaway it's AI generated and just turns off a large group of people.
You should re-evaluate what you consider to be hard work if prompting a chat bot is your idea of it.
Sometimes less is more. This page reads more like a DevOps dashboard than it does an entertaining article about DOOM. It's just overbearing right from the jump and it's a turn off. LLMs are notoriously bad at this, so this just comes off as someone prompting an LLM and posting the output as-is (even if that's not the case).
I concur with the other comment that it's an interesting idea, but you should just write the article yourself. You can even link to the technical details so that those who are interested in exploring that data can do so. But as-is it's really a good example of AI slop.
I found this interesting.
I appreciate that you liked the premise. The repo has my full methodology and data for scrutiny. I thought the voice was alright, just a straight description of the data.
Author here. Fair hits on the writing; I leaned on an LLM too heavily for the prose. The data and method are mine, and they're all in the repo. Happy to answer questions about how the extraction works.
Thanks for being candid about it. LLM text usually is too verbose and unfocused to make it an entertaining reading. For me it would be better to listen to your opinions and ideas that to get a longer more refined prose by an LLM.
It’s truly unbearable to read. I wouldn’t even know what to ask. Keep in mind the end result of a project like this is an entertaining and informative article about code history.