>On my visits to the Bay Area, I would ask AI researchers or interns why they are doing their current research or projects, when in a year or three agentic LLMs could probably do them;
This is such a weird point to make that doesn't become correct just because everyone makes it, all the time. Why clean the ocean if some magic future tech will clean them? Why save the world now if some benevolent AI is 'just around the corner' and will do it for us? And people have been making this point for years now, and it's not like my job got any easier. I just got more AI.
And I say that as someone who uses Claude Code in complex environments almost hourly; I, as the human, still have to do the thinking as Claude still 'can't jump' [1] and I have seen no evidence that they (or similar AI, any time soon) will 'jump' like a human brain does.
I think it is a great point to make, because if everyone really believed that AIs will do everything without human intervention in a handful of years, as the marketing repeats again and again (AGI, singularity, etc.) and have been saying for years... why then get bothered?
Because we DO know LLMs have their hallucinations, limitations, perform tasks not previously seen way worse than humans, etc. And it seems that, for now, there is not a good or magic solution to it, it is inherent limitations of the paradigm.
Yes, you can feed more and more and more (curated data) and eventually make AIs excellent at task X or Y, but then you spend your time specializing those engines. So the work does not really disappear, it just shifts and you make it more replicable for a bound set of problems.
Needless to say that at some point I prefer to learn (and combine with AIs, it is ok) than acritically getting inputs from something until I become totally useless.
Unless we have a paradigm for which a fully autonomous AI can do everything, this will just become improving our productivity in some ways, with all the in-between bottlenecks that it has.
It's a very silly point to make to AI researchers specifically. If they don't work on those projects, the AI won't advance and won't magically be able to replicate the work in "one to three years".
Can you imagine scenarios that would make it less silly? I will give an example:
- The AI researcher might be working for a lab or company with much less funds than the top dogs. Are they likely to discover something that is worth it before a bigger model becomes more capable?
Ask the researchers working on Deepseek. They seem to be doing pretty well for themselves.
Just because a AI will be able to do it in the future does not mean that us plebs will be allowed to have access to it. That alone is enough of a reason for smaller labs to keep going; having a seat at the table.
This is such a weird point to make. We are currently ( only ) discovering that paradigm; we are not inventing anything. We found a bunch of laws that produce rather cool results but our paradigm is incomplete which leads more or less wordy or frame-rich weird stuff like hallucinations, singularity and so on ... it's childish, really and on that funny pseudo-profound, pseudo-intellectual, pseudo-spiritual ( personal opinion, if it gets you horny, you go, baby ) "universe consciousness unity, Rick James, bitch" level ...
Our bodies and minds need proper AI, not all the stuff we already outsource to middle and/or passionate men and women. Other species on the planet would certainly like to see us get augmented by AI so we can solve as many survivability issues as possible to keep as many ecosystems running long enough ... whatever that means but whether animals and plants are aware of chance and potential is another philosophical debate.
To individuals, software is a hammer and chisel, a knife, a brush and canvas, pen and paper, a reading help, and to a good amount of people it's a microscope and a fine scalpel.
To collectives, it's a tool to work on consensus and conventions, to share and gather.
It's baby steps for civilizations and it looks like our particular species is gonna get stuck in a puddle of our own monkey shit, with bottles of champagne in our hands and monkeys grinding up and down the few ivory towers in proximity.
> why then get bothered
Humans are on different levels. Most have decided that "nature realized the/a bug and wanted someone dead" or "their survival is a matter of chance" is not acceptable at all and some people decided that sabotage, poison, abuse, rape, murder are acceptable means to get chicken shit ...
The "paradigm" of life is far from explored/discovered, so we simply can't content ourselves with presumptions about inherent limitations of the LLM and AI paradigm for any other reason than to uncover ( not invent ) other parts of the paradigm.
We are happy with what AI can do for us but "AIs will do everything without human intervention" sounds weird because babies are born and the older they get and the less sabotaged ( vs influence, cultural manipulation ) they get to grow up, the more breadth and depth humans want to experience. For this they need to learn and use their hands & fingers. They need to feed body and mind to find what triggers what, and what excitement and curiosity are inherent and which can or need to be added/acquired/experienced extrinsically.
How many associations will we be able to make if AIs will do everything without human intervention?
The Hallucinations are becoming less, significantly by now.
It also might be already were it is cheaper for one of the big few companies to spend millions and billions to teach the LLM / creating the training data necessary for an LLM to do something which it is not yet good enough due to the fact, that they sell this capability then to everyone who wants to use this capabilitiy.
We have not seen the end of Reinforcement Learning, which does need a lot less training data but more compute.
I'm 'vibing' on the side a handfull of small things, no LLM trained on particular what i'm asking to do. Its very capable of stringing together enough things so it can clearly follow handwavy things i tell it to do, analyse error messages, analysing screenshots etc. all by itself.
There is not a single real ceilling in sight, we only have clear barriers like compute but constant fast progress.
The field of mathematics went from 'useless' to 'you start better using it' to 'gamechanger' in how fast? 1 year after coding? less?
I want signes that we hit a real problem, instead I get cheaper tokens, Chinese models becoming very good as open models, new model updates from the others, mathematicans now saying how good it is etc.
Only half a year ago I had to babysit an LLM, now i tell it 1-3 sentences and it just goes and does it. And that stuff runs without compile errors etc.
If AI makes us 10% or 20% betteer, which is not that much, this alone will lead to companies reduing their expensive staff by 10-20%, which will has real impact on a job area. Some jobs are already hard to sell like cyber security and basic image tasks.
Hallucinations were low hanging fruit in some ways. As someone working on a large-ish complex-ish distributed system that has to be maintained and support customers, it's still very high value to have Claude in the mix, but the core problem of needing to monitor, advise, course correct, and make sure you don't end up with more code and complexity than you need is, at least in my experience, still roughly the same. The sharp edges are being filed off very rapidly, but the core experience of "make and maintain a large system" isn't advancing nearly as fast, IMO.
We see AI factories going in this direction but there is no real 'the open source ai platform' thingy.
It needs connectors to integrate with k8s, hyperscalers etc. it needs to be able to have a basic router, a way of configuring expert agents and interaction options for the human in the loop.
There is for sure things we need to build or change, but it def feels like to me that it would immediadly fix a few things today.
I'm not disappointed that it doesn't advance as fast as it feels
> The Hallucinations are becoming less, significantly by now.
Yes? What is the mega-solid technique that is used for it? Armies of people using curated data and reviewing it by hand? That is exactly one of my points: shifting the work elsewhere for specialized tasks. More replicable, improved, but, it scales infinitely and is autonomous? Can you assert that?
I am not denying there is some use (a lot of uses!) for this, but this is more nuanced than just: oh, they will replace us. Not at all, that day, with the current technology, is not going to arrive. This is just a systematization, fitting and tweaking of human knowledge by curated data. It is not the one true superintelligence they are selling us. To begin with, they do not have a concept of truth, but of probabilistic truth. Only that poses already a very, very big problem for the path to perfection.
> We have not seen the end of Reinforcement Learning, which does need a lot less training data but more compute.
Noone said the opposite, but I would like to know at which cost and if it is feasible. We do not have even enough compute power for current technology.
> . Its very capable of stringing together enough things so it can clearly follow handwavy things i tell it to do, analyse error messages, analysing screenshots etc. all by itself.
I use it every day for these tasks and it works well BECAUSE I review the output and makes me go faster. It finds a lot of things I would have not found and it also hallucinates another handful of them, which confirms my point about AIs not being able to be fully autonomous in any future point in time unless tweaked exactly for the task, and even then, it can still miss judgement a human could have for edge cases. So I am not sure of how bad or good it can be compared to a human but I am pretty sure it cannot be more reliable than an expert in many situations.
> Chinese models becoming very good as open models
I think they will be better in the long term if they follow this path. Not absolutely better but when mixing with economics and the fact that no frontier model is totally reliable anyway... why pay a lot for something that needs human inspection anyway?
> There is not a single real ceilling in sight, we only have clear barriers like compute but constant fast progress.
The ceiling is the paradigm itself, as I mentioned above. There is not a single chance with current technology that something could become "generically knowledgeable" and "reliable" both at the same time. If it becomes generically knowledgeable and reliable, it is bc of data fed into it and curated and tweaked by humans. This is not an original idea from myself, there are armies of people doing this every day around the world, you can check. This is where a lot of improvement comes from. Can this be reused? Of course. It is a generic solution? No way.
> Only half a year ago I had to babysit an LLM, now i tell it 1-3 sentences and it just goes and does it. And that stuff runs without compile errors etc.
Yes, I also do one-off scripts like this and code snippets, even reviews and others. Now go design a full distributed system. Use agents if you want. We come back in six months and compare it to a system that was properly written and tested by humans and we can compare the quality on some grounds:
1. how long it takes to add new features?
2. which ones act more according to spec once added?
3. when adding features, which ones have more bugs?
4. in the face of an error, will the agent delete my whole AWS infra (count the money losses if possible also)?
5. will I understand (or need to understand, but I bet yes) this code at some point in the future?
You have to count all that money also, not just I vibe coded something and it seemed to work. With full systems things become super messy. Now add the human factor of requirements and back and forth (iterations can be admittedly faster with AI, especially prototypes, but that comes with other costs also)...
> shifting the work elsewhere for specialized tasks. More replicable, improved, but, it scales infinitely and is autonomous? Can you assert that?
I would say yes and it will scale. It will either happen through central LLM just paying for it and scaling it up to everyone on the planet (literaly) OR by the agentic layer every big business is building into their systems.
You needed some human to use your tool optimized for their company? With agentic layer you no longer need this. And if you look at companies like Google, they were pushing this notion for ages already because they saw an adoption problem of more 'complex' tools and trying to make it simpler and easier. Now you can act from the other side too.
> We do not have even enough compute power for current technology.
Exactly. Right? There is no ceiling if its clear that we don't even have the hrdware. But the hardware is a bottleneck not a ceiling.
> The ceiling is the paradigm itself, as I mentioned above. There is not a single chance with current technology that something could become "generically knowledgeable" and "reliable" both at the same time.
It doesn't need to be perfect, it only needs to be better than the avg human. And the current LLMs are already better than aat least 1-2 people in my team.
> Now add the human factor of requirements and back and forth
Yeah for now. Grill me skill made it a lot easier. Harness engineering is also being worked on, agentic layer, ai factories etc.
And all of this can be copy and pasted. There is only one harness needed which becomes the expert security reviewer and tomorrow everyone can have it.
I'm still discusing progress with LLMs with people and still not everyone is using it or playing around with harnesses or developgn an agentic layer. We still have a lot of work to do to even see how good it will become while it already is really good.
People are already borred of AI today and making wrong decisions based on the current level of AI while i think we will see continues progress for years.
So you mean AI will be useful for any general job without lots of training for those jobs? How about new tasks? Tasks it has not been tweaked for. When I deviated from the average, and not really weird things, when programming, the output was way worse than average stuff. And this is an explicit target of AIs nowadays.
I think you are missing a lot of details here, honestly.
> Exactly. Right? There is no ceiling if its clear that we don't even have the hrdware. But the hardware is a bottleneck not a ceiling.
No, the hardware is a bottleneck, the paradigm as we know it is a ceiling unless you massively and continuously feed this system with average tasks (which is useful). Which is exactly the opposite of what singularity and AGI have been promising.
The systems we have now (unless the paradigm changes) will keep doing, essentially, fitting. No concept of truth and limited inference. That inference is based on already existing data, not on future data. In fact, there have been experiments about feeding output back to the input of LLMs and the degradation of the quality is very visible. If they are supposed to be so "intelligent", why it happens?
> It doesn't need to be perfect
I can agree that for lots of tasks it does not. But for others it is just not a tool good enough.
> Yeah for now. Grill me skill made it a lot easier. Harness engineering is also being worked on, agentic layer, ai factories etc.
I will not deny there could be progress, but nothing similar to "autonomous", "reliable", "super intelligence" or "singularity" with this paradigm.
In fact, often in my experience, this is a waste of tokens for subpar results that shift the technical debt elsewhere. I mean if you try to develop full systems by "vibe-code like" techniques. If you use them judiciously, you can accelerate your workflow, maybe 2x, but not much beyond that if you want to have something worth to be used. Note that here I am talking about the full thing: with testing, quality, maintenance concerns and everything together.
If you want to ship a sub-par thing that will go to the rubbish in a couple of months, then yes, you can do that. But that will fail commercially any way. Unless your job is convincing enough people that you can go 10x faster every time, deliver some sub-par thing, and find another customer, which, to me, would equal a scam.
> So you mean AI will be useful for any general job without lots of training for those jobs? How about new tasks?
I'm pretty sure we will solve this issue. Either already through World Models or another architecture.
It could also be, that we just need a 10 or 100 Trillion Parameter model to match so many generic ways of solving tasks and keeping the concept in the LLMs 'head' to solve it that it will just emerge with parameter size. Like with fable they said that it can chain together exploits which wouldn't work as standalone exploits.
What if the only real barrier is the depth of understanding of concepts and this is exactly what is getting solved with parameter count?
But look how young this field really is if you start counting it when it became relevant on mass. Its not 'just' an LLM which is changing the world, its machine learning overall. Robotics wouldn't be were it is today if its not for machine learning. Took humans time and energy to take the leap, to start learning what the status quo is and then actually doing more with it.
> the paradigm as we know it is a ceiling unless you massively and continuously feed this system with average tasks
While I do think its doable to achieve AGI in 5-15 years, even if it doesn't happen and it always means that people train an LLM or whatever, if you need 10 experts to teach this to an LLM OR every single senior has to teach this to their juniors every single time, the LLM will always win.
I'm now team lead for 10 years and every single year I teach them the same thing over and over and over again.
the craziest thing about this? If i wouldn't tell them what they are doing wrong, they wouldn't even know it.
Quality is already a very flexible term for a lot of people.
> I can agree that for lots of tasks it does not. But for others it is just not a tool good enough.
Yet.
> If you want to ship a sub-par thing that will go to the rubbish in a couple of months, then yes, you can do that. But that will fail commercially any way.
Now we come to the reality: I have seen so much garbage software its crazy. People using md5 as a password hash in 2024! No clue what coding best practices are, teams without code review, teams without a security expert not even knowing what crazy things they do day in day out.
Just a few month ago a team build an API for my team including a Swagger UI. Half of it didn't work. You pressed a button on the Swagger UI and a 500 returned.
And do'nt underestimate what it means that a lot of business people don't like software people. You know that fruit basket we get? and water and stuff? they don't do it because they like us they do it because thats what you have to do. if a Product Owner starts vibe coding with AI, he will have leadership convinved in no time, then it goes on production and it will run for waaaaaay longer than anyone would have guest.
Besides that there is plenty of software were complexity is less relevant or security is not that big of an issue.
> I'm pretty sure we will solve this issue. Either already through World Models or another architecture.
Please elaborate. How? With which technique? Currently the only path forward is to feed more data and tweak for specific situations (fitting, basically). How does that help in the general case or in new situations with current tecchnology (LLMs, concretely). Noatter how far you get, this is not a general or reliable solution. It van only simulate more generality or more reliability by training and tweaking. Nothing else. At least, with this paradigm.
This does not mean they will not be useful. What I challenge here is the AGI or singularity. We are far from that.
> I'm now team lead for 10 years and every single year I teach them the same thing over and over and over again.
I have been a lead and an architect also for years at different position. I think you miss how much tacit knowledge and judgement there is inside the brains of each of us that an LLM is not capable of. And if it is, then you have to dumo so much context that it is better to go do it yourself. There is a cost to that also actually. It is not just so "dry and technical" the knowledge. Maybe yes to learn Java patterns or C++ constructors or the like.
But not for "given this situation with all these specifics", which solution would you bet on? Probably the LLM will give you a shitty REST API that is not what u need at all.So u tell the AI. It gives u something else generati g 30-50% of "decorated code". Now it seems to workso you use it. Now you do this every day. Come back in 2 months. You generated a lot of fat.
Now you have a bug. You do not know even where to start. Thisis theprice to payfor speed, as usual: technical debt.
Now you tell me you put three agents to talk and burn 2000 usd in tokens. Great! Is the final solution better than what you would have achieved? Not sure at all.
TBH I am not into agents bc I do not trust a tool sniffing all my code and for copyright concerns. But I saw some and use a prompt with limited access and the best I can take out for my speed + control when coding is tech discussions to decide on it, error catching, test generation, one-off scripts... But never "make an app like this or that". If I ever do that (I did it a couple of times) is for scaffolding and later throw away 70%.
Namely, to see something that runs on screen quickly. But later you need to spend time yourself as usual. Not a bad thing, just that this is not what you deliver and need the work done. Iterations etc.
Reinforcement learning can just solve things even if they are new. It doesn't understand how a tool works? Give it a vm with the tool, a thousand agents and let it discover it automatically.
Use the thumbs up/down emoji + chat analysis when a customer is unhappy, feed that to a RL Loop.
The AI Researchers though work on World Models, grounding the AI and letting it simulate. It can do the simulation in parallel (unlimited) and choose what is best.
> I think you miss how much tacit knowledge and judgement there is inside the brains of each of us that an LLM is not capable of.
But thats my problem. Soooo many do not have this even as senior developers.
> Now you have a bug. You do not know even where to start. Thisis theprice to payfor speed, as usual: technical debt.
Yeah now i just ask the LLM to describe to me the bug. Works very well.
> Now you tell me you put three agents to talk and burn 2000 usd in tokens. Great! Is the final solution better than what you would have achieved? Not sure at all.
This is the thing. It only needs to make the team 10-30% better to compensate token budget with one work collegue. We have reached this level in my opinion already. Choosing a head count vs. choosing tokens.
But it becomes cheaper and easier and better. So you will not just be able to do ith with 3 agents but with 20, 50 or 100.
It will be better if your team is an avg team. It will be worse if you have a high profile team, for now. But man our industry has such a weird broad quality spectrum.
> TBH I am not into agents bc I do not trust a tool sniffing all my code and for copyright concerns
In worst case, my team always do code reviews, I do a code review on an ai instead of a human and adjust the harness or the infos the ai can access. I can actually work on making this workflow better and then i can clone it or spin it up for every single PR. For a human? I have to train them and they might leave.
But there are plenty of cases were code doesn't matter. Researchers write a lot of random shitty uggly code as long as it does what it does, it doesn't matter. I have scripts for small tasks, we have microservices which do one thing because it is a tech stack we only need for one use case.
> eah now i just ask the LLM to describe to me the bug. Works very well.
I think you are confusing giving theories about what a bug might be with certainty. It does help bc it csn accelerste things, but many times I had AIs with challenging bugs throwing a lot of misleading theories to me. For the easier bugs, I was just as capable most of the time. Not every time, so there is some potential time saving there. But also time waste.
As for research and fast prototyping you are right: I find it a good tool to explore bc yiu do not need the quality of a final product and researxh is in big part throwaway work.
But I was talking about software that needs features, maintenance, etc. This is just not the same thing.
Yes I double checked the quote is not in the article. HN is probably the best place on the internet for people actually reading the article, but this being the top comment here suggests that the majority of voters still do not read the article
> I, as the human, still have to do the thinking as Claude still 'can't jump'
I still have to do quite a bit of thinking but the amount of of thinking I do per task is trending down. I agree LLMs are not good at abduction but very few humans are either and very few jobs/tasks require it. I can't talk for researchers jobs though. But perhaps fewer researchers would be desired by these labs (not none).
Cognitive offloading is delegating to the AI and still owning the answer. Cognitive surrender is when the AI’s output quietly becomes your output and there is nothing you feel is left to check. For software engineers the line between the two moves under your feet most days, and most of us are crossing it without noticing.
Same reason some think preserving the environment is pointless because the believers will ascend to heaven, either way. It’s a religion. It’s dogmatic nihilism.
> And I say that as someone who uses Claude Code in complex environments almost hourly; I, as the human, still have to do the thinking as Claude still 'can't jump' [1] and I have seen no evidence that they (or similar AI, any time soon) will 'jump' like a human brain does.
Sure it can, turn up the "temperature" a bit.
There's this notion that human "jumping" is magic. It's not. It's all based on inputs. Including unrelated inputs, past inputs, and feeding yourself your own thoughts.
The hard part is not the ability to make conceptual jumps. That's just random search. The hard part is discrimination: whether a given mental jump is "creative" or "insane". Iterated, the problem is that of balancing between the two failure modes: relax your thinking too much, and you'll start thinking nonsense thoughts; tighten it too much, and you'll be just following immediate-term rewards and obvious thought trains. It takes time to find that balance, and plenty of people at various points err in one or the other direction (e.g. small kids in particular tend to err on the "crazy non-sequitur side", but that's because they're learning the basics of reality and social interactions).
100% agree. Plus you can easily instruct it to use inspiration from unspecified unrelated and counterintuitive concepts at random and in parallel. The "jump" is not "on" by default because it would waste tokens on high risk paths, not because it's incapable. It's very capable if you are willing to wast some tokens.
> The hard part is discrimination: whether a given mental jump is "creative" or "insane"
That's what they mean by LLM's can't jump. They mean it can't make a creative jump. Their example is Einstein's Theory of Relativity - It's not a random jump.
Einstein didn't magically one day woke up and jump on relativity theory. He had prerequisites in terms of recent mathematical advancements (notation) and physical discoveries, and a job exposing him to a lot of lateral thinking, and time to bounce ideas around in his head. We don't know how many fruitless jumps he made before making one that we remember him for.
although, if i'm out of tokens and have to wait a full day, i won't bother doing some things manually because the day i'll spend doing something won't take more than 1 hour the next day when tokens are available again.
That seems like a somewhat orthogonal point? Like, if I'm a carpenter and my batteries all run out / I can't actually power my power tools then the best course of action is to go home and recharge all the batteries instead of trying to hand-cut 100 pieces of lumber today. After all, the power tools can do it a lot faster (and with less effort) than I can.
I say this as someone who's watched a bunch of woodworking videos but hasn't actually done this myself :)
I read that more so as, I'm a carpenter and my batteries have all ran flat, so I'll put them on charge and do something else today. I'll cut up the lumber tomorrow when the batteries have charged.
At this point at least half of your day is spent doing work that you shouldn't be doing in the first place, but over past decades companies saw it fit to eliminate specialized roles with legible paychecks, and smear the work they did on everyone else until it disappears from the books.
Self-service and office suite software is largely responsible for this.
A fully automated utopia isn't just going to happen. Even with frontier models, the integrations, the evals, the UX, need a lot of work and someone needs to do it. After I've automated this thing I'll move on to the next task, this is what it means to be a software engineer.
It still makes a massive difference for me if they only need a handfull people now.
Generating a good looking UI for example, is so much easier now with LLM.
For a joke I asked ChatGPT yesterday to make a short promoimage for a 'joke' idea i had, it was above avg. I have for sure seen worse Marketing Images than what ChatGPT generated.
It looked similiar to plenty of other Marketing Images but its not that anyone cares.
AI has made 'jumps' in demanding fields like leading mathematical research and has made advancements in AI research itself. Is now a good time to start a maths career? Is there a field of research (yours?) which is inherently (more) AI proof?
Btw, I think the discussion of Einstein's career in the paper you link is historically wrong in many respects, particularly the argument about 'weak signal'. Einstein was in fact working on some of the most mainstream and widely discussed problems in physics of the day, he is admired for the creativity of his solutions to those problems, and much of his work built incrementally on ideas and breakthroughs that came (long) before (as all research does).
Article suggests that a central motivation of Einstein's work was resolving action-at-a-distance in Newtonian mechanics - yet Maxwell introduced the same Lagrangian field theories for electromagnetism we use today 50 years earlier to solve the same problem for Farraday's laws of electromagnetism. Similar wave equations existed even earlier. Heaviside in 1893 extended this technique to gravity (matching 'weak field' GR) 20 years earlier. So this is perhaps the one aspect of gravity that had actually already been solved before Einstein. Authors might be conflating his work on action-at-a-distance in QM.
Einstein's GR extended the linear 'weak field' understanding of gravity to include the non-linear self-referential case where masses themselves create gravity. This was mathematically incredibly difficult but was necessary precisely because SR's mass energy equivalence created so many strong signals that were unresolved. For example: if finite energy is mass, then mass changes as objects accelerate past a large mass like a start, and hence their propagation in space could not be explained by linear EM style field equations. Many such considerations were causing very 'strong signals' in SR, and there were analogous problems in QM atomic models being developed at the same time.
SR was also a solution to a problem that was actively being worked by many of the leading physicists of the day. SR actually does match Newtonian mechanics for a single observer - it resolves contradictions in the case of separate observers, by allowing them to assign different values to the speeds, masses, etc of objects such that each object appears to follow Newtonian mechanics for each observer. Again, this was necessary because of a lot of contradictions related to the behavior of light that had been well-known for ~20 years at the time.
Personally, I don't consider this kind of reasoning to be beyond the capabilities of future LLMs (even current LLMs if the task was broken into technical rather than philosophical problems). Personally, I doubt that such problems could stand open for 20+ years waiting for a creative genius to solve them in the modern world.
Also people tend to forget that LLMs still just work on compressed data... Where are the MAJOR breakthroughs? Where is all the "crazy" AI output going? Software seemed to degrade in quality a lot in the recent years. All "improvements" LLMs go through are simply improvements on how to burn more tokens out of my pockets given that Claude now want an actual browser extension to "visually" confirm small changes every time I use it for UI.
They are still just data parrots.
From what I see most benefits are for people that work with LLMs, but usually smaller percentages never 50% or more because of the LLMs (OK, unless you were doing basic, repetitive stuff, but then that's not to write about).
Which kind of answers the original question "why bother working?" with "because now, I can do a bit more than before".
I also see bad quality (in code, documents, presentations). It comes from people that had no clue how to do something before and now they imagine that just asking Claude is solving well the problem. And is annoying (and hard) to explain to it them, and then they get frustrated.
> They learn the concept of things and how to do them because this is better compression than learning concepts one by one.
When anthropic looked at how an LLM does addition it found it had some mental math heuristics that might or might not always work. The LLM hadn't learned the concept of addition. It had learned some heuristics that might work for some numbers. The result is that LLM's cannot add numbers reliably because they have not learned the concept of addition.
It learned a concept of a heuristic which made it smart enough for the learning reward.
Might be an architecture issue or a parameter size issue that it didn't learn to do math like a caculator.
But look at your own math skills: How many numbers / how big of numbers can you keep in your head? How far is this heuristic away from how much a human learned until you start using pen and paper or a caculator?
Idk, maybe some people get crazy productivity out of LLMs. To me, going deep into the AI bubble, reading about terms I've never seen before just feels like some crypto bro bubble with people being too deep into the sauce to notice that these things are not the wonder machines they believe so hard in...
By trade I'm a UX Researcher/Designer who designs in code (HTML/CSS) and have done so since 2009. Recently I vibe coded an entire python app with a database and each time I didnt know what to do I would just feed screenshots to Gemini or Codex for guidance (i think i could share my screen with Codex and it can guide me via a voice conversation). I know I could follow up and build a companion iPhone and Android app using these tools.
Overall, I'd like to understand those who have a positive outlook on design and software engineering as a career. Where do you see the opportunity where I just see a bleak one where anyone can do this stuff by typing or talking to AI? Myself, after 17 years in the field I am begrudingly back in school for a new medical career. As well, anytime an IT recruiter reaches out I am getting responses back only after under-cutting the hourly rate I use to demand and what others probably are still trying to get. And with it feels even bleaker as it becomes a race to the bottom!
In my experience, not everyone can really do this stuff by typing. I think you need to be creative, resourceful, inventive, open minded and have ideas how to approach the typing/prompting. I see many people struggle in using AI.
> In my experience, not everyone can really do this stuff by typing. I think you need to be creative, resourceful, inventive, open minded and have ideas how to approach the typing/prompting. I see many people struggle in using AI.
The problem, for the profession, is that the set of people who can really do this stuff by typing is close to "all of them". I'm not seeing anyone struggle with using AI. I see struggles from professional software developers because they are trying to get quality output, but if you don't have a bar for quality, just about everyone can create their own software.
A poster a few months ago had a Show HN about his 7 year old kid, barely able to read, who was happily vibing up games.
I can attest to the other side as well, that I have seen professional software developers outputting code of lower quality than AI. And I would say that during my career (18 or so years) I have met a small number of quality software developers or engineers.
Although that might be because I was not in Silicon Valley where most of the smart/hotshot engineers converge.
The industry has vast (and increasing) oversupply of “programmers” versus diminishing demand. Add to this, the adoption of AI.
> Overall, I'd like to understand those who have a positive outlook on design and software engineering as a career.
I think until the market better achieves some equilibrium, there is no way general software programming (sorry “engineering”) should be considered as a career. That said, there will always be opportunities in particular markets or specialties.
I also work in UX and SWE, and heavily use GenAI in my work. I don’t have a positive outlook for people who limit their career to one of those fields, but I do have a positive outlook for generalist, multi-disciplinary careers. When you have the experience and skill to steer product development from end-to-end, you can produce high-quality products super-quickly. The experience and skills are the differentiator — if you lack those you can still use GenAI to move fast but probably in the wrong direction.
So one person now doing the job a handful use to do. That's what I hear you saying and Ive been thinking since my lay off in Feb. prompting me to be back in school.
It also seems kinda tone deaf. If someone basically told me I was wasting my time and asked what I would do in the future, I would not bother giving them a particularly thoughtful answer because trying to spend effort justifying my life choices to them would be the actual waste of time.
> > Mathematician Richard Hamming used to ask scientists in other fields "What are the most important problems in your field?" partly so he could troll them by asking "Why aren't you working on them?" and partly because getting asked this question is really useful for focusing people's attention on what matters.
> I imagine someone being asked this question, and how they should respond. I think like so - ‘Fuck off Richard’.
> This is partly because I imagine this question being asked in a kind of snarky, gotcha kind of way, with some sort of nerdy superiority. Like ‘ha your behaviour is inconsistent with your implied preferences, you idiot, do you even von Neumann–Morgenstern?’
> We already know developers don’t actually spend most of their time writing code, with studies at Microsoft and elsewhere showing it’s closer to 14 percent.
Anyone else finding they're spending more time writing code (or at least driving agents to write code) now?
14% used to feel about right for me - I'd spend the rest of the time researching approaches and libraries, planning things out in issues, or sometimes just thinking really hard about problems I ran into.
Now... I still do those things, but I'm doing many of them faster - and I'm often doing them while my coding agents are churning away on code.
There's also this weird effect where the harder a problem is the more I can get done in parallel with it, because an agent might need to spend 20 minutes on it without my involvement.
You read my mind. I suspect it is a transient spike while I blow through the backlog and I'll run out of things that AI can vastly accelerate in, say, 6 months. But right now? I'm spending 80-90% of my time blasting through implementing features, finding bugs, fixing old bugs, writing and improving my tools... Code was never the majority of my job. But right now the bar to doing things is so low and the sheer joy of blasting through some previously tedious, low to medium hanging fruit is thrilling.
>> We already know developers don’t actually spend most of their time writing code, with studies at Microsoft and elsewhere showing it’s closer to 14 percent.
> Anyone else finding they're spending more time writing code (or at least driving agents to write code) now?
Not really, as once it is time to write code, the problem has been defined/understood (to the degree possible with knowledge acquired at the time), and encoding it is largely an exercise in typing along with verifying assumptions via test suites.
Does GenAI quicken some portions of the above workflow? Sure, in the same way IDEs with contextual code snippet suggestions can make encoding faster.
I used it to write SQL and make dashboards. Back in the day, I would spend a lot of time doing that, then I changed roles. I dipped my toe in it recently and used AI exclusively. I would send a prompt, see the output, decide if that is what I wanted or not. I kept my brain in "what-if mode" and I let the LLM handle the technical specs.
The cost of prototyping has crashed. So you can A/B/C... various ideas, test them in the real world or by personal preference, to taste, and converge on designs that work for you.
This used to very hard and expensive. Now it's so cheap it makes the idea -> test -> refine loop much tighter and faster.
>The cost of prototyping has crashed. So you can A/B/C... various ideas)
Oh yes! I would think "I wonder how the data would look if I did X" and feel the pain of complicated SQL... but then the LLM would just do it and I got to see if I liked or not! It was a delight
I review the code that matters - anything security adjacent or that's an API that will be used by other code in the future.
I don't review code that either works or doesn't - most HTML and CSS layout code for example. There I test it on desktop and mobile and commit it if it works.
Ditto for stuff that's simple. A JSON endpoint that runs a SQL query and returns some JSON? If it works and a glance at the tests looks OK then I trust my agents wrote it properly.
I'm getting more confident with my judgement over what needs a close look and what doesn't over time, as so far I haven't been majorly burned my any mistakes that snuck through.
Honestly, it's similar to being an engineer on a larger team. You don't review every line of code written by every one of your coworkers.
I think this is THE issue of our time as programmers to be honest: do you review every line of code an agent writes?
An increasing number of expert programmers are moving in the direction of NOT reviewing every line. It's working out OK for a lot of them.
I've found that engineers on a large team do read every line, mainly due to the fact that the skill levels run the gamut from intern to lead, and only 1 or 2 people out of 12 might have knowledge of the application being modified.
It's actually getting worse due to "AI code bloat", for example I have 16k lines of code to review across 3 apps by the end of this week. Normally it would be a quarter of that, but what Claude produces is extremely verbose in some places and anemic in others, and I can't tell at a glance what's right and what looks right with that much ground to cover.
Goodness, how is that being tolerated? I guess it can’t be stopped without a lot of political capital; but 16k lines of code is HUGE, and I cannot imagine that it’s actually 16k lines of value - I’ve written whole new subsystems of a product in fewer lines. Are these all written in an exceptionally verbose language like Go or Java? Are they VERY well documented? Are they doing things they shouldn’t be doing???
I mean, I've done that for a couple of things, but probably 90+ percent was vendored libraries and javascript which could be ignored. Sadly, this is not one of those cases, and Im guessing yours isn't either?
> I don't review code that either works or doesn't - most HTML and CSS layout code for example. There I test it on desktop and mobile and commit it if it works.
Good example of what not to review if you're working on your hobbies. Also exploratory can sometimes be done this way. However, this ultimately boils down to how you approach programming as an engineering discipline, including your responsibility for the outcome.
> I'm getting more confident with my judgement over what needs a close look and what doesn't over time, as so far I haven't been majorly burned my any mistakes that snuck through.
This doesn't generalize well. If you drink raw milk, or if you don't wear your seatbelt, or if you don't escape your user input correctly, you'll probably be fine, but I really hope aspiring programmers/engineers don't take this attitude towards any serious task. One should always examine their biases, tools' failure modes, etc. regardless of how many times something didn't fail.
> Honestly, it's similar to being an engineer on a larger team. You don't review every line of code written by every one of your coworkers.
One [should] review the code they're responsible for. In a team, people usually assign you (or ask you) to review code, and the work is divided accordingly. If the code isn’t reviewed by the code owners, it’s a problem, not something inspiring!
> An increasing number of expert programmers are moving in the direction of NOT reviewing every line. It's working out OK for a lot of them.
Have you considered that the sheer amount of code being generated is what makes thorough review infeasible, not that it’s a desirable approach?
But if you observe that the agent day after day do handle user input safely; and also routinely run an agent that scans for security vulnerabilities and observe it finding cases where input is not handled safely in existing code, you may conclude that the chance of an issue is at the same level at, or probably lower than, if a human wrote it and a human reviewed it.
("Escaping" user input is not good practice though, use parameters, assuming you are talking about SQL.)
> Ditto for stuff that's simple. A JSON endpoint that runs a SQL query and returns some JSON? If it works and a glance at the tests looks OK then I trust my agents wrote it properly.
That is *exactly* the sort of area I *wouldn’t* blindly trust AI, there’s a huge security boundary there. What if the AI is doing string concatenation with user-provided data???
Is this really a rational strategy for something whose nature is to be right most of the time and then spectacularly wrong a much lesser amount of the time?
I agree with a commenter above/below (depending where this comment lands). For some time models won't do this. And any review from review agents would caught this.
For most of the AI programming there needs to be a more stricter (automated) review process now. Most SAST tools would caught this type of security issue.
lol I think you’re setting yourself up for failure. Why? Just because something works doesn’t necessarily know you the boundaries of it.
What scale does it work for?
Will it crumble under load in prod?
It works but allows cross tenant access (security issue) because security checks weren’t in the location you thought… Dangerous!
I think it also depends on what you're building. Some solo project or basic html thing? Sure no need to review every line. It's a bit different when you're working on foundational libraries that a business relies on, anything touching a production database, etc.
> Honestly, it's similar to being an engineer on a larger team. You don't review every line of code written by every one of your coworkers.
We don’t because everyone is accountable for his or her own mistakes. So everyone is incentivized for their recklessness to not be the root cause of some bug.
> An increasing number of expert programmers are moving in the direction of NOT reviewing every line. It's working out OK for a lot of them.
Have you ever asked your users? What about bug reports? Is the amount and rate decreasing?
Yeah. I spend most of my day driving agents to write code, verifying the results, orchestrating work streams, and so on. The rest of the time, a Fable agent is organizing work in Linear/Jira and making sure coworkers are getting their stuff done in a way that won’t conflict.
Yep. ADHD very strong in this one, so LLM code generation takes pretty much all joy out of coding. It’s like watching a computers play chess. Yeah, no thanks.
With GenAI, we can now produce something without caring about it - or while caring about it very little.
And the parts we don't care about aren't necessarily worse, they are just... arbitrary. Could be good, could be bad, no one knows, because no one really cares.
I find that if I care about something a lot, it's a pretty similar time investment than pre LLMs. And it makes me feel invested and proud in the result, motivated to show it and improve it.
If I care about something very little, in the past I just wouldn't have done it at all. Now I might, but I feel that same disconnect you mentioned.
I think being strategic in what we do and do not care about is likely the key skill we'll have to build to actually make the best of the tech.
I feel like all you need to know about how seriously to take this is that they cite that ancient early-2025 METR study, and describe it in the text as "recently one even found..."
Same thought - 80% through reading it occurred to me to check the citations. A few items from 2025 and most well before that.
So much has changed since late 2025 one can’t really draw any conclusions from this.
In fact, I’m guessing things will continue to move so fast that by the time one were to execute a survey of developers, many of the responses and findings are no longer relevant.
Your point really goes both ways, we really don't know anything about how LLM usage is affecting anything. No one knows, it's the wild wild west, which is whatever. But I think no one can really draw conclusions from what's happening in tech right now.
Reminds me of COVID and how everyone was fighting over early trends during that time.
The whole point of the 2025 one is that they found the self-reporting to be significantly inflated, which is why self-reported surveys like this one are hard to trust.
Yes, but their newer write-up discusses that (and shows that the self-reported numbers have gone up radically, in a way that suggests that even if there is some inflation, the numbers are almost certainly positive if you deflate).
They also have an update -- linked from the original study! -- explaining that it's out of date and no longer reliable, and explaining why they had to cancel a follow-up study because it was understating productivity gains (but also was showing wins for the people who carried over from their previous study): https://metr.org/blog/2026-02-24-uplift-update/
The authors of this paper decided to ignore all of METR's follow-up data and discussion, and to report only the ancient number from early 2025 (a time when Windsurf was state of the art). And then, rather than apologizing for it, and caveating it as a number not to be taken seriously, they described it as a study done "recently."
That's either shockingly dishonest or incredibly out-of-touch.
> shows that the self-reported numbers have gone up radically, in a way that suggests that even if there is some inflation, the numbers are almost certainly positive if you deflate
Or the hype machine is even stronger, making people inflate those numbers even more.
If I wanted to be uncharitable, the question of "how much slower you'd be without AI" compares against people's AI-less productivity from 2026, which could have become much less than 2025 AI-less productivity, if hypothetically AI use makes people helpless and deskilled.
Like many others in the comments, I feel there are a lot of assumptions in this piece. Before, coding is only 14% therefore, small slice. I think that's a very superficial assumption. That was because coding was expensive and we needed to be sure we didn't code the wrong thing. If code is as cheap as it is now, we will optimize differently, we will structure around it. Instead of so many meetings we will code 5 different versions of the same thing and choose, etc.
Coding was expensive in the sense that once you decided what to do, it took a few engineers months / years to do moderately complex projects. That's not true anymore. Therefore the risk of "coding the wrong thing" is less.
> Coding was expensive in the sense that once you decided what to do, it took a few engineers months / years to do moderately complex projects. That's not true anymore.
This conflates two orthogonal concerns; understanding what needs to be done with encoding a solution for what needs to be done. I am reminded of an axiom I have long held:
When making software, remember that it is a snapshot of
your understanding of the problem. It states to all,
including your future-self, your approach, clarity, and
appropriateness of the solution for the problem at hand.
Choose your statements wisely.
> Therefore the risk of "coding the wrong thing" is less.
I disagree with the proposition that being able to generate "wrong code faster" has anything to do with understanding what ultimately needs to be done. In fact, exploratory efforts having the intent to refine problem domain understanding (a.k.a. "proof of concept" projects) is an established industry technique whose work product is best discarded.
Is it? The temptation to start without a thorough design is now much stronger because the implementation osnperceived to be cheap and easy to replace. But if you start building the wrong thing fast, you still get the right thing later than when you had checked properly at the start.
CICD and iterative development and agile work because interactive building leads to better product. Otherwise we'd all build perfect software using waterfall methodologies and release only once a year. If implementation is cheap then you implement, review and then re-implement. Not even iterate in the extreme case. Throw it away if needed and start from scratch if needed.
Lots os engineering projects took as long as you mentioned because they were horribly specified. Then the engineer either waits for an answer (which they generally do not get) or takes a decision (which might not be ideal without domain knowledge). Then you release and the customer suddenly starts explaining what they want.
You can improve some parts with LLM (make more prototypes, iterate faster), but if humans are the bottleneck and the problem itself is complex it will not be 10x improvement.
Of course, maybe we talk about different type of project, there are also the type of projects like "a web-shop that sells stuff" that someone considered "expensive to code".
Specification is hard because you are trying to predict a future state in a vacuum. You need to do that because creating the product is very expensive coding wise. If instead you could instantly build to your current specification then you could iterate on the specification against a real product. Even if all that code is thrown out the end result would be a better specification.
For some projects specification is also hard because of interdependence between modules or data or parts of the project. To which you might answer "make more tests". And we turn in circles, because it is hard to test something you did not thought of.
My experience is that users (customers) even when presented with a prototype (done fast by AI, or previously, in the slow way) can take days to "discover" or "think" about things they wish. Then more days to understand why some wishes can't be done because it breaks other stuff they want.
Maybe you have smarter, faster, more experienced and decided users. Then I can only say: good for you, you are lucky!
Basically yes, there will be more coding in that phase, more prototyping, the PRD phases will be shorter too, there will be more pressure to deliver quickly and the PRDs will be under more pressure to move more quickly. This is what I'm already seeing to be honest.
> Before, coding is only 14% therefore, small slice.
But also, no, because they write:
> “coding” (not including bug fixing, testing, etc.)
What if bug fixing includes "coding", or "writing code", or however one would want to define that? Especially in the enterprise setting they evoke, a lot of work will not be "coding" in the sense of churning out new features, but "coding" in the sense of fixing bugs. I know a lot of my "coding" is in this category. But we're not given a number for it. I suspect the slice would be bigger if they included this type of "coding".
I can't remember of any product in my lifetime that was more over hyped than AI.
It's a huge piece of shit and if I wasn't forced to use it at work I would never use it.
It writes dumb, throw-away code and adds thousands of dollars/developer in costs. All this crazy code that we're adding to our projects will come back to bite us in the future, there's no way it won't.
Writing code is indeed the bottleneck for same resource constrained companies.
Rapid code development creates more opportunities for trial and error, providing companies with more information for decision making, that previously might have been addressed by meetings.
Of course, this might bring other problems, but it might not right to generally speaking that writing code is not a bottleneck.
I’m very suspicious of this objection, because when Claude first landed the same people now saying “code is not the bottleneck” were saying “the generated code doesn’t work.” Smacks of moving goalposts.
The only solid objection to “AI is going replace developers” is “AI is an accelerant.” It helps developers move faster. I haven’t seen anywhere it has fully replaced developers.
Whether this leads to a large number of job losses depends on whether you think we can increase software output by the same factor as the acceleration and still be profitable. I think we can, latent software demand is extremely high. I also think we’re nearing the limit of capability with current models.
Situation could change if more advanced models emerge, but some of the more foreseeable advances probably have compute requirements beyond today’s hardware.
I don't understand Myth 1 (Developers Spend Most of Their Time Writing Code).
They quote a study in which developers report to spend 11-14% of their day coding. The rest is stuff like solution design and meetings. The insinuation is that AI can at most automate 14% of your day.
The problem with this argument is that once you have code, some (not all) of the precursors to code go away.
Well, I expect when you've vibed too much and lost track of the code, and can't answer questions in meetings anymore, you'll stop getting invited to them.
How often do you talk about low code details in the meetings? It is mostly about requirements and goals (IMHO) and "architecture"/design, and now I wouldn't even bother my peers with questions about the code as I can let AI tell me how things work exactly as written in the code, not some thing that someone remembers or thinks how it works (in large codebases, most developers only know in detail the things they are working on currently or from recent history)
Anecdotally I find them helpful for design and I don't find them nearly as helpful for coding (strange, given others experiences reported online)
I am working on a compiler for an OCaml inspired language (I am on a sabbatical) and used them a lot in the early phases to check my ideas and search for prior work. I use them less now (mostly debugging at this point), but have made use of LLMs to write some pieces like source map generation, a couple of small and well known algorithms that were new to me, and an analyzer to dump a schema from a typescript module. Other times I've tried to use it for other things, I have ended up regretting it because of subtle bugs
I don't think it made the design phase any shorter but it definitely helped save me energy in researching and made that period feel less like a fever dream. Design phases can be draining for me so that was welcome
It helps me. Case in point, I recently had a API refactoring project that was 80% busywork and only a small amount of code. It's crap work, but it has to be done and it's part of my role. I told AI to generate the initial design doc. It took it upon itself to reference the local headers on my system and found some APIs that should be included that I hadn't even considered would also need to be touched for consistency. It saved me hours typing up the doc, requirements, test plan, etc and then saved me embarrassment by preventing me from making a stupid oversight on a tedious and boring task.
From the patterns I have seen people would tend to use the time to build demo's using AI instead of design and then have a back and forth on the demo itself. One has to consider the fact that demo's look more convincing/complete and give an impression that the design decision has been made. In some ways, how agile got rid of the need for explicit & detailed documentation, the AI created demo's will cut the requirements of high level designs too.
The concept of MVP has been there for a while, it felt deceptive too and incurred lot of tech debt. However one would feel the deception only if they understand the missing details. For a lot of roles making decisions it benefits them to ignore the details unless it really bites them
The article mentions that coding is only a fraction of dev time and thus accelerating that part of the job can only create incremental gains, and oh PS, lines of code and similar metrics are a bad way if measuring dev productivity, anyway, and we've known that for decades.
The OP claims AI accelerates non-coding parts of the job, too, and so the article is misguided.
That's why Microsoft did a study (referenced in the article), where they measured the time spent on things, so they could get to the truth of how much time people spend coding using "a shape of evidence would you find convincing"
The Microsoft study concerns the ideal vs. actual developer workweek. They conclude that developers using AI tools "heavily" are both more productive and more satisfied. And, surprise, that devs hate meetings and want to code more.
Isn't this falling into the "lines of code" trap TFA mentions?
More code written is not a good measure of productivity. It could be garbage, or redundant code, or simply not addressing the real or more pressing needs, it could be building the wrong thing, etc.
Like TFA mentions, it's been known for decades LoC is a misleading metric for productivity. It's one of the lessons of software engineering.
I'm planning an article at the moment in defense of lines of code. Saying "lines of code are a bad measurement" is too easy! It's about time someone presented an opposing argument.
In my specific case, lines of code for my published open source projects is a metric that I trust, because I have high standards for those. I have plenty of other projects where I'll accept poor quality, unreviewed code (almost all of https://github.com/simonw/tools for example), but Datasette, sqlite-utils and LLM are not that.
Of course, that's only useful for me personally and for people who trust me to stick to my own self-declared high standards!
> Saying "lines of code are a bad measurement" is too easy! It's about time someone presented an opposing argument.
Why though? The only entities who would be interested in such an implication is the AI marketing.
No one real cares about more or less lines of code. But everyone cares about decreasing the complexity of the implementation. Sometimes that means more LOC and sometimes it means less.
> Saying "lines of code are a bad measurement" is too easy! It's about time someone presented an opposing argument.
I don't know about easy. It's a hard earned lesson of software engineering, backed by research (some of it cited in TFA). Also, there's a related lesson: the more LoC to solve a given problem, the more bugs. That's also a finding backed by research.
I'll be reading your article when you write it, but I confess I'm skeptical.
The AI companies have a vested interest in using this metric, since it's easy to measure (the reason LoC were used even before AI) and there's no doubt that LLMs are writing tons of code. This makes me doubly skeptical.
> Saying "lines of code are a bad measurement" is too easy! It's about time someone presented an opposing argument.
Maybe because there is none.
One of the main quality of good codebase is simplicity. Which is about how easy for someone else to understand the code. It’s hard to define what simplicity looks like, so the best bet is to avoid the other side, making the code too complex.
And you can make the code complex by shortening variable name, doing code golfing with quicks of the platforms, so smaller LoC. You can also go the other way and increase the LoC by adding unneeded abstractions, repeating slices of code,… There’s a window where the LoC is perfect to attain simplicity, but that amount is an effect of striving for simplicity, not a cause of it. And it’s variable for every problem.
So you got something where the correct value is a different for each case. And trying to manipulate it artificially often results in complex code. And you want to say that is a good metric for productivity?
And in the cases of your projects, there are a lot more info could share that are interesting, like the amount of issues (reported or found by you) that are tied to implementation bugs (coding, libraries API breakage,…) or design issues (requirements conflicts,…), documentation improvement,… Anything that is tied to the actual usefulness of the projects, and not fumbling around with code.
> Lines of code are akshually a good metric now, because that's the only metric LLMs can optimize and I really, really, really love LLMs, they're the bee's knees.
That meeting that you spent an hour in to understand the requirements? You don't need that meeting if you're not writing the code. That sync up with the QA engineer you did to hand it off to them? Don't need that meeting if you're not writing the code. That half hour you spent installing vim extensions? Don't need 'em if you don't open vim anymore.
There are engineers whose jobs go well beyond coding, of course. Staff engineers and principal engineers have had their jobs radically change because of AI, but not because it's writing all their code.
But there are also a lot of engineers -- your standard mid-level engineer, or even senior engineers at a lot of orgs with title inflation -- whose job is almost entirely about delivering code, and who spend all day either writing code or engaging in scaffolding around code-writing activities. Let's not pretend that automating away that code writing is a 15% boost.
In some number of cases, the business partner who would have passed you the requirements will just generate the code themselves.. I’m shocked shin some engineers don’t see a massive disruption coming..
Sure, but why doesn't said business partner just give a requirement to the AI to do the business partner's own job? Since it is hypothetically superhuman at fulfilling requirements at such a point...
PMs seem pretty easy to replace. In fact, given any single role to fixate on, it's probably replaceable
> In some number of cases, the business partner who would have passed you the requirements will just generate the code themselves.. I’m shocked shin some engineers don’t see a massive disruption coming..
It’s because you are engaging in first-order thinking. You imagine the immediate result which is that software engineers are replaced.
If you engaged in second-order thinking you would realize that if software engineers can be replaced, _any_ work can eventually be automated. So why stress about it, we’ll figure out a replacement for capitalism. It seems like the world is starting the slow swing to the left so eventually the problem will be solved.
Of course might be those engineers you mention are engaged in no thinking.
Oh, some of us see a massive disruption coming, just maybe not in the way you think.
Talented, experienced devs leaving the field out of disgust would be pretty disruptive.
Crumbling, faulty technical infrastructure with unmanageable heaps of AI slop that no one understands would definitely disrupt businesses.
Skill atrophy, and an entire generation of "developers" that have never actually written code will be disruptive when something breaks and no one knows how to fix it (this isn't a hypothetical, I'm seeing this phenomenon in a lot of large enterprise groups).
The total elimination of novel solutions, new ideas and new approaches to development and the resulting industry wide stagnation won't be disruptive, but will certainly be a drag.
> Crumbling, faulty technical infrastructure with unmanageable heaps of AI slop that no one understands
Replace "AI slop" with "legacy code", and you've got basically any large enterprise today.
The timescales are different though: it takes years if not decades to accumulate classic legacy cruft, while LLMs can spew up huge amounts of incomprehensible spaghetti in seconds.
>> How are you going to prompt the LLM or validate its output if you don't understand the requirements?
> Your PM can write the ticket, and your QA can test the output.
If your PM can write the ticket and your QA can test the output, why not cut out the middleman by having the PM write the coding agent prompt(s)?
And if your PM can write the coding agent prompt(s), why not cut out the QA group by having the PM write the verification prompt(s)?
And if your PM can write those prompts, why not cut out the PM and have your CSRs write the feature request/verification prompts?
And if your CSRs can write the feature request/verification prompts, why not cut them out and have the organization's customers write the feature request along with acceptance prompts?
And if customers can write those prompts, why would they pay you?
> And finally: Why bother making new software at all when AI can just do the thing you wanted done, directly?
In many ways, the "save costs by LLM-ing everything" mindset is the same logical fallacy companies fell into in the 90's/00's with regard to outsourcing. The end-state was rationalizing outsourcing call centers, then production system support, then software development, then project management, then...
Once the outsourcing raptor came for the accountants (who wrote the cheques) and remaining management (who cashed quarterly bonus cheques), outsourcing became "less appealing."
Also AI is now drafting design docs, generating PLC work products, entering it all in Jira, characterizing and root causing bugs... It's speeding up the 86% of my job that isn't coding. The article is a bit myopic and frankly contradicts itself.
If Claude told you to work on a task that you don't want to work on, or make a design choice that you think is wrong, would you do it? If not, then it can't really replace things like design or meetings. (Note that this is subtly but importantly different than the "vibecoding" model, where you just don't bother to supervise Claude's decisions.)
I don't know about others, but at work, the reason I only spend like 14% of my day coding is because I'm lazy, not because I'm actually doing other stuff.
> f developers spend only about 15 percent of their time typing in the editor
I think this is missing an important detail. Lots of time was spent on non-coding stuff, because coding used to be more committal and hence expensive. With how quickly one can code up a quick prototype or even production-ready code these days, the code becomes the communication tool as well.
how come lines of code (or expressions) by an engineer aren't a good way to measure progress (Gates point etc) but GenAI tokens must count and be paid for?
That's a good point. They're not selling progress, they are claiming to enable progress by selling you (tokens in -> tokens out), making it your responsibility to connect the two. Filling that gap is part of your expertise that you can get paid for.
You forgot to add "coordination" to that pipeline. That is easily far and away the biggest source of delays.
That includes talking to vendors, meetings with every layer of stakeholder when just one person digs their heels, etc.
That is truly the final frontier for "AI", and one that it will likely never cross. That would be when even the execs and upper management feel threatened by "AI". But, since they also delegate so much, you often see someone at the bottom of the totem pole in those meetings. This is why nobody is getting replaced by "AI". We really need to move this discussion away from the scifi stupidity already. There is no singularity or godlike AGI about to take over the world.
I hate to use awful terms like "synergy" and "teamwork", but they do have a lot more substance and truth to them than any perceived threat from "AI".
> A study of more than 450 engineers at Microsoft in 2025 showed developers spend only 14 percent of their time writing code
This feels about right as an average across project cycles and different types of companies, and is the same type of number I've suggested here before.
The obvious conclusion is that even if AI reduced coding time to zero, then it would only reduce software development time by that 14%.
Of course AI may be used for other aspects of the job as well as coding, but on the flip side any serious use of AI requires a human in the loop to give work to the AI, steer the AI, assess the output, etc.
It's interesting that this 14% figure is close to Uber's choice to limit AI spend to 10% of developers salary. I wonder where that Uber number came from?
Another rather startling datapoint on the perceived value of AI comes from Microsoft who are also adopting budgeted AI usage, and are looking for "outcomes that move the needle". Their budget guidelines apparently refer to (current? targeted?) per-developer AI usage of "hundreds of dollars a month to a few thousand dollars in tokens".
>>>Recent research shows that developers—especially women and older engineers—face a “competence penalty” when using AI
and why did this study (performed in China I might add) find that women and “mature age” (which is not defined in the study) have less usage of AI tools?
>>>? We
suggest a new barrier: using technology to assist task completion signals a lack of competence to
perform the task independently.
So basically the study suggests that older engineers and women are reluctant (and especially women) to use AI tooling because they feel they are being judged more on non-technical competencies.
This study may have a strong cultural influence but I would say that one thing they noted I’ve also seen. 41% of engineers in the study had used AI 12 months after the initial rollout. This aligns with my observations. Some people are struggling figuring out how to adopt in their day to day while others are full in.
I look askance at an article that uses a GenAI survey from 2024. Other references could be good. But the dev agent milestone was Nov/Dev 2025, and CoPilot and agents were pretty sketch before then (but helpful at times.)
- In biz development, a dev usually spends 30-40% time on coding, and more time on requirement discussion, integration testing (especially when the tests involves mobilephone or car)
- coding time can be reduced to 30%, which means reduce 20%-30% time of the full pipeline
- meanwhile, every phase and role is using LLM now, for example, product manager can produce longer requirement doc easily (we can use LLM to read it anyway:) Meeting sometimes is more than before, because more document output leads to more reading and discussion.
- I hope to find new ways to express biz requirements, in a more efficient and automatic manner.
- Shorten the requirement-dev-test-deploy loop is very important. OUTPUT is not OUTCOME. It is equal when we can see the final result, instead of intermediate metric.
- Agentic infra is extremely useful, or every one will find a way to access the database, report and ops system, in some weird fragile method.
If you can spin up a new software project with little to no effort, the most important part of the job becomes making the right decisions and this is where we will be spending 99.9% of your time.
I think the paper would have been stronger if it acknowledged how quickly the underlying evidence is becoming outdated. AI-assisted development in 2026 isn't just better models. The way many devs including myself work has changed and matured quite a bit as compared to last year
We switched from looking at an UI (claude webui) and waiting for code generation to using claude exclusivlie on the cli and claude doing a lot more stuff in the background with smaller prompts.
For me it changes in a way that i would like to have a 24/7 workspace vm setup outside of my work laptop for keeping it running if it wants and looking at it remotely if i want.
The workspace thing would also allow it to have more permissions like downloading, configuring and using headless chrome instead of highjacking my chrome session.
I think the tooling around the models themself has improved _a lot_ - they are really good at giving the models the correct context, even in big code bases
I’ve had people tell me, with a completely straight face, that they expected 10-100x productivity improvements. This is at the executive and VC level. The mania is extreme.
I for sure do plenty of things with AI a lot faster.
Instead of searching some linux issue, i will prompt claude to generate a small analyser script for checking wha tlinux i have, i will tell it what hardware i have and it fixed my issue in like 5 minutes? That would have been a lot longer before.
11-18% of time spent in coding is still very high number I think. For a large org with lots of process and risk aversion, this number could be as low as 5%. Even for 14%, the 10x improvement could mean 86+(14/10) => 87.4/100 => 12.6% overall time saved.
A lot of this rings true, but I think it's still too narrow. Sure, coding does not equal productivity, that is well debunked already. But I would argue that productivity is a product of engineering delivery + product decision making. Now where is the line between product and engineering? It varies by company, team and individual, but I don't think productivity can be measured for those functions independently, and in fact I see gains from AI on both the coding AND the product management side.
Basically as a senior tech lead in a large company engineering org, I don't have the bandwidth to individually validate every assertion from engineers on other teams OR from every product manager that comes with a half-baked ask. In the past I would be limited by the influence I could get through human relationships to strong SMEs with good judgment, and those folks always thin out as a company grows and calcifies. The number of creative and innovative thinkers dwindles, and the number of people protecting their turf and doing the minimum not to get fired increases. As a result many good ideas can get blocked by random gatekeeprs with poor imagination, poor expertise or both. However with AI I can follow up on gut instincts and fact check a lot more things, and ask incisive questions that can cut through a lot of organizational bullshit.
That's where I think most of the AI gains are today. Of course once AI plateaus and normalizes I think it will be baked into the org structures of tomorrow. But for now it offers real competitive advantage to those with the expertise to ask the right questions.
> studies at Microsoft and elsewhere showing it’s closer to 14 percent
This is a depressing stat. The real productivity gains come from leaving soul sucking big tech companies where nothing gets done with any sort of urgency.
In my open source work I believe this is the same. I don't have numbers, but I'm sure the vast majority of my time isn't spent writing code. Of course, it depends on how you define "writing code".
It is not urgency. Large production systems mean you are doing mostly unsexy operational planning. If I had a dollar each time I hear the word "data migration" I reckon I could do well.
With all the myths and hyperbolae circulating regarding AI, I'd love to know what it's like at large software companies adjusting to this brave new world.
It's easy for a small team to adjust workflows and roles, but I just imagine the office politics must be a waking nightmare in big organisations right now.
As an individual contributor I do not have insights into office politics. While we have >10k developers, it is not a software company.
Top leadership and/or investors believe in the benefits of AI.
Thus, skeptics stay silent. Who wants to loudly contradict their bosses' boss?
In practice, I do not see a big shift yet in workflows that would require a lot of politics. We just have the agent implement the code, and then it still goes through the usual code review and QA processes.
Only now with more effective models and harnesses do many developers realize how good these tools are at investigating bugs, etc.
Before the price-decreased GPT 5.6 Luna we barely had access to enough cheap AI to last for a month of work.
In my company GitHub CoPilot was rolled out on a global scale last year.
LLMs from Anthropic and co we get through a central tool which buys api tokens from Azure and AWS.
Its surprisingly fast, probably thefastest i have seen technology getting rolled out.
Im more worried about the small/middle sized companies which are software companies but don't get that they are. You know the companies 100% depending on IT but not having the right or good or enough people who often in interviews struggle with basics like code review.
I hope that some of the executives out there will read the list. I know they won't spend the time to read the full blog, but at least the headers should be enough
I mostly agree with the part where it is stated that AI is a tool which received massive investments without knowing how to maximize its utility. I think that we will pay for it in the near future.
> AI Will Turn Individual Developers into 10x Developers
Maybe not 10x, but there's definitely a significant boost in velocity of development and delivery.
We used to be a team of 20+, now only 9, and we are delivering more than before. Besides that, we did some major refactors that were sitting in the backlog for months.
>a “good” workday, engineers spent 18 percent of their time “coding” (not including bug fixing, testing, etc.)
I must be a crap developer, because I probably spend twice as much time bugfixing and testing than "coding". (Both of which actually involve coding stuff, so I really don't like that distinction they make)
This is stuff AI can be really good at, so brushing that part under the table distorts the picture.
Having said that, I do agree with most of the myths they present.
LLMs are also quite good in writing unit tests and understanding bugs a lot faster than I do, now.
Just a few month back i looked at some yaml stuff for like 20 minutes, played around with it, looked at formatting etc. then i asked the LLM, it immediadly told me what was wrong. I was just blind to that particular wrong char.
This reads like a critique of 2023 tooling published in 2026. Their Amdahl-style arithmetic (speed up a 14% slice, cap your gains at 14%) holds only if "AI" means autocomplete. Current frontier models do far more than that: research, code comprehension, review, test authoring, debugging, exploratory prototyping, ideation. That's most of the rest of the working day or "86%".
The only point that still holds is that organizational policies and procedures that automate AI use and lower the barrier to entry are more efficient than leaving it up to each individual. Every other point they make is either stale or was never true to begin with.
All very sensible points which I think all senior programmers who have used AI would largely to agree with.
For those more junior - keep in mind that a lot of the maximalist rhetoric are from people either selling models, or the cottage industry of people selling you courses or tools to help you use the models. Try and keep in mind software is not a mature industry, it's an immature one, and it's prone to hype and fads.
It's kind of weird how we blame agents for hallucinations as if humans don't fall for that as well, while agents can run the build-fail-fix-repeat in much faster cycles than coders.
even an AI assist that makes coding twice as fast would, in theory, improve developers’ overall productivity by less than 15 percent. The other 85 percent of their time remains untouched
I stopped reading after this. AI has massively impacted most aspects of my non-coding work including the mentioned planning, understanding legacy code bases, setting up environments, etc etc.
Either this article is written by people with skill issues or - given the platform - its a biased and protectionist take that will fall quickly under the march of reality.
The 14% coding time figure is one of those stats that sounds surprising until you actually track your own time. When I started building a coding agent with persistent state, I realized how some days are spent with minimal actual typing, most of it is design, reading code, debugging, problem solving, and context-switching.
But I'd push back on one thing the article implies that AI is automatically a productivity win. It's not. Some days I've shipped two months of work in a few days with AI. Other days, like today, I've burned a whole day and gotten almost nothing done because the proper research was not done by me or multiple agents.
The bottleneck for AI can be the human understanding of how to optimally use the tool. While the bottleneck for the human can be not maximizing multiple agents, or the input the user enters, then the retention of the output. If the user's input is lost, the output falters. If the user doesn't understand what the AI output is, there is going to be a problem eventually.
The article touches on adoption barriers (Myth 7), but it doesn't really get into the ego piece. There's still a wave of experienced devs who either refuse to adopt AI, or use it quietly and don't share what they're doing. That slows the whole team's learning curve. At this point, I think it's pretty much understood that you should be using AI as a dev — not to replace your skills, but to accelerate them. That means still learning new languages, still writing code, still troubleshooting. The tools change, but the craft doesn't.
I think the article is right that the real leverage is organizational, not individual. The teams that succeed with AI aren't the ones giving everyone a license — they're the ones rethinking how they review, test, and maintain code.
What I'm still uncertain about is how to measure whether AI is actually making systems better, not just faster. Lines of code is clearly a bad metric, but I haven't seen a good alternative yet. What metrics are people actually using that feel meaningful?
I'm getting tired of these articles telling me what AI will or won't do to my career when every day I see something different first hand. I'm about to stop arguing with people. If you think it's all BS then fine. Good luck.
I don’t trust point estimates like 14%. It’s like calculating an average salary and saying it’s $120k. Completely meaningless. What does the actual distribution look like that this was pulled from? No standard deviation. Is it even symmetric? What’s the 10th and 90th percentiles? Just giving a statistic on its own tells me nothing.
>On my visits to the Bay Area, I would ask AI researchers or interns why they are doing their current research or projects, when in a year or three agentic LLMs could probably do them;
This is such a weird point to make that doesn't become correct just because everyone makes it, all the time. Why clean the ocean if some magic future tech will clean them? Why save the world now if some benevolent AI is 'just around the corner' and will do it for us? And people have been making this point for years now, and it's not like my job got any easier. I just got more AI.
https://www.poetryfoundation.org/poems/51294/waiting-for-the...
And I say that as someone who uses Claude Code in complex environments almost hourly; I, as the human, still have to do the thinking as Claude still 'can't jump' [1] and I have seen no evidence that they (or similar AI, any time soon) will 'jump' like a human brain does.
[1] https://www.tomzahavy.com/files/llms-cant-jump.pdf
> This is such a weird point to make
I think it is a great point to make, because if everyone really believed that AIs will do everything without human intervention in a handful of years, as the marketing repeats again and again (AGI, singularity, etc.) and have been saying for years... why then get bothered?
Because we DO know LLMs have their hallucinations, limitations, perform tasks not previously seen way worse than humans, etc. And it seems that, for now, there is not a good or magic solution to it, it is inherent limitations of the paradigm.
Yes, you can feed more and more and more (curated data) and eventually make AIs excellent at task X or Y, but then you spend your time specializing those engines. So the work does not really disappear, it just shifts and you make it more replicable for a bound set of problems.
Needless to say that at some point I prefer to learn (and combine with AIs, it is ok) than acritically getting inputs from something until I become totally useless.
Unless we have a paradigm for which a fully autonomous AI can do everything, this will just become improving our productivity in some ways, with all the in-between bottlenecks that it has.
It's a very silly point to make to AI researchers specifically. If they don't work on those projects, the AI won't advance and won't magically be able to replicate the work in "one to three years".
Can you imagine scenarios that would make it less silly? I will give an example:
- The AI researcher might be working for a lab or company with much less funds than the top dogs. Are they likely to discover something that is worth it before a bigger model becomes more capable?
That company or their staff is probably focusing its funds on getting itself acquired/acquihired by the top dog.
Ask the researchers working on Deepseek. They seem to be doing pretty well for themselves.
Just because a AI will be able to do it in the future does not mean that us plebs will be allowed to have access to it. That alone is enough of a reason for smaller labs to keep going; having a seat at the table.
Relevant xkcd: https://xkcd.com/989/
Why spend money and time making the new flagship model when a future flagship model can make you the flagship model?
Then why not stop researching and doing the definitve model that will solve every problem? Why some people are not doing it?
Bc they are aware of the marketing and limitations. If they did believe it, then they would switch area of research.
> you can feed more and more and more
I have a meta thought..
Hypothetically what happens once there is no more data to be fed to the system? Are we expecting AI to invent its own data and reach full cognition?
Currently we are feeding it the data that humans created but if we stop (i.e "why bother?") thinking that AI will do it all?
The AI researchers are not the ones making the marketing, much less believing in it.
Is this really true? At least one very headline AI researcher is pretty much an Anthropic spokesperson.
That they believe in the PR is itself PR, they're paid to be spokespeople as well as researchers.
I agree. But this is not what you see on the headlines and what money-incentivized stakeholders are saying.
> inherent limitations of the paradigm
This is such a weird point to make. We are currently ( only ) discovering that paradigm; we are not inventing anything. We found a bunch of laws that produce rather cool results but our paradigm is incomplete which leads more or less wordy or frame-rich weird stuff like hallucinations, singularity and so on ... it's childish, really and on that funny pseudo-profound, pseudo-intellectual, pseudo-spiritual ( personal opinion, if it gets you horny, you go, baby ) "universe consciousness unity, Rick James, bitch" level ...
Our bodies and minds need proper AI, not all the stuff we already outsource to middle and/or passionate men and women. Other species on the planet would certainly like to see us get augmented by AI so we can solve as many survivability issues as possible to keep as many ecosystems running long enough ... whatever that means but whether animals and plants are aware of chance and potential is another philosophical debate.
To individuals, software is a hammer and chisel, a knife, a brush and canvas, pen and paper, a reading help, and to a good amount of people it's a microscope and a fine scalpel.
To collectives, it's a tool to work on consensus and conventions, to share and gather.
It's baby steps for civilizations and it looks like our particular species is gonna get stuck in a puddle of our own monkey shit, with bottles of champagne in our hands and monkeys grinding up and down the few ivory towers in proximity.
> why then get bothered
Humans are on different levels. Most have decided that "nature realized the/a bug and wanted someone dead" or "their survival is a matter of chance" is not acceptable at all and some people decided that sabotage, poison, abuse, rape, murder are acceptable means to get chicken shit ...
The "paradigm" of life is far from explored/discovered, so we simply can't content ourselves with presumptions about inherent limitations of the LLM and AI paradigm for any other reason than to uncover ( not invent ) other parts of the paradigm.
We are happy with what AI can do for us but "AIs will do everything without human intervention" sounds weird because babies are born and the older they get and the less sabotaged ( vs influence, cultural manipulation ) they get to grow up, the more breadth and depth humans want to experience. For this they need to learn and use their hands & fingers. They need to feed body and mind to find what triggers what, and what excitement and curiosity are inherent and which can or need to be added/acquired/experienced extrinsically.
How many associations will we be able to make if AIs will do everything without human intervention?
No we don't know your 'points'
The Hallucinations are becoming less, significantly by now.
It also might be already were it is cheaper for one of the big few companies to spend millions and billions to teach the LLM / creating the training data necessary for an LLM to do something which it is not yet good enough due to the fact, that they sell this capability then to everyone who wants to use this capabilitiy.
We have not seen the end of Reinforcement Learning, which does need a lot less training data but more compute.
I'm 'vibing' on the side a handfull of small things, no LLM trained on particular what i'm asking to do. Its very capable of stringing together enough things so it can clearly follow handwavy things i tell it to do, analyse error messages, analysing screenshots etc. all by itself.
There is not a single real ceilling in sight, we only have clear barriers like compute but constant fast progress.
The field of mathematics went from 'useless' to 'you start better using it' to 'gamechanger' in how fast? 1 year after coding? less?
I want signes that we hit a real problem, instead I get cheaper tokens, Chinese models becoming very good as open models, new model updates from the others, mathematicans now saying how good it is etc.
Only half a year ago I had to babysit an LLM, now i tell it 1-3 sentences and it just goes and does it. And that stuff runs without compile errors etc.
If AI makes us 10% or 20% betteer, which is not that much, this alone will lead to companies reduing their expensive staff by 10-20%, which will has real impact on a job area. Some jobs are already hard to sell like cyber security and basic image tasks.
Hallucinations were low hanging fruit in some ways. As someone working on a large-ish complex-ish distributed system that has to be maintained and support customers, it's still very high value to have Claude in the mix, but the core problem of needing to monitor, advise, course correct, and make sure you don't end up with more code and complexity than you need is, at least in my experience, still roughly the same. The sharp edges are being filed off very rapidly, but the core experience of "make and maintain a large system" isn't advancing nearly as fast, IMO.
I'm waiting for the agentic ai platform layer.
We see AI factories going in this direction but there is no real 'the open source ai platform' thingy.
It needs connectors to integrate with k8s, hyperscalers etc. it needs to be able to have a basic router, a way of configuring expert agents and interaction options for the human in the loop.
There is for sure things we need to build or change, but it def feels like to me that it would immediadly fix a few things today.
I'm not disappointed that it doesn't advance as fast as it feels
> The Hallucinations are becoming less, significantly by now.
Yes? What is the mega-solid technique that is used for it? Armies of people using curated data and reviewing it by hand? That is exactly one of my points: shifting the work elsewhere for specialized tasks. More replicable, improved, but, it scales infinitely and is autonomous? Can you assert that?
I am not denying there is some use (a lot of uses!) for this, but this is more nuanced than just: oh, they will replace us. Not at all, that day, with the current technology, is not going to arrive. This is just a systematization, fitting and tweaking of human knowledge by curated data. It is not the one true superintelligence they are selling us. To begin with, they do not have a concept of truth, but of probabilistic truth. Only that poses already a very, very big problem for the path to perfection.
> We have not seen the end of Reinforcement Learning, which does need a lot less training data but more compute.
Noone said the opposite, but I would like to know at which cost and if it is feasible. We do not have even enough compute power for current technology.
> . Its very capable of stringing together enough things so it can clearly follow handwavy things i tell it to do, analyse error messages, analysing screenshots etc. all by itself.
I use it every day for these tasks and it works well BECAUSE I review the output and makes me go faster. It finds a lot of things I would have not found and it also hallucinates another handful of them, which confirms my point about AIs not being able to be fully autonomous in any future point in time unless tweaked exactly for the task, and even then, it can still miss judgement a human could have for edge cases. So I am not sure of how bad or good it can be compared to a human but I am pretty sure it cannot be more reliable than an expert in many situations.
> Chinese models becoming very good as open models
I think they will be better in the long term if they follow this path. Not absolutely better but when mixing with economics and the fact that no frontier model is totally reliable anyway... why pay a lot for something that needs human inspection anyway?
> There is not a single real ceilling in sight, we only have clear barriers like compute but constant fast progress.
The ceiling is the paradigm itself, as I mentioned above. There is not a single chance with current technology that something could become "generically knowledgeable" and "reliable" both at the same time. If it becomes generically knowledgeable and reliable, it is bc of data fed into it and curated and tweaked by humans. This is not an original idea from myself, there are armies of people doing this every day around the world, you can check. This is where a lot of improvement comes from. Can this be reused? Of course. It is a generic solution? No way.
> Only half a year ago I had to babysit an LLM, now i tell it 1-3 sentences and it just goes and does it. And that stuff runs without compile errors etc.
Yes, I also do one-off scripts like this and code snippets, even reviews and others. Now go design a full distributed system. Use agents if you want. We come back in six months and compare it to a system that was properly written and tested by humans and we can compare the quality on some grounds:
You have to count all that money also, not just I vibe coded something and it seemed to work. With full systems things become super messy. Now add the human factor of requirements and back and forth (iterations can be admittedly faster with AI, especially prototypes, but that comes with other costs also)...Not easy at all.
> shifting the work elsewhere for specialized tasks. More replicable, improved, but, it scales infinitely and is autonomous? Can you assert that?
I would say yes and it will scale. It will either happen through central LLM just paying for it and scaling it up to everyone on the planet (literaly) OR by the agentic layer every big business is building into their systems.
You needed some human to use your tool optimized for their company? With agentic layer you no longer need this. And if you look at companies like Google, they were pushing this notion for ages already because they saw an adoption problem of more 'complex' tools and trying to make it simpler and easier. Now you can act from the other side too.
> We do not have even enough compute power for current technology.
Exactly. Right? There is no ceiling if its clear that we don't even have the hrdware. But the hardware is a bottleneck not a ceiling.
> The ceiling is the paradigm itself, as I mentioned above. There is not a single chance with current technology that something could become "generically knowledgeable" and "reliable" both at the same time.
It doesn't need to be perfect, it only needs to be better than the avg human. And the current LLMs are already better than aat least 1-2 people in my team.
> Now add the human factor of requirements and back and forth
Yeah for now. Grill me skill made it a lot easier. Harness engineering is also being worked on, agentic layer, ai factories etc.
And all of this can be copy and pasted. There is only one harness needed which becomes the expert security reviewer and tomorrow everyone can have it.
I'm still discusing progress with LLMs with people and still not everyone is using it or playing around with harnesses or developgn an agentic layer. We still have a lot of work to do to even see how good it will become while it already is really good.
People are already borred of AI today and making wrong decisions based on the current level of AI while i think we will see continues progress for years.
> I would say yes and it will scale
So you mean AI will be useful for any general job without lots of training for those jobs? How about new tasks? Tasks it has not been tweaked for. When I deviated from the average, and not really weird things, when programming, the output was way worse than average stuff. And this is an explicit target of AIs nowadays.
I think you are missing a lot of details here, honestly.
> Exactly. Right? There is no ceiling if its clear that we don't even have the hrdware. But the hardware is a bottleneck not a ceiling.
No, the hardware is a bottleneck, the paradigm as we know it is a ceiling unless you massively and continuously feed this system with average tasks (which is useful). Which is exactly the opposite of what singularity and AGI have been promising.
The systems we have now (unless the paradigm changes) will keep doing, essentially, fitting. No concept of truth and limited inference. That inference is based on already existing data, not on future data. In fact, there have been experiments about feeding output back to the input of LLMs and the degradation of the quality is very visible. If they are supposed to be so "intelligent", why it happens?
> It doesn't need to be perfect
I can agree that for lots of tasks it does not. But for others it is just not a tool good enough.
> Yeah for now. Grill me skill made it a lot easier. Harness engineering is also being worked on, agentic layer, ai factories etc.
I will not deny there could be progress, but nothing similar to "autonomous", "reliable", "super intelligence" or "singularity" with this paradigm.
In fact, often in my experience, this is a waste of tokens for subpar results that shift the technical debt elsewhere. I mean if you try to develop full systems by "vibe-code like" techniques. If you use them judiciously, you can accelerate your workflow, maybe 2x, but not much beyond that if you want to have something worth to be used. Note that here I am talking about the full thing: with testing, quality, maintenance concerns and everything together.
If you want to ship a sub-par thing that will go to the rubbish in a couple of months, then yes, you can do that. But that will fail commercially any way. Unless your job is convincing enough people that you can go 10x faster every time, deliver some sub-par thing, and find another customer, which, to me, would equal a scam.
> So you mean AI will be useful for any general job without lots of training for those jobs? How about new tasks?
I'm pretty sure we will solve this issue. Either already through World Models or another architecture.
It could also be, that we just need a 10 or 100 Trillion Parameter model to match so many generic ways of solving tasks and keeping the concept in the LLMs 'head' to solve it that it will just emerge with parameter size. Like with fable they said that it can chain together exploits which wouldn't work as standalone exploits.
What if the only real barrier is the depth of understanding of concepts and this is exactly what is getting solved with parameter count?
But look how young this field really is if you start counting it when it became relevant on mass. Its not 'just' an LLM which is changing the world, its machine learning overall. Robotics wouldn't be were it is today if its not for machine learning. Took humans time and energy to take the leap, to start learning what the status quo is and then actually doing more with it.
> the paradigm as we know it is a ceiling unless you massively and continuously feed this system with average tasks
While I do think its doable to achieve AGI in 5-15 years, even if it doesn't happen and it always means that people train an LLM or whatever, if you need 10 experts to teach this to an LLM OR every single senior has to teach this to their juniors every single time, the LLM will always win.
I'm now team lead for 10 years and every single year I teach them the same thing over and over and over again.
the craziest thing about this? If i wouldn't tell them what they are doing wrong, they wouldn't even know it.
Quality is already a very flexible term for a lot of people.
> I can agree that for lots of tasks it does not. But for others it is just not a tool good enough.
Yet.
> If you want to ship a sub-par thing that will go to the rubbish in a couple of months, then yes, you can do that. But that will fail commercially any way.
Now we come to the reality: I have seen so much garbage software its crazy. People using md5 as a password hash in 2024! No clue what coding best practices are, teams without code review, teams without a security expert not even knowing what crazy things they do day in day out.
Just a few month ago a team build an API for my team including a Swagger UI. Half of it didn't work. You pressed a button on the Swagger UI and a 500 returned.
And do'nt underestimate what it means that a lot of business people don't like software people. You know that fruit basket we get? and water and stuff? they don't do it because they like us they do it because thats what you have to do. if a Product Owner starts vibe coding with AI, he will have leadership convinved in no time, then it goes on production and it will run for waaaaaay longer than anyone would have guest.
Besides that there is plenty of software were complexity is less relevant or security is not that big of an issue.
> I'm pretty sure we will solve this issue. Either already through World Models or another architecture.
Please elaborate. How? With which technique? Currently the only path forward is to feed more data and tweak for specific situations (fitting, basically). How does that help in the general case or in new situations with current tecchnology (LLMs, concretely). Noatter how far you get, this is not a general or reliable solution. It van only simulate more generality or more reliability by training and tweaking. Nothing else. At least, with this paradigm.
This does not mean they will not be useful. What I challenge here is the AGI or singularity. We are far from that.
> I'm now team lead for 10 years and every single year I teach them the same thing over and over and over again.
I have been a lead and an architect also for years at different position. I think you miss how much tacit knowledge and judgement there is inside the brains of each of us that an LLM is not capable of. And if it is, then you have to dumo so much context that it is better to go do it yourself. There is a cost to that also actually. It is not just so "dry and technical" the knowledge. Maybe yes to learn Java patterns or C++ constructors or the like.
But not for "given this situation with all these specifics", which solution would you bet on? Probably the LLM will give you a shitty REST API that is not what u need at all.So u tell the AI. It gives u something else generati g 30-50% of "decorated code". Now it seems to workso you use it. Now you do this every day. Come back in 2 months. You generated a lot of fat.
Now you have a bug. You do not know even where to start. Thisis theprice to payfor speed, as usual: technical debt.
Now you tell me you put three agents to talk and burn 2000 usd in tokens. Great! Is the final solution better than what you would have achieved? Not sure at all.
TBH I am not into agents bc I do not trust a tool sniffing all my code and for copyright concerns. But I saw some and use a prompt with limited access and the best I can take out for my speed + control when coding is tech discussions to decide on it, error catching, test generation, one-off scripts... But never "make an app like this or that". If I ever do that (I did it a couple of times) is for scaffolding and later throw away 70%.
Namely, to see something that runs on screen quickly. But later you need to spend time yourself as usual. Not a bad thing, just that this is not what you deliver and need the work done. Iterations etc.
> Please elaborate. How? With which technique?
Reinforcement learning can just solve things even if they are new. It doesn't understand how a tool works? Give it a vm with the tool, a thousand agents and let it discover it automatically.
Use the thumbs up/down emoji + chat analysis when a customer is unhappy, feed that to a RL Loop.
The AI Researchers though work on World Models, grounding the AI and letting it simulate. It can do the simulation in parallel (unlimited) and choose what is best.
> I think you miss how much tacit knowledge and judgement there is inside the brains of each of us that an LLM is not capable of.
But thats my problem. Soooo many do not have this even as senior developers.
> Now you have a bug. You do not know even where to start. Thisis theprice to payfor speed, as usual: technical debt.
Yeah now i just ask the LLM to describe to me the bug. Works very well.
> Now you tell me you put three agents to talk and burn 2000 usd in tokens. Great! Is the final solution better than what you would have achieved? Not sure at all.
This is the thing. It only needs to make the team 10-30% better to compensate token budget with one work collegue. We have reached this level in my opinion already. Choosing a head count vs. choosing tokens.
But it becomes cheaper and easier and better. So you will not just be able to do ith with 3 agents but with 20, 50 or 100.
It will be better if your team is an avg team. It will be worse if you have a high profile team, for now. But man our industry has such a weird broad quality spectrum.
> TBH I am not into agents bc I do not trust a tool sniffing all my code and for copyright concerns
In worst case, my team always do code reviews, I do a code review on an ai instead of a human and adjust the harness or the infos the ai can access. I can actually work on making this workflow better and then i can clone it or spin it up for every single PR. For a human? I have to train them and they might leave.
But there are plenty of cases were code doesn't matter. Researchers write a lot of random shitty uggly code as long as it does what it does, it doesn't matter. I have scripts for small tasks, we have microservices which do one thing because it is a tech stack we only need for one use case.
> eah now i just ask the LLM to describe to me the bug. Works very well.
I think you are confusing giving theories about what a bug might be with certainty. It does help bc it csn accelerste things, but many times I had AIs with challenging bugs throwing a lot of misleading theories to me. For the easier bugs, I was just as capable most of the time. Not every time, so there is some potential time saving there. But also time waste.
As for research and fast prototyping you are right: I find it a good tool to explore bc yiu do not need the quality of a final product and researxh is in big part throwaway work.
But I was talking about software that needs features, maintenance, etc. This is just not the same thing.
I think you meant to post this in response to https://news.ycombinator.com/item?id=49174900?
Yes I double checked the quote is not in the article. HN is probably the best place on the internet for people actually reading the article, but this being the top comment here suggests that the majority of voters still do not read the article
> I, as the human, still have to do the thinking as Claude still 'can't jump'
I still have to do quite a bit of thinking but the amount of of thinking I do per task is trending down. I agree LLMs are not good at abduction but very few humans are either and very few jobs/tasks require it. I can't talk for researchers jobs though. But perhaps fewer researchers would be desired by these labs (not none).
> I still have to do quite a bit of thinking but the amount of of thinking I do per task is trending down
Don't worry, I'm sure you'll hit your goal of zero thinking soon!
Trending towards the Homo Amens Mechanicus: the mechanical thoughtless human. What a goal
Only bc of this I will keep balancing what I do with my brain with what machines can do. dangerous outcome.
The IQ willl drop if we just become mechanical acritical people the same way muscles get worse if you do not exercise.
Cognitive offloading is delegating to the AI and still owning the answer. Cognitive surrender is when the AI’s output quietly becomes your output and there is nothing you feel is left to check. For software engineers the line between the two moves under your feet most days, and most of us are crossing it without noticing.
Source: https://addyosmani.com/blog/cognitive-surrender/
> I agree LLMs are not good at abduction but very few humans are either
I, too, am glad that few humans seem good at abduction.
And I hope LLMs don't get better at it.
"Abduction" being the way the article used it as in "explanatory reasoning in justifying hypotheses". https://plato.stanford.edu/entries/abduction/
Same reason some think preserving the environment is pointless because the believers will ascend to heaven, either way. It’s a religion. It’s dogmatic nihilism.
> And I say that as someone who uses Claude Code in complex environments almost hourly; I, as the human, still have to do the thinking as Claude still 'can't jump' [1] and I have seen no evidence that they (or similar AI, any time soon) will 'jump' like a human brain does.
Sure it can, turn up the "temperature" a bit.
There's this notion that human "jumping" is magic. It's not. It's all based on inputs. Including unrelated inputs, past inputs, and feeding yourself your own thoughts.
The hard part is not the ability to make conceptual jumps. That's just random search. The hard part is discrimination: whether a given mental jump is "creative" or "insane". Iterated, the problem is that of balancing between the two failure modes: relax your thinking too much, and you'll start thinking nonsense thoughts; tighten it too much, and you'll be just following immediate-term rewards and obvious thought trains. It takes time to find that balance, and plenty of people at various points err in one or the other direction (e.g. small kids in particular tend to err on the "crazy non-sequitur side", but that's because they're learning the basics of reality and social interactions).
100% agree. Plus you can easily instruct it to use inspiration from unspecified unrelated and counterintuitive concepts at random and in parallel. The "jump" is not "on" by default because it would waste tokens on high risk paths, not because it's incapable. It's very capable if you are willing to wast some tokens.
> The hard part is discrimination: whether a given mental jump is "creative" or "insane"
That's what they mean by LLM's can't jump. They mean it can't make a creative jump. Their example is Einstein's Theory of Relativity - It's not a random jump.
Einstein didn't magically one day woke up and jump on relativity theory. He had prerequisites in terms of recent mathematical advancements (notation) and physical discoveries, and a job exposing him to a lot of lateral thinking, and time to bounce ideas around in his head. We don't know how many fruitless jumps he made before making one that we remember him for.
No one’s saying that relativity appeared out of thin air, but Einstein still made a conceptual leap that LLMs are incapable of replicating.
although, if i'm out of tokens and have to wait a full day, i won't bother doing some things manually because the day i'll spend doing something won't take more than 1 hour the next day when tokens are available again.
That seems like a somewhat orthogonal point? Like, if I'm a carpenter and my batteries all run out / I can't actually power my power tools then the best course of action is to go home and recharge all the batteries instead of trying to hand-cut 100 pieces of lumber today. After all, the power tools can do it a lot faster (and with less effort) than I can.
I say this as someone who's watched a bunch of woodworking videos but hasn't actually done this myself :)
I read that more so as, I'm a carpenter and my batteries have all ran flat, so I'll put them on charge and do something else today. I'll cut up the lumber tomorrow when the batteries have charged.
According to the article that part being automated isn’t more than 18% of your day anyway
At this point at least half of your day is spent doing work that you shouldn't be doing in the first place, but over past decades companies saw it fit to eliminate specialized roles with legible paychecks, and smear the work they did on everyone else until it disappears from the books.
Self-service and office suite software is largely responsible for this.
A fully automated utopia isn't just going to happen. Even with frontier models, the integrations, the evals, the UX, need a lot of work and someone needs to do it. After I've automated this thing I'll move on to the next task, this is what it means to be a software engineer.
An actual utopia would require never-before-seen democratic mandate from people who are currently on the brink of hot civil war
It still makes a massive difference for me if they only need a handfull people now.
Generating a good looking UI for example, is so much easier now with LLM.
For a joke I asked ChatGPT yesterday to make a short promoimage for a 'joke' idea i had, it was above avg. I have for sure seen worse Marketing Images than what ChatGPT generated.
It looked similiar to plenty of other Marketing Images but its not that anyone cares.
Robot powered factories still have humans there, the gist is that they are a tiny fraction of what a classical factory would require 50 years ago.
I completely agree. There's no guarantee that researchers won't still be needed despite AI.
AI has made 'jumps' in demanding fields like leading mathematical research and has made advancements in AI research itself. Is now a good time to start a maths career? Is there a field of research (yours?) which is inherently (more) AI proof?
Btw, I think the discussion of Einstein's career in the paper you link is historically wrong in many respects, particularly the argument about 'weak signal'. Einstein was in fact working on some of the most mainstream and widely discussed problems in physics of the day, he is admired for the creativity of his solutions to those problems, and much of his work built incrementally on ideas and breakthroughs that came (long) before (as all research does).
Article suggests that a central motivation of Einstein's work was resolving action-at-a-distance in Newtonian mechanics - yet Maxwell introduced the same Lagrangian field theories for electromagnetism we use today 50 years earlier to solve the same problem for Farraday's laws of electromagnetism. Similar wave equations existed even earlier. Heaviside in 1893 extended this technique to gravity (matching 'weak field' GR) 20 years earlier. So this is perhaps the one aspect of gravity that had actually already been solved before Einstein. Authors might be conflating his work on action-at-a-distance in QM.
Einstein's GR extended the linear 'weak field' understanding of gravity to include the non-linear self-referential case where masses themselves create gravity. This was mathematically incredibly difficult but was necessary precisely because SR's mass energy equivalence created so many strong signals that were unresolved. For example: if finite energy is mass, then mass changes as objects accelerate past a large mass like a start, and hence their propagation in space could not be explained by linear EM style field equations. Many such considerations were causing very 'strong signals' in SR, and there were analogous problems in QM atomic models being developed at the same time.
SR was also a solution to a problem that was actively being worked by many of the leading physicists of the day. SR actually does match Newtonian mechanics for a single observer - it resolves contradictions in the case of separate observers, by allowing them to assign different values to the speeds, masses, etc of objects such that each object appears to follow Newtonian mechanics for each observer. Again, this was necessary because of a lot of contradictions related to the behavior of light that had been well-known for ~20 years at the time.
Personally, I don't consider this kind of reasoning to be beyond the capabilities of future LLMs (even current LLMs if the task was broken into technical rather than philosophical problems). Personally, I doubt that such problems could stand open for 20+ years waiting for a creative genius to solve them in the modern world.
And don't get me started on the philosophy.
Sorry where in the article does it make this point? I cannot find the text "On my visits to the Bay Area" anywhere in the original article.
It doesn't, GP was responding to the Gwern story at https://news.ycombinator.com/item?id=49174900 but posted in the wrong place.
Also people tend to forget that LLMs still just work on compressed data... Where are the MAJOR breakthroughs? Where is all the "crazy" AI output going? Software seemed to degrade in quality a lot in the recent years. All "improvements" LLMs go through are simply improvements on how to burn more tokens out of my pockets given that Claude now want an actual browser extension to "visually" confirm small changes every time I use it for UI. They are still just data parrots.
From what I see most benefits are for people that work with LLMs, but usually smaller percentages never 50% or more because of the LLMs (OK, unless you were doing basic, repetitive stuff, but then that's not to write about).
Which kind of answers the original question "why bother working?" with "because now, I can do a bit more than before".
I also see bad quality (in code, documents, presentations). It comes from people that had no clue how to do something before and now they imagine that just asking Claude is solving well the problem. And is annoying (and hard) to explain to it them, and then they get frustrated.
LLM don't work on 'compressed data'. LLM compress data into their latent space which allows them to become general.
They learn the concept of things and how to do them because this is better compression than learning concepts one by one.
Which means, if an LLM 'learns' the concept of a poem, it can put everything into the formad of a poem instead of learning a billion poems.
> They learn the concept of things and how to do them because this is better compression than learning concepts one by one.
When anthropic looked at how an LLM does addition it found it had some mental math heuristics that might or might not always work. The LLM hadn't learned the concept of addition. It had learned some heuristics that might work for some numbers. The result is that LLM's cannot add numbers reliably because they have not learned the concept of addition.
It learned a concept of a heuristic which made it smart enough for the learning reward.
Might be an architecture issue or a parameter size issue that it didn't learn to do math like a caculator.
But look at your own math skills: How many numbers / how big of numbers can you keep in your head? How far is this heuristic away from how much a human learned until you start using pen and paper or a caculator?
There’s a small group of established architects talking about harness engineering, but I’m not sure anyone is actually listening to them.
And those same architects are quietly extracting real productivity from GenAI.
And even this write up skips that info by waving, “Some people…”
Mind sharing some names or something else we can learn about?
https://martinfowler.com/articles/harness-engineering.html
https://developers.redhat.com/articles/2026/04/07/harness-en...
https://loiane.com/2026/04/harness-engineering-missing-layer...
https://devarch.ai/
Thanks
Idk, maybe some people get crazy productivity out of LLMs. To me, going deep into the AI bubble, reading about terms I've never seen before just feels like some crypto bro bubble with people being too deep into the sauce to notice that these things are not the wonder machines they believe so hard in...
The only thing I noticed in some of the software I use is more design changes, but they are often not better than before.
By trade I'm a UX Researcher/Designer who designs in code (HTML/CSS) and have done so since 2009. Recently I vibe coded an entire python app with a database and each time I didnt know what to do I would just feed screenshots to Gemini or Codex for guidance (i think i could share my screen with Codex and it can guide me via a voice conversation). I know I could follow up and build a companion iPhone and Android app using these tools.
Overall, I'd like to understand those who have a positive outlook on design and software engineering as a career. Where do you see the opportunity where I just see a bleak one where anyone can do this stuff by typing or talking to AI? Myself, after 17 years in the field I am begrudingly back in school for a new medical career. As well, anytime an IT recruiter reaches out I am getting responses back only after under-cutting the hourly rate I use to demand and what others probably are still trying to get. And with it feels even bleaker as it becomes a race to the bottom!
In my experience, not everyone can really do this stuff by typing. I think you need to be creative, resourceful, inventive, open minded and have ideas how to approach the typing/prompting. I see many people struggle in using AI.
> In my experience, not everyone can really do this stuff by typing. I think you need to be creative, resourceful, inventive, open minded and have ideas how to approach the typing/prompting. I see many people struggle in using AI.
The problem, for the profession, is that the set of people who can really do this stuff by typing is close to "all of them". I'm not seeing anyone struggle with using AI. I see struggles from professional software developers because they are trying to get quality output, but if you don't have a bar for quality, just about everyone can create their own software.
A poster a few months ago had a Show HN about his 7 year old kid, barely able to read, who was happily vibing up games.
I can attest to the other side as well, that I have seen professional software developers outputting code of lower quality than AI. And I would say that during my career (18 or so years) I have met a small number of quality software developers or engineers. Although that might be because I was not in Silicon Valley where most of the smart/hotshot engineers converge.
The industry has vast (and increasing) oversupply of “programmers” versus diminishing demand. Add to this, the adoption of AI.
> Overall, I'd like to understand those who have a positive outlook on design and software engineering as a career.
I think until the market better achieves some equilibrium, there is no way general software programming (sorry “engineering”) should be considered as a career. That said, there will always be opportunities in particular markets or specialties.
I also work in UX and SWE, and heavily use GenAI in my work. I don’t have a positive outlook for people who limit their career to one of those fields, but I do have a positive outlook for generalist, multi-disciplinary careers. When you have the experience and skill to steer product development from end-to-end, you can produce high-quality products super-quickly. The experience and skills are the differentiator — if you lack those you can still use GenAI to move fast but probably in the wrong direction.
So one person now doing the job a handful use to do. That's what I hear you saying and Ive been thinking since my lay off in Feb. prompting me to be back in school.
It also seems kinda tone deaf. If someone basically told me I was wasting my time and asked what I would do in the future, I would not bother giving them a particularly thoughtful answer because trying to spend effort justifying my life choices to them would be the actual waste of time.
What kind of answers were they expecting to get?
It reminds me of Richard Hamming’s notorious question. I like the summary at https://bestjelly.substack.com/p/hamming-questions (which starts out with a quote from another site):
> > Mathematician Richard Hamming used to ask scientists in other fields "What are the most important problems in your field?" partly so he could troll them by asking "Why aren't you working on them?" and partly because getting asked this question is really useful for focusing people's attention on what matters.
> I imagine someone being asked this question, and how they should respond. I think like so - ‘Fuck off Richard’.
> This is partly because I imagine this question being asked in a kind of snarky, gotcha kind of way, with some sort of nerdy superiority. Like ‘ha your behaviour is inconsistent with your implied preferences, you idiot, do you even von Neumann–Morgenstern?’
Also, if you believe your well-paying job is eventually going to be automated you would be prudent to bank the money while you prepare for the future.
> We already know developers don’t actually spend most of their time writing code, with studies at Microsoft and elsewhere showing it’s closer to 14 percent.
Anyone else finding they're spending more time writing code (or at least driving agents to write code) now?
14% used to feel about right for me - I'd spend the rest of the time researching approaches and libraries, planning things out in issues, or sometimes just thinking really hard about problems I ran into.
Now... I still do those things, but I'm doing many of them faster - and I'm often doing them while my coding agents are churning away on code.
There's also this weird effect where the harder a problem is the more I can get done in parallel with it, because an agent might need to spend 20 minutes on it without my involvement.
You read my mind. I suspect it is a transient spike while I blow through the backlog and I'll run out of things that AI can vastly accelerate in, say, 6 months. But right now? I'm spending 80-90% of my time blasting through implementing features, finding bugs, fixing old bugs, writing and improving my tools... Code was never the majority of my job. But right now the bar to doing things is so low and the sheer joy of blasting through some previously tedious, low to medium hanging fruit is thrilling.
>> We already know developers don’t actually spend most of their time writing code, with studies at Microsoft and elsewhere showing it’s closer to 14 percent.
> Anyone else finding they're spending more time writing code (or at least driving agents to write code) now?
Not really, as once it is time to write code, the problem has been defined/understood (to the degree possible with knowledge acquired at the time), and encoding it is largely an exercise in typing along with verifying assumptions via test suites.
Does GenAI quicken some portions of the above workflow? Sure, in the same way IDEs with contextual code snippet suggestions can make encoding faster.
I used it to write SQL and make dashboards. Back in the day, I would spend a lot of time doing that, then I changed roles. I dipped my toe in it recently and used AI exclusively. I would send a prompt, see the output, decide if that is what I wanted or not. I kept my brain in "what-if mode" and I let the LLM handle the technical specs.
'What-if' mode is a game changer. And underrated.
The cost of prototyping has crashed. So you can A/B/C... various ideas, test them in the real world or by personal preference, to taste, and converge on designs that work for you.
This used to very hard and expensive. Now it's so cheap it makes the idea -> test -> refine loop much tighter and faster.
>The cost of prototyping has crashed. So you can A/B/C... various ideas)
Oh yes! I would think "I wonder how the data would look if I did X" and feel the pain of complicated SQL... but then the LLM would just do it and I got to see if I liked or not! It was a delight
Does the code get reviewed? How do you deal with increased amount of code that may need to be looked at?
I review the code that matters - anything security adjacent or that's an API that will be used by other code in the future.
I don't review code that either works or doesn't - most HTML and CSS layout code for example. There I test it on desktop and mobile and commit it if it works.
Ditto for stuff that's simple. A JSON endpoint that runs a SQL query and returns some JSON? If it works and a glance at the tests looks OK then I trust my agents wrote it properly.
I'm getting more confident with my judgement over what needs a close look and what doesn't over time, as so far I haven't been majorly burned my any mistakes that snuck through.
Honestly, it's similar to being an engineer on a larger team. You don't review every line of code written by every one of your coworkers.
I think this is THE issue of our time as programmers to be honest: do you review every line of code an agent writes?
An increasing number of expert programmers are moving in the direction of NOT reviewing every line. It's working out OK for a lot of them.
I've found that engineers on a large team do read every line, mainly due to the fact that the skill levels run the gamut from intern to lead, and only 1 or 2 people out of 12 might have knowledge of the application being modified.
It's actually getting worse due to "AI code bloat", for example I have 16k lines of code to review across 3 apps by the end of this week. Normally it would be a quarter of that, but what Claude produces is extremely verbose in some places and anemic in others, and I can't tell at a glance what's right and what looks right with that much ground to cover.
Goodness, how is that being tolerated? I guess it can’t be stopped without a lot of political capital; but 16k lines of code is HUGE, and I cannot imagine that it’s actually 16k lines of value - I’ve written whole new subsystems of a product in fewer lines. Are these all written in an exceptionally verbose language like Go or Java? Are they VERY well documented? Are they doing things they shouldn’t be doing???
Brother at my startup one person shipped 75k lines in two weeks. We’re living in the golden age of slop
software """engineering"""
I mean, I've done that for a couple of things, but probably 90+ percent was vendored libraries and javascript which could be ignored. Sadly, this is not one of those cases, and Im guessing yours isn't either?
> I don't review code that either works or doesn't - most HTML and CSS layout code for example. There I test it on desktop and mobile and commit it if it works.
Good example of what not to review if you're working on your hobbies. Also exploratory can sometimes be done this way. However, this ultimately boils down to how you approach programming as an engineering discipline, including your responsibility for the outcome.
> I'm getting more confident with my judgement over what needs a close look and what doesn't over time, as so far I haven't been majorly burned my any mistakes that snuck through.
This doesn't generalize well. If you drink raw milk, or if you don't wear your seatbelt, or if you don't escape your user input correctly, you'll probably be fine, but I really hope aspiring programmers/engineers don't take this attitude towards any serious task. One should always examine their biases, tools' failure modes, etc. regardless of how many times something didn't fail.
> Honestly, it's similar to being an engineer on a larger team. You don't review every line of code written by every one of your coworkers.
One [should] review the code they're responsible for. In a team, people usually assign you (or ask you) to review code, and the work is divided accordingly. If the code isn’t reviewed by the code owners, it’s a problem, not something inspiring!
> An increasing number of expert programmers are moving in the direction of NOT reviewing every line. It's working out OK for a lot of them.
Have you considered that the sheer amount of code being generated is what makes thorough review infeasible, not that it’s a desirable approach?
But if you observe that the agent day after day do handle user input safely; and also routinely run an agent that scans for security vulnerabilities and observe it finding cases where input is not handled safely in existing code, you may conclude that the chance of an issue is at the same level at, or probably lower than, if a human wrote it and a human reviewed it.
("Escaping" user input is not good practice though, use parameters, assuming you are talking about SQL.)
> observe that the agent day after day do handle user input safely
How do you observe the issues that aren’t apparent via a GUI? Do you notice the circular logic in your reasoning?
What do you mean GUI? Of course the observation of the agents behaviour happens by reading the code it emits.
> Ditto for stuff that's simple. A JSON endpoint that runs a SQL query and returns some JSON? If it works and a glance at the tests looks OK then I trust my agents wrote it properly.
That is *exactly* the sort of area I *wouldn’t* blindly trust AI, there’s a huge security boundary there. What if the AI is doing string concatenation with user-provided data???
Once you've seen the AI not make mistakes like that a few dozen times you start to trust it not to mess that up in the future.
Is this really a rational strategy for something whose nature is to be right most of the time and then spectacularly wrong a much lesser amount of the time?
Yes, because the mistakes in code are easy to spot. I wouldn't use this to write me an unreviewed legal brief.
But you said that in this case you don't review the code because you trust the AI having seen it write that kind of code before.
you literally just said you don't review this code!
I agree with a commenter above/below (depending where this comment lands). For some time models won't do this. And any review from review agents would caught this. For most of the AI programming there needs to be a more stricter (automated) review process now. Most SAST tools would caught this type of security issue.
We're pretty far past this if you're using anything close to the sota models.
But you could be defensive with a security checklist in agents.md and have adversarial review, if you wanted.
lol I think you’re setting yourself up for failure. Why? Just because something works doesn’t necessarily know you the boundaries of it. What scale does it work for? Will it crumble under load in prod? It works but allows cross tenant access (security issue) because security checks weren’t in the location you thought… Dangerous!
I've been waiting with baited breath for this to bite me for more than six months now. So far unbitten.
meaningless without telling us what you're working on
I'm pretty public about what I work on: https://simonwillison.net/elsewhere/release/
I think it also depends on what you're building. Some solo project or basic html thing? Sure no need to review every line. It's a bit different when you're working on foundational libraries that a business relies on, anything touching a production database, etc.
> Honestly, it's similar to being an engineer on a larger team. You don't review every line of code written by every one of your coworkers.
We don’t because everyone is accountable for his or her own mistakes. So everyone is incentivized for their recklessness to not be the root cause of some bug.
> An increasing number of expert programmers are moving in the direction of NOT reviewing every line. It's working out OK for a lot of them.
Have you ever asked your users? What about bug reports? Is the amount and rate decreasing?
If you're going to be reading the code you might as well write it by hand instead of using AI.
Yeah. I spend most of my day driving agents to write code, verifying the results, orchestrating work streams, and so on. The rest of the time, a Fable agent is organizing work in Linear/Jira and making sure coworkers are getting their stuff done in a way that won’t conflict.
I've found that using LLMs for significant amounts of code generation completely drain the result from any dopamine I would get doing it myself.
Have others noticed this as well? This is going so far as to me losing interest in side projects because I have "lost touch" with the code base.
Yep. ADHD very strong in this one, so LLM code generation takes pretty much all joy out of coding. It’s like watching a computers play chess. Yeah, no thanks.
My ADHD gets dopamine from getting something working, not from the tedious process of getting there. So YMMV.
With GenAI, we can now produce something without caring about it - or while caring about it very little.
And the parts we don't care about aren't necessarily worse, they are just... arbitrary. Could be good, could be bad, no one knows, because no one really cares.
I find that if I care about something a lot, it's a pretty similar time investment than pre LLMs. And it makes me feel invested and proud in the result, motivated to show it and improve it.
If I care about something very little, in the past I just wouldn't have done it at all. Now I might, but I feel that same disconnect you mentioned.
I think being strategic in what we do and do not care about is likely the key skill we'll have to build to actually make the best of the tech.
I feel like all you need to know about how seriously to take this is that they cite that ancient early-2025 METR study, and describe it in the text as "recently one even found..."
Same thought - 80% through reading it occurred to me to check the citations. A few items from 2025 and most well before that.
So much has changed since late 2025 one can’t really draw any conclusions from this.
In fact, I’m guessing things will continue to move so fast that by the time one were to execute a survey of developers, many of the responses and findings are no longer relevant.
Your point really goes both ways, we really don't know anything about how LLM usage is affecting anything. No one knows, it's the wild wild west, which is whatever. But I think no one can really draw conclusions from what's happening in tech right now.
Reminds me of COVID and how everyone was fighting over early trends during that time.
Exactly. I saw them using things from 2025... AI sorta sucked then and didn't really "take off" until that Opus drop in December or whatever it was.
I felt the same and why didn’t the authors look over METR’s recent material?
https://metr.org/blog/2026-05-11-ai-usage-survey/
The whole point of the 2025 one is that they found the self-reporting to be significantly inflated, which is why self-reported surveys like this one are hard to trust.
Yes, but their newer write-up discusses that (and shows that the self-reported numbers have gone up radically, in a way that suggests that even if there is some inflation, the numbers are almost certainly positive if you deflate).
They also have an update -- linked from the original study! -- explaining that it's out of date and no longer reliable, and explaining why they had to cancel a follow-up study because it was understating productivity gains (but also was showing wins for the people who carried over from their previous study): https://metr.org/blog/2026-02-24-uplift-update/
The authors of this paper decided to ignore all of METR's follow-up data and discussion, and to report only the ancient number from early 2025 (a time when Windsurf was state of the art). And then, rather than apologizing for it, and caveating it as a number not to be taken seriously, they described it as a study done "recently."
That's either shockingly dishonest or incredibly out-of-touch.
> shows that the self-reported numbers have gone up radically, in a way that suggests that even if there is some inflation, the numbers are almost certainly positive if you deflate
Or the hype machine is even stronger, making people inflate those numbers even more.
If I wanted to be uncharitable, the question of "how much slower you'd be without AI" compares against people's AI-less productivity from 2026, which could have become much less than 2025 AI-less productivity, if hypothetically AI use makes people helpless and deskilled.
Like many others in the comments, I feel there are a lot of assumptions in this piece. Before, coding is only 14% therefore, small slice. I think that's a very superficial assumption. That was because coding was expensive and we needed to be sure we didn't code the wrong thing. If code is as cheap as it is now, we will optimize differently, we will structure around it. Instead of so many meetings we will code 5 different versions of the same thing and choose, etc.
> That was because coding was expensive and we needed to be sure we didn't code the wrong thing.
Coding has never been expensive as it is nothing more than a reification of a solution to a problem as it is understood at that time.
It is the underlying understanding of the problem which has always been expensive and remains so.
Coding was expensive in the sense that once you decided what to do, it took a few engineers months / years to do moderately complex projects. That's not true anymore. Therefore the risk of "coding the wrong thing" is less.
> Coding was expensive in the sense that once you decided what to do, it took a few engineers months / years to do moderately complex projects. That's not true anymore.
This conflates two orthogonal concerns; understanding what needs to be done with encoding a solution for what needs to be done. I am reminded of an axiom I have long held:
> Therefore the risk of "coding the wrong thing" is less.I disagree with the proposition that being able to generate "wrong code faster" has anything to do with understanding what ultimately needs to be done. In fact, exploratory efforts having the intent to refine problem domain understanding (a.k.a. "proof of concept" projects) is an established industry technique whose work product is best discarded.
Is it? The temptation to start without a thorough design is now much stronger because the implementation osnperceived to be cheap and easy to replace. But if you start building the wrong thing fast, you still get the right thing later than when you had checked properly at the start.
CICD and iterative development and agile work because interactive building leads to better product. Otherwise we'd all build perfect software using waterfall methodologies and release only once a year. If implementation is cheap then you implement, review and then re-implement. Not even iterate in the extreme case. Throw it away if needed and start from scratch if needed.
Lots os engineering projects took as long as you mentioned because they were horribly specified. Then the engineer either waits for an answer (which they generally do not get) or takes a decision (which might not be ideal without domain knowledge). Then you release and the customer suddenly starts explaining what they want.
You can improve some parts with LLM (make more prototypes, iterate faster), but if humans are the bottleneck and the problem itself is complex it will not be 10x improvement.
Of course, maybe we talk about different type of project, there are also the type of projects like "a web-shop that sells stuff" that someone considered "expensive to code".
Specification is hard because you are trying to predict a future state in a vacuum. You need to do that because creating the product is very expensive coding wise. If instead you could instantly build to your current specification then you could iterate on the specification against a real product. Even if all that code is thrown out the end result would be a better specification.
For some projects specification is also hard because of interdependence between modules or data or parts of the project. To which you might answer "make more tests". And we turn in circles, because it is hard to test something you did not thought of.
My experience is that users (customers) even when presented with a prototype (done fast by AI, or previously, in the slow way) can take days to "discover" or "think" about things they wish. Then more days to understand why some wishes can't be done because it breaks other stuff they want.
Maybe you have smarter, faster, more experienced and decided users. Then I can only say: good for you, you are lucky!
So you're suggesting that coding will take more of the PRD phase?
Basically yes, there will be more coding in that phase, more prototyping, the PRD phases will be shorter too, there will be more pressure to deliver quickly and the PRDs will be under more pressure to move more quickly. This is what I'm already seeing to be honest.
> Before, coding is only 14% therefore, small slice.
But also, no, because they write:
> “coding” (not including bug fixing, testing, etc.)
What if bug fixing includes "coding", or "writing code", or however one would want to define that? Especially in the enterprise setting they evoke, a lot of work will not be "coding" in the sense of churning out new features, but "coding" in the sense of fixing bugs. I know a lot of my "coding" is in this category. But we're not given a number for it. I suspect the slice would be bigger if they included this type of "coding".
I can't remember of any product in my lifetime that was more over hyped than AI.
It's a huge piece of shit and if I wasn't forced to use it at work I would never use it.
It writes dumb, throw-away code and adds thousands of dollars/developer in costs. All this crazy code that we're adding to our projects will come back to bite us in the future, there's no way it won't.
> Myth 2: Writing Code Is the Bottleneck
Writing code is indeed the bottleneck for same resource constrained companies.
Rapid code development creates more opportunities for trial and error, providing companies with more information for decision making, that previously might have been addressed by meetings.
Of course, this might bring other problems, but it might not right to generally speaking that writing code is not a bottleneck.
I’m very suspicious of this objection, because when Claude first landed the same people now saying “code is not the bottleneck” were saying “the generated code doesn’t work.” Smacks of moving goalposts.
The only solid objection to “AI is going replace developers” is “AI is an accelerant.” It helps developers move faster. I haven’t seen anywhere it has fully replaced developers.
Whether this leads to a large number of job losses depends on whether you think we can increase software output by the same factor as the acceleration and still be profitable. I think we can, latent software demand is extremely high. I also think we’re nearing the limit of capability with current models.
Situation could change if more advanced models emerge, but some of the more foreseeable advances probably have compute requirements beyond today’s hardware.
I don't understand Myth 1 (Developers Spend Most of Their Time Writing Code).
They quote a study in which developers report to spend 11-14% of their day coding. The rest is stuff like solution design and meetings. The insinuation is that AI can at most automate 14% of your day.
The problem with this argument is that once you have code, some (not all) of the precursors to code go away.
Okay.
Show me the evidence that AI has an impact on productivity when doing design work. Or reducing meeting load.
My own experience is that AI doesn't tighten the design cycle, and in fact might extend it by encouraging gold plating.
> Or reducing meeting load.
Well, I expect when you've vibed too much and lost track of the code, and can't answer questions in meetings anymore, you'll stop getting invited to them.
How often do you talk about low code details in the meetings? It is mostly about requirements and goals (IMHO) and "architecture"/design, and now I wouldn't even bother my peers with questions about the code as I can let AI tell me how things work exactly as written in the code, not some thing that someone remembers or thinks how it works (in large codebases, most developers only know in detail the things they are working on currently or from recent history)
> It is mostly about requirements and goals and architecture/design
People vibe code because they have no clue about any of that. Not because they're slow typers.
Anecdotally I find them helpful for design and I don't find them nearly as helpful for coding (strange, given others experiences reported online)
I am working on a compiler for an OCaml inspired language (I am on a sabbatical) and used them a lot in the early phases to check my ideas and search for prior work. I use them less now (mostly debugging at this point), but have made use of LLMs to write some pieces like source map generation, a couple of small and well known algorithms that were new to me, and an analyzer to dump a schema from a typescript module. Other times I've tried to use it for other things, I have ended up regretting it because of subtle bugs
I don't think it made the design phase any shorter but it definitely helped save me energy in researching and made that period feel less like a fever dream. Design phases can be draining for me so that was welcome
It helps me. Case in point, I recently had a API refactoring project that was 80% busywork and only a small amount of code. It's crap work, but it has to be done and it's part of my role. I told AI to generate the initial design doc. It took it upon itself to reference the local headers on my system and found some APIs that should be included that I hadn't even considered would also need to be touched for consistency. It saved me hours typing up the doc, requirements, test plan, etc and then saved me embarrassment by preventing me from making a stupid oversight on a tedious and boring task.
AI is helpful in such projects. Less so in other projects where design is heavy
From the patterns I have seen people would tend to use the time to build demo's using AI instead of design and then have a back and forth on the demo itself. One has to consider the fact that demo's look more convincing/complete and give an impression that the design decision has been made. In some ways, how agile got rid of the need for explicit & detailed documentation, the AI created demo's will cut the requirements of high level designs too.
> demo's look more convincing/complete ...
In the case of an LLM generated demo, usually deceptively so.
> and give an impression that the design decision has been made.
In the case of a vibed design, this is the opposite of useful for the team.
The concept of MVP has been there for a while, it felt deceptive too and incurred lot of tech debt. However one would feel the deception only if they understand the missing details. For a lot of roles making decisions it benefits them to ignore the details unless it really bites them
What kind of shape of evidence would you find convincing?
Self-reported or observational data capturing time spent for categories of task ala every other similar study in this space?
This isn't exactly novel territory, here, Simon. Let's not pretend I'm asking for something strange, unprecedented, or unreasonable.
How's this? https://simonwillison.net/2026/Jul/13/datasette-code-frequen...
I have similar charts across my three main open source projects:
https://github.com/simonw/datasette/graphs/code-frequency
https://github.com/simonw/llm/graphs/code-frequency
https://github.com/simonw/sqlite-utils/graphs/code-frequency
The article mentions that coding is only a fraction of dev time and thus accelerating that part of the job can only create incremental gains, and oh PS, lines of code and similar metrics are a bad way if measuring dev productivity, anyway, and we've known that for decades.
The OP claims AI accelerates non-coding parts of the job, too, and so the article is misguided.
I ask for evidence.
In response you give me... code output metrics?
I don't have anything else to hand that I can think of. I don't keep a time tracking diary.
> I don't keep a time tracking diary.
That's why Microsoft did a study (referenced in the article), where they measured the time spent on things, so they could get to the truth of how much time people spend coding using "a shape of evidence would you find convincing"
The Microsoft study concerns the ideal vs. actual developer workweek. They conclude that developers using AI tools "heavily" are both more productive and more satisfied. And, surprise, that devs hate meetings and want to code more.
I don't think volume of code changed was the metric the other commenter was asking for.
Isn't this falling into the "lines of code" trap TFA mentions?
More code written is not a good measure of productivity. It could be garbage, or redundant code, or simply not addressing the real or more pressing needs, it could be building the wrong thing, etc.
Like TFA mentions, it's been known for decades LoC is a misleading metric for productivity. It's one of the lessons of software engineering.
I'm planning an article at the moment in defense of lines of code. Saying "lines of code are a bad measurement" is too easy! It's about time someone presented an opposing argument.
In my specific case, lines of code for my published open source projects is a metric that I trust, because I have high standards for those. I have plenty of other projects where I'll accept poor quality, unreviewed code (almost all of https://github.com/simonw/tools for example), but Datasette, sqlite-utils and LLM are not that.
Of course, that's only useful for me personally and for people who trust me to stick to my own self-declared high standards!
> Saying "lines of code are a bad measurement" is too easy! It's about time someone presented an opposing argument.
Why though? The only entities who would be interested in such an implication is the AI marketing.
No one real cares about more or less lines of code. But everyone cares about decreasing the complexity of the implementation. Sometimes that means more LOC and sometimes it means less.
> Saying "lines of code are a bad measurement" is too easy! It's about time someone presented an opposing argument.
I don't know about easy. It's a hard earned lesson of software engineering, backed by research (some of it cited in TFA). Also, there's a related lesson: the more LoC to solve a given problem, the more bugs. That's also a finding backed by research.
I'll be reading your article when you write it, but I confess I'm skeptical.
The AI companies have a vested interest in using this metric, since it's easy to measure (the reason LoC were used even before AI) and there's no doubt that LLMs are writing tons of code. This makes me doubly skeptical.
> Saying "lines of code are a bad measurement" is too easy! It's about time someone presented an opposing argument.
Maybe because there is none.
One of the main quality of good codebase is simplicity. Which is about how easy for someone else to understand the code. It’s hard to define what simplicity looks like, so the best bet is to avoid the other side, making the code too complex.
And you can make the code complex by shortening variable name, doing code golfing with quicks of the platforms, so smaller LoC. You can also go the other way and increase the LoC by adding unneeded abstractions, repeating slices of code,… There’s a window where the LoC is perfect to attain simplicity, but that amount is an effect of striving for simplicity, not a cause of it. And it’s variable for every problem.
So you got something where the correct value is a different for each case. And trying to manipulate it artificially often results in complex code. And you want to say that is a good metric for productivity?
And in the cases of your projects, there are a lot more info could share that are interesting, like the amount of issues (reported or found by you) that are tied to implementation bugs (coding, libraries API breakage,…) or design issues (requirements conflicts,…), documentation improvement,… Anything that is tied to the actual usefulness of the projects, and not fumbling around with code.
> Lines of code are akshually a good metric now, because that's the only metric LLMs can optimize and I really, really, really love LLMs, they're the bee's knees.
Really now?
No.
People working fewer hours :)
Damn, looks like we have lower productivity than cavemen!
Sir, this is Capitalism.
It certainly helps as a research assistant for design work. But it can’t do it for you.
It does because you can now just steamroll features out the door and make everyone look bad that’s sitting around in meetings all day
Are you aware literally everyone else also has access to AI stuff? You’re not special for using an agent. You’re part of the crowd
Yeah, this seriously drives me nuts.
That meeting that you spent an hour in to understand the requirements? You don't need that meeting if you're not writing the code. That sync up with the QA engineer you did to hand it off to them? Don't need that meeting if you're not writing the code. That half hour you spent installing vim extensions? Don't need 'em if you don't open vim anymore.
There are engineers whose jobs go well beyond coding, of course. Staff engineers and principal engineers have had their jobs radically change because of AI, but not because it's writing all their code.
But there are also a lot of engineers -- your standard mid-level engineer, or even senior engineers at a lot of orgs with title inflation -- whose job is almost entirely about delivering code, and who spend all day either writing code or engaging in scaffolding around code-writing activities. Let's not pretend that automating away that code writing is a 15% boost.
> That meeting that you spent an hour in to understand the requirements? You don't need that meeting if you're not writing the code.
How are you going to prompt the LLM or validate its output if you don't understand the requirements?
In some number of cases, the business partner who would have passed you the requirements will just generate the code themselves.. I’m shocked shin some engineers don’t see a massive disruption coming..
Is this actually happening anywhere in a medium-large business with an established product and real customers?
Sure, but why doesn't said business partner just give a requirement to the AI to do the business partner's own job? Since it is hypothetically superhuman at fulfilling requirements at such a point...
PMs seem pretty easy to replace. In fact, given any single role to fixate on, it's probably replaceable
> In some number of cases, the business partner who would have passed you the requirements will just generate the code themselves.. I’m shocked shin some engineers don’t see a massive disruption coming..
It’s because you are engaging in first-order thinking. You imagine the immediate result which is that software engineers are replaced.
If you engaged in second-order thinking you would realize that if software engineers can be replaced, _any_ work can eventually be automated. So why stress about it, we’ll figure out a replacement for capitalism. It seems like the world is starting the slow swing to the left so eventually the problem will be solved.
Of course might be those engineers you mention are engaged in no thinking.
Oh, some of us see a massive disruption coming, just maybe not in the way you think.
Talented, experienced devs leaving the field out of disgust would be pretty disruptive.
Crumbling, faulty technical infrastructure with unmanageable heaps of AI slop that no one understands would definitely disrupt businesses.
Skill atrophy, and an entire generation of "developers" that have never actually written code will be disruptive when something breaks and no one knows how to fix it (this isn't a hypothetical, I'm seeing this phenomenon in a lot of large enterprise groups).
The total elimination of novel solutions, new ideas and new approaches to development and the resulting industry wide stagnation won't be disruptive, but will certainly be a drag.
You will have artisans or software SWAT teams on one side and code factories on the other. For most problems code factories will be good enough.
> Crumbling, faulty technical infrastructure with unmanageable heaps of AI slop that no one understands
Replace "AI slop" with "legacy code", and you've got basically any large enterprise today.
The timescales are different though: it takes years if not decades to accumulate classic legacy cruft, while LLMs can spew up huge amounts of incomprehensible spaghetti in seconds.
Your PM can write the ticket, and your QA can test the output.
I think we would, at some point realize that the roles of PM and QA are best done by an LLM, while coding is left to humans.
So an 180 from the current coding-automation frenzy..
>> How are you going to prompt the LLM or validate its output if you don't understand the requirements?
> Your PM can write the ticket, and your QA can test the output.
If your PM can write the ticket and your QA can test the output, why not cut out the middleman by having the PM write the coding agent prompt(s)?
And if your PM can write the coding agent prompt(s), why not cut out the QA group by having the PM write the verification prompt(s)?
And if your PM can write those prompts, why not cut out the PM and have your CSRs write the feature request/verification prompts?
And if your CSRs can write the feature request/verification prompts, why not cut them out and have the organization's customers write the feature request along with acceptance prompts?
And if customers can write those prompts, why would they pay you?
And finally: Why bother making new software at all when AI can just do the thing you wanted done, directly?
> And finally: Why bother making new software at all when AI can just do the thing you wanted done, directly?
In many ways, the "save costs by LLM-ing everything" mindset is the same logical fallacy companies fell into in the 90's/00's with regard to outsourcing. The end-state was rationalizing outsourcing call centers, then production system support, then software development, then project management, then...
Once the outsourcing raptor came for the accountants (who wrote the cheques) and remaining management (who cashed quarterly bonus cheques), outsourcing became "less appealing."
Because customers have better things to do.
Do they though? I assume they’re also getting replaced by LLM in this hypothetical.
Indeed they would, as the decision chain I posited is a form of "argumentum ad absurdum"[0].
0 - https://en.wikipedia.org/wiki/Reductio_ad_absurdum
My mates are not, their code editor are not either : I am.
Yes, I could be a clown and be accountable for something I know nothing about. I choose not to.
Not everybody is a cruft-engineer .
Also AI is now drafting design docs, generating PLC work products, entering it all in Jira, characterizing and root causing bugs... It's speeding up the 86% of my job that isn't coding. The article is a bit myopic and frankly contradicts itself.
I pity the poor souls who need to read that.
If Claude told you to work on a task that you don't want to work on, or make a design choice that you think is wrong, would you do it? If not, then it can't really replace things like design or meetings. (Note that this is subtly but importantly different than the "vibecoding" model, where you just don't bother to supervise Claude's decisions.)
I don't know about others, but at work, the reason I only spend like 14% of my day coding is because I'm lazy, not because I'm actually doing other stuff.
Not lazy it is a taxing task like doing an exam.
> f developers spend only about 15 percent of their time typing in the editor
I think this is missing an important detail. Lots of time was spent on non-coding stuff, because coding used to be more committal and hence expensive. With how quickly one can code up a quick prototype or even production-ready code these days, the code becomes the communication tool as well.
> Myth 3: Lines of Code Written by AI...
how come lines of code (or expressions) by an engineer aren't a good way to measure progress (Gates point etc) but GenAI tokens must count and be paid for?
That's a good point. They're not selling progress, they are claiming to enable progress by selling you (tokens in -> tokens out), making it your responsibility to connect the two. Filling that gap is part of your expertise that you can get paid for.
|--------|-------|------|------|-------|------|
|Contract|Product|Design|Coding|Testing|Deploy|
Writing Code Isn't the Bottleneck, until writing code is the bottleneck, until it's not again.
Getting a usable PRD is often the bottleneck.
The real bottleneck is Omega Star getting their shit together. And I ain't joking.
Or SPS Commerce. Evidence is becoming unconvincing that they're even in business anymore.
You forgot to add "coordination" to that pipeline. That is easily far and away the biggest source of delays.
That includes talking to vendors, meetings with every layer of stakeholder when just one person digs their heels, etc.
That is truly the final frontier for "AI", and one that it will likely never cross. That would be when even the execs and upper management feel threatened by "AI". But, since they also delegate so much, you often see someone at the bottom of the totem pole in those meetings. This is why nobody is getting replaced by "AI". We really need to move this discussion away from the scifi stupidity already. There is no singularity or godlike AGI about to take over the world.
I hate to use awful terms like "synergy" and "teamwork", but they do have a lot more substance and truth to them than any perceived threat from "AI".
> A study of more than 450 engineers at Microsoft in 2025 showed developers spend only 14 percent of their time writing code
This feels about right as an average across project cycles and different types of companies, and is the same type of number I've suggested here before.
The obvious conclusion is that even if AI reduced coding time to zero, then it would only reduce software development time by that 14%.
Of course AI may be used for other aspects of the job as well as coding, but on the flip side any serious use of AI requires a human in the loop to give work to the AI, steer the AI, assess the output, etc.
It's interesting that this 14% figure is close to Uber's choice to limit AI spend to 10% of developers salary. I wonder where that Uber number came from?
Another rather startling datapoint on the perceived value of AI comes from Microsoft who are also adopting budgeted AI usage, and are looking for "outcomes that move the needle". Their budget guidelines apparently refer to (current? targeted?) per-developer AI usage of "hundreds of dollars a month to a few thousand dollars in tokens".
https://www.techradar.com/pro/tokenmaxxing-is-not-what-we-ar...
>>>Recent research shows that developers—especially women and older engineers—face a “competence penalty” when using AI
and why did this study (performed in China I might add) find that women and “mature age” (which is not defined in the study) have less usage of AI tools?
>>>? We suggest a new barrier: using technology to assist task completion signals a lack of competence to perform the task independently.
So basically the study suggests that older engineers and women are reluctant (and especially women) to use AI tooling because they feel they are being judged more on non-technical competencies.
This study may have a strong cultural influence but I would say that one thing they noted I’ve also seen. 41% of engineers in the study had used AI 12 months after the initial rollout. This aligns with my observations. Some people are struggling figuring out how to adopt in their day to day while others are full in.
I look askance at an article that uses a GenAI survey from 2024. Other references could be good. But the dev agent milestone was Nov/Dev 2025, and CoPilot and agents were pretty sketch before then (but helpful at times.)
It seems that this could have been expanded or contracted to any Fibonacci number of myths.
>a “good” workday, engineers spent 18 percent of their time “coding” (not including bug fixing, testing, etc.)
that's such a weird metric, why exclude bug fixing and testing? depending on the phase of the project I might spend 100% of my coding time bug fixing
Some experience from my work:
- In biz development, a dev usually spends 30-40% time on coding, and more time on requirement discussion, integration testing (especially when the tests involves mobilephone or car)
- coding time can be reduced to 30%, which means reduce 20%-30% time of the full pipeline
- meanwhile, every phase and role is using LLM now, for example, product manager can produce longer requirement doc easily (we can use LLM to read it anyway:) Meeting sometimes is more than before, because more document output leads to more reading and discussion.
- I hope to find new ways to express biz requirements, in a more efficient and automatic manner.
- Shorten the requirement-dev-test-deploy loop is very important. OUTPUT is not OUTCOME. It is equal when we can see the final result, instead of intermediate metric.
- Agentic infra is extremely useful, or every one will find a way to access the database, report and ops system, in some weird fragile method.
If you can spin up a new software project with little to no effort, the most important part of the job becomes making the right decisions and this is where we will be spending 99.9% of your time.
I think the paper would have been stronger if it acknowledged how quickly the underlying evidence is becoming outdated. AI-assisted development in 2026 isn't just better models. The way many devs including myself work has changed and matured quite a bit as compared to last year
how so?
We switched from looking at an UI (claude webui) and waiting for code generation to using claude exclusivlie on the cli and claude doing a lot more stuff in the background with smaller prompts.
For me it changes in a way that i would like to have a 24/7 workspace vm setup outside of my work laptop for keeping it running if it wants and looking at it remotely if i want.
The workspace thing would also allow it to have more permissions like downloading, configuring and using headless chrome instead of highjacking my chrome session.
I think the tooling around the models themself has improved _a lot_ - they are really good at giving the models the correct context, even in big code bases
Many devs now work in YOLO mode letting LLMs automate their tasks.
This is actually true at my company. They expect employees to be 10× more productive now that we have AI.
I’ve had people tell me, with a completely straight face, that they expected 10-100x productivity improvements. This is at the executive and VC level. The mania is extreme.
I for sure do plenty of things with AI a lot faster.
Instead of searching some linux issue, i will prompt claude to generate a small analyser script for checking wha tlinux i have, i will tell it what hardware i have and it fixed my issue in like 5 minutes? That would have been a lot longer before.
> … with a completely straight face, that they expected 10-100x productivity improvements.
A great alternative future to look forward to - 1 employee expected to do the work of 10-100. Those others fired to save money.
So then they should get 10-100x more revenue, now that AI does all the marketing and selling.
You can be 10x more productive if you don’t verify anything the LLM generates and just YOLO it into your codebase. I wouldn’t recommend it.
11-18% of time spent in coding is still very high number I think. For a large org with lots of process and risk aversion, this number could be as low as 5%. Even for 14%, the 10x improvement could mean 86+(14/10) => 87.4/100 => 12.6% overall time saved.
And time in coding is like time on highways for taxi drivers. A fairly useless metric.
A lot of this rings true, but I think it's still too narrow. Sure, coding does not equal productivity, that is well debunked already. But I would argue that productivity is a product of engineering delivery + product decision making. Now where is the line between product and engineering? It varies by company, team and individual, but I don't think productivity can be measured for those functions independently, and in fact I see gains from AI on both the coding AND the product management side.
Basically as a senior tech lead in a large company engineering org, I don't have the bandwidth to individually validate every assertion from engineers on other teams OR from every product manager that comes with a half-baked ask. In the past I would be limited by the influence I could get through human relationships to strong SMEs with good judgment, and those folks always thin out as a company grows and calcifies. The number of creative and innovative thinkers dwindles, and the number of people protecting their turf and doing the minimum not to get fired increases. As a result many good ideas can get blocked by random gatekeeprs with poor imagination, poor expertise or both. However with AI I can follow up on gut instincts and fact check a lot more things, and ask incisive questions that can cut through a lot of organizational bullshit.
That's where I think most of the AI gains are today. Of course once AI plateaus and normalizes I think it will be baked into the org structures of tomorrow. But for now it offers real competitive advantage to those with the expertise to ask the right questions.
> studies at Microsoft and elsewhere showing it’s closer to 14 percent
This is a depressing stat. The real productivity gains come from leaving soul sucking big tech companies where nothing gets done with any sort of urgency.
In my open source work I believe this is the same. I don't have numbers, but I'm sure the vast majority of my time isn't spent writing code. Of course, it depends on how you define "writing code".
It is not urgency. Large production systems mean you are doing mostly unsexy operational planning. If I had a dollar each time I hear the word "data migration" I reckon I could do well.
With all the myths and hyperbolae circulating regarding AI, I'd love to know what it's like at large software companies adjusting to this brave new world.
It's easy for a small team to adjust workflows and roles, but I just imagine the office politics must be a waking nightmare in big organisations right now.
As an individual contributor I do not have insights into office politics. While we have >10k developers, it is not a software company.
Top leadership and/or investors believe in the benefits of AI.
Thus, skeptics stay silent. Who wants to loudly contradict their bosses' boss?
In practice, I do not see a big shift yet in workflows that would require a lot of politics. We just have the agent implement the code, and then it still goes through the usual code review and QA processes.
Only now with more effective models and harnesses do many developers realize how good these tools are at investigating bugs, etc.
Before the price-decreased GPT 5.6 Luna we barely had access to enough cheap AI to last for a month of work.
In my company GitHub CoPilot was rolled out on a global scale last year.
LLMs from Anthropic and co we get through a central tool which buys api tokens from Azure and AWS.
Its surprisingly fast, probably thefastest i have seen technology getting rolled out.
Im more worried about the small/middle sized companies which are software companies but don't get that they are. You know the companies 100% depending on IT but not having the right or good or enough people who often in interviews struggle with basics like code review.
I hope that some of the executives out there will read the list. I know they won't spend the time to read the full blog, but at least the headers should be enough
I mostly agree with the part where it is stated that AI is a tool which received massive investments without knowing how to maximize its utility. I think that we will pay for it in the near future.
> AI Will Turn Individual Developers into 10x Developers
Maybe not 10x, but there's definitely a significant boost in velocity of development and delivery.
We used to be a team of 20+, now only 9, and we are delivering more than before. Besides that, we did some major refactors that were sitting in the backlog for months.
>a “good” workday, engineers spent 18 percent of their time “coding” (not including bug fixing, testing, etc.)
I must be a crap developer, because I probably spend twice as much time bugfixing and testing than "coding". (Both of which actually involve coding stuff, so I really don't like that distinction they make)
This is stuff AI can be really good at, so brushing that part under the table distorts the picture.
Having said that, I do agree with most of the myths they present.
Then yeah you might not be a good developer.
LLMs are also quite good in writing unit tests and understanding bugs a lot faster than I do, now.
Just a few month back i looked at some yaml stuff for like 20 minutes, played around with it, looked at formatting etc. then i asked the LLM, it immediadly told me what was wrong. I was just blind to that particular wrong char.
Logic bugs? Yeah it can find them too.
> A June 2025 study of Microsoft developers
A year ago feels like forever
Not so much due to length of time, but because of the Opus shockwave than fell within that period.
This reads like a critique of 2023 tooling published in 2026. Their Amdahl-style arithmetic (speed up a 14% slice, cap your gains at 14%) holds only if "AI" means autocomplete. Current frontier models do far more than that: research, code comprehension, review, test authoring, debugging, exploratory prototyping, ideation. That's most of the rest of the working day or "86%".
The only point that still holds is that organizational policies and procedures that automate AI use and lower the barrier to entry are more efficient than leaving it up to each individual. Every other point they make is either stale or was never true to begin with.
one still has to think. Also about this ai automating stuff and humans playing around. I believe there is a time for this and time for that
All very sensible points which I think all senior programmers who have used AI would largely to agree with.
For those more junior - keep in mind that a lot of the maximalist rhetoric are from people either selling models, or the cottage industry of people selling you courses or tools to help you use the models. Try and keep in mind software is not a mature industry, it's an immature one, and it's prone to hype and fads.
It's kind of weird how we blame agents for hallucinations as if humans don't fall for that as well, while agents can run the build-fail-fix-repeat in much faster cycles than coders.
> Despite this decade-old research, many organizations still rely on lines of code as a measure of developer productivity.
No, they don't! It's easy to dispel myths when the myths are built on straw men. Dumb article.
even an AI assist that makes coding twice as fast would, in theory, improve developers’ overall productivity by less than 15 percent. The other 85 percent of their time remains untouched
I stopped reading after this. AI has massively impacted most aspects of my non-coding work including the mentioned planning, understanding legacy code bases, setting up environments, etc etc.
Either this article is written by people with skill issues or - given the platform - its a biased and protectionist take that will fall quickly under the march of reality.
Have you considered the theory that, perhaps, you are the one with skill issue ?
ChatGPT, summarize
The 14% coding time figure is one of those stats that sounds surprising until you actually track your own time. When I started building a coding agent with persistent state, I realized how some days are spent with minimal actual typing, most of it is design, reading code, debugging, problem solving, and context-switching.
But I'd push back on one thing the article implies that AI is automatically a productivity win. It's not. Some days I've shipped two months of work in a few days with AI. Other days, like today, I've burned a whole day and gotten almost nothing done because the proper research was not done by me or multiple agents.
The bottleneck for AI can be the human understanding of how to optimally use the tool. While the bottleneck for the human can be not maximizing multiple agents, or the input the user enters, then the retention of the output. If the user's input is lost, the output falters. If the user doesn't understand what the AI output is, there is going to be a problem eventually.
The article touches on adoption barriers (Myth 7), but it doesn't really get into the ego piece. There's still a wave of experienced devs who either refuse to adopt AI, or use it quietly and don't share what they're doing. That slows the whole team's learning curve. At this point, I think it's pretty much understood that you should be using AI as a dev — not to replace your skills, but to accelerate them. That means still learning new languages, still writing code, still troubleshooting. The tools change, but the craft doesn't.
I think the article is right that the real leverage is organizational, not individual. The teams that succeed with AI aren't the ones giving everyone a license — they're the ones rethinking how they review, test, and maintain code.
What I'm still uncertain about is how to measure whether AI is actually making systems better, not just faster. Lines of code is clearly a bad metric, but I haven't seen a good alternative yet. What metrics are people actually using that feel meaningful?
I'm getting tired of these articles telling me what AI will or won't do to my career when every day I see something different first hand. I'm about to stop arguing with people. If you think it's all BS then fine. Good luck.
I don’t trust point estimates like 14%. It’s like calculating an average salary and saying it’s $120k. Completely meaningless. What does the actual distribution look like that this was pulled from? No standard deviation. Is it even symmetric? What’s the 10th and 90th percentiles? Just giving a statistic on its own tells me nothing.