The article claims, that senior developers with over 10 years of experience are more than twice as likely to heavily rely on AI tools compared to their junior counterparts. No p-values or statistical significance tests are reported in either The Register article or Fastly's original blog post.
I have over 30 years of experience and recently used Claude Opus 4.1 (via browser and claude.ai) to generate an ECMA-335 and an LLVM code generator for a compiler, and a Qt adapter for the Mono soft debugging protocol. Each task resulted in 2-3kLOC of C++.
The Claude experience was mixed; there is a high probability that the system doesn't respond or just quickly shows an overloaded message and does nothing. If it generates code, I quckly run in some output limitation and have to manually press "continue", and then often the result gets scrambled (i.e. the order of the generated code fragments gets mixed up, which requires another round with Claude to fix).
After this process, the resulting code then compiled immediately, which impressed me. But it is full of omissions and logical errors. I am still testing and correcting. All in all, I can't say at this point that Claude has really taken any work off my hands. In order to understand the code and assess the correctness of the intermediate results, I need to know exactly how to implement the problem myself. And you have to test everything in detail and do a lot of redesigning and correcting. Some implementations are just stubs, and even after several attempts, there was still no implementation.
In my opinion, what is currently available (via my $20 subscription) is impressive, but it neither replaces experience nor does it really save time.
So yes, now I'm one of the 30% seniors who used AI tools, but I didn't really benefit from them in these specific tasks. Not surprisingly, also the original blog states, that nearly 30% of senior developers report "editing AI output enough to offset most of the time savings". So not really a success so far. But all in all I'm still impressed.
Imho your post summarizes 90% of the posts I see about AI coding on HN: not understanding the tools, not understanding their strenghts and weaknesses, not being good at prompting or context management yet forming strong(ish) opinions.
If you don't know what they are good at and how to use them of course you may end up with mixed results and yes, you may waste time.
That's a criticism I have also towards AI super enthusiasts (especially vibe coders, albeit you won't find much here), they often confuse the fact that LLMs often one shot 80% of the solutions with the idea that LLMs are 80% there, whereas the Pareto principle well applies to software development where it's the hardest 20% that's gonna prove difficult.
I'm pretty good at prompting and I successfully use Perplexity (mostly with Claude Sonnet 4) to develop concepts, sometimes with the same session expanded over several days. I think the user interface is much superior over Claude.ai. My hope was that the newer Claude Opus 4.1 would be much better in solving complicated coding tasks, which doesn't seem to be the case. For this I had to subscribe to claude.ai. Actually I didn't see much difference in performance, but a much worse UI and availability experience. When it comes to developing a complex topic in a factual dialogue, Claude Sonnet Thinking seems to me to be even more suitable than Claude Opus.
1) Your original post asks a lot if not too much out the LLM, the expectation you have is too big, to the point that to get anywhere near decent results you need a super detailed prompt (if not several spec documents) and your conclusion stands true: it might be faster to just do it manually. That's the state of LLMs as of today. Your post neither hints at such detailed and laborious prompting nor seem to recognize you've asked it too much, displaying that you are not very comfortable with the limitations of the tool. You're still exploring what it can and what it can't do. But that also implies you're yet not an expert.
2) The second takeaway that you're not yet as comfortable with the tools as you think you are is clearly context management. 2/3k locs of code are way too much. It's a massive amount of output to hope for good results (this also ties with the quality of the prompt, with the guidelines and code practices provided, etc, etc).
3) Neither 1 or 2 are criticisms of your conclusions or opinions, if anything, they are confirmations of your point that LLMs are not there. But what I disagree with is the rush into concluding that AI coding provides net 0 benefits out of your experience. That I don't share. Instead of settling on what it could do (help with planning, writing a spec file, writing unit tests, providing the more boilerplate-y part of the code) and use the LLM to reduce the friction (and thus provide a net benefit), you essentially asked it to replace you and found out the obvious: that LLMs cannot take care of non-trivial business logic yet, and even when they can the results are nowhere near being satisfactory. But that doesn't mean that AI-assisted coding is useless and the net benefit is 0, or negative, it only becomes so as the expectations on the tool are too big and the amount of information provided is either too small to return consistent results or too large for the context to be an issue.
I don't know where your confidence or assumptions come from. Do you work for Anthropic? My prompts for the code generators included an 1.2kLOC code file plus detailed instructions (as described elsewhere), with more details during the session. So I don't think your points apply.
This is a kind of nuts take;
- Senior engineer
- Uses tools for non trivial undertaking
- Didn't find value in it
Your conclusion from that is "but they are doing it wrong", while also claiming they are saying things they didn't say (0 net benefits, useless, etc).
Do you see how that might undermine your point? That you feel they haven't take the time to understand the tools, but you didn't actually read what what wrote?
The situation has improved a little bit over the last few months but LLMs are still only barely usable in languages like C/C++/Zig - and it's not about prompting. I would say that LLMs are usable for JS/Python and while the code is not always what I'd write myself, it can be used and improved later (unless you are working on perf-sensitive JS app, then it's useless again).
And it might be also something with GC, because I suppose the big boys are doing some GRPO over synthetically generated/altered source code (I would!) but obviously doing that in C++ is much more challenging - and I'd expect Rust to be straight impossible.
How do you know that your humble opinion is right about who knows what tool and how deep?
Even if you know better than themselves how musch they know, isn't the tool inadequate just yet for power use then when it is sooo easy to misuse?
Too much tweeking and adapting users to the needs of the tool (vs. the other way around) and there is little point using those (which is a bit of the sickness of modern day computing: 'with computers you can solve problems lightning fast that you wouldn't have without them')
Would you agree with the claim that emacs/vim is an inadequate tool, since it has such a high learning curve?
Prior to LLMs, my impression was "high learning curve, high results" was a pretty popular sweet-spot with a large portion of the tech crowd. It seems weird how much LLMs seem to be an exception to this.
Emacs and vim have complex interfaces that have been stable for decades. Seems like every new flavor of LLM requires learning its warts and blind spots from scratch.
Hey! I would encourage you to try our Claude code instead, which is also part of your subscription. It's a CLI that takes care of many of the issues you encountered, as it works directly on the code files in a directory. No more copy pasting or unscrambling results. Likewise, it can run commands itself to e.g. compile or even test code.
I'm working on old hardware and not-recent Linux and compiler versions, and I have no confidence yet in allowing AI direct (write) access to my repositories.
Instead I provided Claude with the source code of a transpiler to C (one file) which is known to work, uses the same IR as the new code generators were supposed to use.
This is a controlled experiment with a clear and complete input and clear expectations and specifications of the output. I don't think I would be able to clearly isolate the contributions and assess the performance of Claude when it has access to arbitrary parts of the source code.
I use Claude Code with the Max plan, and the experience isn't far off from what you describe. You still need to understand the system and review the implementation, because it makes many mistakes.
That's not the part it saves me time in, it saves me time in looking up the documentation. Other than that, it might be slower, because the larger the code change is, the more time I need to spend reviewing, and past a point I just can't be bothered.
The best way I've found is to have it write small functions, and then I tell it to compose them together. That way, I know exactly what's happening in the code, and I can trust that it works correctly. Cursor is probably a better way to do that than Claude Code, though.
So, I am paying $20, for a glorified code generator, that may or may not be correct, to write a small function that I can do for free, and be confident about the correctness, if I have not been lazy to implement a test for it.
If you point out, with test it's also the same with any AI tool available, but to come to that result, I have to continuously prompt it till it gives me the desired output, while I may be able to do it in 2/3 iterations.
Reading documentation always made me little bit knowledgeable than before, while prompting the LLM, gives me nothing of knowledge.
And, I also have to decide which LLM would be good for the task at hand, and most of them will not be free (unless I use a local, but that will also use GPU, and add an energy cost)
I may be nitpicking, but I see too many holes with this approach
The biggest hole you don't see is that it's worth the $20 to make me overcome my laziness, because I don't like writing code, but I like making stuff, and this way I can make stuff while fooling my brain into thinking I'm not writing code.
Your time isn't free, and I'd certainly with more than $20/month.
I find it extremely useful as a smarter autocomplete, especially for the tedious work - changing function definitions, updating queries when DB schema changes, and writing http requests/api calls from vendor/library documentation.
Certainly, So I use an IDE, IntelliJ Ultimate to be precise.
None of the use-cases you mention requires LLM. Just available as IDE functionalities.
IntelliJ has LLM based auto complete, with which I am okay, But it still wrong too many times. Works extremely well with Rust. Their non-llm autocomplete is also superb, which uses ML for suggesting closest, relevant match, IIRC.
It also makes refactoring a breeze, I know what it's going to do exactly.
Also, it can handle database refactoring to a certain capacity! And for that it does not require LLM, so no nondeterministic behavior.
Also, the IDE have its own way of doing http requests, and it's really nice! But, I can use their live template to do autocomplete any boilerplate code. It only requires setting once. No need to fiddle with prompts.
Not only that, but the process described is how you train a junior dev.
There, at least, the wasted time results in the training of a human being who can become sophisticated enough to become a trusted independent implementer in a relatively short duration
> The best way I've found is to have it write small functions, and then I tell it to compose them together.
Pretty much how I code without AI, except making my brain break the problem down into small functions and expressing them in code rather than as a chat.
> it saves me time in looking up the documentation
I have a Perplexity subscription which I heavily use for such purpose, just asking how something works or should be used, with a response just on the point and with examples. Very useful indeed. Perplexity gives me access to Claude Sonnet 4 w/o Thinking which I consider great models, and it can also generate decent code. My intention was to find out how good the recent Claude Opus is in comparison and how much of my work I'm able to delegate. Personally I much prefer the user interface features, performance and availability of Perplexity to Claude.ai.
I end up using Perplexity a lot too, especially when I'm doing something unfamiliar. It's also a good way to quickly find out what are best practices for a given framework/language I'm not that familiar with (I usually ask it to link to examples in the wild and it find opensource projects illustrating those points)
You can select the model. I very much appreciate the Claude Sonnet models which are very good and rational discussion partners, responding to arguments in detail and critically, allowing for the dialectical exploration of complex topics. I have also experimented with other models including ChatGPT, Gemini or Grok, but the resulting discussions were only a fraction as useful (i.e. more optimized towards affirmative feel-good small talk, from my humble point of view).
claude-code asks you to allow it to do anything before it does them. Once you start trusting it and get comfortable with its behavior it gets annoying being prompted all the time, so you can whitelist specific commands it wants to run. You can also interactively toggle into (and out of) "accept changes without asking" mode.
(It wasn't clear to me that I would be able to toggle out of accept changes mode, so I resisted for a loooooong time. But turns out it's just a toggle on/off and can be changed in real-time as it's chugging along. There's also a planning state but haven't looked into that yet)
It always asks before running commands unless you whitelist them. I have whitelisted running testsuites and linters, for example so it can iterate on those corners with minimal interaction. I have had to learn to let it go ahead and make small obvious mistakes rather than intervene immediately because the linters and tests will catch them and Claude will diagnose the failure and fix them at that point.
Anyway I took a small toy project and used that to get a feel for claude-code. In my experience using the /init command to create CLAUDE.md (or asking Claude to interview you to create it) is vital for consistent behavior.
I haven't had good "vibe" experiences yet. Mostly I know what I want to do and just basically delegate implementation. Some things that have worked well for me is to ask Claude to propose a few ways to improve or implement a feature. It's come up with a few things I hadn't thought of that way.
Anyway, claude-code was very good at slowly and incrementally earning my trust. I resisted trying it because I expected it would just run hogwild doing bewildering things, but that's not what it does. It tends to be a bit of an asskisser in it's communication style in a way that would annoy me if it were a real person. But I've managed to look past that.
On Claude you specifically accept any attempt to use a terminal command (optionally whitelisting) so there’s no risk that it will push force something or whatever. You can also whitelist with granularity, for example to enable it to use git to view git logs but not commit.
You can just let it work, see what’s uncommitted after it’s over, and get rid of the result if you don’t like it.
I've been trying it for a couple of months, I can't recommend it either tbh. It's frustrating as hell to work with: super inconsistent, very bad at following its own instructions, wasteful and generally unreliable.
The problem is, it's like a very, very junior programmer that knows the framework well, but won't use it consistently and doesn't learn from mistakes AT ALL. And has amnesia. Fine for some trivial things, but anything more complicated the hand-holding becomes so involved you are better off doing it yourself. That way you internalise some of the solutions as well, which is nice because then you can debug it later! Now I have a huge PR that even I myself don't really grasp as much as I would want.
But for me the nail in the coffin was the terrible customer service. ymmv.
In what specific programming language/toolchain/technology is your experience? Why do you think that "everybody can tell that chat gpt wrote your code"? Meanwhile I looked at a lot of LLM generated code in different languages, and I wouldn't generally subscribe to your statement. And you still didn't explain why Claude should fail. I think it is rather an advantage (when it reliably works in future).
I'm just shy of 30 years experience. I think I've spent more time learning how to use these tools than any other technology I've learned, and I still don't know the best way to use them.
They certainly weren't a time-saver right away but they became one after some time giving them a real shot. I tested their + my limits on small projects, working out how to get them to do the whole project, figuring out when they stop working and why, figuring out which technology they work best with, figuring out the right size problems to give them, figuring out how to recognize if I'm asking them something they can't do well a ask something different instead, guide them into creating code that they can't actually continue to be successful with.
I started last December in Cursor's agentic mode and have been in Claude Code ever since probably March or April. It's definitely been a huge boost all year for side projects - but only in the last couple months have I been having success in a large codebase.
Even with all this experience I don't know that I would really be able to get much value out of
the chat interface for them. They need to be proposing changes I can just hit accept or reject on (this is how both Claude Code and Cursor work btw - you don't have to allow it to write to any file you don't want or execute any command you don't want).
I don't think it is true that GPT-5 is much better than Claude 4.1. And more important is that light year is distance measure and I am sure openAI data centers is still on earth.
I hate to play the "you're holding it wrong" card, but when I started, I had more or less the same experience. Eventually you start to learn how to better talk to it in order to get better results.
Something I've found useful with Claude Code is that it works a lot better if I give it many small tasks to perform to eventually get the big thing done, rather than just dumping the big thing in its lap. You can do this interactively (prompt, output, prompt, output, prompt, output...) or by writing a big markdown file with the steps to build it laid out.
While this matches my experience, it's worth mentioning thar the act of breaking a task up into the correct chunk size and describing it in English is itself a non trivial task which can be more time consuming than simply writing the actual code.
The fact that it works is amazing, but I'm less convinced that it's enhancing my productivity.
(I think they real productivity boost for me is if I still write the code and have the assistant write test coverage based on diffs, which is trivial to prompt for good results)
This. For me (senior as in I am in the field since last century), that's how I use it. "I want to do that, with this data, to obtain that"
I still do the part of my job that I got experience on, analyze the need, and use the AI like an assistant to do small libraries or part of code. Like these, errors have less chance to appear.
Then I glue that together.
For me the time ratio is best use like that. If I have to describe the whole thing, I'm not far from doing it myself, so there's no need for me.
Important: I work alone, not in a team, so maybe it has an impact on my thought
I just tried to use it for something I consider it to provide the most benefit (for my case). Being able to fully delegate a complicated (and boring) part to a machine would give me more time for the things I'm really interested in. I think we are on the right track in this regard, but we still have a long way to go.
Similar here. AI works much better as a consultant than as a developer. I ask it all kinds of things I have suspicions and intuitions about and it provides clarity and examples. It's great for subroutines. Trying to make full programs is just too large of a space. It's difficult to communicate all the implicit requirements.
Seems about right for me (older developer at a big tech company). But we need to define what it means that the code is AI-generated. In my case, I typically know how I want the code to look like, and I'm writing a prompt to tell the agent to do it. The AI doesn't solve any problem, it just does the typing and helps with syntax. I'm not even sure I'm ultimately more productive.
For all the folks on the "reduce mental burden", "reduce cognitive load" train, are you all aware that this basically means you are exercising less your brain day in and day out, and in the end you will forget how to do things? You will learn how to guide an AI agent, but until the day an AI agent is perfect (and we don't know if we will ever see that day), you are just losing inch by inch your ability to actually understand what the agent is writing and what is going on.
I'm pretty radical on this topic but for me cognitive load is good, you are making your neurons work and keep synapses in place where they matter (at least for your job). I totally accept writing down doc or howto to make doing some action in future easier and reduce that cognitive load, but using AI agent IMO is like going to bike in the mountain with an electrical bike.
Yes, you keep seeing the wonderful vistas but you are not really training your legs.
This, to me, feels like you're complaining to the 45 year old builder that they should be using a hammer instead of a nail gun.
I know how to nail a nail, I've nailed so many nails that I can't remember them all.
My job is to build a house efficiently, not nail nails. Anything that can make me more efficient at it is a net positive.
Now I've saved 2 hours in the framing process by using a nail gun, I have 2 extra hours to do things that need my experience. Maybe spot the contractor using a nail plate in the wrong way or help the apprentice on their hammering technique.
IMO it's different. That's why I brought the e-bike similitude: climbing even mild mountains or hills with your own legs will actually make your legs, heart and lungs stronger in the process. So you get both the wonderful views (building the house or delivering the software) but also you get improved health (keeping your mind trained on both high level thinking and low level implementation vs high level only). We might say that using a hammer constantly will develop more your muscles, but in carpentry there are still plenty of manual work that will develop your muscles anyway. (and we still don't have bricks laying machines)
Ironically, e-bikes, at least in the EU, are having the exact opposite effect. More people that don't normally ride bikes are using e-bikes to get about. The motor functions not as a replacement, but as a force multiplier. It also makes "experimenting" easier, because the motor can make up for any mistakes or wrong turns.
Caveat: In the EU, an e-bike REQUIRES some physical effort any time for the motor to run. Throttles are illegal.
> Ironically, e-bikes, at least in the EU, are having the exact opposite effect. More people that don't normally ride bikes are using e-bikes to get about.
At least in Germany people rather joke that the moment e-bikes became popular, people began to realize that they suddenly became too unathletic to be capable of pedaling a bicycle. I know of no person who uses an e-bike who did not ride an ordinary bicycle before.
> In the EU, an e-bike REQUIRES some physical effort any time for the motor to run.
The motor must shut off when 25 km/h is reached - which is basically the speed that a trained cyclist can easily attain. So because of this red tape stuff, e-bikes are considered to be useless and expensive by cyclists who are not couch potatoes.
Yes, you pedal with almost zero effort when flat and with a little more effort when uphill. Obviously if you are going to go up the Tourmalet you will run out of battery pretty soon, but that's not the context most e-bikers use them.
In that it fits the LLM situation quite well. LLMs remove the anxieties around coding for newbies at scale better than they make indisputable productivity gains for senior developers, similar to how e-bikes help with newbies more than cyclists.
I know that's what many people, especially elder one, say but this is still a hill I will die on :) They are mostly used to go in mostly flat roads, like some slow-speed motorcycle that needs some low effort. The ones using them outside paved road, using them as multipliers, are the ones that already did mountain biking when they were younger and they want to continue doing it at a higher level their age would permit without (which it's perfectly fine!).
But if my goal is to ride Downhill[0] me exhausting myself riding up the hill isn't bringing any extra enjoyment for me.
With electric assist, I can get up faster and to the business of coming down the hill really fast.
We have ski-lifts for the exact same reason. People doing downhill skiing would make their legs, heart and lungs stronger in the process of walking up the hill with their skis. But who wants to do that, that's not the fun part.
And to step back from analogy-land.
I'm paid to solve problems, I'm good at architecture, I can design services, I can also write the code to do so. But in most cases the majority of the code I write is just boilerplate crap with minor differences.
With LLMs I can have them write the Terraform deployment code, write the C# CRUD Controller classes and data models and apply the Entity Framework migrations. It'll do all that and add all the fancy Swagger annotation while it's at it. It'll even whip up a Github build action for me.
Now I've saved a few days of mindless typing and I can get into solving the actual problem at hand, the one I'm paid to do. In reality I'm doing it _while_ I'm instructing the LLM to do the menial crap + reviewing the code it produced so I'm moving at twice the speed I would be normally.
"But can't you just..." nope, if every project was _exactly_ the same, I'd have a template already, there are just enough differences to not make it worth my time[1]. If I had infinite time and money, I could build a DSL to do this, but again, referring to [1] - there's no point =)
It's more cost efficient to pay the LLM tax to OpenAI, Anthropic or whatever and use it as a bespoke bootstrapping system for projects.
> So you get both the wonderful views (building the house or delivering the software) but also you get improved health (keeping your mind trained on both high level thinking and low level implementation vs high level only).
The vast majority of developers aren't summitting beautiful mountains of code, but are instead are sifting through endless corporate slop.
> We might say that using a hammer constantly will develop more your muscles, but in carpentry there are still plenty of manual work that will develop your muscles anyway.
The trades destroy human bodies over time and lead to awful health outcomes.
Most developers will and should take any opportunity to reduce cognitive load, and will instead spend their limited cognitive abilities on things that matter: family, sport, art, literature, civics.
Very few developers are vocational. If that is you and your job is your identity, then that's good for you. But don't fall into the trap of thinking that's a normal or desirable situation for others.
> The vast majority of developers aren't summitting beautiful mountains
I'm not sure you're approaching this metaphor the right away. The point is that coding manually is great cognitive exercise which keeps the mind sharp for doing the beautiful stuff.
> The trades destroy human bodies over time and leads to awful health outcomes.
Again, you're maybe being too literal and missing the point. No one is destroying their minds by coding. Exercise is good.
I am using LLMs, too, and I do not consider myself thinking less. I still have to be part of the whole process, incl. architectural process among other things that require my knowledge and my thinking.
I use them too and actually agree with you that the cognitive load is somewhat comparable. I was only pointing out what seemed like an abuse of the metaphor.
> I'm not sure you're approaching this metaphor the right away. The point is that coding manually is great cognitive exercise which keeps the mind sharp for doing the beautiful stuff.
No, I'm challenging the metaphor. Working the trades isn't exercise - it's a grind that wears people out.
> Again, you're maybe being too literal and missing the point. No one is destroying their minds by coding. Exercise is good.
We actually have good evidence that the effects of heavy cognitive load are detrimental to both the brain and mental health. We know that overwork and stress are extremely damaging to both.
So reducing cognitive load in the workplace is an unambiguous good, and protects the brain and mind for the important parts of life, which are not in front of a screen.
> We actually have good evidence that the effects of heavy cognitive load are detrimental to both the brain and mental health. We know that overwork and stress are extremely damaging to both.
I don't think this is fair either, you're comparing "overwork and stress" to "work." It's like saying we have evidence that extreme physical stress is detrimental ergo it's "unambiguously" healthier to drive than to walk.
Maybe you could share your good evidence so we can see if normal coding tasks would fall under the umbrella of overwork and stress?
We have plentiful evidence and studies on the effect of even moderate day-long cognitive work has on cognitive ability and on the effect of stress.
This is so well-founded that I do not have to provide individual sources - it is the current global accepted reality. I wouldn't provide sources for the effect of CO2 emissions on the climate or gravity, either.
However, the opposite is not true. If you have evidence that routine coding itself improves adult brain health or cognitive ability, please share RCTs or large longitudinal studies showing net cognitive gains under typical workloads.
Again you're conflating things and are now also moving goalposts (overwork->moderate work) and asking me for very precise kinds of studies while refusing to even point towards the basis for your own claims. On top of this you're implying that I'm some kind of lunatic by associating my questions with climate denial.
It's clear that you're more interested in "winning" than actually have a reasonable discussion so goodbye. I've had less frustrating exchanges with leprechauns.
Come on. We’ve had decades of occupational-health research on cognitive load, stress, and hours. The pattern is clear. Higher demands and longer hours raise depression risk. Lab and field work shows day-long cognitive tasks produce measurable fatigue, decision drift, and brain chemistry changes. These are universally accepted.
And yet, you now want me to source individual studies on those effects in a HN thread? Yes, in this instance you are approaching flat-earth/climate-change-denial levels of discourse. Reducing cognitive load is an unambiguous good.
If you think routine coding itself improves brain health or cognitive ability, produce the studies showing as you demanded from me, because that is a controversial claim. Or you can crash out of the conversation.
> No, I'm challenging the metaphor. Working the trades isn't exercise - it's a grind that wears people out.
If your job is just grinding out code in a stressful and soul-crushing manner, the issue lies elsewhere. You will be soon either grinding out prompts to create software you don't even understand anymore or you will be replaced by an agent.
And by no way I'm implying you are part of the issue.
But LLMs can make the soul crushing part so much easier.
I need to add a FooController to an existing application, to store FooModels to the database. The controller needs the basic CRUD endpoints, etc.
I can spend a day doing it (crushing my soul) or I can just tell any Agentic LLM to do it and no something that doesn't crush my soul, like talk with the customer about how the FooModels will be used after storing.
"But it'll produce bad code!"
No it doesn't. It knows _exactly_ how to do a basic CRUD HTTP API controller in C#. It's not an art form, it's just rote typing and adding Attributes to functions.
Because it's an Agentic LLM, it'll go look at another controller and copy its structure (which is not standard globally, but this project has specific static attributes in every call).
Then I review the code, maybe add a few comments for actual humans, commit and push.
My soul remains uncrushed, client is happy when I delivered the feature on time and I have half the day off for other smaller tasks that would become technical debt otherwise.
> My soul remains uncrushed, client is happy when I delivered the feature on time and I have half the day off for other smaller tasks that would become technical debt otherwise.
This is a very optimistic take. If you are in the type of company that just gives you boring code and tasks, you will required to use the other half-day to work on some other boring feature. This will not give us time to pay tech debt. Maybe it will do in the beginning when using AI agents has not been institutionalized yet, but once it has, you will be asked to churn out more "features"
If the mechanics of the work are “soul-crushing,” isn’t that the root cause, and the LLM is just a bandaid? I’m not saying every professional dev is enthused with all their tasks. But if you’re so eager to avoid parts of the job (however rote they are), then maybe it’s time for something new?
I can’t write this without feeling preachy, and I apologize for that. But I keep reading a profound lack of agency in comments like these.
> If your job is just grinding out code in a stressful and soul-crushing manner, the issue lies elsewhere.
The vast majority of developers are in or near this category. Most software developers never write code outside of education or employment and would avoid doing so if an AI provided the opportunity. Any opportunity to reduce cognitive load is welcome
I think you don't recognise how much of an outlier you are in believing that your work improves your cognitive abilities.
I'd like to counter argue we need to shift the notion of efficiency.
While the nailgun is more efficient than the hammer, both need to be operated by a real human. Including all limitations of physical and mental health.
While we can improve the efficiency of the tools we should consider not to burn out humans to match the goal.
The builder still needs to study the plans and build a mental model of what they’re building.
A nailgun isn’t automated in the way an LLM is, maybe if it moved itself around and fired nails where it thought they should go based on things it had seen in the past it would be a better comparison.
On the contrary, I am a lot more willing to think through the contours of the problems I need to solve because I haven't drained my mental energy writing five repetitive - but slightly different - log lines and tweaking the wording slightly to be correct.
I'm training smarter, and exercising better, instead of wasting all the workout/training time on warmups, as it were.
It completely depends on how you use AI. If you turn off your brain entirely and just coast as much as possible, then yeah your comment would apply.
But I think of work as essentially two things - creative activity and toil. I simply use AI for toil, and let my brain focus on creativity and problem solving.
Writing my 100,000th for loop is not going to preserve my brain.
I have been coding and athletic training for about as long as eachother, your anecdote works. However just like in physical training, you should really spare your energy for stuff that you enjoy and actually progresses you.
By using LLMs to do some of the stuff I have long gotten over, I have a bit more mental energy to tackle new problems I wouldn't have previously.
As well LLMs just aren't actually that competent yet, so it's not like devs are completely hands off. Since I barely do boilerplate as I work across legacy projects, there's no way Claude Code today is writing half my code.
My current pattern is to manually craft during the first half of the day when I enjoy that, and during the second half when I'd be normally burnt on hard thought and not quite up for another coffee, pomodoro, theanine deep dive, I can start tackling tests, exploratory data analysis, or small bugs, and these tasks are 50% or more LLM.
So yeah, 30%-50% seems right, but it's not like I lost any part of my job that I love.
That's a good approach, I like it and probably adopt it as well. I'm not dogmatically against LLMs, I just need we should think about possible consequences, and not thinking about them like a holy grail of everything.
You're right, but usually at the end of the day I'm completely mentally exhausted and dont want to talk to anyone. Its something I've realized is a big problem in my life. I'm actively trying to reduce mental load to leave stuff for other hobbies and social activities.
you are obviously not past 50 years old. At that age, eventhough I "train" (as you imply it) a lot (I code about 50 hours a week, 40 at work, the rest for my own projects; not of them easy, my job is writing numerical code for simulation and my pet project is writinga very accurate emulator (which implies physics modelling, tons of research, etc)). I can definitely feel I'm not as productive as before (before I could sustain 60 hours a week) eventhough I do feel I'm using my brain to the maximum...
So yeah, past a certain age, you'll be happy to reduce your mental load. No question about it. And I feel quite relieved when Claud writes this classic algorith I have understood long ago and don't want to re-activate in my brain. And I feel quite disappointed when Claude misses the point and I have to code review it...
Do you sit there multiplying two digit numbers in your head for fun, for the practice, to keep operating at peek mental capacity on weekends? In the name of operating at peek mental capacity, that seems like the most logical thing to do. Just wake up at 6 am Saturday morning and start multiplying numbers on your head.
If you don't wanna use AI, that's entirely up to you, but "you're gonna forget how to program if you use AI and then whatever are you going to do if the AI is down" reeks of motivated reasoning.
Strangely I've found myself more exhausted at the end of the week and I think it's because of the constant supervision necessary to stop Claude from colouring outside the lines when I don't watch it like a hawk.
Also I tend to get more done at a time, it makes it easier to get started on "gruntwork" tasks that I would have procrastinated on. Which in turn can lead to burnout quite quickly.
I think in the end it's just as much "work", just a different kind of work and with more quantity as a result.
No, what I want better is tooling that makes supervising and getting insight into what it's doing a superior experience.
A far more interactive coding "agent" that makes sure it walks through every change it makes with you, and doesn't just rush through tasks. That helps team members come up to speed on a repository by working through it with them.
> Strangely I've found myself more exhausted at the end of the week and I think it's because of the constant supervision necessary to stop Claude from colouring outside the lines when I don't watch it like a hawk.
Welcome to management. Computers and code are easy. People and people wannabes like LLMs are a pain.
I find AI is most useful at the ancillary extra stuff. Things that I'd never get to myself. Little scripts of course, but more like "it'd be nice to rename this entire feature / db table / tests to better match the words that the business has started to use to discuss it".
In the past, that much nitpicky detail just wouldn't have gotten done, my time would have been spent on actual features. But what I just described was a 30 minute background thing in claude code. Worked 95%, and needed just one reminder tweak to make it deployable.
The actual work I do is too deep in business knowledge to be AI coded directly, but I do use it to write tests to cover various edge cases, trace current usage of existing code, and so on. I also find AI code reviews really useful to catch 'dumb errors' - nil errors, type mismatches, style mismatch with existing code, and so on. It's in addition to human code reviews, but easy to run on every PR.
Wow, 30 minutes to rename functions and tests? I wonder how much energy and water that llm wasted for something that any lsp supporting editor can do in a second.
For me, this is the biggest benefit of AI coding. And it's energy saved that I can use to focus on higher level problems e.g. architecture thereby increasing my productivity.
What kind of codebases do you work on if you don't mind me asking?
I've found a huge boost from using AI to deal with APIs (databases, k8s, aws, ...) but less so on large codebases that needed conceptual improvements. But at worst, i'm getting more than 10% benefit, just cause the AI's can read files so quickly and answer questions and propose reasonable ideas.
Do you feel like you lack mental energy at 40? I do not feel any different from 30. I think the main difference between 30 and 40 is I am much more efficient at doing things and therefore am able to do more.
I do, but I’m not sure it’s age related. Ever since 2020, it feels like my physical and mental energy has been on a downward trajectory. I feel like I’ve lost several IQ points. What’s interesting is that I’ve heard the same from a lot of people. Not sure what the root cause is, but I do need to take better care of myself.
Is the alleviating of the mental energy going to make you a worst programmer in the long run? Is this like skipping mental workouts that were ultimately keeping you sharp?
Also in my 40s and above senior level. Theres not many mental workouts in day to day coding because the world is just not a new challenge every day. What I consider 'boilerplate' just expands to include things I've written a dozen times before in a different context. AI can write that to my taste and I can tackle the few actual challenges.
At 51, no one hires me because of my coding ability. They hire me because I know how to talk to the “business” and lead (larger projects) or implement (smaller projects) and to help sales close deals.
Don’t get me wrong, I care very deeply about the organization and maintainability of my code and I don’t use “agents”. I carefully build my code (and my infrastructure as code based architecture) piece by piece through prompting.
And I do have enough paranoia about losing my coding ability - and I have lost some because of LLMs - that I keep a year in savings to have time to practice coding for three months while looking for a job.
I know a couple of people whose mental faculties took a sharp nosedive after they started relying on LLMs too much. They might be outliers but just a few years ago I considered them to be really sharp and these days they often struggle with pretty basic reasoning and problem solving.
Do coding in non-assembly programming languages make you a worse programmer in the long run because you are not exposed to the deepest level of complexity?
My guess is if we assume the high level and low level programmers are equally proficient in their mediums, they would use the same amount of effort to tackle problems, but the kinds of problems they can tackle are vastly different
I will say that some weeks it make me 10% more productive, some weeks -10%.
I came to the conclusion that I need to do all the hard work and only ask it to fill the gaps otherwise it will generate too much crap.
I didn't see much mention of tab-completions in the survey and comments here. To me that's the ultimate coding AI is doing at my end, even though it seems to pass unnoticed nowadays. It's massive LOC (and comments!), and that's were I find AI immensely productive.
Don't you have to keep dismissing incorrect auto-complete? For me I have a particular idea in mind, and I find auto-complete to be incredibly annoying.
It breaks flow. It has no idea my intention, but very eagerly provides suggestions I have to stop and swat away.
have tons of text dumped into your text area. Sometimes it looks plausibly right, but with subtle little issues. And you have to carefully analyze whatever it output for correctness (like constant code review).
That we're trying to replace entry/mediocre/expert level code writing with entry/mediocre/expert level code reading is one of the strangest aspects of this whole AI paradigm.
There's literally no way I can see that resulting in better quality, so either that is not what is happening or we're in for a rude awakening at some point.
there is no "we" or at least not sufficiently differentiated. Another layer is inserted into .. everything? think MSFT Teams. Your manager's manager is being empowered.. you become a truck driver who must stay on the route, on schedule or be replaced.
This is how you know the "AI" proselytizers are completely full of shit. They're trying to bend the narrative with a totally unrealistic scenario where reading and reviewing code is somehow "more efficient" than writing it. This is only true if you
(a) don't know what you're doing and just approve everything you see or
Does it actually replace it? I can still get intellisense style suggestions on my ides (various jetbrains and visual studio) and it's still just as useful (what can this object do I wonder...)
My experience with Supermaven has been close to perfect. Imho, the smaller the suggestion, the better it generally is. 99% of supermaven suggestions are just one line.
Does it even fall into "AI-generated" category? GitHub Copilot has been around for years, I certainly remember using it long before the recent AI boom, and at that time it wasn't even thought of as any kind of a breakthrough.
And at this point it's not just a productivity booster, it's as essential as using a good IDE. I feel extremely uncomfortable and slow writing any code without auto-completion.
I think there is a difference between type system or Language Server completions and AI generated completion.
When the AI tab completion fills in full functions based on the function definition you have half typed, or completes a full test case the moment you start type - mock data values and all, that just feels mind-reading magical.
Tab completions simply hit the bottleneck problem. I dont want to press tab on every line it makes no sense. I would rather AI generate a function block and then integrate it back. Saves me the typing hassle and I can focus in design and business logic.
Can you be specific about what you're asking it to write?
I'm pretty confidient that I couldn't get it to implement a element in a web browser. I'm talking about C++ in WebKit or Chromium, not a custom element in HTML/JS. Let's say browsers wanted a new data-table element that natively implemented a scrolling window such that you registered events to supply elements and it asked for only the portion that were visible. I feel like those code bases are too complex for LLMs to add this (could certainly be wrong).
In any case, more concrete examples would help these discussions.
I can give a concrete example: I just asked ChatGPT (should have asked something more integrated into my editor), to give me JavaScript to directly parse meta data out of an MP4 file in JS in the browser. It gave my typescript but told me I should look at mp4box.
I spent 15-20 minutes getting a mp4box example setup (probably should have asked for that too). Only to find that I think mp4box does not get me the data I wanted.
So I went back to ChatGPT and asked for JavaScript because I was hacking in something like jsfiddle. It gave me that and it worked!
I then said I wanted the title and comments meta data as that was missing from what it gave me. That worked too, first try.
I’m not yet up to half (because my corporate code base is a mess that doesn’t lend itself well to AI)
But your approach sounds familiar to me. I find sometimes it may be slower and lower quality to use AI, but it requires less mental bandwidth from me, which is sometimes a worthwhile trade off.
> because my corporate code base is a mess that doesn’t lend itself well to AI
What language? I picked up an old JS project that had several developers fail over weeks to upgrade to newer versions of react. But I got it done in a day by using AI to generate a ton of unit tests then loop an upgrade / test / build. Was 9 years out of date and it’s running in prod now with less errors than before.
Also upgraded rails 4 app to rails 8 over a few days.
Done other apps too. None of these are small. Found a few memory leaks in a C++ app that our senior “experts” who have spent 20 years doing c++ couldn’t find.
Yeah, Claude has gotten much better with Elixir. I think too many people expect one-shot miracles, but you have to guide it along like you would a junior, including directing it towards the proper context.
I've been coding professionally for almost 30 years. I use Claude Code heavily, even for larger features. To get that to work half-decent, you have to take on a PM/Tech-lead role, you're no longer a senior engineer.
For large pieces of work, I will iterate with CC to generate a feature spec. It's usually pretty good at getting you most of the way there first shot and then either have it tweak things or manually do so.
Implementation is having CC first generate a plan, and iterating with it on the plan - a bit like mentoring a junior, except CC won't remember anything after a little while. Once you get the plan in place, then CC is generally pretty good at getting through code and tests, etc. You'll still have to review it after for all the reasons others have mentioned, but in my experience, it'll get through it way faster than I would on my own.
To parallelize some of the work, I often have Visual Studio Code open to monitor what's happening while it's working so I can redirect early if necessary. It also allows me to get a head start on the code review.
I will admit that I spent a lot of time iterating on my way of working to get to where I am, and I don't feel at all done (CC has workflows and subagents to help with common tasks that I haven't fully explored yet). I think the big thing is that tools like CC allow us to work in new ways but we need to shift our mindset and invest time in learning how to use these tools.
* brainstorm all the ideas, get Claude to write docs + code for all them, and then throw away the code
* ask it to develop architecture and design principles based on the contents of those docs
* get it to write a concise config spec doc that incorporates all the features, respects the architecture and design as appropriate
* iterate over that for a while until I get it into a state I like
* ask it to write an implementation plan for the config spec
* babysit it as I ask it to implement phase by phase of the implementation plan while adhering to the config spec
It’s a bit slower to than what I’d hoped originally, but it’s a lot better in terms of end result and gives me more opportunity to verify tests, tweak implementation, briefly segue or explore enhancements, etc.
> To get that to work half-decent, you have to take on a PM/Tech-lead role, you're no longer a senior engineer.
But you’re saying it can be half-decent?
The problem is that about 75% of HN commenters have their identities tightly wound up in being a (genuflect) senior engineer and putting down PM/tech-lead type roles.
They’ll do anything to avoid losing that identity including writing non-stop about how bad AI code is. There’s an Upton Sinclair quote that fits the situation quite nicely.
I'd agree that 75% you speak of is generally hostile to the mere concept of PMs, but that's usually from a misapplication of PMs as proxy-bosses for absentee product owners/directors who don't want to talk to nerds - flow interruptions, beancounting perceived as useless, pointless ceremonies, even more pointless(er) meetings etc, and the further defiling of the definition of "agile".
But a deep conceptual product and roadmap understanding that helps one steer Claude Code is invaluable for both devs and PMs, and I don't think most of that 75% would begrudge that quality in a PM
I wouldn't say I'm old, but I suddenly fell into the coding agent rabbit hole when I had to write some Python automations against Google APIs.
Found myself having 3-4 different sites open for documentation, context switching between 3 different libraries. It was a lot to take in.
So I said, why not give AI a whirl. It helped me a lot! And since then I have published at least 6 different projects with the help of AI.
It refactors stuff for me, it writes boilerplate for me, most importantly it's great at context switching between different topics. My work is pretty broadly around DevOps, automation, system integration, so the topics can be very wide range.
So no I don't mind it at all, but I'm not old. The most important lesson I learned is that you never trust the AI. I can't tell you how often it has hallucinated things for me. It makes up entire libraries or modules that don't even exist.
It's a very good tool if you already know the topic you have it work on.
But it also hit me that I might be training my replacement. Every time I correct its mistakes I "teach" the database how to become a better AI and eventually it won't even need me. Thankfully I'm very old and will have retired by then.
When it comes to dealing with shitty platforms AI is really the best thing ever. I have had the misfortune of writing automations for Atlassian with their weird handling of refresh keys and had AI not pointed out that Atlassian had the genius idea of invalidating refresh keys after single use, I would have wasted a lot more of my time. For this sort of manual labout, AI is the best tool there is.
This article goes completely against my experience so far.
I teach at an internship program and the main problem with interns since 2023 has been their over reliance on AI tools. I feel like I have to teach them to stop using AI for everything and think through the problem so that they don't get stuck.
Meanwhile many of the seniors around me are stuck in their ways, refusing to adopt interactive debuggers to replace their printf() debug habits, let alone AI tooling...
> Meanwhile many of the seniors around me are stuck in their ways, refusing to adopt interactive debuggers to replace their printf() debug habits, let alone AI tooling...
When I was new to the business, I used interactive debugging a lot. The more experienced I got, the less I used it. printf() is surprisingly useful, especially if you upgrade it a little bit to a log-level aware framework. Then you can leave your debugging lines in the code and switch it on or off with loglevel = TRACE or INFO, something like that.
I kind of had the opposite experience. I used to rely mostly on printfs and etc. but started using debugger more.
printf doesn't improve going up and down the call stacks in the debugger to analyze their chain (you'd have to spam debug printfs all around you expect this chain to happen to replace the debugger which would waste time). debugger is really powerful if you use it more than superficially.
> you'd have to spam debug printfs all around you expect this chain to happen to replace the debugger which would waste time
It's not wasting time, it's narrowing in on the things you know you need to look for and hiding everything else. With a debugger you have to do this step mentally every time you look at the debugger output.
Trying to guess what that chain is and putting printf's all around that path feels like doing a poor simulation of what debugger can do out of the box and unlike us - precisely. So I'd say it's exactly the opposite.
If you only care about some specific spot, then sure - printf is enough, but you also need to recompile things every time you add a new one or change debug related details, while debugger can do it re-running things without recompilation. So if anything, printf method can take more time.
Also, in debugger you can reproduce printf using REPL.
This is absolutely true. If anything, interactive debuggers are a crutch and actual logging is the real way of debugging. You really can't debug all sorts of things in an interactive debugger, things like timing issues, thread problems, and you certainly can't find the actual hard bugs that are in running services in production, you know, where the bugs actually happen and are found. Or on other people's machines that you can't just attach a debugger. You need good logging with a good logging library that doesn't affect performance too much when it's turned off, and those messages can also provide very useful context to what things are going on, many times as good if not better than a comment, because at least the log messages are compiled in and type checked, as opposed to comments, which can easily go stale.
Both are valid, if your code is slightly complex it's invaluable to run it at least once with a debugger to verify that your logic is all good. And using logs for this is highly inefficient. E.g. if you have huge data structures that are a pain to print, or if after starting the program you notice that you forgot to add some print somewhere needed.
And obviously when you can't hook the debugger, logs are mandatory. Doesn't have to be one or the other.
This is why I trigger a segfault which dumps core at the spot where I had the printf when the conditions aren't what I want, so I can then open the debugger on the core (obviously: not if I have a copy of the input which can recreate it, if so then a debugger with a conditional breakpoint at the same spot is even better)
Interactive debuggers and printf() are both completely valid and have separate use-cases with some overlap. If you're trying to use, or trying to get people to use, exclusively one, you've got some things to think about.
I almost exclusively debug via printf / logging, and I am so stupendously ignorant that I have even written and published a multi-platform interactive debugger, to go with the compiler I also wrote (one of several, but this one was the most successful). Make of that what you will, I suppose.
That's funny. I remember using interactive debuggers all the time back in the '90s, but it's been a long time since I've bothered. Logging, reading, and thinking is just... easier.
Really ? I find myself thinking the opposite. My program always runs in debug mode, and when there's some issue I put a breakpoint, trigger it, and boom I can check what is wrong. I don't need to stop the program, insert a new line to print what i _guess_ is wrong, restart the program from scratch etc.
Properly debugging my stack is probably one of the first things I setup because I find it way less tedious. Like, for example, if you have an issue in a huge Object or Array, will you actually print all the content, paste it somewhere else and search through the logs ? And by the way, most debuggers also have ability to setup a log points anyways, without having to restart your program. Genuinely curious to know how writing extra lines and having to restart makes things easier.
Of course I'm not saying that I never débug with logs, sometimes it's require or even more efficient, but it's often my second choice.
I imagine that it depends on the kind of software you are working on. I found debuggers helpful back when I was working on interactive GUI programs which had a lot of complex state, built in classic OOP style with lots of objects pointing at each other, but I have not done that sort of thing in a long time. In part, that's because I got seriously into functional programming, which left me with a much more rigorous approach to state transitions; but then I shifted from conventional software into embedded firmware, where you can sometimes stop the microcontroller and use a debugger via JTAG, but it's usually easier to just stream data out over a serial port, or hook an LED up to a spare pin and make it blink. The world doesn't stop even if you want the software to stop, so the utility of a debugger is limited.
After that I went to work for Google, building distributed software running across many machines in a datacenter, and I have no idea how you would hook up a debugger even if you wanted to. It's all logs all the time, there.
By the time that was over, I was thoroughly accustomed to logging, and attaching a debugger had come to seem like a nuisance. Since then I've mostly worked on compilers, or ML pipelines, or both: pure data-processing engines, with no interactivity. If I'm fixing a bug, I'm certainly also writing a regression test about it, which lends itself to a logging-based workflow. I don't mind popping into gdb if that's what would most directly answer my question, but that only happens a couple of times a year.
Thanks for the detailed answer. Indeed I also worked on embedded a long time ago, and the debugger was often the last resort because the debug pins weren't always accessible.
Yeah, I remember learning to use gdb when I was beginning in the early 2000s. I totally thought I was "levelling up" as a programmer and to be honest felt kinda badass with all those windows open in Emacs. But I've found that the number of times I actually resorted to using the debugger has been so small I don't remain fluent in it's use. What am I supposed to do? Write more bugs? On the other hand, I'm always ready to read, think and put in some print/log statements.
The right tool for the right job. If someone gets the job done with printf() then that would be good enough for me.
Interactive debuggers are a great way to waste a ton of time and get absolutely nowhere. They do have their uses but those are not all that common. The biggest usecase for me for GDB has been to inspect stacktraces, having a good mental model of the software you are working on is usually enough to tell you exactly what went wrong if you know where it went wrong.
Lots of people spend way too much time debugging code instead of thinking about it before writing.
Nitpicking a bit here but theres nothing wrong with printf debugging. Its immensely helpful to debug concurrent programs where stopping one part would mess up the state and maybe even avoid the bug you were trying to reproduce.
As for tooling, I really love AI coding. My workflow is pasting interfaces in ChatGPT and then just copy pasting stuff back. I usually write the glue code by hand. I also define the test cases and have AI take over those laborious bits. I love solving problems and I genuinely hate typing :)
I guess it's not a massive surprise given it's an HN forum and a reasonable percentage of HN candidates are doing LLM/AI stuff in recent cohorts, but it still means I have to apply a very big filter every time I open an article and people wax lyrical about how amazing Claude-GPT-super-codez is and how it has made them twice the engineer they were yesterday at the bargain price of $200 a month...
May it all die in a fire very soon. Butlerian jihad now.
I've tried the interactive debuggers but I'm yet to find a situation where they worked better than just printing. I use an interactive console to test what stuff does, but inline in the app I've never had anything that printing wasn't the straightforward fast solution.
I'm not above the old print here or there but the value of an interactive debugger is being able to step and inspect the state of variables at all the different call sites, for instance.
I’m only found them to be useful in gargantuan OOP piles where the context is really hard to keep in your head and getting to any given point in execution can take minutes. In those cases interactive debugging has been invaluable.
The old fogeys don't rely on printf because they can't use a debugger, but because a debugger stops the entire program and requires you to go step by step.
Printf gives you an entire trace or log you can glance at, giving you a bird's eye view of entire processes.
The other thing they do is conveniently not mention all the negative stuff about AI that the source article mentions, they only report on the portion of content from the source that's in any way positive of AI.
And of course, its an article based on a source article based on a survey (of a single company), with the source article written by a "content marketing manager", and the raw data of the survey isn't released/published, only some marketing summary of what the results (supposedly) were. Very trustworthy.
I'm using Zed as my editor, and maybe 18 months ago I upgraded my system. I didn't miss the AI autocomplete at the time, so I didn't bother to set it up again. However, around two weeks ago I figured I'd give it another go.
I set up GitHub Copilot in Zed and... it's horrible. It seems like most of its suggestions are completely misguided and incorrect, usually just duplicating the code immediately above or below the cursor location while updating the name of a single identifier to match some perceived pattern. Not remotely close to what I'd consider useful; I'm definitely a faster & better programmer without it.
I also tried setting up some local models on ollama: I kept getting random tokens inserted that seemed to be markup from the model output that Zed didn't know how to parse. (On mobile rn, will post sample output when I am back at work if I remember to.)
Is paying Anthropic an arm and a leg for the privilege of granting them first-party access to train on my user data really the competitive move as a modern developer?
P.S. I know Zed has their own AI (and it seems like it should be really good!), but when they first introduced it, I tried it out and it immediately consumed the entire free tier's worth of credits in just a few minutes of normal coding: suggestions are proactively generated and count against your account credit even if not accepted, so I didn't really feel like I'd gotten a good sense of the tool by the time the trial ran out. Even if it's really good, it burns through credits extremely fast.
I've used the 200 bucks Claude Code account, and while that's pretty great, it's definitely not "fire and forget". It often requires a completely different skill set. You basically need to develop a feeling for what the agent can do automatically and what needs extra prompting.
If you want a perfect investment imho, get Supermaven. It's autocomplete is 99% perfect.
For autocompletion I suggest giving Zed's own offering another try. It's guesses somewhat correctly at least half the time and is very convenient when it get's what you're trying to do durin a refactor and suggest valid multiline changes. It's still rather annoying when it replaces regular autocompletion for import paths with it's plausible sounding guesses. Before that I was using Cody, but I don't think that they have free tier any more.
> immediately consumed the entire free tier's worth of credits in just a few minutes of normal coding
Happened to me too when I've tried to do a single edit with connected gemini pro. It somehow ran out of free quota even before finishing it.
I don’t think you should look at the Copilot autocomplete as anything other than autocomplete, as far as I know they use worse but faster models for that.
I’m not big on AI but I get occasional use out of Copilot with the chat / agent mode and Claude Sonnet 4.
> suggestions are proactively generated and count against your account credit even if not accepted
This is no longer the case. They only count if you accept them.
Ime their model is not great, but better than copilot (ime copilot is just too slow and disruptive).
Also the options you have for tab-complete, afaik, are zed/zeta, guthub copilot, supermaven, and local models. I dont think there are other providers right now, glad if I am wrong in that.
In terms of LOCs maybe, in terms of importance I think is much less. At least that's how I use LLMs.
While I understand that <Enter model here> might produce the meaty bits as well, I believe that having a truck factor of basically 0 (since no-one REALLY understands the code) is a recipe for a disaster and I dare say long term maintainability of a code base.
I feel that you need to have someone in any team that needs to have that level of understanding to fix non trivial issues.
However, by all means, I use the LLM to create all the scaffolding, test fixtures, ... because that is mental energy that I can use elsewhere.
Agreed. If I use an LLM to generate fairly exhaustive unit tests of a trivial function just because I can, that doesn’t mean those lines are as useful as core complex business logic that it would almost certainly make subtle mistakes in.
I think the parent commentors point was that it is nearly trivial to generate variations on unit tests in most (if not all) unit test frameworks. For example:
A belief that the ability of LLMs to generate parameterizations is intrinsically helpful to a degree which cannot be trivially achieved in most mainstream programming languages/test frameworks may be an indicator that an individual has not achieved a substantial depth of experience.
The useful part is generating the mocks. The various auto mocking frameworks are so hit or miss I end up having to manually make mocks which is time consuming and boring. LLMs help out dramatically and save literally hours of boring error prone work.
Why mock at all? Spend the time making integration tests fast. There is little reason a database, queue, etc. can't be set up in a per-test group basis and be made fast. Reliable software is built upon (mostly) reliable foundations.
Because if part of my tests involve calling an OpenAI endpoint, I don't want to pay .01 cent every time I run my tests.
Because my tests shouldn't fail when a 3rd party dependency is down.
Because I want to be able to fake failure conditions from my dependencies.
Because unit tests have value and mocks make unit tests fast and useful.
Even my integration tests have some mocks in them, especially for any services that have usage based pricing.
But in general I'm going to mock out things that I want to simulate failure states for, and since I'm paranoid, I generally want to simulate failure states for everything.
> Because if part of my tests involve calling an OpenAI endpoint, I don't want to pay .01 cent every time I run my tests.
This is a good time to think to yourself: do I need these dependencies? Can I replace them with something that doesn't expose vendor risk?
These are very real questions that large enterprises grapple with. In general (but not always), orgs that view technology as the product (or product under test) will view the costs of either testing or inhousing technology as acceptable, and cost centers will not.
> But in general I'm going to mock out things that I want to simulate failure states for, and since I'm paranoid, I generally want to simulate failure states for everything.
This can be achieved with an instrumented version of the service itself.
> This is a good time to think to yourself: do I need these dependencies? Can I replace them with something that doesn't expose vendor risk?
Given that my current projects all revolve solely around using LLMs to do things, yes I need them.
The entire purpose of the code is to call into LLMs and do something useful with the output. That said I need to gracefully handle failures, handle OpenAI giving me back trash results (forgetting fields even though they are marked required in the schema, etc), or just the occasional service outage.
Also integration tests only make sense once I have an entire system to integrate. Unit tests let me know that the file I just wrote works.
There are thousands of projects out there that use mocks for various reasons, some good, some bad, some ugly. But it doesn't matter: most engineers on those projects do not have the option to go another direction, they have to push forward.
In this context, why not refactor (and have your LLM of choice) write and optimize the integration tests for you? If the crux of the argument for LLMs is that it is capable of producing sufficient quality software and dramatically reduced costs, why not have it rewrite tests?
hmmmm. I do like integration tests, but I often tell people the art of modern software is to make reliable systems on top of unreliable components. And the integration tests should 100% include times when the network flakes out and drops 1/2 of replies and corrupts msgs and the like.
Minor nit. I wouldn't call those failing systems tests integration tests.
Unit tests are for validation of error paths. Unit tests can leverage mocks or fakes. Need 3 retires with exponential back off, use unit tests and fakes. Integration tests should use real components. Typically, integration tests are happy path and unit are error paths.
Making real components fail and having tests validate failure handling in a more complete environment jumps from integration testing to resilience or chaos testing. Being able to accurately validate backoffs and retries may diminish, but validating intermediate or ending state can be done with artifact monitoring via sinks.
There is unit-integration testing which fakes out as little as possible but still fakes out some edges. The difference being that the failures are introduced via fake vs managing actual system components. If you connect to a real db on unit-integration tests, you typically wouldn't kill the db or use Comcast to slow the network artificially. That would be reserved for the next layer in the test pyramid.
> I do like integration tests, but I often tell people the art of modern software is to make reliable systems on top of unreliable components.
There is a dramatic difference between unreliable in the sense of S3 or other services and unreliable as in "we get different sets of logical outputs when we provide the same input to a LLM". In the first, you can prepare for what are logical outcomes -- network failures, durability loss, etc. In the latter, unless you know the total space of outputs for a LLM you cannot prepare. In the operational sense, LLMs are not a system component, they are a system builder. And a rather poor one, at that.
> And the integration tests should 100% include times when the network flakes out and drops 1/2 of replies and corrupts msgs and the like.
Yeah, it's not that hard to include that in modern testing.
Parameterized tests are good, but I think he might be talking about exercising all the corner cases in the logic of your function, which to my knowledge almost no languages can auto-generate for but LLMs can sorta-ish figure it out.
We are talking about basic computing for CRUD apps. When you start needing to rely upon "sorta-ish" to describe the efficacy or a tool for such a straightforward and deterministic use case, it may be an indicator you need to rethink your approach.
If you want to discount a tool that may save you an immense amount of time because you might have to help it along the fast few feet, thats up to you.
If you can share a tool that can analyze a function and create a test for all corner cases in a popular language, I'm sure some people would be interested in that.
Maybe you should have brought that up earlier instead of acting smug and burying the lede? It's also pretty telling that you didn't elaborate this further and kept your comment short.
What you should have said is that some parameterized test generators do automated white box testing where they look at your code similar to a fuzzer and try to find the test cases automatically. Your first link is literally just setting up an array with test cases, which basically means you'd have to use an LLM to quickly produce the test cases anyway, which makes parameterized testing sound exceedingly pathetic.
>IntelliTest explores your .NET code to generate test data and a suite of unit tests. For every statement in the code, a test input is generated that will execute that statement. A case analysis is performed for every conditional branch in the code. For example, if statements, assertions, and all operations that can throw exceptions are analyzed. This analysis is used to generate test data for a parameterized unit test for each of your methods, creating unit tests with high code coverage. Think of it as smart fuzz testing that trims down the inputs and test cases to what executes all your logic branches and checks for exceptions.
> Maybe you should have brought that up earlier instead of acting smug and burying the lede? It's also pretty telling that you didn't elaborate this further and kept your comment short.
I thought people were generally competent within the areas they discuss and are aware of the tooling within their preferred ecosystem. I apologize if that is not the case.
99% of the work in testing is coming up with test scenarios and test cases. 95% of the code is just dealing with setting up input and output data, 4% is calling the code you want to test and the final assert is often just a single line of code.
I'm not sure what depth of experience has to do with any of this, since it is busy work that costs a lot of time. A form with 120 fields is a form with 120 fields. There is no way around coming up with the several dozens of test cases that you're going to test without filling out almost all of the fields, even the ones that are not relevant to the test itself, otherwise you're not really testing your application.
I use CodeCompanion on neovim. My workflow for text fixtures is basically:
"""
Given these files: file1, file2, ... (these are pulled entirely into the LLM context)
Create a test fixture by creating a type that implements the trait A and should use an in memory SQLite DB, another one that implements Trait B, ...
"""
Of course there is a bit of back and forth, but I find that using Interfaces/Traits/ABCs extensively makes LLMs perform better at these tasks (but I believe it is a nice side-effect of having more testable code to begin with).
However, wiring with IoC frameworks are a bit of a hit and miss to be honest so often I still have to do these parts manually.
“AI” is great for coding in the small, it’s like having a powerful semantic code editor, or pairing with a junior developer who can lookup some info online quickly. The hardest part of the job was never typing or figuring out some API bullshit anyway.
But trying to use it like “please write this entire feature for me” (what vibe coding is supposed to mean) is the wrong way to handle the tool IMO. It turns into a specification problem.
Yes, but in my experience actually no. At least not with the bleeding edge models today. I've been able to get LLMs to write whole features to the point that I'm quite surprised at the result. Perhaps I'm talking to it right (the new "holding it right"?). I tend to begin asking for an empty application with the characteristics I want (CLI, has subcommands, ...) then I ask it to add a simple feature. Get that working then ask it to enhance functionality progressively, testing as we go. Then when functionality is working I ask for a refactor (often it puts 1500 loc in one file, for example), doc, improve help text, and so on. Basically the same way you'd manage a human.
I've also been close to astonished at the capability LLMs have to draw conclusions from very large complex codebases. For example I wanted to understand the details of a distributed replication mechanism in a project that is enormous. Pre-LLM I'd spent a couple of days crawling through the code using grep and perhaps IDE tools, making notes on paper. I'd probably have to run the code or instrument it with logging then look at the results in a test deployment. But I've found I can ask the LLM to take a look at the p2p code and tell me how it works. Then ask it how the peer set is managed. I can ask it if all reachable peers are known at all nodes. It's almost better than me at this, and it's what I've done for a living for 30 years. Certainly it's very good for very low cost and effort. While it's chugging I can think about higher order things.
I say all this as a massive AI skeptic dating back to the 1980s.
> I tend to begin asking for an empty application with the characteristics I want (CLI, has subcommands, ...) then I ask it to add a simple feature.
That makes sense, as you're breaking the task into smaller achievable tasks. But it takes an already experienced developer to think like this.
Instead, a lot of people in the hype train are pretending an AI can work an idea to production from a "CEO level" of detail – that probably ain't happening.
> you're breaking the task into smaller achievable tasks.
this is the part that I would describe as engineering in the first place. This is the part that separates a script kiddie or someone who "knows" one language and can be somewhat dangerous with it, from someone who commands a $200k/year salary, and it is the important part
and so far there is no indication that language models can do this part at. all.
for someone who CAN do the part of breaking down a problem into smaller abstractions, though, some of these models can save you a little time, sometimes, in cases where it's less effort to type an explanation to the problem than it is to type the code directly..
All the hype is about asking an LLM to start with an empty project with loose requirements. Asking it to work on a million lines of legacy code (inadequately tested, as all legacy code) with ancient and complex contracts is a completely different experience.
Very large projects are an area where AI tools can really empower developers without replacing them.
It is very useful to be able to ask basic questions about the code that I am working on, without having to read through dozens of other source files. It frees up a lot of time to actually get stuff done.
I find this half state kind of useless. If I have to know and understand the code being generated, it's easier to just write it myself. The AI tools can just spit out function names and tools I don't know off the top of my head, and the only way to check they are correct is to go look up the documentation, and at that point I've just done the hard work I wanted to avoid.
Feels like a similar situation to self driving where companies want to insist that you should be fully aware and ready to take over in an instant when things go wrong. That's just not how your brain works. You either want to fully disengage, or be actively doing the work.
> The AI tools can just spit out function names and tools I don't know off the top of my head, and the only way to check they are correct is to go look up the documentation, and at that point I've just done the hard work I wanted to avoid.
This is exactly my experience, but I guess generating code with depreciated methods is useful for some people.
This, IMHO, is the critical point and why a lot of “deep” development work doesn’t benefit much from the current generation of AI tools.
Last week, I was dealing with some temporal data. I often find working in this area a little frustrating because you spend so much time dealing with the inherent traps and edge cases, so using an AI code generator is superficially attractive. However, the vast majority of my time wasn’t spent writing code, it was getting my head around what the various representations of certain time-based events in this system actually mean and what should happen when they interact. I probably wrote about 100 test cases next, each covering a distinct real world scenario, and working out how to parameterise them so the coverage was exhaustive for certain tricky interactions also required a bit of thought. Finally, I wrote the implementation of this algorithm that had a lot of essential complexity, which means code with lots of conditionals that needs to be crystal clear about why things are being done in a certain order and decisions made a certain way, so anyone reading it later has a fighting chance of understanding it. Which of those three stages would current AI tools really have helped with?
I find AI code generators can be quite helpful for low-level boilerplate stuff, where the required behaviour is obvious and the details tend to be a specific database schema or remote API spec. No doubt some applications consist almost entirely of this kind of code, and I can easily believe that people working on those find AI coding tools much more effective than I typically do. But as 'manoDev says in the parent comment, deeper work is often a specification problem. The valuable part is often figuring out the what and the why rather than the how, and so far that isn’t something AI has been very good at.
Joking aside, if he is one of the top developers in the company and if he is "actually" a good developer, when compared to others outside of the company, then I can see this bill.
The current feature that I'm working on, required 100 messages to finalize things and I would say the context window was around 35k - 50k per "chat completion". My model of choice is Gemini 2.5 Flash which has an input cost of $0.30/1M. Compare this to Sonnet which is $3.00/1M.
If the person was properly designing and instructing the LLM to build something advanced correctly, I can see the bill being quite high. I personally don't think you need to use Sonnet 99% of the time, but if somebody else is willing to pay the bill, why not.
Bear in mind those are revenue figures, they're costing claude hundreds a day.
One imagines Leadership won't be so pleased after the inevitably price hike (which, given the margins software uses, is going to be in the 1-3 thousands a day) and the hype wears off enough for them to realize they're spending a full salary automating a partial FTE.
But, by the looks of things, models will be more efficient by then and a cheaper-to-run model will produce comparable output. At least that's how it's been with OSS models, or with the Openai api model. So maybe the inevitable price hike (or rate limiting) may lead to switching models / providers and the results being just as good.
> Senior developers were also more likely to say they invest time fixing AI-generated code.
That quote is the key - even if 1/3 of senior devs are pushing mostly AI-driven code, they are checking it first. And while the survey did not cover it, I suspect they are using their experience to decide which areas of a codebase are commonplace enough that AI can handle it vs. which areas are unique and require coding without AI.
People are learning when to use AI as a helpful tool and when not to.
Most of code that "needs to be written" is just a copy
of something standard, "Do X in the simplest way possible"
code that doesn't need optimizations, and writing
it by hand is just waste of time.
AI is good enough to write megabytes of that code,
since its statistically common and part of tons of codebases.
Its the other half of code that AI can't handle,
that you need to manually verify it doesn't hallucinate
fantastic stuff that manages to compile, but doesn't work.
This is self reported unless I missed something. I bet that skews these results quite a bit. Many are very hesitant to say they use AI, and I suspect that's much more likely to be the case when you are new to the field.
Also, green coding? That's new to me. I guess we'll see optional carbon offset purchasing in our subs soon.
Disclaimer: not only am I a Fastly employee, I'm the Fastly employee quoted in the register article originally linked.
We'd argue our stance and focus are a bit different. Our stance is -- we help eng teams and the open web with whatever their preference for AI might be.
If you want to go all in and monetize AI traffic or sign an agreement with a foundation model for training, we have tools that can help you do that. If you want to go scorched-earth on AI bots that try to scrape your systems, we can help you do that too.
It's all about helping engineers tasked with tough problems get their jobs done, regardless of whether the flavor of the week is AI or something else.
LLMs are tools. If you master your tools, you become more productive. I've had the same mixed experiences with LLMs that others have. Some good, some not so great. But I've been merging quite a bit of codex created PRs in the last weeks. Some needed manual intervention. Some were tedious to create. And some clearly saved me a lot of work.
I always have more work than I can handle. Part of my job is deciding what not to do and what to drop because it hasn't got the right priority. Me spending an afternoon on a thing that is fun but not valuable is usually a bad use of my time. With LLMs, I'm taking on a few more of the things that I previously wouldn't have. That started with a few hobby projects that I'm now doing that I previously wasn't. And it's creeping into work as well. LLMs struggle on larger code bases. But less so with recent model releases.
> LLMs are tools. If you master your tools, you become more productive.
LLMs are unpredictable tools which change all the time, let’s not pretend otherwise. You can’t “master” them in the same way as previous tools. You can learn some tricks to trick them to be closer to what you want, and that’s about it.
Imagine if every time you did the exact same movement to hammer a nail, you had to check your work. Maybe this time it hammered it in perfectly, or maybe it smashed your finger, or maybe it only went half-way through. You could never develop muscle memory for such a tool. You could use it, sure, but never master it.
a crack pipe is a tool. if you master your tools you become more productive. i've had the same mixed experiences with crack cocaine that others have. some good, some not so great. But i've been writing a lot of crack-fueled PRs in the last weeks. some needed manual intervention. some were tedious to create. and some clearly saved me a lot of work.
Article did not say what kind languages/applications thosd 791 developers were working on. I work on a legacy Java code base (which looks more like C than Java, thankfully) and I cant imagine AI doing any of it. It can do small isolated, well formulated chunks (functions that do a very specific task) but even that will require very verbose explanation.
I just can't fathom shipping a big percentage of work using LLMs.
But I’ve come full circle and have gone back to hand coding after a couple years of fighting LLMs. I’m tired of coaxing their style and fixing their bugs - some of which are just really dumb and some are devious.
I've also just turned off copilot now. I had several cases where bugs in the generated code slipped through and ended up deployed. Bugs I never would have written myself. Reviewing code properly is so much harder than writing it from scratch.
"write it right the first time" is my new mantra/challenge to myself.
along with improving my skills in vim, this approach has made me significantly more productive and has made my code much simpler compared to when i was using LLM code generation tools.
there is no shortcut around hard work.
there is no shortcut to thoroughly interrogating the constraints of the software problem at hand.
developers who rely on LLM code generation are poor coworkers because they don't understand what they've written and why they've written it.
I'm in the same exact boat. I started with a lot of different tools but eventually went back to hand coding everything. When using tools like co-pilot I noticed I would ship a lot mode dumb mistakes. I even experimented with not even using a chat interface and it turns out that a lot of answers to problems are indeed found with a web search.
Even then I’ve mostly given up. I’ve seen LLMs change from snake case to camel case for a single method and leave the rest untouched. I’ve seen them completely fabricate APIs to non existent libraries. I’ve seen them get mathematical formulae completely wrong. I’ve seen it make entire methods for things that are builtins of a library I’m already using.
It’s just not worth it anymore for anything that is part of an actual product.
Occasionally I will still churn out little scripts or methods from scratch that are low risk - but anything that gets to prod is pretty much hand coded again.
It basically uses multiple different LLMs from different providers to debate a change or code review. Opus 4.1, Gemini 2.5 Pro, and GPT-5 all have a go at it before it writes out plans or makes changes.
it's a lot faster and a lot easier and it gives you a much better understanding of the problem if you just think about the changes you want to make and write them yourself.
why are you looking for a shortcut? just do the work.
I’m not surprised. AI tools have become such a huge time saver. I use them a lot too, but I still double check everything to keep quality high. I read an article on finance gossips called “The Rise of AI in Banking: Friend or Foe”, and it really put this into perspective. AI’s growth in finance shows how quickly it is reshaping industries, so seeing devs rely on AI for half their code makes total sense. Like in banking, it’s not about replacing humans but creating smarter systems, as long as we manage trust, ethics, and transparency, it’s a tool to level us up, not phase us out.
better code == less code? In my mind that would be highly succinct purely functional code in scala or haskell. Those really allow you to "compress" code down to its barest essential domain logic? Perhaps the future is writing highly domain specific DSLs and have the LLM generate the runtime for it.
Naive question but wouldn't it could as having AI write 50%+ of your code if you just use an unintelligent complete-the-line AI tool? In this case the AI is hardly doing anything intelligent, but is still getting credit for doing most of the work.
Yes there is even a small business that champions an small LLM that is trained on the language Lsp, your code base, and your recent coding history (not necessarily commit, but any time one presses ctrl + s). How it works is essentially autocomplete. This functionality is packaged as an IDE plugin: TabNine
However now they try to sell subscriptions to LLMs.
Tabnine has been in the scene since at least 2018.
I've been at this for many years. If I want to implement a new feature that ties together various systems and delivers an expected output, I know the general steps that I need to take. About 80% of those steps are creating and stubbing out new files with the general methods and objects I know will be needed, and all the test cases. So... I could either spend the next 4 hours doing that, or spend 3 minutes filling out a CLAUDE.md with the specs and 5 minutes having Claude do it (and fairly well).
I feel no shame in doing the later. I've also learned enough about LLMs that I know how to write that CLAUDE.md so it sticks to best practices. YMMV.
I tried it - didn't like it. Had an LLM work on a backup script since I don't use Bash very often. Took a bunch of learning the quirks of bash to get the code working properly.
While I'll say it got me started, it wasn't a snap of the fingers and a quick debug to get something done. Took me quite a while to figure out why something worked but really it didn't (LLM using command line commands where Bash doesn't interpret the results the same).
If its something I know, probably wont use LLM (as it doesn't do my style). If it's something I don't know, might use it to get me started but I expect that's all I'll it for.
Can I ask which agent/model you used?
I'm similarly irritated with shell script coding, but find I have to make scripts fairly often. My experience using various models but latterly Claude Code has been quite different -- it churned out pretty much what I was looking for. Also old, fwiw. I'm older than all shells.
Validity of the sample size is not determined by its fraction of the whole population. I don't know the formulas and I'm not a statistician. Maybe someone can drop some citations.
I generally agree with this just from a perspective of personal sentiment, it does feel wrong.
But statistically speaking, at a 95% confidence level you'd be within a +/- 3.5% margin of error given the 791 sample size, irrespective of whether the population is 30k or 30M.
You should read more about statistical significance. Under some reasonable assumptions, you can confidently certain deduce things with small sample sizes.
From another perspective: we've deduced a lot of things about how atoms work without any given experiment inspecting more than an insignificant fraction of all atoms.
TL;DR: The population size (25e6 total devs, 1e80 atoms in observable universe) is almost entirely irrelevant to hypothesis testing.
As a developer with over 30 years of experience, I face a dilemma when asked "how much AI code do you commit"?
a) Answer "a lot". This answer supports the notion that developers are a dying breed and soon everyone will be able to vibe code their own personal software. Which at this point is obviously false. This answer is detrimental to my job.
b) Answer "not much". Then this gets interpreted as "he's an old fart, he can't learn new things, we should be thinking of retiring him". Which is (hopefully) false. Which is - again - detrimental to my job.
This agrees with my experience on a project I’ve been working on this year, in particular related to porting the code. I’ve developed a strategy that I’m calling “Polyglot Mirroring” where the code is written in multiple languages at once, with LLMs handling the mirroring.
I know this is late to the party but I think, given that this topic comes up.every day, we should start including statistics in our posts to get a better sense of how AI is working or not for us. I imagine a years of experience / primary language used / tool used / AI positive or negative as a percentile estimate.
Personally for me it's 20 years of experience / Python / Copilot / 85% positive experience.
I think they're being really loose with the term "vibe coding", and what they really mean is AI-assisted coding.
Older devs are not letting the AI do everything for them. Assuming they're like me, the planning is mostly done by a human, while the coding is largely done by the AI, but in small sections with the human giving specific instructions.
Then there's debugging, which I don't really trust the AI to do very well. Too many times I've seen it miss the real problem, then try to rewrite large sections of the code unnecessarily. I do most of the debugging myself, with some assistance from the AI.
> Assuming they're like me, the planning is mostly done by a human, while the coding is largely done by the AI
I've largely settled on the opposite. AI has become very good at planning what to do and explaining it in plain English, but its command of programming languages still leaves a lot to be desired.
Yes, much like many of the humans I have worked with, sometimes bad choices are introduced. But those bad choices are caught during the writing of the code, so that's not really that big of a deal when it does happen. It is still a boon to have it do most of the work.
And remains markably better than when AI makes bad choices while writing code. That is much harder to catch and requires pouring over the code with a fine tooth comb to the point that you may as well have just written it yourself, negating all the potential benefits of using it to generate code in the first place.
When debugging, I'll coax the AI to determine what went wrong first - to my satisfaction - and have it go from there. Otherwise it's a descent into madness.
I’m still yet to find a use-case for AI-generated code in my workflow.
Even when I am building tools that heavily utilize modern AI, I haven’t found it. Recently, I disabled the AI-powered code completion in my IDE because I found that the cognitive load required to evaluate the suggestions it provided was greater and more time consuming than just writing the code I was already going to write anyways.
I don’t know if this is an experience thing or not - I mainly work on a tech stack I have over a decade of experience in - but I just don’t see it.
Others have suggested generating tests with AI but I find that horrifying. Tests are the one thing you should be the most anal about accuracy on compared to anything else in your codebase.
One thing I don't hear a lot of people talk about is building prototypes. That's where I see a gigantic time savings. It doesn't have to be beautiful code, it just has to help me answer a question so I can make a decision about where to go next. That and tools. There have been many times where I've wanted to build a task-specific tool but justifying the time would be hard. Now I can create little tools like that, and it's a huge productivity boost.
I often use LLMs for method level implementation work. Anything beyond the scope of a single function call I have very little confidence in. This is OK though, since everything is a function and I can perfectly control the blast radius as long as I keep my hands on the steering wheel. I don't ever let the LLM define method signatures for me.
If I don't know how to structure functions around a problem, I will also use the LLM, but I am asking it to write zero code in this case. I am just having a conversation about what would be good paths to consider.
The nice thing about having AI generated code is that you get more time to focus on designing the system and it's edge cases. My code coverage (the significant metric, not the vain metric) and documentation coverage is really high because it's a very strong productivity boost when coupled with LLM generated code.
Been in the industry professionally since 1996 writing code and before that 10 years as a hobbyist between a little BASIC, C and a lot of assembly (65C02, 68k, PPC and x86).
In the last 6 months, when I have had an assignment that involved coding, AI has generated 100% of my code. I just described the abstractions I wanted and reusable modules/classes I needed and built on it.
It's 80% at least for me. I've hit a groove for sure. It's not only 80% is very well tested, and almost exactly how I would have preferred. Big tips are to design your CLAUDE.md files how you would actually code from a high level perspective, and not try the usual "You are an expert Google Distributed Engineer" and all that embarassing AI hype bro crap.
I think at this point it's whoever can get the most useful work out of AI which is actually really hard due to their 'incomplete' state. Finding uses which require very little user input is going to be the next big thing in my opinion since it seems that LLMs are currently at a wall where they require technical advancements before they can overcome it.
Is "vibe coding" synonymous with using AI code-generation tools now?
I thought vibe coding meant very little direct interaction with the code, mostly telling the LLM what you want and iterating using the LLM. Which is fun and worth trying, but probably not a valid professional tool.
I think what happened is that a lot of people started dismissing all LLM code creation as "vibe coding" because those people were anti-LLM, and so the term itself became an easy umbrella pejorative.
And then, more people saw these critics using "vibe coding" to refer to all LLM code creation, and naturally understood it to mean exactly that. Which means the recent articles we've seen about how good vibe coding starts with a requirements file, then tests that fail, then tests that pass, etc.
Like so many terms that started out being used pejoratively, vibe coding got reclaimed. And it just sounds cool.
Also because we don't really have any other good memorable term for describing code built entirely with LLM's from the ground up, separate from mere autocomplete AI or using LLM's to work on established codebases.
“Agentic coding” is probably more accurate, though many people (fairly) find the term “Agentic” to be buzz-wordy and obnoxious.
I’m willing to vibe code a spike project. That is to say, I want to see how well some new tool or library works, so I’ll tell the LLM to build a proof of concept, and then I’ll study that and see how I feel about it. Then I throw it away and build the real version with more care and attention.
I have "vibe coded" a few internal tools now that are very low risk in terms of negative business impact but nonetheless valuable for our team's efficiency.
E.g one tool packages a debug build of an iOS simulator app with various metadata and uploads it to a specified location.
Another tool spits out my team's github velocity metrics.
These were relatively small scripting apps, that yes, I code reviewed and checked for security issues.
I don't see why this wouldn't be a valid professional tool? It's working well, saves me time, is fun, and safe (assuming proper code review, and LLM tool usage).
With these little scripts it creates it's actually pretty quick to validate their safety and efficacy. They're like validating NP problems.
The original definition of vibe coding meant that you just let the agent write everything, and if it works then you commit it. Your code review and security check turned this from vibe coding into something else.
This is complicated by the fact that some people use “vibe coding” to mean any kind of LLM-assisted coding.
My personal definition of "vibe coding" is when a developer delegates -- abdicates, really -- responsibility for understanding & testing what AI-generated code is doing and/or how that result is achieved. I consider it something that's separate from & inferior to using AI as a development tool.
Yeah, for some reason the term has been used interchangeably for a while, which is making it very hard to have a conversation about it since many people think vibe coding is just using AI to assist you.
From Karpathy's original post I understood it to be what you're describing. It is getting confusing.
I think there is actually a pressure to show thst you are using AI (stories of ceos firing employees who supposedly did not "embrace" ai). So people are over attributing to AI. Though originally VC was meant to be infinite monkey style button smashing, people are attributing to VC just to avoid the cross hairs.
I'm not a coder but a sysadmin. 35 years or so. I'm conversant with Perl, Python, (nods to C), BASIC, shell, Powershell, AutoIT (int al)
I muck about with CAD - OpenSCAD, FreeCAD, and 3D printing.
I'm not a senior developer - I pay them.
LLMs are handy in the same way I still have my slide rules and calculators (OK kids I use a calc app) but I do still have my slide rules.
ChatGPT does quite well with the basics for a simple OpenSCAD effort but invents functions within libraries. That is to be expected - its a next token decider function and not a real AI.
I just got back into OpenSCAD after recently getting my first new 3D Printer in 10 years, so I basically had to relearn it. ChatGPT got the syntax wrong for the most basic of operations.
Chat did work for some operations for me but not many.
My screwdrivers, drills, planes, and all the rest don't tell me how to use them and I think that you need to treat LLMs in the same way.
They can provide an insight or a prod or whatever that gets you over the line but I do not think that they are ... AI of any sort.
I can cut say 12mm of plywood with a panel saw within an accuracy of about 1mm if I am careless. If I take more care, I can do way better. Depends on the job. However, I have to keep my eye in.
Just like a slightly blunt chisel, you have to treat Chat and co in the same way - they are tools. No more and no less. You need to know how to sharpen them up (oil and a whetstone).
I can be massively more ambitious when coding with AI, but most importantly I have zero emotional investment in the code so I can throw it away and start again whenever I want.
Jesus, that's a staggering figure to me coming from senior developers. I guess I'm the odd one out here, but ChatGPT is nothing more than an index of Stack Overflow (and friends) for me. It's essentially replaced Googling, but once I get the answer I need I'm still just slinging code like an asshole. Copying the output wholesale from any of these LLMs just seems crazy to me.
If you’re using ChatGPT directly for work then I believe that you are doing it so profoundly wrong, at this point, that you’re going to make really incorrect assumptions.
As we have all observed, the models get things wrong, and if you’re wrong 5% of the time, then ten edits in you’re at 60-40. So you need to run them in a loop where they’re constantly sanity checking themselves—-linting, styling, typing and testing. In other words; calling tools in a loop. Agents are so much better than any other approach it’s comical precisely because they’re scaffolding to let models self-correct.
This is likely somewhat domain-specific; I can’t imagine the models are that great at domains they haven’t seen much code in, so they probably suck at HFT infrastructure for example, though they are decent at reading docs by this point. There’s also a lot of skill in setting up the right documentation, testing structure, interfaces, etc etc etc to make the agents more reliable and productive (fringe benefit; your LLM-wielding colleagues actually write docs now, even if they’re full of em-dashes and emoji). You also need to be willing to let it write a bunch of code, look at it, work out why it’s structurally deficient, throw it away, and build the structure you want to guide it - but the typing is essentially free, so that’s tractable. Don’t view it as bad code, view it as a useful null result.
But if you’re not using Claude Code or Codex or Roo or relatives, you’re living in an entirely different world to the people who have gone messianic about these things.
Yes, I was using "ChatGPT" colloquially. I’ve tried Claude Code, Copilot - I currently have Gemini running though madox2's vim-ai plug - and a few of the others, but I’ve never personally seen the kind of 10x productivity gains some people here talk about. In my experience, it’s typically been quicker to just write the code myself than to wrangle with the model’s output, iterate, and debug. That’s not to say the tools are useless - like I said I think they're great at exposing docs that are disparate or obtuse (looking at you Google) but for actual problem-solving and building, they’ve unfortunately rarely felt like a net win.
I'm sure you're correct that a lot of this comes down to workflow and domain - in mine, the overhead of prompting, reviewing, and correcting usually outweighs the benefits. Of course, as always, this is just one asshole's opinion!
Various projects have proposed a "Co-Developed-By" tag. Even if that were not used I'd expect some open source developers to have become much more productive, and written about it. We'd also see widespread adoption of ancillary tools like AI code reviewers (which have had some adoption but in my experience make suggestions that are worse than useless).
Haha right. I would imagine a "Senior Developer" who is super into AI assisted coding would be more likely to come across this survey and want to participate.
Indeed. This is my biggest fear for engineers as a whole. LLMs can be a great productivity boost in the very short term, but can so easily be abused. If you build a product with it, suddenly everyone is an engineering manager and no one is an expert on it. And growth as an engineer is stunted. It reminds me of abusing energy drinks or grinding to the point of burnout... But worse.
I think we'll find a middle ground though. I just think it hasn't happened yet. I'm cautiously optimistic.
I don't particularly care what this, or any, LLM spits out, and given the pervasive problems that have bedeviled social-science research, I also don't care what the results show, let alone what they show on average.
Anybody who has developed software should understand the value of struggling with a difficult problem. I'm obviously not talking about classroom exercises where the problem sets are expected to match a given skill level or cultivate a specific skill set, so the very idea of individualized, calibrated learning is irrelevant.
As a teacher I'm also 100% uninterested in highly individualized, calibrated challenges for what I teach -- or for what I do professionally. The people who need those highly individualized, wildly different, more gently graduated increases in difficulty, for general problem solving or for the study of any area of programming or computer science, simply should not become engineers.
Apparently vibe coding now just means ai assisted coding beyond immediate code completion?
For me, success with LLM-assisted coding comes when I have a clear idea of what I want to accomplish and can express it clearly in a prompt. The relevant key business and technical concerns come into play, including complexities like balancing somewhat conflicting shorter and longer term concerns.
Juniors are probably all going to have to be learning this kind of stuff at an accelerated rate now (we don't need em cranking out REST endpoints or whatever anymore), but at this point this takes a senior perspective and senior skills.
Anyone can get an LLM and agentic tool to crank out code now. But you really need to have them crank out code to do something useful.
This is the usual propaganda from private equity that invest in LLMs to make you feel like you're missing out for not generating AI slop yourself. I am not surprised that this gets so much traction on YC-owned website since they invest only in AI generating slop companies lately.
It would be nice if we could avoid blog posts like this with extraordinary claims but nearly zero methodological information written by a "content marketing manager" from a company that sells AI products.
My programming background - started from machine codes. Did everything - business desktop, high performance game like apps, firmware for microcontrollers, web front ends, high performance enterprise grade backends, distributed etc. etc. Multiple languages. So lots of experience. Now I use AI quite extensively to generate snippets / classes of code with small and well defined functionality. Most of the time it works just fine and due to my extensive experience I could quickly spot wrong approach, inefficiency, potential issues etc. After telling AI to fix whatever if any problems I've found I usually satisfied with the solution. So overall AI has become a very valuable tool for me, saves gobbles of time. How much exactly - do not really know / care.
This is nonsense, if they are getting this AI slop past code review that is a disgrace. When I take solutions from AI I need to rewrite almost every line for style and project alignment reasons. There is something uncanny valley about AI code that just needs to be deleted or changed too, for example the comments it writes are usually terrible.
Maybe I can potentially believe at one point AI code was used to create a starting point for 50% of PRs…
around a third of senior developers with more than a decade of experience are using AI code-generation tools such as Copilot, Claude, and Gemini to produce over half of their finished software, compared to 13 percent for those devs who've only been on the job for up to two years.
A third? I would expect at least a majority based on the headline and tone of the article... Isn't this saying 66% are down on vibe coding?
This submission was written by the marketing department of a company with commercial interests on the given topic and with almost no information about the "survey" itself, it's essentially blogspam.
The article claims, that senior developers with over 10 years of experience are more than twice as likely to heavily rely on AI tools compared to their junior counterparts. No p-values or statistical significance tests are reported in either The Register article or Fastly's original blog post.
I have over 30 years of experience and recently used Claude Opus 4.1 (via browser and claude.ai) to generate an ECMA-335 and an LLVM code generator for a compiler, and a Qt adapter for the Mono soft debugging protocol. Each task resulted in 2-3kLOC of C++.
The Claude experience was mixed; there is a high probability that the system doesn't respond or just quickly shows an overloaded message and does nothing. If it generates code, I quckly run in some output limitation and have to manually press "continue", and then often the result gets scrambled (i.e. the order of the generated code fragments gets mixed up, which requires another round with Claude to fix).
After this process, the resulting code then compiled immediately, which impressed me. But it is full of omissions and logical errors. I am still testing and correcting. All in all, I can't say at this point that Claude has really taken any work off my hands. In order to understand the code and assess the correctness of the intermediate results, I need to know exactly how to implement the problem myself. And you have to test everything in detail and do a lot of redesigning and correcting. Some implementations are just stubs, and even after several attempts, there was still no implementation.
In my opinion, what is currently available (via my $20 subscription) is impressive, but it neither replaces experience nor does it really save time.
So yes, now I'm one of the 30% seniors who used AI tools, but I didn't really benefit from them in these specific tasks. Not surprisingly, also the original blog states, that nearly 30% of senior developers report "editing AI output enough to offset most of the time savings". So not really a success so far. But all in all I'm still impressed.
Imho your post summarizes 90% of the posts I see about AI coding on HN: not understanding the tools, not understanding their strenghts and weaknesses, not being good at prompting or context management yet forming strong(ish) opinions.
If you don't know what they are good at and how to use them of course you may end up with mixed results and yes, you may waste time.
That's a criticism I have also towards AI super enthusiasts (especially vibe coders, albeit you won't find much here), they often confuse the fact that LLMs often one shot 80% of the solutions with the idea that LLMs are 80% there, whereas the Pareto principle well applies to software development where it's the hardest 20% that's gonna prove difficult.
I'm pretty good at prompting and I successfully use Perplexity (mostly with Claude Sonnet 4) to develop concepts, sometimes with the same session expanded over several days. I think the user interface is much superior over Claude.ai. My hope was that the newer Claude Opus 4.1 would be much better in solving complicated coding tasks, which doesn't seem to be the case. For this I had to subscribe to claude.ai. Actually I didn't see much difference in performance, but a much worse UI and availability experience. When it comes to developing a complex topic in a factual dialogue, Claude Sonnet Thinking seems to me to be even more suitable than Claude Opus.
I'll be more detailed in my second reply.
1) Your original post asks a lot if not too much out the LLM, the expectation you have is too big, to the point that to get anywhere near decent results you need a super detailed prompt (if not several spec documents) and your conclusion stands true: it might be faster to just do it manually. That's the state of LLMs as of today. Your post neither hints at such detailed and laborious prompting nor seem to recognize you've asked it too much, displaying that you are not very comfortable with the limitations of the tool. You're still exploring what it can and what it can't do. But that also implies you're yet not an expert.
2) The second takeaway that you're not yet as comfortable with the tools as you think you are is clearly context management. 2/3k locs of code are way too much. It's a massive amount of output to hope for good results (this also ties with the quality of the prompt, with the guidelines and code practices provided, etc, etc).
3) Neither 1 or 2 are criticisms of your conclusions or opinions, if anything, they are confirmations of your point that LLMs are not there. But what I disagree with is the rush into concluding that AI coding provides net 0 benefits out of your experience. That I don't share. Instead of settling on what it could do (help with planning, writing a spec file, writing unit tests, providing the more boilerplate-y part of the code) and use the LLM to reduce the friction (and thus provide a net benefit), you essentially asked it to replace you and found out the obvious: that LLMs cannot take care of non-trivial business logic yet, and even when they can the results are nowhere near being satisfactory. But that doesn't mean that AI-assisted coding is useless and the net benefit is 0, or negative, it only becomes so as the expectations on the tool are too big and the amount of information provided is either too small to return consistent results or too large for the context to be an issue.
I don't know where your confidence or assumptions come from. Do you work for Anthropic? My prompts for the code generators included an 1.2kLOC code file plus detailed instructions (as described elsewhere), with more details during the session. So I don't think your points apply.
This is a kind of nuts take; - Senior engineer - Uses tools for non trivial undertaking - Didn't find value in it
Your conclusion from that is "but they are doing it wrong", while also claiming they are saying things they didn't say (0 net benefits, useless, etc).
Do you see how that might undermine your point? That you feel they haven't take the time to understand the tools, but you didn't actually read what what wrote?
The situation has improved a little bit over the last few months but LLMs are still only barely usable in languages like C/C++/Zig - and it's not about prompting. I would say that LLMs are usable for JS/Python and while the code is not always what I'd write myself, it can be used and improved later (unless you are working on perf-sensitive JS app, then it's useless again).
And it might be also something with GC, because I suppose the big boys are doing some GRPO over synthetically generated/altered source code (I would!) but obviously doing that in C++ is much more challenging - and I'd expect Rust to be straight impossible.
How do you know that your humble opinion is right about who knows what tool and how deep?
Even if you know better than themselves how musch they know, isn't the tool inadequate just yet for power use then when it is sooo easy to misuse?
Too much tweeking and adapting users to the needs of the tool (vs. the other way around) and there is little point using those (which is a bit of the sickness of modern day computing: 'with computers you can solve problems lightning fast that you wouldn't have without them')
Would you agree with the claim that emacs/vim is an inadequate tool, since it has such a high learning curve?
Prior to LLMs, my impression was "high learning curve, high results" was a pretty popular sweet-spot with a large portion of the tech crowd. It seems weird how much LLMs seem to be an exception to this.
Emacs and vim have complex interfaces that have been stable for decades. Seems like every new flavor of LLM requires learning its warts and blind spots from scratch.
Hey! I would encourage you to try our Claude code instead, which is also part of your subscription. It's a CLI that takes care of many of the issues you encountered, as it works directly on the code files in a directory. No more copy pasting or unscrambling results. Likewise, it can run commands itself to e.g. compile or even test code.
I'm working on old hardware and not-recent Linux and compiler versions, and I have no confidence yet in allowing AI direct (write) access to my repositories.
Instead I provided Claude with the source code of a transpiler to C (one file) which is known to work, uses the same IR as the new code generators were supposed to use.
This is a controlled experiment with a clear and complete input and clear expectations and specifications of the output. I don't think I would be able to clearly isolate the contributions and assess the performance of Claude when it has access to arbitrary parts of the source code.
I use Claude Code with the Max plan, and the experience isn't far off from what you describe. You still need to understand the system and review the implementation, because it makes many mistakes.
That's not the part it saves me time in, it saves me time in looking up the documentation. Other than that, it might be slower, because the larger the code change is, the more time I need to spend reviewing, and past a point I just can't be bothered.
The best way I've found is to have it write small functions, and then I tell it to compose them together. That way, I know exactly what's happening in the code, and I can trust that it works correctly. Cursor is probably a better way to do that than Claude Code, though.
So, I am paying $20, for a glorified code generator, that may or may not be correct, to write a small function that I can do for free, and be confident about the correctness, if I have not been lazy to implement a test for it.
If you point out, with test it's also the same with any AI tool available, but to come to that result, I have to continuously prompt it till it gives me the desired output, while I may be able to do it in 2/3 iterations.
Reading documentation always made me little bit knowledgeable than before, while prompting the LLM, gives me nothing of knowledge.
And, I also have to decide which LLM would be good for the task at hand, and most of them will not be free (unless I use a local, but that will also use GPU, and add an energy cost)
I may be nitpicking, but I see too many holes with this approach
The biggest hole you don't see is that it's worth the $20 to make me overcome my laziness, because I don't like writing code, but I like making stuff, and this way I can make stuff while fooling my brain into thinking I'm not writing code.
Sure, that can be a point, which is helping you overcome your personal barrier, But that can be anything,
That is not you were vouching for on the original comment. It was about saving time.
Your time isn't free, and I'd certainly with more than $20/month.
I find it extremely useful as a smarter autocomplete, especially for the tedious work - changing function definitions, updating queries when DB schema changes, and writing http requests/api calls from vendor/library documentation.
Certainly, So I use an IDE, IntelliJ Ultimate to be precise.
None of the use-cases you mention requires LLM. Just available as IDE functionalities.
IntelliJ has LLM based auto complete, with which I am okay, But it still wrong too many times. Works extremely well with Rust. Their non-llm autocomplete is also superb, which uses ML for suggesting closest, relevant match, IIRC.
It also makes refactoring a breeze, I know what it's going to do exactly.
Also, it can handle database refactoring to a certain capacity! And for that it does not require LLM, so no nondeterministic behavior.
Also, the IDE have its own way of doing http requests, and it's really nice! But, I can use their live template to do autocomplete any boilerplate code. It only requires setting once. No need to fiddle with prompts.
Not only that, but the process described is how you train a junior dev.
There, at least, the wasted time results in the training of a human being who can become sophisticated enough to become a trusted independent implementer in a relatively short duration
> The best way I've found is to have it write small functions, and then I tell it to compose them together.
Pretty much how I code without AI, except making my brain break the problem down into small functions and expressing them in code rather than as a chat.
> it saves me time in looking up the documentation
I have a Perplexity subscription which I heavily use for such purpose, just asking how something works or should be used, with a response just on the point and with examples. Very useful indeed. Perplexity gives me access to Claude Sonnet 4 w/o Thinking which I consider great models, and it can also generate decent code. My intention was to find out how good the recent Claude Opus is in comparison and how much of my work I'm able to delegate. Personally I much prefer the user interface features, performance and availability of Perplexity to Claude.ai.
I end up using Perplexity a lot too, especially when I'm doing something unfamiliar. It's also a good way to quickly find out what are best practices for a given framework/language I'm not that familiar with (I usually ask it to link to examples in the wild and it find opensource projects illustrating those points)
I have both, and Perplexity is much more like a search engine than a chat companion (or at least that's how I use it). I like both, though.
You can select the model. I very much appreciate the Claude Sonnet models which are very good and rational discussion partners, responding to arguments in detail and critically, allowing for the dialectical exploration of complex topics. I have also experimented with other models including ChatGPT, Gemini or Grok, but the resulting discussions were only a fraction as useful (i.e. more optimized towards affirmative feel-good small talk, from my humble point of view).
Hmm, I've never tried that, even though I prefer Claude in general too. I'll try that, thanks!
claude-code asks you to allow it to do anything before it does them. Once you start trusting it and get comfortable with its behavior it gets annoying being prompted all the time, so you can whitelist specific commands it wants to run. You can also interactively toggle into (and out of) "accept changes without asking" mode.
(It wasn't clear to me that I would be able to toggle out of accept changes mode, so I resisted for a loooooong time. But turns out it's just a toggle on/off and can be changed in real-time as it's chugging along. There's also a planning state but haven't looked into that yet)
It always asks before running commands unless you whitelist them. I have whitelisted running testsuites and linters, for example so it can iterate on those corners with minimal interaction. I have had to learn to let it go ahead and make small obvious mistakes rather than intervene immediately because the linters and tests will catch them and Claude will diagnose the failure and fix them at that point.
Anyway I took a small toy project and used that to get a feel for claude-code. In my experience using the /init command to create CLAUDE.md (or asking Claude to interview you to create it) is vital for consistent behavior.
I haven't had good "vibe" experiences yet. Mostly I know what I want to do and just basically delegate implementation. Some things that have worked well for me is to ask Claude to propose a few ways to improve or implement a feature. It's come up with a few things I hadn't thought of that way.
Anyway, claude-code was very good at slowly and incrementally earning my trust. I resisted trying it because I expected it would just run hogwild doing bewildering things, but that's not what it does. It tends to be a bit of an asskisser in it's communication style in a way that would annoy me if it were a real person. But I've managed to look past that.
On Claude you specifically accept any attempt to use a terminal command (optionally whitelisting) so there’s no risk that it will push force something or whatever. You can also whitelist with granularity, for example to enable it to use git to view git logs but not commit.
You can just let it work, see what’s uncommitted after it’s over, and get rid of the result if you don’t like it.
> I have no confidence yet in allowing AI direct (write) access to my repositories.
You don't need to give it write access to your repositories, just to a source tree.
I've been trying it for a couple of months, I can't recommend it either tbh. It's frustrating as hell to work with: super inconsistent, very bad at following its own instructions, wasteful and generally unreliable.
The problem is, it's like a very, very junior programmer that knows the framework well, but won't use it consistently and doesn't learn from mistakes AT ALL. And has amnesia. Fine for some trivial things, but anything more complicated the hand-holding becomes so involved you are better off doing it yourself. That way you internalise some of the solutions as well, which is nice because then you can debug it later! Now I have a huge PR that even I myself don't really grasp as much as I would want.
But for me the nail in the coffin was the terrible customer service. ymmv.
[flagged]
Do you mean Claude code should fail? Why?
[flagged]
In what specific programming language/toolchain/technology is your experience? Why do you think that "everybody can tell that chat gpt wrote your code"? Meanwhile I looked at a lot of LLM generated code in different languages, and I wouldn't generally subscribe to your statement. And you still didn't explain why Claude should fail. I think it is rather an advantage (when it reliably works in future).
I can tell when my coworkers Go code is generated by an llm. I hate it very much.
Wow, shots fired! Would you add something to that?
i spend a lot of time fixing unacceptably poor code that LLM platforms have tricked human coworkers into finding adequate.
my coworkers are increasingly ignorant about the software products they work on.
LLM-informed software development is organizationally poisonous.
Businesses selling LLM coding tools occupy the same place in my mind as drug dealers.
Feels like that's more of a problem with the competence of your coworkers than with the LLM. The LLM is just exposing how bad they are.
I'm just shy of 30 years experience. I think I've spent more time learning how to use these tools than any other technology I've learned, and I still don't know the best way to use them.
They certainly weren't a time-saver right away but they became one after some time giving them a real shot. I tested their + my limits on small projects, working out how to get them to do the whole project, figuring out when they stop working and why, figuring out which technology they work best with, figuring out the right size problems to give them, figuring out how to recognize if I'm asking them something they can't do well a ask something different instead, guide them into creating code that they can't actually continue to be successful with.
I started last December in Cursor's agentic mode and have been in Claude Code ever since probably March or April. It's definitely been a huge boost all year for side projects - but only in the last couple months have I been having success in a large codebase.
Even with all this experience I don't know that I would really be able to get much value out of the chat interface for them. They need to be proposing changes I can just hit accept or reject on (this is how both Claude Code and Cursor work btw - you don't have to allow it to write to any file you don't want or execute any command you don't want).
The fact that you use Claude, not gpt5 which is light-years ahead for coding, tells me all I need to know.
I don't think it is true that GPT-5 is much better than Claude 4.1. And more important is that light year is distance measure and I am sure openAI data centers is still on earth.
I hate to play the "you're holding it wrong" card, but when I started, I had more or less the same experience. Eventually you start to learn how to better talk to it in order to get better results.
Something I've found useful with Claude Code is that it works a lot better if I give it many small tasks to perform to eventually get the big thing done, rather than just dumping the big thing in its lap. You can do this interactively (prompt, output, prompt, output, prompt, output...) or by writing a big markdown file with the steps to build it laid out.
While this matches my experience, it's worth mentioning thar the act of breaking a task up into the correct chunk size and describing it in English is itself a non trivial task which can be more time consuming than simply writing the actual code.
The fact that it works is amazing, but I'm less convinced that it's enhancing my productivity.
(I think they real productivity boost for me is if I still write the code and have the assistant write test coverage based on diffs, which is trivial to prompt for good results)
And one that a lot of people skip, so that forcing function might make for better code, even if it isn’t faster.
This. For me (senior as in I am in the field since last century), that's how I use it. "I want to do that, with this data, to obtain that"
I still do the part of my job that I got experience on, analyze the need, and use the AI like an assistant to do small libraries or part of code. Like these, errors have less chance to appear. Then I glue that together.
For me the time ratio is best use like that. If I have to describe the whole thing, I'm not far from doing it myself, so there's no need for me.
Important: I work alone, not in a team, so maybe it has an impact on my thought
I just tried to use it for something I consider it to provide the most benefit (for my case). Being able to fully delegate a complicated (and boring) part to a machine would give me more time for the things I'm really interested in. I think we are on the right track in this regard, but we still have a long way to go.
Similar here. AI works much better as a consultant than as a developer. I ask it all kinds of things I have suspicions and intuitions about and it provides clarity and examples. It's great for subroutines. Trying to make full programs is just too large of a space. It's difficult to communicate all the implicit requirements.
People who consistently consult LLMs for product direction or software feature design overwhelmingly appear to me as willfully ignorant dullards.
I mean it's even further than willful ignorance. It's delight in one's own ignorance.
It’d be nice if we could “pipe” prompts directly similar to how we pipe multiple Unix commands to eventually get what we really want.
Then we can give someone that entire string of prompts as a repeatable recipe.
You can send prompts on the command line to Claude, I typically save prompts in the repo. But note it won’t produce deterministic output.
Seems about right for me (older developer at a big tech company). But we need to define what it means that the code is AI-generated. In my case, I typically know how I want the code to look like, and I'm writing a prompt to tell the agent to do it. The AI doesn't solve any problem, it just does the typing and helps with syntax. I'm not even sure I'm ultimately more productive.
Yeah I’m still not more productive. Maybe 10% more. But it alleviates a lot of mental energy, which is very nice at the age of 40.
For all the folks on the "reduce mental burden", "reduce cognitive load" train, are you all aware that this basically means you are exercising less your brain day in and day out, and in the end you will forget how to do things? You will learn how to guide an AI agent, but until the day an AI agent is perfect (and we don't know if we will ever see that day), you are just losing inch by inch your ability to actually understand what the agent is writing and what is going on.
I'm pretty radical on this topic but for me cognitive load is good, you are making your neurons work and keep synapses in place where they matter (at least for your job). I totally accept writing down doc or howto to make doing some action in future easier and reduce that cognitive load, but using AI agent IMO is like going to bike in the mountain with an electrical bike.
Yes, you keep seeing the wonderful vistas but you are not really training your legs.
This, to me, feels like you're complaining to the 45 year old builder that they should be using a hammer instead of a nail gun.
I know how to nail a nail, I've nailed so many nails that I can't remember them all.
My job is to build a house efficiently, not nail nails. Anything that can make me more efficient at it is a net positive.
Now I've saved 2 hours in the framing process by using a nail gun, I have 2 extra hours to do things that need my experience. Maybe spot the contractor using a nail plate in the wrong way or help the apprentice on their hammering technique.
IMO it's different. That's why I brought the e-bike similitude: climbing even mild mountains or hills with your own legs will actually make your legs, heart and lungs stronger in the process. So you get both the wonderful views (building the house or delivering the software) but also you get improved health (keeping your mind trained on both high level thinking and low level implementation vs high level only). We might say that using a hammer constantly will develop more your muscles, but in carpentry there are still plenty of manual work that will develop your muscles anyway. (and we still don't have bricks laying machines)
Ironically, e-bikes, at least in the EU, are having the exact opposite effect. More people that don't normally ride bikes are using e-bikes to get about. The motor functions not as a replacement, but as a force multiplier. It also makes "experimenting" easier, because the motor can make up for any mistakes or wrong turns.
Caveat: In the EU, an e-bike REQUIRES some physical effort any time for the motor to run. Throttles are illegal.
> Ironically, e-bikes, at least in the EU, are having the exact opposite effect. More people that don't normally ride bikes are using e-bikes to get about.
At least in Germany people rather joke that the moment e-bikes became popular, people began to realize that they suddenly became too unathletic to be capable of pedaling a bicycle. I know of no person who uses an e-bike who did not ride an ordinary bicycle before.
> In the EU, an e-bike REQUIRES some physical effort any time for the motor to run.
The motor must shut off when 25 km/h is reached - which is basically the speed that a trained cyclist can easily attain. So because of this red tape stuff, e-bikes are considered to be useless and expensive by cyclists who are not couch potatoes.
But they still cannot assist 100%, there needs to be effort from the user.
Otherwise they would be considered e-scooters and would have different rules and regulations applied.
Yes, you pedal with almost zero effort when flat and with a little more effort when uphill. Obviously if you are going to go up the Tourmalet you will run out of battery pretty soon, but that's not the context most e-bikers use them.
In that it fits the LLM situation quite well. LLMs remove the anxieties around coding for newbies at scale better than they make indisputable productivity gains for senior developers, similar to how e-bikes help with newbies more than cyclists.
I know that's what many people, especially elder one, say but this is still a hill I will die on :) They are mostly used to go in mostly flat roads, like some slow-speed motorcycle that needs some low effort. The ones using them outside paved road, using them as multipliers, are the ones that already did mountain biking when they were younger and they want to continue doing it at a higher level their age would permit without (which it's perfectly fine!).
But if my goal is to ride Downhill[0] me exhausting myself riding up the hill isn't bringing any extra enjoyment for me.
With electric assist, I can get up faster and to the business of coming down the hill really fast.
We have ski-lifts for the exact same reason. People doing downhill skiing would make their legs, heart and lungs stronger in the process of walking up the hill with their skis. But who wants to do that, that's not the fun part.
And to step back from analogy-land.
I'm paid to solve problems, I'm good at architecture, I can design services, I can also write the code to do so. But in most cases the majority of the code I write is just boilerplate crap with minor differences.
With LLMs I can have them write the Terraform deployment code, write the C# CRUD Controller classes and data models and apply the Entity Framework migrations. It'll do all that and add all the fancy Swagger annotation while it's at it. It'll even whip up a Github build action for me.
Now I've saved a few days of mindless typing and I can get into solving the actual problem at hand, the one I'm paid to do. In reality I'm doing it _while_ I'm instructing the LLM to do the menial crap + reviewing the code it produced so I'm moving at twice the speed I would be normally.
"But can't you just..." nope, if every project was _exactly_ the same, I'd have a template already, there are just enough differences to not make it worth my time[1]. If I had infinite time and money, I could build a DSL to do this, but again, referring to [1] - there's no point =)
It's more cost efficient to pay the LLM tax to OpenAI, Anthropic or whatever and use it as a bespoke bootstrapping system for projects.
[0] https://en.wikipedia.org/wiki/Downhill_mountain_biking
[1] https://xkcd.com/1319/
> So you get both the wonderful views (building the house or delivering the software) but also you get improved health (keeping your mind trained on both high level thinking and low level implementation vs high level only).
The vast majority of developers aren't summitting beautiful mountains of code, but are instead are sifting through endless corporate slop.
> We might say that using a hammer constantly will develop more your muscles, but in carpentry there are still plenty of manual work that will develop your muscles anyway.
The trades destroy human bodies over time and lead to awful health outcomes.
Most developers will and should take any opportunity to reduce cognitive load, and will instead spend their limited cognitive abilities on things that matter: family, sport, art, literature, civics.
Very few developers are vocational. If that is you and your job is your identity, then that's good for you. But don't fall into the trap of thinking that's a normal or desirable situation for others.
> The vast majority of developers aren't summitting beautiful mountains
I'm not sure you're approaching this metaphor the right away. The point is that coding manually is great cognitive exercise which keeps the mind sharp for doing the beautiful stuff.
> The trades destroy human bodies over time and leads to awful health outcomes.
Again, you're maybe being too literal and missing the point. No one is destroying their minds by coding. Exercise is good.
I am using LLMs, too, and I do not consider myself thinking less. I still have to be part of the whole process, incl. architectural process among other things that require my knowledge and my thinking.
I use them too and actually agree with you that the cognitive load is somewhat comparable. I was only pointing out what seemed like an abuse of the metaphor.
> I'm not sure you're approaching this metaphor the right away. The point is that coding manually is great cognitive exercise which keeps the mind sharp for doing the beautiful stuff.
No, I'm challenging the metaphor. Working the trades isn't exercise - it's a grind that wears people out.
> Again, you're maybe being too literal and missing the point. No one is destroying their minds by coding. Exercise is good.
We actually have good evidence that the effects of heavy cognitive load are detrimental to both the brain and mental health. We know that overwork and stress are extremely damaging to both.
So reducing cognitive load in the workplace is an unambiguous good, and protects the brain and mind for the important parts of life, which are not in front of a screen.
> We actually have good evidence that the effects of heavy cognitive load are detrimental to both the brain and mental health. We know that overwork and stress are extremely damaging to both.
I don't think this is fair either, you're comparing "overwork and stress" to "work." It's like saying we have evidence that extreme physical stress is detrimental ergo it's "unambiguously" healthier to drive than to walk.
Maybe you could share your good evidence so we can see if normal coding tasks would fall under the umbrella of overwork and stress?
We have plentiful evidence and studies on the effect of even moderate day-long cognitive work has on cognitive ability and on the effect of stress.
This is so well-founded that I do not have to provide individual sources - it is the current global accepted reality. I wouldn't provide sources for the effect of CO2 emissions on the climate or gravity, either.
However, the opposite is not true. If you have evidence that routine coding itself improves adult brain health or cognitive ability, please share RCTs or large longitudinal studies showing net cognitive gains under typical workloads.
Again you're conflating things and are now also moving goalposts (overwork->moderate work) and asking me for very precise kinds of studies while refusing to even point towards the basis for your own claims. On top of this you're implying that I'm some kind of lunatic by associating my questions with climate denial.
It's clear that you're more interested in "winning" than actually have a reasonable discussion so goodbye. I've had less frustrating exchanges with leprechauns.
Come on. We’ve had decades of occupational-health research on cognitive load, stress, and hours. The pattern is clear. Higher demands and longer hours raise depression risk. Lab and field work shows day-long cognitive tasks produce measurable fatigue, decision drift, and brain chemistry changes. These are universally accepted.
And yet, you now want me to source individual studies on those effects in a HN thread? Yes, in this instance you are approaching flat-earth/climate-change-denial levels of discourse. Reducing cognitive load is an unambiguous good.
If you think routine coding itself improves brain health or cognitive ability, produce the studies showing as you demanded from me, because that is a controversial claim. Or you can crash out of the conversation.
> No, I'm challenging the metaphor. Working the trades isn't exercise - it's a grind that wears people out.
If your job is just grinding out code in a stressful and soul-crushing manner, the issue lies elsewhere. You will be soon either grinding out prompts to create software you don't even understand anymore or you will be replaced by an agent.
And by no way I'm implying you are part of the issue.
But LLMs can make the soul crushing part so much easier.
I need to add a FooController to an existing application, to store FooModels to the database. The controller needs the basic CRUD endpoints, etc.
I can spend a day doing it (crushing my soul) or I can just tell any Agentic LLM to do it and no something that doesn't crush my soul, like talk with the customer about how the FooModels will be used after storing.
"But it'll produce bad code!"
No it doesn't. It knows _exactly_ how to do a basic CRUD HTTP API controller in C#. It's not an art form, it's just rote typing and adding Attributes to functions.
Because it's an Agentic LLM, it'll go look at another controller and copy its structure (which is not standard globally, but this project has specific static attributes in every call).
Then I review the code, maybe add a few comments for actual humans, commit and push.
My soul remains uncrushed, client is happy when I delivered the feature on time and I have half the day off for other smaller tasks that would become technical debt otherwise.
> My soul remains uncrushed, client is happy when I delivered the feature on time and I have half the day off for other smaller tasks that would become technical debt otherwise.
This is a very optimistic take. If you are in the type of company that just gives you boring code and tasks, you will required to use the other half-day to work on some other boring feature. This will not give us time to pay tech debt. Maybe it will do in the beginning when using AI agents has not been institutionalized yet, but once it has, you will be asked to churn out more "features"
If the mechanics of the work are “soul-crushing,” isn’t that the root cause, and the LLM is just a bandaid? I’m not saying every professional dev is enthused with all their tasks. But if you’re so eager to avoid parts of the job (however rote they are), then maybe it’s time for something new?
I can’t write this without feeling preachy, and I apologize for that. But I keep reading a profound lack of agency in comments like these.
It’s a necessary part of the job.
Like no mechanic gets pleasure from an oil change or tire rotation. They’d rather figure out an issue with an old V8
But they do it because that’s a part of the job.
> If your job is just grinding out code in a stressful and soul-crushing manner, the issue lies elsewhere.
The vast majority of developers are in or near this category. Most software developers never write code outside of education or employment and would avoid doing so if an AI provided the opportunity. Any opportunity to reduce cognitive load is welcome
I think you don't recognise how much of an outlier you are in believing that your work improves your cognitive abilities.
Exactly, not every programmer is an artesan who hones their craft on and off the clock.
Some people just write code for 8 hours, go home and never think about it on their free time.
I'd like to counter argue we need to shift the notion of efficiency. While the nailgun is more efficient than the hammer, both need to be operated by a real human. Including all limitations of physical and mental health. While we can improve the efficiency of the tools we should consider not to burn out humans to match the goal.
The builder still needs to study the plans and build a mental model of what they’re building.
A nailgun isn’t automated in the way an LLM is, maybe if it moved itself around and fired nails where it thought they should go based on things it had seen in the past it would be a better comparison.
On the contrary, I am a lot more willing to think through the contours of the problems I need to solve because I haven't drained my mental energy writing five repetitive - but slightly different - log lines and tweaking the wording slightly to be correct.
I'm training smarter, and exercising better, instead of wasting all the workout/training time on warmups, as it were.
It completely depends on how you use AI. If you turn off your brain entirely and just coast as much as possible, then yeah your comment would apply.
But I think of work as essentially two things - creative activity and toil. I simply use AI for toil, and let my brain focus on creativity and problem solving.
Writing my 100,000th for loop is not going to preserve my brain.
This is why I recommend the functional programming map pattern ;)
I have been coding and athletic training for about as long as eachother, your anecdote works. However just like in physical training, you should really spare your energy for stuff that you enjoy and actually progresses you.
By using LLMs to do some of the stuff I have long gotten over, I have a bit more mental energy to tackle new problems I wouldn't have previously.
As well LLMs just aren't actually that competent yet, so it's not like devs are completely hands off. Since I barely do boilerplate as I work across legacy projects, there's no way Claude Code today is writing half my code.
My current pattern is to manually craft during the first half of the day when I enjoy that, and during the second half when I'd be normally burnt on hard thought and not quite up for another coffee, pomodoro, theanine deep dive, I can start tackling tests, exploratory data analysis, or small bugs, and these tasks are 50% or more LLM.
So yeah, 30%-50% seems right, but it's not like I lost any part of my job that I love.
That's a good approach, I like it and probably adopt it as well. I'm not dogmatically against LLMs, I just need we should think about possible consequences, and not thinking about them like a holy grail of everything.
You're right, but usually at the end of the day I'm completely mentally exhausted and dont want to talk to anyone. Its something I've realized is a big problem in my life. I'm actively trying to reduce mental load to leave stuff for other hobbies and social activities.
you are obviously not past 50 years old. At that age, eventhough I "train" (as you imply it) a lot (I code about 50 hours a week, 40 at work, the rest for my own projects; not of them easy, my job is writing numerical code for simulation and my pet project is writinga very accurate emulator (which implies physics modelling, tons of research, etc)). I can definitely feel I'm not as productive as before (before I could sustain 60 hours a week) eventhough I do feel I'm using my brain to the maximum...
So yeah, past a certain age, you'll be happy to reduce your mental load. No question about it. And I feel quite relieved when Claud writes this classic algorith I have understood long ago and don't want to re-activate in my brain. And I feel quite disappointed when Claude misses the point and I have to code review it...
> are you all aware that this basically means you are exercising less your brain day in and day out, and in the end you will forget how to do things?
IDE did the same thing and we found other ways to exercise our brains. This one is really something unreasonable to worry about.
Do you sit there multiplying two digit numbers in your head for fun, for the practice, to keep operating at peek mental capacity on weekends? In the name of operating at peek mental capacity, that seems like the most logical thing to do. Just wake up at 6 am Saturday morning and start multiplying numbers on your head.
If you don't wanna use AI, that's entirely up to you, but "you're gonna forget how to program if you use AI and then whatever are you going to do if the AI is down" reeks of motivated reasoning.
Strangely I've found myself more exhausted at the end of the week and I think it's because of the constant supervision necessary to stop Claude from colouring outside the lines when I don't watch it like a hawk.
Also I tend to get more done at a time, it makes it easier to get started on "gruntwork" tasks that I would have procrastinated on. Which in turn can lead to burnout quite quickly.
I think in the end it's just as much "work", just a different kind of work and with more quantity as a result.
Now we just need an AI that helps with more quality.
No, what I want better is tooling that makes supervising and getting insight into what it's doing a superior experience.
A far more interactive coding "agent" that makes sure it walks through every change it makes with you, and doesn't just rush through tasks. That helps team members come up to speed on a repository by working through it with them.
> Strangely I've found myself more exhausted at the end of the week and I think it's because of the constant supervision necessary to stop Claude from colouring outside the lines when I don't watch it like a hawk.
Welcome to management. Computers and code are easy. People and people wannabes like LLMs are a pain.
I find AI is most useful at the ancillary extra stuff. Things that I'd never get to myself. Little scripts of course, but more like "it'd be nice to rename this entire feature / db table / tests to better match the words that the business has started to use to discuss it".
In the past, that much nitpicky detail just wouldn't have gotten done, my time would have been spent on actual features. But what I just described was a 30 minute background thing in claude code. Worked 95%, and needed just one reminder tweak to make it deployable.
The actual work I do is too deep in business knowledge to be AI coded directly, but I do use it to write tests to cover various edge cases, trace current usage of existing code, and so on. I also find AI code reviews really useful to catch 'dumb errors' - nil errors, type mismatches, style mismatch with existing code, and so on. It's in addition to human code reviews, but easy to run on every PR.
Wow, 30 minutes to rename functions and tests? I wonder how much energy and water that llm wasted for something that any lsp supporting editor can do in a second.
Nice, which reviewer do you use? I was using CodeRabbi now switched to LiveReview.
> it alleviates a lot of mental energy
For me, this is the biggest benefit of AI coding. And it's energy saved that I can use to focus on higher level problems e.g. architecture thereby increasing my productivity.
What kind of codebases do you work on if you don't mind me asking?
I've found a huge boost from using AI to deal with APIs (databases, k8s, aws, ...) but less so on large codebases that needed conceptual improvements. But at worst, i'm getting more than 10% benefit, just cause the AI's can read files so quickly and answer questions and propose reasonable ideas.
How are you quantifying that 10% ?
Do you feel like you lack mental energy at 40? I do not feel any different from 30. I think the main difference between 30 and 40 is I am much more efficient at doing things and therefore am able to do more.
I do, but I’m not sure it’s age related. Ever since 2020, it feels like my physical and mental energy has been on a downward trajectory. I feel like I’ve lost several IQ points. What’s interesting is that I’ve heard the same from a lot of people. Not sure what the root cause is, but I do need to take better care of myself.
I think it may be just age and stress. Also apparently even pretty mild Covid could cause some vasculatory damage?
On the flip side physical excercise can cause even very old people to improve radically.
Also creatine is good for everyone for mind and body.
This could be long COVID.
Is the alleviating of the mental energy going to make you a worst programmer in the long run? Is this like skipping mental workouts that were ultimately keeping you sharp?
Also in my 40s and above senior level. Theres not many mental workouts in day to day coding because the world is just not a new challenge every day. What I consider 'boilerplate' just expands to include things I've written a dozen times before in a different context. AI can write that to my taste and I can tackle the few actual challenges.
At 51, no one hires me because of my coding ability. They hire me because I know how to talk to the “business” and lead (larger projects) or implement (smaller projects) and to help sales close deals.
Don’t get me wrong, I care very deeply about the organization and maintainability of my code and I don’t use “agents”. I carefully build my code (and my infrastructure as code based architecture) piece by piece through prompting.
And I do have enough paranoia about losing my coding ability - and I have lost some because of LLMs - that I keep a year in savings to have time to practice coding for three months while looking for a job.
I know a couple of people whose mental faculties took a sharp nosedive after they started relying on LLMs too much. They might be outliers but just a few years ago I considered them to be really sharp and these days they often struggle with pretty basic reasoning and problem solving.
Do coding in non-assembly programming languages make you a worse programmer in the long run because you are not exposed to the deepest level of complexity?
My guess is if we assume the high level and low level programmers are equally proficient in their mediums, they would use the same amount of effort to tackle problems, but the kinds of problems they can tackle are vastly different
I will say that some weeks it make me 10% more productive, some weeks -10%. I came to the conclusion that I need to do all the hard work and only ask it to fill the gaps otherwise it will generate too much crap.
I am pleasantly surprised when it is able to figure out the root cause of some bugs which at times require a lot of mental energy
Exactly
I didn't see much mention of tab-completions in the survey and comments here. To me that's the ultimate coding AI is doing at my end, even though it seems to pass unnoticed nowadays. It's massive LOC (and comments!), and that's were I find AI immensely productive.
Don't you have to keep dismissing incorrect auto-complete? For me I have a particular idea in mind, and I find auto-complete to be incredibly annoying.
It breaks flow. It has no idea my intention, but very eagerly provides suggestions I have to stop and swat away.
Yeah, [autocomplete: I totally agree]
it's so [great to have auto-complete]
annoying to constantly [have to type]
have tons of text dumped into your text area. Sometimes it looks plausibly right, but with subtle little issues. And you have to carefully analyze whatever it output for correctness (like constant code review).
That we're trying to replace entry/mediocre/expert level code writing with entry/mediocre/expert level code reading is one of the strangest aspects of this whole AI paradigm.
There's literally no way I can see that resulting in better quality, so either that is not what is happening or we're in for a rude awakening at some point.
https://www.joelonsoftware.com/2000/05/26/reading-code-is-li...
https://www.joelonsoftware.com/2000/04/06/things-you-should-... (read the bold text in the middle of the article)
These articles are 25 years old.
there is no "we" or at least not sufficiently differentiated. Another layer is inserted into .. everything? think MSFT Teams. Your manager's manager is being empowered.. you become a truck driver who must stay on the route, on schedule or be replaced.
This is how you know the "AI" proselytizers are completely full of shit. They're trying to bend the narrative with a totally unrealistic scenario where reading and reviewing code is somehow "more efficient" than writing it. This is only true if you
(a) don't know what you're doing and just approve everything you see or
(b) don't care how bad things get
fwiw, VS Code has a snooze auto complete button. Each press is 5m, a decently designed feature imo
Copilot replacing intellisense is a huge shame. Why get actual auto complete when you can get completely hallucinated methods and properties instead.
Does it actually replace it? I can still get intellisense style suggestions on my ides (various jetbrains and visual studio) and it's still just as useful (what can this object do I wonder...)
No it does not replace it. I still get intellisense in VS Code. GP is misinformed
My experience with Supermaven has been close to perfect. Imho, the smaller the suggestion, the better it generally is. 99% of supermaven suggestions are just one line.
For those about to look it up: Supermaven is dead.
Does it even fall into "AI-generated" category? GitHub Copilot has been around for years, I certainly remember using it long before the recent AI boom, and at that time it wasn't even thought of as any kind of a breakthrough.
And at this point it's not just a productivity booster, it's as essential as using a good IDE. I feel extremely uncomfortable and slow writing any code without auto-completion.
I think there is a difference between type system or Language Server completions and AI generated completion.
When the AI tab completion fills in full functions based on the function definition you have half typed, or completes a full test case the moment you start type - mock data values and all, that just feels mind-reading magical.
I haven’t tried it lately but how well are these models in generating property based tests?
Tab completions simply hit the bottleneck problem. I dont want to press tab on every line it makes no sense. I would rather AI generate a function block and then integrate it back. Saves me the typing hassle and I can focus in design and business logic.
Can you be specific about what you're asking it to write?
I'm pretty confidient that I couldn't get it to implement a element in a web browser. I'm talking about C++ in WebKit or Chromium, not a custom element in HTML/JS. Let's say browsers wanted a new data-table element that natively implemented a scrolling window such that you registered events to supply elements and it asked for only the portion that were visible. I feel like those code bases are too complex for LLMs to add this (could certainly be wrong).
In any case, more concrete examples would help these discussions.
I can give a concrete example: I just asked ChatGPT (should have asked something more integrated into my editor), to give me JavaScript to directly parse meta data out of an MP4 file in JS in the browser. It gave my typescript but told me I should look at mp4box.
I spent 15-20 minutes getting a mp4box example setup (probably should have asked for that too). Only to find that I think mp4box does not get me the data I wanted.
So I went back to ChatGPT and asked for JavaScript because I was hacking in something like jsfiddle. It gave me that and it worked!
I then said I wanted the title and comments meta data as that was missing from what it gave me. That worked too, first try.
I’m not yet up to half (because my corporate code base is a mess that doesn’t lend itself well to AI)
But your approach sounds familiar to me. I find sometimes it may be slower and lower quality to use AI, but it requires less mental bandwidth from me, which is sometimes a worthwhile trade off.
> because my corporate code base is a mess that doesn’t lend itself well to AI
What language? I picked up an old JS project that had several developers fail over weeks to upgrade to newer versions of react. But I got it done in a day by using AI to generate a ton of unit tests then loop an upgrade / test / build. Was 9 years out of date and it’s running in prod now with less errors than before.
Also upgraded rails 4 app to rails 8 over a few days.
Done other apps too. None of these are small. Found a few memory leaks in a C++ app that our senior “experts” who have spent 20 years doing c++ couldn’t find.
100%. My instructions are often pretty exact "i want a genserver that fetches the data every X sec and updates Y if new data"
Yeah, Claude has gotten much better with Elixir. I think too many people expect one-shot miracles, but you have to guide it along like you would a junior, including directing it towards the proper context.
I typed less using AI, so it is a net win I guess
I've been coding professionally for almost 30 years. I use Claude Code heavily, even for larger features. To get that to work half-decent, you have to take on a PM/Tech-lead role, you're no longer a senior engineer.
For large pieces of work, I will iterate with CC to generate a feature spec. It's usually pretty good at getting you most of the way there first shot and then either have it tweak things or manually do so.
Implementation is having CC first generate a plan, and iterating with it on the plan - a bit like mentoring a junior, except CC won't remember anything after a little while. Once you get the plan in place, then CC is generally pretty good at getting through code and tests, etc. You'll still have to review it after for all the reasons others have mentioned, but in my experience, it'll get through it way faster than I would on my own.
To parallelize some of the work, I often have Visual Studio Code open to monitor what's happening while it's working so I can redirect early if necessary. It also allows me to get a head start on the code review.
I will admit that I spent a lot of time iterating on my way of working to get to where I am, and I don't feel at all done (CC has workflows and subagents to help with common tasks that I haven't fully explored yet). I think the big thing is that tools like CC allow us to work in new ways but we need to shift our mindset and invest time in learning how to use these tools.
I’ve arrived at something similar:
* brainstorm all the ideas, get Claude to write docs + code for all them, and then throw away the code
* ask it to develop architecture and design principles based on the contents of those docs
* get it to write a concise config spec doc that incorporates all the features, respects the architecture and design as appropriate
* iterate over that for a while until I get it into a state I like
* ask it to write an implementation plan for the config spec
* babysit it as I ask it to implement phase by phase of the implementation plan while adhering to the config spec
It’s a bit slower to than what I’d hoped originally, but it’s a lot better in terms of end result and gives me more opportunity to verify tests, tweak implementation, briefly segue or explore enhancements, etc.
> To get that to work half-decent, you have to take on a PM/Tech-lead role, you're no longer a senior engineer.
But you’re saying it can be half-decent?
The problem is that about 75% of HN commenters have their identities tightly wound up in being a (genuflect) senior engineer and putting down PM/tech-lead type roles.
They’ll do anything to avoid losing that identity including writing non-stop about how bad AI code is. There’s an Upton Sinclair quote that fits the situation quite nicely.
Kinda but kinda not.
I'd agree that 75% you speak of is generally hostile to the mere concept of PMs, but that's usually from a misapplication of PMs as proxy-bosses for absentee product owners/directors who don't want to talk to nerds - flow interruptions, beancounting perceived as useless, pointless ceremonies, even more pointless(er) meetings etc, and the further defiling of the definition of "agile".
But a deep conceptual product and roadmap understanding that helps one steer Claude Code is invaluable for both devs and PMs, and I don't think most of that 75% would begrudge that quality in a PM
I wouldn't say I'm old, but I suddenly fell into the coding agent rabbit hole when I had to write some Python automations against Google APIs.
Found myself having 3-4 different sites open for documentation, context switching between 3 different libraries. It was a lot to take in.
So I said, why not give AI a whirl. It helped me a lot! And since then I have published at least 6 different projects with the help of AI.
It refactors stuff for me, it writes boilerplate for me, most importantly it's great at context switching between different topics. My work is pretty broadly around DevOps, automation, system integration, so the topics can be very wide range.
So no I don't mind it at all, but I'm not old. The most important lesson I learned is that you never trust the AI. I can't tell you how often it has hallucinated things for me. It makes up entire libraries or modules that don't even exist.
It's a very good tool if you already know the topic you have it work on.
But it also hit me that I might be training my replacement. Every time I correct its mistakes I "teach" the database how to become a better AI and eventually it won't even need me. Thankfully I'm very old and will have retired by then.
I love the split personality vibe here.
Or perhaps the commenter just aged a lot while writing the post.
First line: "I wouldn't say I'm old"
Last line: "Thankfully I'm very old"
Hmm.....
Maybe he meant "I'll be very old..." the second time.
You probably jest, but I'm sure some HN users do actually have split personalities. (or dissociative identities, as they're called nowadays)
And I’m sure some HN users are autistic or have other traits that make them unable to appreciate a joke.
Okay?
When it comes to dealing with shitty platforms AI is really the best thing ever. I have had the misfortune of writing automations for Atlassian with their weird handling of refresh keys and had AI not pointed out that Atlassian had the genius idea of invalidating refresh keys after single use, I would have wasted a lot more of my time. For this sort of manual labout, AI is the best tool there is.
One time use refresh keys is not all that uncommon, probably more so than not, but lots of clients handle that for you
> invalidating refresh keys after single use
That's called refresh token rotation and is a valid security practice.
I know but the RFC doesnt mandate it. https://datatracker.ietf.org/doc/html/rfc6749#section-6
Not sure why Google doesnt do this but Atlassian does.
Google OAuth2 refresh tokens are definitely singe use.
Atleast not documented here https://developers.google.com/identity/protocols/oauth2#5.-r.... They have a limit on the number of tokens but not on number of uses per token.
This article goes completely against my experience so far.
I teach at an internship program and the main problem with interns since 2023 has been their over reliance on AI tools. I feel like I have to teach them to stop using AI for everything and think through the problem so that they don't get stuck.
Meanwhile many of the seniors around me are stuck in their ways, refusing to adopt interactive debuggers to replace their printf() debug habits, let alone AI tooling...
> Meanwhile many of the seniors around me are stuck in their ways, refusing to adopt interactive debuggers to replace their printf() debug habits, let alone AI tooling...
When I was new to the business, I used interactive debugging a lot. The more experienced I got, the less I used it. printf() is surprisingly useful, especially if you upgrade it a little bit to a log-level aware framework. Then you can leave your debugging lines in the code and switch it on or off with loglevel = TRACE or INFO, something like that.
I kind of had the opposite experience. I used to rely mostly on printfs and etc. but started using debugger more.
printf doesn't improve going up and down the call stacks in the debugger to analyze their chain (you'd have to spam debug printfs all around you expect this chain to happen to replace the debugger which would waste time). debugger is really powerful if you use it more than superficially.
> you'd have to spam debug printfs all around you expect this chain to happen to replace the debugger which would waste time
It's not wasting time, it's narrowing in on the things you know you need to look for and hiding everything else. With a debugger you have to do this step mentally every time you look at the debugger output.
Trying to guess what that chain is and putting printf's all around that path feels like doing a poor simulation of what debugger can do out of the box and unlike us - precisely. So I'd say it's exactly the opposite.
If you only care about some specific spot, then sure - printf is enough, but you also need to recompile things every time you add a new one or change debug related details, while debugger can do it re-running things without recompilation. So if anything, printf method can take more time.
Also, in debugger you can reproduce printf using REPL.
> printf() is surprisingly useful, especially if you upgrade it a little bit to a log-level aware framework.
What do you mean by this? Do you mean using a logging framework instead of printf()?
This is absolutely true. If anything, interactive debuggers are a crutch and actual logging is the real way of debugging. You really can't debug all sorts of things in an interactive debugger, things like timing issues, thread problems, and you certainly can't find the actual hard bugs that are in running services in production, you know, where the bugs actually happen and are found. Or on other people's machines that you can't just attach a debugger. You need good logging with a good logging library that doesn't affect performance too much when it's turned off, and those messages can also provide very useful context to what things are going on, many times as good if not better than a comment, because at least the log messages are compiled in and type checked, as opposed to comments, which can easily go stale.
Both are valid, if your code is slightly complex it's invaluable to run it at least once with a debugger to verify that your logic is all good. And using logs for this is highly inefficient. E.g. if you have huge data structures that are a pain to print, or if after starting the program you notice that you forgot to add some print somewhere needed.
And obviously when you can't hook the debugger, logs are mandatory. Doesn't have to be one or the other.
> verify your logic
This is what unit tests are for.
And no one has ever written a buggy unit test.
That’s what the code is for.
This is why I trigger a segfault which dumps core at the spot where I had the printf when the conditions aren't what I want, so I can then open the debugger on the core (obviously: not if I have a copy of the input which can recreate it, if so then a debugger with a conditional breakpoint at the same spot is even better)
Timing and concurrency issues are actually easier to discover in a debugger than using printf logging.
Really? I’ve been using interactive debuggers since the days of Turbo C/Turbo Pascal in the mid 1990s.
Yes you need good logging also.
it can get even better
https://rustc-dev-guide.rust-lang.org/tracing.html#i-dont-wa...
Interactive debuggers and printf() are both completely valid and have separate use-cases with some overlap. If you're trying to use, or trying to get people to use, exclusively one, you've got some things to think about.
printf() users in this thread are very proud of their own ignorance
I almost exclusively debug via printf / logging, and I am so stupendously ignorant that I have even written and published a multi-platform interactive debugger, to go with the compiler I also wrote (one of several, but this one was the most successful). Make of that what you will, I suppose.
That's funny. I remember using interactive debuggers all the time back in the '90s, but it's been a long time since I've bothered. Logging, reading, and thinking is just... easier.
Really ? I find myself thinking the opposite. My program always runs in debug mode, and when there's some issue I put a breakpoint, trigger it, and boom I can check what is wrong. I don't need to stop the program, insert a new line to print what i _guess_ is wrong, restart the program from scratch etc.
Properly debugging my stack is probably one of the first things I setup because I find it way less tedious. Like, for example, if you have an issue in a huge Object or Array, will you actually print all the content, paste it somewhere else and search through the logs ? And by the way, most debuggers also have ability to setup a log points anyways, without having to restart your program. Genuinely curious to know how writing extra lines and having to restart makes things easier.
Of course I'm not saying that I never débug with logs, sometimes it's require or even more efficient, but it's often my second choice.
I imagine that it depends on the kind of software you are working on. I found debuggers helpful back when I was working on interactive GUI programs which had a lot of complex state, built in classic OOP style with lots of objects pointing at each other, but I have not done that sort of thing in a long time. In part, that's because I got seriously into functional programming, which left me with a much more rigorous approach to state transitions; but then I shifted from conventional software into embedded firmware, where you can sometimes stop the microcontroller and use a debugger via JTAG, but it's usually easier to just stream data out over a serial port, or hook an LED up to a spare pin and make it blink. The world doesn't stop even if you want the software to stop, so the utility of a debugger is limited.
After that I went to work for Google, building distributed software running across many machines in a datacenter, and I have no idea how you would hook up a debugger even if you wanted to. It's all logs all the time, there.
By the time that was over, I was thoroughly accustomed to logging, and attaching a debugger had come to seem like a nuisance. Since then I've mostly worked on compilers, or ML pipelines, or both: pure data-processing engines, with no interactivity. If I'm fixing a bug, I'm certainly also writing a regression test about it, which lends itself to a logging-based workflow. I don't mind popping into gdb if that's what would most directly answer my question, but that only happens a couple of times a year.
Thanks for the detailed answer. Indeed I also worked on embedded a long time ago, and the debugger was often the last resort because the debug pins weren't always accessible.
Also conditional breakpoints. I.e. break on this line if foo==5
I couldn't imagine going back to print statement based debugging. Would be a massive waste of time.
Yeah, I remember learning to use gdb when I was beginning in the early 2000s. I totally thought I was "levelling up" as a programmer and to be honest felt kinda badass with all those windows open in Emacs. But I've found that the number of times I actually resorted to using the debugger has been so small I don't remain fluent in it's use. What am I supposed to do? Write more bugs? On the other hand, I'm always ready to read, think and put in some print/log statements.
The right tool for the right job. If someone gets the job done with printf() then that would be good enough for me.
Interactive debuggers are a great way to waste a ton of time and get absolutely nowhere. They do have their uses but those are not all that common. The biggest usecase for me for GDB has been to inspect stacktraces, having a good mental model of the software you are working on is usually enough to tell you exactly what went wrong if you know where it went wrong.
Lots of people spend way too much time debugging code instead of thinking about it before writing.
Oh, and testing >> debugging.
Nitpicking a bit here but theres nothing wrong with printf debugging. Its immensely helpful to debug concurrent programs where stopping one part would mess up the state and maybe even avoid the bug you were trying to reproduce.
As for tooling, I really love AI coding. My workflow is pasting interfaces in ChatGPT and then just copy pasting stuff back. I usually write the glue code by hand. I also define the test cases and have AI take over those laborious bits. I love solving problems and I genuinely hate typing :)
Yeah, I'm not putting stock in this at all. The methodology of this survey seems dubious.
30% of the articles on this forum are covert LLM advertising
Good, it's not just me.
I guess it's not a massive surprise given it's an HN forum and a reasonable percentage of HN candidates are doing LLM/AI stuff in recent cohorts, but it still means I have to apply a very big filter every time I open an article and people wax lyrical about how amazing Claude-GPT-super-codez is and how it has made them twice the engineer they were yesterday at the bargain price of $200 a month...
May it all die in a fire very soon. Butlerian jihad now.
I agree. Burn the machines-that-think.
To anyone reading, if you work on LLM software, I hope that your business fails.
Every single last link here is some form of an advertisement, why single out LLM-related ads?
because the underlying technology is so egregiously overhyped in relation to the quality of the services offered
Sure, but why this technology. It's not like that's the first time that's happened, and it won't be the last.
I've tried the interactive debuggers but I'm yet to find a situation where they worked better than just printing. I use an interactive console to test what stuff does, but inline in the app I've never had anything that printing wasn't the straightforward fast solution.
I'm not above the old print here or there but the value of an interactive debugger is being able to step and inspect the state of variables at all the different call sites, for instance.
it's genuinely embarassing that the printf() diehards in this thread are not clear on this detail.
I’m only found them to be useful in gargantuan OOP piles where the context is really hard to keep in your head and getting to any given point in execution can take minutes. In those cases interactive debugging has been invaluable.
I guess that’s the difference. I do rails dev mostly and it’s just put a print statement in, then run the unit test. It’s a fast feedback loop.
The old fogeys don't rely on printf because they can't use a debugger, but because a debugger stops the entire program and requires you to go step by step.
Printf gives you an entire trace or log you can glance at, giving you a bird's eye view of entire processes.
Most decent debuggers have condițional breakpoints.
Not to mention tracepoints (logging breakpoints), which are functionally the same as printf but don't require compiling/restarting.
And have since the mid 1990s at least…
So how many developers in that survey are those?
They surveyed 791 developers (:D) and "a third of senior developers" do that. That's... generiously, what... 20 people?
It's amazing how everyone can massage numbers when they're trying to sell something.
The other thing they do is conveniently not mention all the negative stuff about AI that the source article mentions, they only report on the portion of content from the source that's in any way positive of AI.
And of course, its an article based on a source article based on a survey (of a single company), with the source article written by a "content marketing manager", and the raw data of the survey isn't released/published, only some marketing summary of what the results (supposedly) were. Very trustworthy.
>It's amazing how everyone can massage numbers when they're trying to sell something.
More like massage words.
Like one fairly smart person told me- Doing math with words is pointless.
I think I'm using it wrong.
I'm using Zed as my editor, and maybe 18 months ago I upgraded my system. I didn't miss the AI autocomplete at the time, so I didn't bother to set it up again. However, around two weeks ago I figured I'd give it another go.
I set up GitHub Copilot in Zed and... it's horrible. It seems like most of its suggestions are completely misguided and incorrect, usually just duplicating the code immediately above or below the cursor location while updating the name of a single identifier to match some perceived pattern. Not remotely close to what I'd consider useful; I'm definitely a faster & better programmer without it.
I also tried setting up some local models on ollama: I kept getting random tokens inserted that seemed to be markup from the model output that Zed didn't know how to parse. (On mobile rn, will post sample output when I am back at work if I remember to.)
Is paying Anthropic an arm and a leg for the privilege of granting them first-party access to train on my user data really the competitive move as a modern developer?
P.S. I know Zed has their own AI (and it seems like it should be really good!), but when they first introduced it, I tried it out and it immediately consumed the entire free tier's worth of credits in just a few minutes of normal coding: suggestions are proactively generated and count against your account credit even if not accepted, so I didn't really feel like I'd gotten a good sense of the tool by the time the trial ran out. Even if it's really good, it burns through credits extremely fast.
I've used the 200 bucks Claude Code account, and while that's pretty great, it's definitely not "fire and forget". It often requires a completely different skill set. You basically need to develop a feeling for what the agent can do automatically and what needs extra prompting.
If you want a perfect investment imho, get Supermaven. It's autocomplete is 99% perfect.
For autocompletion I suggest giving Zed's own offering another try. It's guesses somewhat correctly at least half the time and is very convenient when it get's what you're trying to do durin a refactor and suggest valid multiline changes. It's still rather annoying when it replaces regular autocompletion for import paths with it's plausible sounding guesses. Before that I was using Cody, but I don't think that they have free tier any more.
> immediately consumed the entire free tier's worth of credits in just a few minutes of normal coding
Happened to me too when I've tried to do a single edit with connected gemini pro. It somehow ran out of free quota even before finishing it.
I don’t think you should look at the Copilot autocomplete as anything other than autocomplete, as far as I know they use worse but faster models for that.
I’m not big on AI but I get occasional use out of Copilot with the chat / agent mode and Claude Sonnet 4.
> suggestions are proactively generated and count against your account credit even if not accepted
This is no longer the case. They only count if you accept them.
Ime their model is not great, but better than copilot (ime copilot is just too slow and disruptive).
Also the options you have for tab-complete, afaik, are zed/zeta, guthub copilot, supermaven, and local models. I dont think there are other providers right now, glad if I am wrong in that.
In terms of LOCs maybe, in terms of importance I think is much less. At least that's how I use LLMs.
While I understand that <Enter model here> might produce the meaty bits as well, I believe that having a truck factor of basically 0 (since no-one REALLY understands the code) is a recipe for a disaster and I dare say long term maintainability of a code base.
I feel that you need to have someone in any team that needs to have that level of understanding to fix non trivial issues.
However, by all means, I use the LLM to create all the scaffolding, test fixtures, ... because that is mental energy that I can use elsewhere.
Agreed. If I use an LLM to generate fairly exhaustive unit tests of a trivial function just because I can, that doesn’t mean those lines are as useful as core complex business logic that it would almost certainly make subtle mistakes in.
> If I … generate fairly exhaustive unit tests of a trivial function
… then you are not a senior software engineer
Neither are you if that's your understanding of a senior engineer
I think the parent commentors point was that it is nearly trivial to generate variations on unit tests in most (if not all) unit test frameworks. For example:
Java: https://docs.parasoft.com/display/JTEST20232/Creating+a+Para...
C# (nunit, but xunit has this too): https://docs.nunit.org/articles/nunit/technical-notes/usage/...
Python: https://docs.pytest.org/en/stable/example/parametrize.html
cpp: https://google.github.io/googletest/advanced.html
A belief that the ability of LLMs to generate parameterizations is intrinsically helpful to a degree which cannot be trivially achieved in most mainstream programming languages/test frameworks may be an indicator that an individual has not achieved a substantial depth of experience.
The useful part is generating the mocks. The various auto mocking frameworks are so hit or miss I end up having to manually make mocks which is time consuming and boring. LLMs help out dramatically and save literally hours of boring error prone work.
Why mock at all? Spend the time making integration tests fast. There is little reason a database, queue, etc. can't be set up in a per-test group basis and be made fast. Reliable software is built upon (mostly) reliable foundations.
Because if part of my tests involve calling an OpenAI endpoint, I don't want to pay .01 cent every time I run my tests.
Because my tests shouldn't fail when a 3rd party dependency is down.
Because I want to be able to fake failure conditions from my dependencies.
Because unit tests have value and mocks make unit tests fast and useful.
Even my integration tests have some mocks in them, especially for any services that have usage based pricing.
But in general I'm going to mock out things that I want to simulate failure states for, and since I'm paranoid, I generally want to simulate failure states for everything.
End to End tests are where everything is real.
> Because if part of my tests involve calling an OpenAI endpoint, I don't want to pay .01 cent every time I run my tests.
This is a good time to think to yourself: do I need these dependencies? Can I replace them with something that doesn't expose vendor risk?
These are very real questions that large enterprises grapple with. In general (but not always), orgs that view technology as the product (or product under test) will view the costs of either testing or inhousing technology as acceptable, and cost centers will not.
> But in general I'm going to mock out things that I want to simulate failure states for, and since I'm paranoid, I generally want to simulate failure states for everything.
This can be achieved with an instrumented version of the service itself.
> This is a good time to think to yourself: do I need these dependencies? Can I replace them with something that doesn't expose vendor risk?
Given that my current projects all revolve solely around using LLMs to do things, yes I need them.
The entire purpose of the code is to call into LLMs and do something useful with the output. That said I need to gracefully handle failures, handle OpenAI giving me back trash results (forgetting fields even though they are marked required in the schema, etc), or just the occasional service outage.
Also integration tests only make sense once I have an entire system to integrate. Unit tests let me know that the file I just wrote works.
There are thousands of projects out there that use mocks for various reasons, some good, some bad, some ugly. But it doesn't matter: most engineers on those projects do not have the option to go another direction, they have to push forward.
In this context, why not refactor (and have your LLM of choice) write and optimize the integration tests for you? If the crux of the argument for LLMs is that it is capable of producing sufficient quality software and dramatically reduced costs, why not have it rewrite tests?
hmmmm. I do like integration tests, but I often tell people the art of modern software is to make reliable systems on top of unreliable components. And the integration tests should 100% include times when the network flakes out and drops 1/2 of replies and corrupts msgs and the like.
Minor nit. I wouldn't call those failing systems tests integration tests.
Unit tests are for validation of error paths. Unit tests can leverage mocks or fakes. Need 3 retires with exponential back off, use unit tests and fakes. Integration tests should use real components. Typically, integration tests are happy path and unit are error paths.
Making real components fail and having tests validate failure handling in a more complete environment jumps from integration testing to resilience or chaos testing. Being able to accurately validate backoffs and retries may diminish, but validating intermediate or ending state can be done with artifact monitoring via sinks.
There is unit-integration testing which fakes out as little as possible but still fakes out some edges. The difference being that the failures are introduced via fake vs managing actual system components. If you connect to a real db on unit-integration tests, you typically wouldn't kill the db or use Comcast to slow the network artificially. That would be reserved for the next layer in the test pyramid.
> I do like integration tests, but I often tell people the art of modern software is to make reliable systems on top of unreliable components.
There is a dramatic difference between unreliable in the sense of S3 or other services and unreliable as in "we get different sets of logical outputs when we provide the same input to a LLM". In the first, you can prepare for what are logical outcomes -- network failures, durability loss, etc. In the latter, unless you know the total space of outputs for a LLM you cannot prepare. In the operational sense, LLMs are not a system component, they are a system builder. And a rather poor one, at that.
> And the integration tests should 100% include times when the network flakes out and drops 1/2 of replies and corrupts msgs and the like.
Yeah, it's not that hard to include that in modern testing.
Parameterized tests are good, but I think he might be talking about exercising all the corner cases in the logic of your function, which to my knowledge almost no languages can auto-generate for but LLMs can sorta-ish figure it out.
We are talking about basic computing for CRUD apps. When you start needing to rely upon "sorta-ish" to describe the efficacy or a tool for such a straightforward and deterministic use case, it may be an indicator you need to rethink your approach.
If you want to discount a tool that may save you an immense amount of time because you might have to help it along the fast few feet, thats up to you.
If you can share a tool that can analyze a function and create a test for all corner cases in a popular language, I'm sure some people would be interested in that.
You should look up intellitest and reshaper test generator. Products exist for this.
Maybe you should have brought that up earlier instead of acting smug and burying the lede? It's also pretty telling that you didn't elaborate this further and kept your comment short.
What you should have said is that some parameterized test generators do automated white box testing where they look at your code similar to a fuzzer and try to find the test cases automatically. Your first link is literally just setting up an array with test cases, which basically means you'd have to use an LLM to quickly produce the test cases anyway, which makes parameterized testing sound exceedingly pathetic.
https://learn.microsoft.com/en-us/visualstudio/test/generate...
>IntelliTest explores your .NET code to generate test data and a suite of unit tests. For every statement in the code, a test input is generated that will execute that statement. A case analysis is performed for every conditional branch in the code. For example, if statements, assertions, and all operations that can throw exceptions are analyzed. This analysis is used to generate test data for a parameterized unit test for each of your methods, creating unit tests with high code coverage. Think of it as smart fuzz testing that trims down the inputs and test cases to what executes all your logic branches and checks for exceptions.
> Maybe you should have brought that up earlier instead of acting smug and burying the lede? It's also pretty telling that you didn't elaborate this further and kept your comment short.
I thought people were generally competent within the areas they discuss and are aware of the tooling within their preferred ecosystem. I apologize if that is not the case.
Reshaper huh...
"With AI Assistant, you can generate unit tests for C# methods."
https://www.jetbrains.com/help/resharper/Generate_tests.html
99% of the work in testing is coming up with test scenarios and test cases. 95% of the code is just dealing with setting up input and output data, 4% is calling the code you want to test and the final assert is often just a single line of code.
I'm not sure what depth of experience has to do with any of this, since it is busy work that costs a lot of time. A form with 120 fields is a form with 120 fields. There is no way around coming up with the several dozens of test cases that you're going to test without filling out almost all of the fields, even the ones that are not relevant to the test itself, otherwise you're not really testing your application.
We're not a licensed profession with universally defined roles. It's whatever the speaker wants it to be given how wildly it varies.
> test fixtures
I'm curious- how does the AI know what you want?
I use CodeCompanion on neovim. My workflow for text fixtures is basically:
""" Given these files: file1, file2, ... (these are pulled entirely into the LLM context)
Create a test fixture by creating a type that implements the trait A and should use an in memory SQLite DB, another one that implements Trait B, ...
"""
Of course there is a bit of back and forth, but I find that using Interfaces/Traits/ABCs extensively makes LLMs perform better at these tasks (but I believe it is a nice side-effect of having more testable code to begin with).
However, wiring with IoC frameworks are a bit of a hit and miss to be honest so often I still have to do these parts manually.
“AI” is great for coding in the small, it’s like having a powerful semantic code editor, or pairing with a junior developer who can lookup some info online quickly. The hardest part of the job was never typing or figuring out some API bullshit anyway.
But trying to use it like “please write this entire feature for me” (what vibe coding is supposed to mean) is the wrong way to handle the tool IMO. It turns into a specification problem.
Yes, but in my experience actually no. At least not with the bleeding edge models today. I've been able to get LLMs to write whole features to the point that I'm quite surprised at the result. Perhaps I'm talking to it right (the new "holding it right"?). I tend to begin asking for an empty application with the characteristics I want (CLI, has subcommands, ...) then I ask it to add a simple feature. Get that working then ask it to enhance functionality progressively, testing as we go. Then when functionality is working I ask for a refactor (often it puts 1500 loc in one file, for example), doc, improve help text, and so on. Basically the same way you'd manage a human.
I've also been close to astonished at the capability LLMs have to draw conclusions from very large complex codebases. For example I wanted to understand the details of a distributed replication mechanism in a project that is enormous. Pre-LLM I'd spent a couple of days crawling through the code using grep and perhaps IDE tools, making notes on paper. I'd probably have to run the code or instrument it with logging then look at the results in a test deployment. But I've found I can ask the LLM to take a look at the p2p code and tell me how it works. Then ask it how the peer set is managed. I can ask it if all reachable peers are known at all nodes. It's almost better than me at this, and it's what I've done for a living for 30 years. Certainly it's very good for very low cost and effort. While it's chugging I can think about higher order things.
I say all this as a massive AI skeptic dating back to the 1980s.
> I tend to begin asking for an empty application with the characteristics I want (CLI, has subcommands, ...) then I ask it to add a simple feature.
That makes sense, as you're breaking the task into smaller achievable tasks. But it takes an already experienced developer to think like this.
Instead, a lot of people in the hype train are pretending an AI can work an idea to production from a "CEO level" of detail – that probably ain't happening.
> you're breaking the task into smaller achievable tasks.
this is the part that I would describe as engineering in the first place. This is the part that separates a script kiddie or someone who "knows" one language and can be somewhat dangerous with it, from someone who commands a $200k/year salary, and it is the important part
and so far there is no indication that language models can do this part at. all.
for someone who CAN do the part of breaking down a problem into smaller abstractions, though, some of these models can save you a little time, sometimes, in cases where it's less effort to type an explanation to the problem than it is to type the code directly..
which is to say.. sometimes.
All the hype is about asking an LLM to start with an empty project with loose requirements. Asking it to work on a million lines of legacy code (inadequately tested, as all legacy code) with ancient and complex contracts is a completely different experience.
Very large projects are an area where AI tools can really empower developers without replacing them.
It is very useful to be able to ask basic questions about the code that I am working on, without having to read through dozens of other source files. It frees up a lot of time to actually get stuff done.
AI makes so many mistakes, I cannot trust it with telling me the truth about how a large codebase works.
I find this half state kind of useless. If I have to know and understand the code being generated, it's easier to just write it myself. The AI tools can just spit out function names and tools I don't know off the top of my head, and the only way to check they are correct is to go look up the documentation, and at that point I've just done the hard work I wanted to avoid.
Feels like a similar situation to self driving where companies want to insist that you should be fully aware and ready to take over in an instant when things go wrong. That's just not how your brain works. You either want to fully disengage, or be actively doing the work.
> The AI tools can just spit out function names and tools I don't know off the top of my head, and the only way to check they are correct is to go look up the documentation, and at that point I've just done the hard work I wanted to avoid.
This is exactly my experience, but I guess generating code with depreciated methods is useful for some people.
It turns into a specification problem.
This, IMHO, is the critical point and why a lot of “deep” development work doesn’t benefit much from the current generation of AI tools.
Last week, I was dealing with some temporal data. I often find working in this area a little frustrating because you spend so much time dealing with the inherent traps and edge cases, so using an AI code generator is superficially attractive. However, the vast majority of my time wasn’t spent writing code, it was getting my head around what the various representations of certain time-based events in this system actually mean and what should happen when they interact. I probably wrote about 100 test cases next, each covering a distinct real world scenario, and working out how to parameterise them so the coverage was exhaustive for certain tricky interactions also required a bit of thought. Finally, I wrote the implementation of this algorithm that had a lot of essential complexity, which means code with lots of conditionals that needs to be crystal clear about why things are being done in a certain order and decisions made a certain way, so anyone reading it later has a fighting chance of understanding it. Which of those three stages would current AI tools really have helped with?
I find AI code generators can be quite helpful for low-level boilerplate stuff, where the required behaviour is obvious and the details tend to be a specific database schema or remote API spec. No doubt some applications consist almost entirely of this kind of code, and I can easily believe that people working on those find AI coding tools much more effective than I typically do. But as 'manoDev says in the parent comment, deeper work is often a specification problem. The valuable part is often figuring out the what and the why rather than the how, and so far that isn’t something AI has been very good at.
I looked at our anthropic bill this week. Saw that one of our best engineers was spending $300/day on Claude. Leadership was psyched about it.
I was told that I wasnt using it enough by one arm of the company and that I was spending too much by another.
Meanwhile, try as I might I couldnt prevent it from being useless.
I know of no better metaphor than that of what it's like being a developer in 2025.
Is he using it to code or just chatting up his AI girlfriend.
Joking aside, if he is one of the top developers in the company and if he is "actually" a good developer, when compared to others outside of the company, then I can see this bill.
The current feature that I'm working on, required 100 messages to finalize things and I would say the context window was around 35k - 50k per "chat completion". My model of choice is Gemini 2.5 Flash which has an input cost of $0.30/1M. Compare this to Sonnet which is $3.00/1M.
If the person was properly designing and instructing the LLM to build something advanced correctly, I can see the bill being quite high. I personally don't think you need to use Sonnet 99% of the time, but if somebody else is willing to pay the bill, why not.
Meanwhile an entire lake has evaporated in Northern Canada to deliver me a slightly different version of yet another payment options page.
Awesome. That must be on the same planet where 300% of the worlds power goes to processing a single bitcoin transaction.
Claude is making $72k a year for a consistent $300/day spend.
Bear in mind those are revenue figures, they're costing claude hundreds a day.
One imagines Leadership won't be so pleased after the inevitably price hike (which, given the margins software uses, is going to be in the 1-3 thousands a day) and the hype wears off enough for them to realize they're spending a full salary automating a partial FTE.
But, by the looks of things, models will be more efficient by then and a cheaper-to-run model will produce comparable output. At least that's how it's been with OSS models, or with the Openai api model. So maybe the inevitable price hike (or rate limiting) may lead to switching models / providers and the results being just as good.
There is an interesting substack post about this. LLM costs are dropping 10x/year but the amount of tokens used have gone up like crazy https://open.substack.com/pub/ethanding/p/ai-subscriptions-g...
> But, by the looks of things, models will be more efficient by then and a cheaper-to-run model will produce comparable output
So far there's negative evidence of this. Things are getting more expensive for similar outputs.
And this varies widely between models based on how they are told to reason.
> Senior developers were also more likely to say they invest time fixing AI-generated code.
That quote is the key - even if 1/3 of senior devs are pushing mostly AI-driven code, they are checking it first. And while the survey did not cover it, I suspect they are using their experience to decide which areas of a codebase are commonplace enough that AI can handle it vs. which areas are unique and require coding without AI.
People are learning when to use AI as a helpful tool and when not to.
Most of code that "needs to be written" is just a copy of something standard, "Do X in the simplest way possible" code that doesn't need optimizations, and writing it by hand is just waste of time. AI is good enough to write megabytes of that code, since its statistically common and part of tons of codebases. Its the other half of code that AI can't handle, that you need to manually verify it doesn't hallucinate fantastic stuff that manages to compile, but doesn't work.
This is self reported unless I missed something. I bet that skews these results quite a bit. Many are very hesitant to say they use AI, and I suspect that's much more likely to be the case when you are new to the field.
Also, green coding? That's new to me. I guess we'll see optional carbon offset purchasing in our subs soon.
Fastly is in the AI business like Cloudflare:
https://www.fastly.com/products/ai
https://www.fastly.com/products/fastly-ai-bot-management
https://www.fastly.com/documentation/guides/compute/about-th...
Disclaimer: not only am I a Fastly employee, I'm the Fastly employee quoted in the register article originally linked.
We'd argue our stance and focus are a bit different. Our stance is -- we help eng teams and the open web with whatever their preference for AI might be.
If you want to go all in and monetize AI traffic or sign an agreement with a foundation model for training, we have tools that can help you do that. If you want to go scorched-earth on AI bots that try to scrape your systems, we can help you do that too.
It's all about helping engineers tasked with tough problems get their jobs done, regardless of whether the flavor of the week is AI or something else.
LLMs are tools. If you master your tools, you become more productive. I've had the same mixed experiences with LLMs that others have. Some good, some not so great. But I've been merging quite a bit of codex created PRs in the last weeks. Some needed manual intervention. Some were tedious to create. And some clearly saved me a lot of work.
I always have more work than I can handle. Part of my job is deciding what not to do and what to drop because it hasn't got the right priority. Me spending an afternoon on a thing that is fun but not valuable is usually a bad use of my time. With LLMs, I'm taking on a few more of the things that I previously wouldn't have. That started with a few hobby projects that I'm now doing that I previously wasn't. And it's creeping into work as well. LLMs struggle on larger code bases. But less so with recent model releases.
> LLMs are tools. If you master your tools, you become more productive.
LLMs are unpredictable tools which change all the time, let’s not pretend otherwise. You can’t “master” them in the same way as previous tools. You can learn some tricks to trick them to be closer to what you want, and that’s about it.
Imagine if every time you did the exact same movement to hammer a nail, you had to check your work. Maybe this time it hammered it in perfectly, or maybe it smashed your finger, or maybe it only went half-way through. You could never develop muscle memory for such a tool. You could use it, sure, but never master it.
I didn't say they were perfect tools; quite the opposite actually.
> You can’t “master” them in the same way as previous tools.
It seems I'm getting better and better results over time. I think you want this to be true but that doesn't make it true.
a crack pipe is a tool. if you master your tools you become more productive. i've had the same mixed experiences with crack cocaine that others have. some good, some not so great. But i've been writing a lot of crack-fueled PRs in the last weeks. some needed manual intervention. some were tedious to create. and some clearly saved me a lot of work.
Article did not say what kind languages/applications thosd 791 developers were working on. I work on a legacy Java code base (which looks more like C than Java, thankfully) and I cant imagine AI doing any of it. It can do small isolated, well formulated chunks (functions that do a very specific task) but even that will require very verbose explanation.
I just can't fathom shipping a big percentage of work using LLMs.
I guess I’m an older developer.
But I’ve come full circle and have gone back to hand coding after a couple years of fighting LLMs. I’m tired of coaxing their style and fixing their bugs - some of which are just really dumb and some are devious.
Artisanal hand craft for me!
I've also just turned off copilot now. I had several cases where bugs in the generated code slipped through and ended up deployed. Bugs I never would have written myself. Reviewing code properly is so much harder than writing it from scratch.
"write it right the first time" is my new mantra/challenge to myself.
along with improving my skills in vim, this approach has made me significantly more productive and has made my code much simpler compared to when i was using LLM code generation tools.
there is no shortcut around hard work.
there is no shortcut to thoroughly interrogating the constraints of the software problem at hand.
developers who rely on LLM code generation are poor coworkers because they don't understand what they've written and why they've written it.
I'm in the same exact boat. I started with a lot of different tools but eventually went back to hand coding everything. When using tools like co-pilot I noticed I would ship a lot mode dumb mistakes. I even experimented with not even using a chat interface and it turns out that a lot of answers to problems are indeed found with a web search.
By all means, if my goal is actually crafting anything.
Usually it isn't, though - I just want to pump out code changes ASAP (but not sooner).
Even then I’ve mostly given up. I’ve seen LLMs change from snake case to camel case for a single method and leave the rest untouched. I’ve seen them completely fabricate APIs to non existent libraries. I’ve seen them get mathematical formulae completely wrong. I’ve seen it make entire methods for things that are builtins of a library I’m already using.
It’s just not worth it anymore for anything that is part of an actual product.
Occasionally I will still churn out little scripts or methods from scratch that are low risk - but anything that gets to prod is pretty much hand coded again.
This changed my experience significantly:
https://github.com/BeehiveInnovations/zen-mcp-server/blob/ma...
It basically uses multiple different LLMs from different providers to debate a change or code review. Opus 4.1, Gemini 2.5 Pro, and GPT-5 all have a go at it before it writes out plans or makes changes.
it's a lot faster and a lot easier and it gives you a much better understanding of the problem if you just think about the changes you want to make and write them yourself.
why are you looking for a shortcut? just do the work.
I feel like it’s my job to understand new tools and keep abreast of the way things are changing in my industry.
It is faster, right now, to just write it myself; at least for most things. I definitely want to understand when and how that might change.
Judging from the downvotes, not everybody shares that curiosity.
I'm in this boat. On top of this though, I genuinely think less of developers who rely on LLMs. I very seriously believe that they are beneath me.
The article is saying older devs vibe code: I think you misunderstood
(Article was https://www.theregister.com/2025/08/28/older_developers_ai_c... when this was posted; we've since changed it)
I didn’t misunderstand. I tried to vibe code, and now I don’t. Not sure how you misinterpreted that.
Key word: "But"
I’m not surprised. AI tools have become such a huge time saver. I use them a lot too, but I still double check everything to keep quality high. I read an article on finance gossips called “The Rise of AI in Banking: Friend or Foe”, and it really put this into perspective. AI’s growth in finance shows how quickly it is reshaping industries, so seeing devs rely on AI for half their code makes total sense. Like in banking, it’s not about replacing humans but creating smarter systems, as long as we manage trust, ethics, and transparency, it’s a tool to level us up, not phase us out.
> as long as we manage trust, ethics, and transparency
Trust, ethics and transparency are at all time lows in every corner of every capitalist industry imo
I don't know how anyone could look at the current state of things and think AI isn't being positioned to gut employment across many industries
Naive, childish and short sighted mentality imo
Brute forcing a problem by writing more lines with an LLM instead of designing better code is a step the wrong direction
better code == less code? In my mind that would be highly succinct purely functional code in scala or haskell. Those really allow you to "compress" code down to its barest essential domain logic? Perhaps the future is writing highly domain specific DSLs and have the LLM generate the runtime for it.
I don't know about half, but I do use AI a lot these days to speed run the boilerplate to get to the good parts.
I think it's important to have a least the level of experience to be able to quickly verify that the generated code will work as expected.
Naive question but wouldn't it could as having AI write 50%+ of your code if you just use an unintelligent complete-the-line AI tool? In this case the AI is hardly doing anything intelligent, but is still getting credit for doing most of the work.
Yes there is even a small business that champions an small LLM that is trained on the language Lsp, your code base, and your recent coding history (not necessarily commit, but any time one presses ctrl + s). How it works is essentially autocomplete. This functionality is packaged as an IDE plugin: TabNine
However now they try to sell subscriptions to LLMs.
Tabnine has been in the scene since at least 2018.
At a minimum, 30-50% of bogus surveys are bogus but I'm willing to bet it's a lot more.
That doesn't worry me much. Whats way more concerning is junior/middle developers have likely way more than 1/2 of their code "AI" generated.
At least seniour developer allegedly know how to error-proof their code to some extend.
I've been at this for many years. If I want to implement a new feature that ties together various systems and delivers an expected output, I know the general steps that I need to take. About 80% of those steps are creating and stubbing out new files with the general methods and objects I know will be needed, and all the test cases. So... I could either spend the next 4 hours doing that, or spend 3 minutes filling out a CLAUDE.md with the specs and 5 minutes having Claude do it (and fairly well).
I feel no shame in doing the later. I've also learned enough about LLMs that I know how to write that CLAUDE.md so it sticks to best practices. YMMV.
> I've also learned enough about LLMs that I know how to write that CLAUDE.md so it sticks to best practices.
Could you share some examples / tips about this?
I tried it - didn't like it. Had an LLM work on a backup script since I don't use Bash very often. Took a bunch of learning the quirks of bash to get the code working properly.
While I'll say it got me started, it wasn't a snap of the fingers and a quick debug to get something done. Took me quite a while to figure out why something worked but really it didn't (LLM using command line commands where Bash doesn't interpret the results the same).
If its something I know, probably wont use LLM (as it doesn't do my style). If it's something I don't know, might use it to get me started but I expect that's all I'll it for.
Can I ask which agent/model you used? I'm similarly irritated with shell script coding, but find I have to make scripts fairly often. My experience using various models but latterly Claude Code has been quite different -- it churned out pretty much what I was looking for. Also old, fwiw. I'm older than all shells.
It was Gemini, I picked the point of least resistance. I've heard Claude does better but haven't looked at it.
> survey of 791 developers
We have got to stop. In a universe of well over 25 million programmers a sample of 791 is not significant enough to justify such headlines.
We’ve got to do better than this, whatever this is.
Validity of the sample size is not determined by its fraction of the whole population. I don't know the formulas and I'm not a statistician. Maybe someone can drop some citations.
I generally agree with this just from a perspective of personal sentiment, it does feel wrong.
But statistically speaking, at a 95% confidence level you'd be within a +/- 3.5% margin of error given the 791 sample size, irrespective of whether the population is 30k or 30M.
You should read more about statistical significance. Under some reasonable assumptions, you can confidently certain deduce things with small sample sizes.
From another perspective: we've deduced a lot of things about how atoms work without any given experiment inspecting more than an insignificant fraction of all atoms.
TL;DR: The population size (25e6 total devs, 1e80 atoms in observable universe) is almost entirely irrelevant to hypothesis testing.
As a consultant, I’m seeing this trend firsthand. A few thoughts:
Productivity boost: AI-generated code accelerates prototyping and routine tasks, letting developers focus on higher-value architecture and problem-solving.
Quality depends on oversight: The best results come when experienced engineers review, refine, and adapt AI output—it’s not “set and forget.”
Business impact: Faster development cycles mean reduced time-to-market, which is especially critical for startups and innovation-driven companies.
Skill evolution: Developers are shifting from “just coding” to becoming solution architects who guide AI tools effectively.
In short, AI is becoming a powerful co-pilot, but human expertise remains the differentiator.
As a developer with over 30 years of experience, I face a dilemma when asked "how much AI code do you commit"?
a) Answer "a lot". This answer supports the notion that developers are a dying breed and soon everyone will be able to vibe code their own personal software. Which at this point is obviously false. This answer is detrimental to my job.
b) Answer "not much". Then this gets interpreted as "he's an old fart, he can't learn new things, we should be thinking of retiring him". Which is (hopefully) false. Which is - again - detrimental to my job.
This agrees with my experience on a project I’ve been working on this year, in particular related to porting the code. I’ve developed a strategy that I’m calling “Polyglot Mirroring” where the code is written in multiple languages at once, with LLMs handling the mirroring.
I actually made a Ask HN about it just today https://news.ycombinator.com/item?id=45091607 but for some reason the HN algorithm never even showed it on the Ask page :/
I know this is late to the party but I think, given that this topic comes up.every day, we should start including statistics in our posts to get a better sense of how AI is working or not for us. I imagine a years of experience / primary language used / tool used / AI positive or negative as a percentile estimate.
Personally for me it's 20 years of experience / Python / Copilot / 85% positive experience.
I think they're being really loose with the term "vibe coding", and what they really mean is AI-assisted coding.
Older devs are not letting the AI do everything for them. Assuming they're like me, the planning is mostly done by a human, while the coding is largely done by the AI, but in small sections with the human giving specific instructions.
Then there's debugging, which I don't really trust the AI to do very well. Too many times I've seen it miss the real problem, then try to rewrite large sections of the code unnecessarily. I do most of the debugging myself, with some assistance from the AI.
> Assuming they're like me, the planning is mostly done by a human, while the coding is largely done by the AI
I've largely settled on the opposite. AI has become very good at planning what to do and explaining it in plain English, but its command of programming languages still leaves a lot to be desired.
It's good at checking plans, and helping with plans, but I've seen it make really really bad choices. I don't think it can replace a human architect.
It can't replace a human anything, yet, but that doesn't seem to be stopping anyone from trying unfortunately:(
Yes, much like many of the humans I have worked with, sometimes bad choices are introduced. But those bad choices are caught during the writing of the code, so that's not really that big of a deal when it does happen. It is still a boon to have it do most of the work.
And remains markably better than when AI makes bad choices while writing code. That is much harder to catch and requires pouring over the code with a fine tooth comb to the point that you may as well have just written it yourself, negating all the potential benefits of using it to generate code in the first place.
They're also being really loose with the term "older developers" by describing it as anybody with more than ten years of experience.
When debugging, I'll coax the AI to determine what went wrong first - to my satisfaction - and have it go from there. Otherwise it's a descent into madness.
I’m still yet to find a use-case for AI-generated code in my workflow.
Even when I am building tools that heavily utilize modern AI, I haven’t found it. Recently, I disabled the AI-powered code completion in my IDE because I found that the cognitive load required to evaluate the suggestions it provided was greater and more time consuming than just writing the code I was already going to write anyways.
I don’t know if this is an experience thing or not - I mainly work on a tech stack I have over a decade of experience in - but I just don’t see it.
Others have suggested generating tests with AI but I find that horrifying. Tests are the one thing you should be the most anal about accuracy on compared to anything else in your codebase.
Big assumption here, is that developers don’t spend time fixing their own work or other developers work.
I feel the benchmark is some engineer who shoots out perfect work the first time they hit the keyboard.
as a human being i take it as massive disrespect when another human developer makes me wipe up their AI diarrhea.
What are you replying to
Their inner demons?
[dead]
One thing I don't hear a lot of people talk about is building prototypes. That's where I see a gigantic time savings. It doesn't have to be beautiful code, it just has to help me answer a question so I can make a decision about where to go next. That and tools. There have been many times where I've wanted to build a task-specific tool but justifying the time would be hard. Now I can create little tools like that, and it's a huge productivity boost.
I often use LLMs for method level implementation work. Anything beyond the scope of a single function call I have very little confidence in. This is OK though, since everything is a function and I can perfectly control the blast radius as long as I keep my hands on the steering wheel. I don't ever let the LLM define method signatures for me.
If I don't know how to structure functions around a problem, I will also use the LLM, but I am asking it to write zero code in this case. I am just having a conversation about what would be good paths to consider.
The nice thing about having AI generated code is that you get more time to focus on designing the system and it's edge cases. My code coverage (the significant metric, not the vain metric) and documentation coverage is really high because it's a very strong productivity boost when coupled with LLM generated code.
Been in the industry professionally since 1996 writing code and before that 10 years as a hobbyist between a little BASIC, C and a lot of assembly (65C02, 68k, PPC and x86).
In the last 6 months, when I have had an assignment that involved coding, AI has generated 100% of my code. I just described the abstractions I wanted and reusable modules/classes I needed and built on it.
It's 80% at least for me. I've hit a groove for sure. It's not only 80% is very well tested, and almost exactly how I would have preferred. Big tips are to design your CLAUDE.md files how you would actually code from a high level perspective, and not try the usual "You are an expert Google Distributed Engineer" and all that embarassing AI hype bro crap.
I would certainly hope junior developers don't rely too much on AI; they need the opportunity to learn to do this stuff themselves.
I think at this point it's whoever can get the most useful work out of AI which is actually really hard due to their 'incomplete' state. Finding uses which require very little user input is going to be the next big thing in my opinion since it seems that LLMs are currently at a wall where they require technical advancements before they can overcome it.
> This also: AI slowing down completion time by 19% - https://arxiv.org/abs/2507.09089
Not sure which one is true
Still negative interest in being in the users of AI-generated group. Make it work, make it not steal other people's work and we'll talk then. :)
Is "vibe coding" synonymous with using AI code-generation tools now?
I thought vibe coding meant very little direct interaction with the code, mostly telling the LLM what you want and iterating using the LLM. Which is fun and worth trying, but probably not a valid professional tool.
I think what happened is that a lot of people started dismissing all LLM code creation as "vibe coding" because those people were anti-LLM, and so the term itself became an easy umbrella pejorative.
And then, more people saw these critics using "vibe coding" to refer to all LLM code creation, and naturally understood it to mean exactly that. Which means the recent articles we've seen about how good vibe coding starts with a requirements file, then tests that fail, then tests that pass, etc.
Like so many terms that started out being used pejoratively, vibe coding got reclaimed. And it just sounds cool.
Also because we don't really have any other good memorable term for describing code built entirely with LLM's from the ground up, separate from mere autocomplete AI or using LLM's to work on established codebases.
“Agentic coding” is probably more accurate, though many people (fairly) find the term “Agentic” to be buzz-wordy and obnoxious.
I’m willing to vibe code a spike project. That is to say, I want to see how well some new tool or library works, so I’ll tell the LLM to build a proof of concept, and then I’ll study that and see how I feel about it. Then I throw it away and build the real version with more care and attention.
I have "vibe coded" a few internal tools now that are very low risk in terms of negative business impact but nonetheless valuable for our team's efficiency.
E.g one tool packages a debug build of an iOS simulator app with various metadata and uploads it to a specified location.
Another tool spits out my team's github velocity metrics.
These were relatively small scripting apps, that yes, I code reviewed and checked for security issues.
I don't see why this wouldn't be a valid professional tool? It's working well, saves me time, is fun, and safe (assuming proper code review, and LLM tool usage).
With these little scripts it creates it's actually pretty quick to validate their safety and efficacy. They're like validating NP problems.
The original definition of vibe coding meant that you just let the agent write everything, and if it works then you commit it. Your code review and security check turned this from vibe coding into something else.
This is complicated by the fact that some people use “vibe coding” to mean any kind of LLM-assisted coding.
My personal definition of "vibe coding" is when a developer delegates -- abdicates, really -- responsibility for understanding & testing what AI-generated code is doing and/or how that result is achieved. I consider it something that's separate from & inferior to using AI as a development tool.
Yeah, for some reason the term has been used interchangeably for a while, which is making it very hard to have a conversation about it since many people think vibe coding is just using AI to assist you.
From Karpathy's original post I understood it to be what you're describing. It is getting confusing.
The term sounds funny and quirky, so got overused. Also simply the term pushes emotional buttons on a lot of people so it's good for clickbait.
I think there is actually a pressure to show thst you are using AI (stories of ceos firing employees who supposedly did not "embrace" ai). So people are over attributing to AI. Though originally VC was meant to be infinite monkey style button smashing, people are attributing to VC just to avoid the cross hairs.
What does it mean, "AI-generated"? Does "fancy auto-complete" by an LLM count?
What does it mean, "half their code"? Lines of code? Number of methods/functions? Classes? Number of whole refactors?
Sounds about right in my experience. Not every piece of code has to be elite John Carmack tier quality
90% of my code is auto generated using custom code generation tools. No AI needed.
Really?
I'm not a coder but a sysadmin. 35 years or so. I'm conversant with Perl, Python, (nods to C), BASIC, shell, Powershell, AutoIT (int al)
I muck about with CAD - OpenSCAD, FreeCAD, and 3D printing.
I'm not a senior developer - I pay them.
LLMs are handy in the same way I still have my slide rules and calculators (OK kids I use a calc app) but I do still have my slide rules.
ChatGPT does quite well with the basics for a simple OpenSCAD effort but invents functions within libraries. That is to be expected - its a next token decider function and not a real AI.
I find it handy for basics, very basic.
I just got back into OpenSCAD after recently getting my first new 3D Printer in 10 years, so I basically had to relearn it. ChatGPT got the syntax wrong for the most basic of operations.
Chat did work for some operations for me but not many.
My screwdrivers, drills, planes, and all the rest don't tell me how to use them and I think that you need to treat LLMs in the same way.
They can provide an insight or a prod or whatever that gets you over the line but I do not think that they are ... AI of any sort.
I can cut say 12mm of plywood with a panel saw within an accuracy of about 1mm if I am careless. If I take more care, I can do way better. Depends on the job. However, I have to keep my eye in.
Chat will get the basics right but will hallucinate badly when it comes to libraries like https://github.com/BelfrySCAD/BOSL2/wiki
Just like a slightly blunt chisel, you have to treat Chat and co in the same way - they are tools. No more and no less. You need to know how to sharpen them up (oil and a whetstone).
I can be massively more ambitious when coding with AI, but most importantly I have zero emotional investment in the code so I can throw it away and start again whenever I want.
Claude writes 99% of my code, I’m just a manager and architect and QC now.
What sort of coding do you do? I can't imagine getting that level of code from AI
Jesus, that's a staggering figure to me coming from senior developers. I guess I'm the odd one out here, but ChatGPT is nothing more than an index of Stack Overflow (and friends) for me. It's essentially replaced Googling, but once I get the answer I need I'm still just slinging code like an asshole. Copying the output wholesale from any of these LLMs just seems crazy to me.
If you’re using ChatGPT directly for work then I believe that you are doing it so profoundly wrong, at this point, that you’re going to make really incorrect assumptions.
As we have all observed, the models get things wrong, and if you’re wrong 5% of the time, then ten edits in you’re at 60-40. So you need to run them in a loop where they’re constantly sanity checking themselves—-linting, styling, typing and testing. In other words; calling tools in a loop. Agents are so much better than any other approach it’s comical precisely because they’re scaffolding to let models self-correct.
This is likely somewhat domain-specific; I can’t imagine the models are that great at domains they haven’t seen much code in, so they probably suck at HFT infrastructure for example, though they are decent at reading docs by this point. There’s also a lot of skill in setting up the right documentation, testing structure, interfaces, etc etc etc to make the agents more reliable and productive (fringe benefit; your LLM-wielding colleagues actually write docs now, even if they’re full of em-dashes and emoji). You also need to be willing to let it write a bunch of code, look at it, work out why it’s structurally deficient, throw it away, and build the structure you want to guide it - but the typing is essentially free, so that’s tractable. Don’t view it as bad code, view it as a useful null result.
But if you’re not using Claude Code or Codex or Roo or relatives, you’re living in an entirely different world to the people who have gone messianic about these things.
Yes, I was using "ChatGPT" colloquially. I’ve tried Claude Code, Copilot - I currently have Gemini running though madox2's vim-ai plug - and a few of the others, but I’ve never personally seen the kind of 10x productivity gains some people here talk about. In my experience, it’s typically been quicker to just write the code myself than to wrangle with the model’s output, iterate, and debug. That’s not to say the tools are useless - like I said I think they're great at exposing docs that are disparate or obtuse (looking at you Google) but for actual problem-solving and building, they’ve unfortunately rarely felt like a net win.
I'm sure you're correct that a lot of this comes down to workflow and domain - in mine, the overhead of prompting, reviewing, and correcting usually outweighs the benefits. Of course, as always, this is just one asshole's opinion!
Where is all this code? If the tools are so useful it should show up in widely used open source projects, but I'm not seeing anything.
How would you know?
Various projects have proposed a "Co-Developed-By" tag. Even if that were not used I'd expect some open source developers to have become much more productive, and written about it. We'd also see widespread adoption of ancillary tools like AI code reviewers (which have had some adoption but in my experience make suggestions that are worse than useless).
Of course. We are the most well equipped to run with it. Others will quickly create a sloppy mess while wise developers can keep the beast tame.
Im not sure I believe this. It's the exact opposite in my experience - the young'uns are all over vibe coding.
I use the inline commands to convert pseudo code to normal code. It works great.
LMFTFY: a third of senior developers who answer surveys say over half of their code is AI-generated
Haha right. I would imagine a "Senior Developer" who is super into AI assisted coding would be more likely to come across this survey and want to participate.
I’m glad to see people still employed… I remember what that was like.
Developers are lazy. Anything that makes development faster or easier is going to be welcomed by a good developer.
If you find it is quicker not to use it then you might hate it, but I think it is probably better in some cases and worse in other cases.
as a developer my first priority is whether the software works, not whether it is fast or easy to develop
I think we can assume that what daft_pink means by "development" includes that the software works.
("Please respond to the strongest plausible interpretation of what someone says, not a weaker one that's easier to criticize." - https://news.ycombinator.com/newsguidelines.html)
> Anything that makes development faster or easier is going to be welcomed by a good developer.
I strongly disagree. Struggling with a problem creates expertise. Struggle is slow, and it's hard. Good developers welcome it.
Indeed. This is my biggest fear for engineers as a whole. LLMs can be a great productivity boost in the very short term, but can so easily be abused. If you build a product with it, suddenly everyone is an engineering manager and no one is an expert on it. And growth as an engineer is stunted. It reminds me of abusing energy drinks or grinding to the point of burnout... But worse.
I think we'll find a middle ground though. I just think it hasn't happened yet. I'm cautiously optimistic.
> Struggling with a problem creates expertise. Struggle is slow, and it's hard. Good developers welcome it.
There is significant evidence that shows mixed results for struggle-based learning - it’s highly individualized and has to be calibrated carefully: https://consensus.app/search/challenge-based-learning-outcom...
I don't particularly care what this, or any, LLM spits out, and given the pervasive problems that have bedeviled social-science research, I also don't care what the results show, let alone what they show on average.
Anybody who has developed software should understand the value of struggling with a difficult problem. I'm obviously not talking about classroom exercises where the problem sets are expected to match a given skill level or cultivate a specific skill set, so the very idea of individualized, calibrated learning is irrelevant.
As a teacher I'm also 100% uninterested in highly individualized, calibrated challenges for what I teach -- or for what I do professionally. The people who need those highly individualized, wildly different, more gently graduated increases in difficulty, for general problem solving or for the study of any area of programming or computer science, simply should not become engineers.
Nah, don't agree that struggle adds much.
I'm in the 10<x<20 range of experience.
People will learn if they find the lesson valuable and interesting, no matter how much struggle and time wasting was involved.
So many words to say nothing. Maybe it wan generated by an AI tool?
Apparently vibe coding now just means ai assisted coding beyond immediate code completion?
For me, success with LLM-assisted coding comes when I have a clear idea of what I want to accomplish and can express it clearly in a prompt. The relevant key business and technical concerns come into play, including complexities like balancing somewhat conflicting shorter and longer term concerns.
Juniors are probably all going to have to be learning this kind of stuff at an accelerated rate now (we don't need em cranking out REST endpoints or whatever anymore), but at this point this takes a senior perspective and senior skills.
Anyone can get an LLM and agentic tool to crank out code now. But you really need to have them crank out code to do something useful.
Not mine.
I mean maybe if you include tests, which for some projects are almost a repeating pattern.
Breaking news: 1/3 of devs use autocomplete. Breaking news: autocomplete now autocompletes more code.
This is the usual propaganda from private equity that invest in LLMs to make you feel like you're missing out for not generating AI slop yourself. I am not surprised that this gets so much traction on YC-owned website since they invest only in AI generating slop companies lately.
these companies are destined to fail, just tell the truth about this dog shit technology and watch them flounder.
It would be nice if we could avoid blog posts like this with extraordinary claims but nearly zero methodological information written by a "content marketing manager" from a company that sells AI products.
Related: "MIT Study Finds Artificial Intelligence Use Reprograms the Brain, Leading to Cognitive Decline"
https://publichealthpolicyjournal.com/mit-study-finds-artifi...
Checks out
My programming background - started from machine codes. Did everything - business desktop, high performance game like apps, firmware for microcontrollers, web front ends, high performance enterprise grade backends, distributed etc. etc. Multiple languages. So lots of experience. Now I use AI quite extensively to generate snippets / classes of code with small and well defined functionality. Most of the time it works just fine and due to my extensive experience I could quickly spot wrong approach, inefficiency, potential issues etc. After telling AI to fix whatever if any problems I've found I usually satisfied with the solution. So overall AI has become a very valuable tool for me, saves gobbles of time. How much exactly - do not really know / care.
Survey: a third of surveys say that over half of them are bullshit
I’ve been writing code for more than four decades.
But today, roughly 80% of it is generated and maintained by prompting Roo Code. It’s still coding. Just at a higher level of abstraction.
This is nonsense, if they are getting this AI slop past code review that is a disgrace. When I take solutions from AI I need to rewrite almost every line for style and project alignment reasons. There is something uncanny valley about AI code that just needs to be deleted or changed too, for example the comments it writes are usually terrible.
Maybe I can potentially believe at one point AI code was used to create a starting point for 50% of PRs…
Many young developers’ careers are fucked for at least a decade. Their lifetime earnings will never recover from this.
“I don't know half of you half as well as I should like; and I like less than half of you half as well as you deserve.”
around a third of senior developers with more than a decade of experience are using AI code-generation tools such as Copilot, Claude, and Gemini to produce over half of their finished software, compared to 13 percent for those devs who've only been on the job for up to two years.
A third? I would expect at least a majority based on the headline and tone of the article... Isn't this saying 66% are down on vibe coding?
(Article was https://www.theregister.com/2025/08/28/older_developers_ai_c... when this was posted; we've since changed it. We also changed the title.)
This submission was written by the marketing department of a company with commercial interests on the given topic and with almost no information about the "survey" itself, it's essentially blogspam.
I guess the article was vibe coded.
"This one developer was down with the vibe coding."
[dead]