Ok so Samsung, SK Hynix and Micron do not have the capacity to meet demand. Also, what little capacity they do have they are allocating to HBM over DRAM. Based on my limited knowledge HBM can not be easily repurposed for consumer electronics. Translation: main street is cooked for the next 3-4 years.
It doesn't stop there though. OpenAI is currently mired in a capital crunch. Their last round just about sucked all the dry powder out of the private markets. Folks are now starting to ask difficult questions about their burn rate and revenue. It is increasingly looking like they might not commit to the purchase order they made which kick-started this whole panic over RAM.
Soo ... how sure are we that the memory makers themselves are not going to be the ones holding the bag?
The Radeon VII came out in 2019 as a $700 consumer GPU with an 1TB/s HBM2 memory subsystem which is more than any consumer GPU you can get today, including the high-end ones afaik. At that point in time, there was a whole lineup of AMD GPUs with HBM going down into the midrange.
If they could make this stuff and sell it to regular people a decade ago for very palatable prices, why do they come up with the idea that this is the technology of the gods, unaffordable by mere mortals?
I have been wondering this recently. It was the convention that if you wanted to keep costs down, try to keep the memory bus size down as low as possible. Still remember the awful Radeon 9200 SE - 64bit data bus that strangled an already slow GPU.
Heck, I have a phone with a 16bit memory bus for instance. The high(ish) clock rate only makes up the difference slightly.
But with general prices on all components going up, it might not be such a big factor any more.
HBM migght make sense for higher end products which can free up space for the lower end that will never use the tech.
Yes, it's interesting that HBM was invented by a collaboration between AMD and SK Hynix. It seems, HBM is the way to go for GPUs, anyway.
The GB202 die that's in the GDDR7 based RTX 5090 and RTX 6000 Pro literally needed to be this big to support the 512bit memory bus. It's probably only getting worse with smaller node sizes. (see https://www.youtube.com/watch?v=rCwgAGG2sZQ&t=65s).
BTW: The 1TB/s is matched by RTX4090 and surpassed by the RTX5090 (1,79 TB/s).
5090 is an overpriced outlier. A typical consumer GPU, like RTX 5070, has a 3-times lower memory throughput.
Even a RTX 5080 has a lower memory throughput than a Radeon VII from 2019, 7 years ago, while being much more expensive.
The memory throughput of GPUs per dollar has regressed greatly during the last 5 years, despite the fact that the widths of the GPU memory interfaces have been reduced, in order to decrease the production costs.
RTX 5080 has a 256-bit memory interface, while the much cheaper Radeon VII had an 1024-bit memory interface. RTX 5080 has almost 4-times faster memories than Radeon VII, but it has not used this to increase the memory throughput, but only to reduce the production costs, while simultaneously increasing the product price.
Modern GPUs like RTX 5080 are much faster for the applications that are limited by computational capabilities, mainly because they have more execution units, whose clock frequencies have also increased.
I suppose that most games are limited by computation, so they are indeed much faster on modern GPUs.
However, there are applications that are limited by memory throughput, not by computation, including AI inference and many scientific/technical computing applications.
For such applications, old GPUs with higher memory throughput are still faster.
This is why I am still using an old Radeon VII and a couple of other ancient AMD GPUs with high memory throughput.
Last year I have bought an Intel GPU, which is still slower than my old GPUs, but it at least had very good performance per dollar, competitive with that of the old GPUs, because it was very cheap, while the current AMD and especially NVIDIA GPUs have poor performance per dollar.
5090s are certainly expensive compared to most other GPUs, but not expensive enough to be unobtanium for nearly any professional who could utilize one as part of their job
Vega was a card with decent perf/$ for the consumer, but from a pure technical point of view (perf/mm2, perf/BW, perf/W) it was a major failure. Both Vega (and Fiji before it) showed that excess memory BW alone is not sufficient to win.
The fury X was a beast for a consumer card, sadly limited by having a mere 4gb of (hbm) VRAM, and a non-refillable AIO. But back when games could fit in 4gb, it was incredible.
I'd absolutely buy another hbm consumer GPU if it had at least 8gb (and if I got the vibe/hope AMD will actually support for a couple years...)
FWIW that depends on the stores you're looking at. There are three models from different manufacturers available here in a few shops. The prices are a bit ouchier than what i paid for mine around Christmas 2024 though (i got mine on a sale).
After NVIDIA essentially removed FP64 from consumer GPUs (their 1:64 performance ratio is worse than what you can obtain by software emulation, so it is useless, except for testing programs intended to run on datacenter GPUs), AMD persisted for a few years, but then they also followed NVIDIA.
AMD Hawaii GPUs still had 1:2 FP64:FP32, while the consumer variant of Radeon VII dropped to 1:4. The following AMD consumer GPUs dropped the FP64 performance to levels that are not competitive with CPUs.
Nowadays the only consumer GPUs with decent FP64 performance are the Intel Battlemage GPUs, which have a 1:8 performance ratio, which provides very good performance per dollar.
Only indirectly. They have most of the money, so if they want something that’s in short supply, the price will rise to the point that it becomes unaffordable to everyone else.
Reason number 7,322 why US-style ultracapitalism is self-destructive, anti-social, and dystopian.
To add a more local hurdle as well, the Dutch power grid is at capacity and its managing company is now telling companies that planned to build a datacenter that they can't be connected to the grid until 2030, even though said companies already paid for and got guarantees about that connection.
That is, memory capacity is reserved for datacenters yet to be built, but this will do weird things if said datacenter construction is postponed or cancelled altogether.
That guarantee is not as much of a guarantee as stated in the media. You get a guarantee it will be planned at a certain time (as in looked at), not that it will be build. The cost of doing business is taking risks and mitigating them. There is a reason the nuclear plant in Borsele was build: an aluminium smelter. Maybe you should arrange for something similar as a datacenter (no politician will fall on a sword for that but you can try). The (original) power draw is about the same 80-100MW.
It says that in 2025, Netherlands was a net exporter of electricity (~14,000 GWh). My guess: Where they want to build data centers, the grid cannot handle it, but the overall system has more than enough power to build data centers. Do you think that sounds like a resonable guess?
> the Dutch power grid is at capacity and its managing company is now telling companies that planned to build a datacenter that they can't be connected to the grid until 2030, even though said companies already paid for and got guarantees about that connection.
Are the Netherlands a large proportion of global datacenters?
Amsterdam hosts a major internet exchange. It's not a bad place to build a datacenter and there are many. Northern latitude brings free air cooling, but also additional distance to clients. Lots of peers in AMS-IX, but not a lot of oceananic cable landings (one with two paths to the US, but most of the submarine cables land nearby in Europe)
London is about 3X bigger than Amsterdam in terms of capacity. If you look at the core western europe market (so called FLAP-D) which I am rather familiar with London holds 35% of that market and Amsterdam about 12%. I'm not at all sold that the old rule of thumb that this market is about 25% of global capacity has been true since about 2023 because of the AI buildout in NA/ME.
Yes. Amsterdam has one of the largest IXPs (AMS-IX) in Europe and is also one of the largest European markets for Internet Infrastructure services (i.e. hosting, DNS provision, domain name registration, etc.)
Do AI data centers not need internet connectivity anymore?
The value of an IX isn't just in the IX itself, but also in the presence of hundreds of parties for direct peering, and excellent connectivity to the rest of the world.
It makes a lot of sense to build your DC near one - even if you have no intention of actually participating in the IX itself.
> Do AI data centers not need internet connectivity anymore?
They don't need entire IX worth of connectivity. You're sending mostly text back and forth and any media is in far lower volume than even normal far less dense DC would generate, all the major traffic is inside the AI DC.
What the grid looks like in different countries is very different. The Dutch power grid is already almost 50% renewables, which is an inconvenience for adding capacity because that's around where you have to start really dealing with storage in order to add more.
In most other places the percentage is significantly less than that and then you can easily add more of the cheap-but-intermittent stuff because a cloudy day only requires you to make up a 10% shortfall instead of a 50% one, which existing hydro or natural gas plants can handle without new storage when there are more of them to begin with.
I don’t think the source of the electricity is particularly relevant to whether or not you have the transport capacity to add tens of megawatts of demand to the grid. The problem is generally not the supply but whether your local transformers have capacity left.
When you're talking about something that draws megawatts existing transformers are pretty irrelevant because you're going to run high voltage lines directly to the site itself and install new dedicated transformers on site.
What's more common is that they don't have the transmission capacity itself, but that one's pretty easy in this case too, because what that means is that you have an existing transmission line which is already near capacity with generation on one end and customers on the other. So then you just build the data center on the end of the transmission line where the generation is rather than the end where the existing customers are, at which point you can add new generation anywhere you want -- and if you put it near the existing customers you've just freed up transmission capacity because you now have new customers closer to the existing generation and new generation closer to the existing customers.
High-level, I would agree with you. One thing that blows me away: I think I read that Northern Virginia, USA has the highest data center density in the world. Mostly it is due to demand from US gov't, military, and spy agencies (like NSA). How did they do it? In mainstream media, I don't see any news about a stressed power grid in this area. I guess the US gov't carefully coordinated with local power providers to continuously upgrade their power grid? This is a real question. It makes no sense to me. No shilling/trolling here.
> I think I read that Northern Virginia, USA has the highest data center density in the world. Mostly it is due to demand from US gov't, military, and spy agencies (like NSA).
That's where AWS us-east-1 is, i.e. the oldest AWS region where they got started to begin with. Google and Microsoft also have a large presence there. It's not just the US government, it's everybody, and it's not new.
> How did they do it?
Here's the US nuclear plant map, guess where a bunch of them are:
The area around Virginia is also a major coal producer and when this was getting started it was a source of cheap electricity, but coal is quickly being replaced with natural gas via pipelines from the Gulf coast. Their current power mix is ~30% nuclear, ~12% renewables (solar) and almost all the rest natural gas.
Well, they have done pretty well for 20 years of planning. Google tells me that AWS us-east-1 region (Northern Virginia) was started in 2006!
EDIT
The opening paragraph:
> State regulators’ review of the controversial power line proposed to stretch across three rural Maryland counties will extend to at least February 2027, officials announced Thursday, a timeline that prevents developers from meeting the grid operator’s deadline to ensure reliable electricity.
I bet this is pure NIMBYism. Just this phrase alone is a dead giveaway: "controversial power line". LOL: What is controversial about a power line? Hint: They aren't, but NIMBYism exists.
The grid is constantly "stressed" everywhere, because they're designed to be. You don't build five times as much capacity as you need at huge expense for no reason, you design the thing to have only slightly more capacity than you expect to need.
If demand increases you have to build more generation and power lines etc. This is not a problem but for NIMBYs, it's just the logical consequence. If the local population increases and you don't have enough grocery stores you don't say "grocery stores are stressed" and regard it as an insurmountable problem, people just open more of them.
Chef's kiss for this reply. It is perfect. I like how you reframed the argument around grocery stores. It would be ridiculous to write a newspaper article about "grocery stores are stressed" and there is a newly proposed "controversial grocery store". NIMBYists will perform all kinds of mental gymnastics to defend their positions.
I know that you comment is midly off topic, but I am going through the same out of body experience each time I see a major project announces an opening date of 2030 or later.
This just highlights what an utter failure and self-inflicted wound the green policies of Euro countries have been. Europe has already lost the AI race to the U.S. and China.
Renewables is the only realistic path to energy independence. Today's global situation should show the absolute necessity of that, even if you dont give a sh*t about the environment.
Had we done more 10 years ago we would have been better of. The second best time to start is now.
(We used to build it at a fraction of the cost and less than half of the time that we do with our modern fuckups and fuel can come from just about anywhere if need be. It might be a lot more expensive than the stuff kazachstan and still be a fraction of the cost.)
I think ideally we would've done both to press the cost of nuclear down and given the fact that the renewables rollout turned out to be a lot lot more expensive than proponents claimed it would be whilst still tying us up into gass to cover winter.
When reading an article written by Bjorn Lomborg, you should also do the effort to read the cited sources. This is not an ad hominem attack, just an observation. Do it and you will see.
It does not. That is not economically mined. Last big hard coal producer in EU - Poland, has extraction cost x2 or x3 of the mountain top removal mining in US. This sector is shrinking rapidly. Poland coal production came back to ~1915 levels (taking into account current PL territory). This sector would be closed already if not for massive subsidies.
Last year, China's coal use decreased, while China installed 300x more renewables than nuclear. Coal and nuclear aren't cost competitive with renewables, either in a free market or a technocratic top-down economy. Coal and gas still maintain a valid niche of firming intermittency. But that niche is temporary and shrinking.
The free market installs a tiny amount of coal, and a lot of renewable energy. Whether you believe this means "coal is/isn't cost competitive with renewables in a free market" is a debate about word definitions that I'm not terribly interested in.
China, like Brussels, is trying to reduce coal for similar reasons. They don't like the air pollution health hazard (fully believable), and they say they don't like global warming (somewhat believable).
China and Texas are both installing silly amounts of renewables. They install very little new fossil fuels or nuclear. They both maintain cheap electricity prices through abundance.
The problem in the EU is not renewables, it's the same problem that Democratic states in the US face. Regulations and permitting hurdles that block private renewable energy developers.
Their coal generation decreased last year. They're building on the order of 70GW of new coal while they decomission or underutilized more than 70GW of pre-existing coal. Meanwhile they installed 450GW of new renewables energy.
Not relevant to the question of which energy source makes sense to build in the year 2026. But sure China has many coal plants left over from 2003 when renewables was more expensive, nobody would dispute that this is a fact, however irrelevant.
There appears to be zero advantage to having the datacenter actually in your country apart from minor local property tax, in exchange for which it will put up the electricity bills of every single citizen, who already hate how much they're paying.
The problem in this case seems to have sprung from a lack of collusion. Altman reportedly approached Samsung and SK independently to strike deals for a large chunk of both companies' production. Neither party apparently knew he was negotiating with the other.
If they had actually been communicating or colluding with each other, they would have put the screws to him, making it harder for OpenAI to assert control over the vast majority of the DRAM market.
Failing that, you'd like to think a regulatory agency somewhere would step in to keep a single player from hosing everybody else, but...
> Neither party apparently knew he was negotiating with the other.
I don’t buy it that two of the largest manufacturers of DRAM in the world, from the same country, didn’t know this. Even of you ignore each company’s intelligence teams, that’s also the job of the country’s internal intelligence services, to make sure they know what all companies are doing and then make it so they have the best leverage to gain as much as possible. Both companies would have known “somehow” and played hardball.
How would the country’s internal intelligence services know what’s happening? Yes, by spying. That’s literally their job and they have assets in every critical area in a country. Every institution, every major industry player, they are monitored to a degree by the internal intelligence in every country in the world. There are more nefarious reasons to do this but the ostensible one is that if it’s of strategic importance the country needs to know everything there is to know.
The companies also do a lot of spying themselves, every bit of info could give them an edge.
There are a lot of cyclical businesses that make money every year. It requires careful management. Factories can produce less than full capacity - but you better design for that. you can make money in the worst years without laying anyone off even - but it requires careful attention to details and not over hiring in good times as if they will never end.
Factories working at (significantly) less than full capacity gets a bit harder when you've got one of the most expensive machines on earth working in them, and production lines that'll be out of date in a couple of years
the normal way to do that is by hiring/firing to meet demand, but in the fab business, you have 10s of billions of dollars of capex with relatively little opex. if you're running at <90% capacity, you're losing money.
That is the common way, but there are companies that manage without hiring/firing. (or they hire temp workers). There is a minimum level of capacity you need to run just to keep the lights on, and figuring out how to get that low without impacting your ability to serve the highs is hard. Memory manufactures have not gotten very low, probably for good reasons, but it is something they should work on.
Last year ai folks are all over wallstreet and articles, decrying how hardware folks are a roadblock to new frontiers in AI. They just couldn't print and pack the new chips faster.
but if you don't collude during times of feast you will have famine, and during times of famine you will have famine, in an economy based on feast/famine you must sometimes feast or die.
All of the capital intensive businesses face this issue. Chemicals, Shipping, Semiconductors etc.
You get market signals that the demand is there, you acquire the necessary capital, you spend 5 years to build capacity, but guess what, 5 other market players did the same thing. So now you are doomed, because the market is flooded and you have low cash flow since you need to drop prices to compete for pennies.
Now you cannot find capital, you don't invest, but guess what, neither your competitors did. So now the demand is higher than the supply. Your price per unit sold skyrocketed, but you don't have enough capacity!
Forecasting demand 5 years into the future is intrinsically highly unreliable. It doesn’t matter if it is capitalism or a command economy. The bet is always going to be risky and someone will have to pay for that risk.
At least with capitalism you have many different people with different perspectives on the risk making independent bets. That mitigates the more extreme negative outcomes.
To add to that, investors who do make the bet get punished for over-building, which is better than tax payers paying for it. And before someone says it, big corps do get bailed out by gov't, but that's definitely goes against capitalist ideas.
Also it is government job to regulate. Monopolies should be busted, and behaviour like that should be culled. But US govt is full on AI to mask them cratering their own economy.
Is the DRAM industry really capitalist? Focusing on just the Korean parties, it functions like a command economy. I would say the same about most high end semi-conductor manufacturing, TSMC, Intel, ASML are being commanded and driven by nation-state level decision making. Right now the command is to focus on high wattage centralized AI systems at the expense of everything else.
No one at high levels is capitalist, in ideology or action. An ideological capitalist would be in favor of competition, but these people disdain it and collude regularly. The only 'capitalist' actions they take are by accident, the real goal is as much power/money as possible as fast as possible.
We don't even expect companies to plan long-term anymore, it's just moving wealth as fast as possible.
That isn't really a change, very few people could ever have been said to be ideological capitalists. (capitalist is not a word with a hard definition, but I'm considering it a different thing than the more modern pure libertarian zero-regulation ideology)
I think the common use of the term capitalist is as a participant in capitalism.
This is distinct from someone who is a proponent of capitalism as a system, which appears to be the way you are using capitalist. For which I don't blame you.
Liberalism (in the traditional economic sens) likes competition. Capitalism is a mode of production, and capitalists notoriously don’t like competition when they are the incumbents
Because that does not happen exactly as you say for all players. The demand signals will be processed and long-term risk is balanced against short-term gain in a distributed fashion, so not everyone will do the same.
It's more optimal than planned economies until we have AI planned economies with realtime feedback, I guess.
Consumers get cheap goods during oversupply and most inefficient companies get elliminated during bust while consolidation leads to economies of scale.
Why is the opposite of capitalist markets automatically assumed to be a command economy? Co-op style businesses aren't really capitalist orientated but are also not reliant on government action.
That's not what capitalists claim. Capitalists claim communism is responsible for tens, if not hundreds, of millions of death due to famine. And overall miserable ways of life, as if humans were termites deprived of any individualism and any freedom.
Capitalists claim that France importing 500 000 people from the third world each year, out of which only 10% are ever going to work (and these are official numbers) and yet offering them all a safety net is unsustainable. And that that socialism is only going to lead to one thing: running out of taxpayers' money.
Capitalists don't claim they have the best system: what they claim is that they haven't seen a less worse one.
I don't know who these "capitalists" are, but generally speaking business regularizes as entrepreneurs learn to read new signals after unexpected market shifts.
It is unlike a socialist system in that there are signals to read in the first place. What, do socialists claim that failure is optimal when you can't even tell if you've failed?
The sheer fucking blazing ignorance of this comment
> Capitalists claim that this is optimal.
Compared to starving under communism coz someone at top got the number wrong, yes. And it only really happens when there are massive, unpredictable market movements and governments not doing their job. Govt should look at the whole thing and just say "no", blame them.
No market system self regulates well enough, and it's government job to file down the edge cases like this. But the revolution happened in country which has two utterly incompetent parties, both in pockets of billionaires, fighting for power, and the clowns from one that won last battle use AI to smokescreen the economic growth their actions cratered
I am betting the pendulum swings faster to the other side to excess capacity as all the construction lies of Altman fall through with financiers waking up the the fact they can't build the infrasctructure as fast nor make any profits on that infrastructure that will get built.
What if it takes 100 years to get to AGI or we never achieve it? All bets on AGI will just fail over and over again for decades in that case. It seems a bit like saying financiers can't risk not being involved with Faster-than-light travel technology. Yeah, it would change everything if we got it, but betting that we'll get it soon over and over again is probably not going to get you a lot of money.
We've been projecting both FTL and AGI as future possibilities for almost 100 years now. Do LLMs get us a lot closer to AGI? I think they get us a little closer and Moore's "law" making compute faster probably is a much bigger factor, but I think we're still a very very long ways away.
Who has been projecting FTL as a realistic technology ever? FTL is not possible according to the current laws of Physics, while AGI is at least not forbidden by them.
I think this should be something you can answer for yourself by looking at human media and news over the lasy 100 years. I find it hard to belive you haven't ever noticed anyone seriously saying we may possibly have FTL sometime in the future. Incidentially I think I read on HackerNews that Sam Altman has been talking about building Dyson Spheres in the future. I suppose they're not forbidden by the current laws of phyisics either, but I don't know if I would call them a realistic technology.
The IQ of the smartest human, the perfect memory storing and recollection of computers, the fact that it never tires. I don't know if it's AGI but it's already something greater than us.
To show ads you need people to stay on your platform. This is especially true once ads become more intrusive or of lower quality, something the big players seem to gravitate towards to keep revenue up. Google and Meta have ways to lock in users (networking effects, the best search engine available, having your data stored there).
I am not sure if OpenAI has that. Their edge regarding models is small, their strategy currently seems to be "buy ALL the hardware so nobody else can". Users can quite easily switch to other models.
If only there were privacy laws and working antitrust laws. There could also be a law straight up banning chat uses of llms and only allow agentic uses with human review. Would solve a lot of problems for lawmakers worried about AI I think.
Memory makers did get themselves into this situation by selling all wafers for empty promises and alienating everyone but OpenAI tbh. I do hope they end up holding the bag once again, cause after covid and the cartel thing they don't seem to ever learn their lesson on how to have the tiniest amount of integrity.
> Memory makers did get themselves into this situation by selling all wafers for empty promises and alienating everyone but OpenAI tbh.
Wasn't the problem here that OpenAI was negotiating with Samsung and SK Hynix at the same time without the other one knowing about it? People only realized the implications when they announced both deals at once.
Permanent public ownership of (very large stakes in) these companies doesn't seem like such a bad idea anymore, does it? It's what we used to have for most of the 20th century at least in Europe.
That's only sound is it not. If you take away the hype - nothing critical actually depends on LLMs. You can remove them all today, and nothing bad would happen.
> Soo ... how sure are we that the memory makers themselves are not going to be the ones holding the bag?
I hope they do, they did not have to agree to sell so much RAM to one customer. They’ve been caught colluding and price fixing more than once, I hope they take it in the shorts and new competitors arise or they go bankrupt and new management takes over the existing plants.
Don’t put all your eggs in the one basket is how the old saying goes.
> Soo ... how sure are we that the memory makers themselves are not going to be the ones holding the bag?
We aren't. The remaining memory manufacturers fear getting caught in a "pork cycle" yet again - that is why there's only the three large ones left anyway.
If they don't expand capacity much, the only negative consequences I foresee happening for them is that they might lose spending discipline, and that systems will be set up to make do with a little less memory. Apart from that, it's just very high profits followed by more or less regular profits.
They could wind up losing all their business to China though.
China has memory makers who are creeping up through the stages of production maturity, and once they hit then there's no going back.
If the existing makers can't meet supply such that Chinese exports get their foot in the door, they may find they never get ahead again due to volume - that domestic market is huge so they have scale, and the gaming market isn't going to care because they get anything at the moment, which is all you'll need for enterprise to say "are we really afraid of memory in this business?"
Good point, it's a risk but so far the Chinese competition isn't up to par and it's unclear whether they'll be able to exploit the current window of opportunity.
You think this window is short? We've been dealing with this for years and years, and to me it seems more like incumbent manufacturers are too comfortable milking cash cows.
That's only for normal RAM. I'm talking about broader aspect: HBM shortage and high prices have been around for longer, and Chinese manufacturers are also climbing up and expanding there.
> If the existing makers can't meet supply such that Chinese exports get their foot in the door, they may find they never get ahead again due to volume
The answer to that is government regulation. Ban anything Chinese or slap it with tariffs. That is what tariffs are intended for - not for the BS the current administration has done.
Good luck actually pulling that off, it'll just kneecap US/Western industry, just as the current tariff and war madness has helped drag the US economy into the gutter.
Surely this can be solved with financial engineering. The memory makers build more capacity, but they finance it with something like floating-rate notes linked to an index of memory prices, or even catastrophe bonds or AT1s. Or more crudely, set up special purpose vehicles to build the extra capacity, and issue convertible bonds from those; if the memory market collapses, investors don't get paid, but they do get a memory factory.
But wouldn’t you rather hbm prices come down first ? Memory makers will be fine. There is practically infinite demand.
Unless you get china style rationing of compute per person world wide.
The real issue is everyone wanting to upgrade to hbm, ddr5, and nvme5 at the same time.
HBM is just normal DDR RAM that's been packaged with (much) wider-than-usual data buses. That's where the high bandwidth comes from, not from high clock rates or any other innovation or improvement in core specifications.
I think the datacenters will simple be repurposed for hardware as a service, specially cloud gaming, sense it will be to expensive to buy a new computer anyway.
Anything where there's _at worst_ solder and traces between the compute and the memory. That's why you see it on GPUs (and Apple hardware). DRAMs advantage is modularity.
I'm under the impression OAI wrote Letters of Intent, and are not actually on the hook for the RAM they requested.
The others...they did not. Memory makers won't be holding the bag because Apple/Google/Samsung are contractually obligated to purchase, after the panic OAI caused.
Not a rec, but just my source: Atrioc (streamer, YouTuber) is good at gathering all the facts for the rest of us. There's many other things in play, like the Strait of Hormuz (helium, bromine). Ultimately it works out that the shortage, and shortage profits, will continue; the chip makers are probably going to continue to see record profits (as Samsung has).
The specific mix of factors could change at any time, but the supply chain is relatively inelastic, it will take some time to show up on price labels.
It's not going to last until 2028, it'll last until 'min(AI_bubble_burst, 2028)', which I expect will be a lot smaller than just '2028'. So the real question is, how long will it take to retool for non-HBM, and will there be a fire sale as they scramble to recover?
Which also explains why production is falling behind demand, companies aren't going to sink billions into creating product for a market that could dry up overnight.
I think I’m missing something. Financially, what bag would the memory makers be holding here? I don’t think I’m well informed regarding how these deals were structured.
Memory makers make capital investements (build different factories, convert physical production lines, etc.) to meet orders that have been place for the next ~5 years.
OpenAI (or whoever) crashes and can't pay for the order leaving the memory makers in a tough spot.
The amount of money flowing both from the AI bubble and from quite literally scalping both the server and consumer market... They gambled on the opportunity and if they fail - it's their problem.
And how is that a problem and more importantly how's that a problem of Average Joe?
Capitalists did their gamble things. If they fail in that gamble what forbids them to sell the regular RAM they made for AI bubbleists to the regular consumers? Besides HBM it's just the regular chips which are exactly the same for the consumer/server market, why it would be any different?
The market is already stagnated. Even if OpenAI doesn’t buy what they reserved other players will do so. SK Hynix CEO said there is a 20% gap between supply and demand per year. And that doesn’t account the shock effect that will take place the moment prices normalize and everyone and their dog will go out and start buying inventory to avoid the next crisis. I for one would certainly buy more than I currently need just in case.
> OpenAI’s rapid growth, fueled by the success of ChatGPT and other AI products, led to a landmark agreement in October to purchase 900,000 DRAM wafers per month from Samsung and SK Hynix—amounting to roughly 40% of global supply. This surge in demand, coupled with limited manufacturing capacity, sent prices for memory kits skyrocketing. [0]
They ordered 40% of the global RAM production for 2025/26. It was a non-binding agreement that either side could easily withdraw from but they're essentially trying to buy about half of all the RAM.
If I booked half a hotel's rooms then suddenly said "yeah never mind. Half my friends cancelled and we're not staying", basically any hotel would be coming at me for my money because there's no way they can fill their rooms now and they're losing revenue. But OpenAI can really get the whole world to pivot towards it then say "cool but we don't need your product anymore" and RAM makers are just going to let it go.
Whoever decided that was a good idea needs to be fired and publicly shamed.
Well if that hotel was then able to sell the other half of hotel rooms for 10x the old price. Then the hotel might actually be happy as they can now charge 10x for the other half or slowly lower prices back down over years.
Do the memory makers not have a contract in place for an order this large? I assume that they aren't going to take "trust us bro" as good enough for several million dollars in orders, and even if there is a way to cancel the order it won't be free. I would assume so at least, but i would like if anyone knew for certain.
I was interning at a company that made networking gear that was put out of business when their largest customer canceled an order within a week of the delivery date.
The customer ran out of money. In terms of where you are in line of debtors when you haven't even delivered the product to a customer, it's so far back as to be assured you won't get your money.
If the memory makers got a deposit from OpenAI as part of this deal, that is likely to be the only money they will get for any undelivered memory, particularly if OpenAI runs out of capital.
It's a business with huge up-front capital expenses and typically very low margins. Supply is scaling up slowly because it's hard, and if you overshoot, you go out of business.
Nobody is "allowing" this. It's a natural property of being both advanced technology and a commodity at the same time.
The strange deals on the entire future output are what was allowed. Try to do the same thing with onions and the government understands you are a criminal.
That is quite the amusing read but it seems like a poorly constructed law. It wasn't futures themselves that were the problem there. The duo engaged in blatant market manipulation and severely disrupted part of the food supply in the process.
Cornering the market with the intent to flip the goods is not quite the same as cornering the market because you actually want the goods and intend to use them yourself.
And it just so happens that many people will now have to use OpenAI’s products because they can’t get enough RAM to run a local LLM. What a coincidence.
It has the makings of a natural monopoly, except its compounded by RAM cartels colluding to shut out the last of the competitors.
Recently they had a second price fixing lawsuit thrown out (in the US).
Now with the state of things I'm sure another lawsuit will arrive and be thrown out because the government will do anything to keep the AI bubble rolling and a price fixing suit will be a threat to national security, somehow. Obviously thats speculative and opinion but to be clear, people are allowing it. There are and more so were things that could be done.
Allowed? We live in a neoliberal world where corporate monopolies / oligopolies aren’t even remotely regulated. If you try to do even the gentlest regulation of companies people scream about communism and totalitarianism. Unless the regulation serves the monopolies by making it harder to enter the market.
It started with raegan, and even parties on the “left” in the west believe in it with very few exceptions.
> We live in a neoliberal world where corporate monopolies / oligopolies aren’t even remotely regulated. If you try to do even the gentlest regulation of companies people scream about communism and totalitarianism. Unless the regulation serves the monopolies by making it harder to enter the market.
The thing that enables this is pretty obvious. The population is divided into two camps, the first of which holds the heuristic that regulations are "communism and totalitarianism" and this camp is used to prevent e.g. antitrust rules/enforcement. The second camp holds the heuristic that companies need to be aggressively "regulated" and this camp is used to create/sustain rules making it harder to enter the market.
The problem is that ordinary people don't have the resources to dive into the details of any given proposal but the companies do. So what we need is a simple heuristic for ordinary people to distinguish them: Make the majority of "regulations" apply only to companies with more than 20% market share. No one is allowed to dump industrial waste in the river but only dominant companies have bureaucratic reporting requirements etc. Allow private lawsuits against dominant companies for certain offenses but only government-initiated prosecutions against smaller ones, the latter preventing incumbents from miring new challengers in litigation and requiring proof beyond a reasonable doubt.
This even makes logical sense, because most of the rules are attempts to mitigate an uncompetitive market, so applying them to new entrants or markets with >5 competitors is more likely to be deleterious, i.e. drive further consolidation. Whereas if the market is already consolidated then the thicket of rules constrains the incumbents from abusing their dominance in the uncompetitive market while encouraging new entrants who are below the threshold.
Arguably a more efficient approach might just be to have a tax that adds on to corporate tax incrementally for every % of market share a company has above say 7-8%. Then dominant companies are incentivised to re-invest in improving their efficiencies rather than just buying/squeezing out competitors. A more evenly spread market would then, as a result, be against regulations that make smaller market participants less competitive, as they'd all be in relatively less table positions.
> Arguably a more efficient approach might just be to have a tax that adds on to corporate tax incrementally for every % of market share a company has above say 7-8%.
How is this more efficient? You'd still be applying all of the inefficient regulatory rules intended to mitigate a lack of competition to the smaller companies trying to sustain a competitive market, and those rules are much more deleterious for smaller entities than higher tax rates.
If you have $100M in fixed regulatory overhead for a larger company with $10B in profit, it's only equivalent to a 1% tax. The same $100M for a smaller company with $50M in profit is a 200% tax. There is no tax rate you can impose on the larger company to make up for it because the overhead destroys the smaller company regardless of what you do to the larger one.
First, on the surface, this sounds like a terrible idea. Almost all ideas that I see on HN about economics fail with even the tiniest amount of common sense.
As a counterpoint: Look at very high value goods, like jet engines and MRI machines. I went for an MRI the other day and wondered to myself (then asked an LLM) what the international MRI market looks like. They are vanishingly small number of manufacturers and are usually dominated by a few international players. How are you going to apply this tax to non-domiciled (international) companies? Also, companies like General Electric, Mitsubishi Heavy, and Seimens are enormous and incredibly diverse. This idea falls apart quickly.
Maybe it’s time for a refresher on what neoliberal means? It’s not simply “new liberalism”. Reaganomics was the start of neoliberalism in the US, tho of course it shifted and developed its character further over time into the monster that drives 99% of our problems today
This will result in demand destruction which will starve the enterprise which will starve the hyperscaler. theres no situation where people not being able to afford hardware for 4 years results in the bubble not popping
I'd expect unaffordable hardware to drive demand for thin clients connected to cloud services which is something that had already been happening gradually prior to this.
They won't be, prices are high because they are refusing to build capacity for demand that may evaporated by the time they are done. They are holding back and building only enough so when the bubble pops they will be fine.
You can't build capacity overnight, and even with that in mind, it's hard to say if it is sensible to increase capacity now that we are in an AI bubble. For all we know, the bubble might burst.
So the ML hate is weaponized in the form of memory demand collapse FUD, and the public at large has to pay through their nose for it... thanks party poopers!
I don't think its from the ML collapse FUD, its most likely from the multiple time's in the past when they overbuilt and it resulted in a memory oversupply and price collapses. The 1985–1988, 1993–1994, 1998–2002 and the post pandemic oversupply. These were all cases where shortages followed by over corrections caused oversupply, financial losses due to low prices and fewer surviving companies. I think they're taking their time and are cautiously adding more capacity in such a way that prices won't end up collapsing again. Regardless, the result is still that we the consumers have to pay more.
At this point the remaining memory companies are… the ones that didn’t die during an over-supply collapse, right? I guess there’s been a strong evolutionary pressure against giving consumers what we want, haha.
If they gradually increase production capacity then prices stay high for 10+ years (or for as long as it takes for demand to crash) because a gradual increase in production takes that long for them to add enough capacity for current demand.
If they add enough capacity to meet current demand quickly then if demand crashes they still have billions of dollars in loans used to build capacity for demand that no longer exists and then they go bankrupt.
The biggest problem is predicting future demand, because it often declines quickly rather than gradually.
do we have evidence of RAM manufacturers going bankrupt? do we have evidence that the increased capacities after the mentioned past shortages went unused or were operated at a loss?
Machines take up space in buildings (factories); both of which are discrete rather than continuous functions. If your factory is already full of memory-making machines, and want to add one more, it will cost you billions and many months to build another factory.
If you suppose you have cracked the smooth-ramping problem, perhaps you should throw your hat in the ring and soak up all the pent-up demand that SK Hynix, Samsung and Micron are neglecting.
Think of the factory problem from physics first principals instead, as Elon would say. Musk says he will outcompete earth fabs by building them on the moon in just a few years, deploy radiation harden versions of the chips into space, and beat out TCO vs doing this on earth.
If he can do all that that fast, the RAM makers should be able to at least 1000X their fab capacity on earth in one year. One year for scaling up existing tech is an eternity compared to Elon's timeframe for moon-fabs given the relative complexity of the challenge.
The most well known thing about Musk besides being an asshat, is that his timelines are almost always imaginary. He is not building fabs on the Moon in "just a few years".
He doesn't have to build them to sell them. FSD has been sold for almost a decade, more than half the depreciation time of the cars. With first principles sales techniques it's possible X could sell these moon chips now and deliver them made on earth 6 or 7 Moore's law cycles later by a different company, like with the solar roof or hydraulic brick machine from alibaba, way cheaper.
But how is that at all related to the DRAM manufacturers' short term production limitations? Unless I'm being dumb and you're saying that these manufacturers could also just kick the can down the road for a few fab generations?
I would expect that OpenAI gets as much money as they ask for for the next 10 years.
There’s virtually infinite capital: if needed, more can be reallocated from the federal government (funded with debt), from public companies (funded with people’s retirement funds), from people’s pockets via wealth redistribution upwards, from offshore investment.
They will be allowed to strangle any part of the supply chain they want.
China already has a well developed DRAM industry, as DRAM is somewhat easier than logic, and can tolerate a much higher defect rate. The industry will figure this out.
Another point is I often see the money argument - like country X has more money, so they can afford to do more and better R&D, make more stuff.
This stuff comes out of factories, that need to be built, the machinery procured, engineers trained and hired.
I think the article has a giant blind spot as far as China is concerned , considering they have already a mature enough memory ecosystem via YMTC that Apple was considering sourcing from them. As well as continued expansion in the DRAM and HBM Fabs [1].
It feels like the memory cartel once again trying to incentivise their various govt to cough up some more tax breaks/funding to cushion the AI buildout bet that they made and the bubble seeming about to pop.
In any case if they leave the consumer market underserved it should be no surprise if before that 2030 prediction we are all on cheaper YMTC memory modules.
I think you're massively overestimating how much money is really accessible here. The parent comment's right that all of the easily available VC & private equity investment is basically used up. OpenAI was struggling to sell $600M of private equity, the big multi-billion dollar investment packages had lots of conditions and non-cash in it.
> more can be reallocated from the federal government (funded with debt)
While this is the most reliable funding, it's still not very accessible. OpenAI is a money pit, and their demands are growing quickly. The US government has started a bunch of very expensive spending. If OpenAI were to require yearly bundles of it's recent "$120B" deal, that's 6% of the US' discretionary budget. 12.5% of the non-military discretionary budget. (And the military is going to ask for a lot more money this year) Even the idea of just issuing more debt is dubious because they're going to want to do that to pay for the wars that are rapidly spiralling out of control.
None of this is saying that the US government can't or wouldn't pay for it, but it's non trivial and it's unclear how much Altman can threaten the US government "give me a trillion dollars or the economy explodes" without consequences.
Further deficit-spending isn't without it's risks for the US government either. Interests rates are already creeping up, and a careless explosion of deficit may well trigger a debt crisis.
> from public companies (funded with people’s retirement funds)
This would be at great cost. OpenAI would need to open up about it's financial performance to go public itself. With it's CFO being put on what is effectively Administrative Leave for pushing against going public, we can assume the financials are so catastrophic an IPO might bomb and take the company down with it. Nobody's going to be investing privately in a company that has no public takers.
Getting money through other companies is also running into limits. Big Tech has deep pockets but they've already started slowing down, switching to debt to finance AI investment, and similarly are increasingly pressured by their own shareholders to show results.
> from people’s pockets via wealth redistribution upwards
The practical mechanism of this is "AI companies raise their prices". That might also just crash the bubble if demand evaporates. For all the hype, the productivity benefit hasn't really shown up in economy-wide aggregates. The moment AI becomes "expensive", all the casual users will drop it. And the non-casual users are likely to follow. The idea of "AI tokens" as a job perk is cute, but exceedingly few are going to accept lower salary in order to use AI at their job.
There's simply not much money to take out of people's pockets these days, with how high cost of living has gotten.
> from offshore investment.
This is a pretty good source of money. The wealthy Arabian oil states have very deep slush funds, extensively investing in AI to get ties to US businesses and in the hope of diversifying their resource economies.
The "no food in other countries" is because of failed/corrupt governments, not because people use AI to generate cat pictures in the West. The economy is not a "fixed pie" that needs to be allocated among people of the world.
Just look at Cuba, which could be a very rich country and one of the prime tourist destinations of the world.
I'm a bit surprised the article makes no mention of Google's TurboQuant[0] introduced 26 days prior.
Given that TurboQuant results in a 6x reduction in memory usage for KV caches and up to 8x boost in speed, this optimization is already showing up in llama.cpp, enabling significantly bigger contexts without having to run a smaller model to fit it all in memory.
Some people thought it might significantly improve the RAM situation, though I remain a bit skeptical - the demand is probably still larger than the reduction turboquant brings.
TurboQuant is known across the industry to not be state of the art. There are superior schemes for KV quant at every bitrate. Eg, SpectralQuant: https://github.com/Dynamis-Labs/spectralquant among many, many papers.
> Given that TurboQuant results in a 6x reduction in memory usage for KV caches
All depends on baseline. The "6x" is by stylistic comparison to a BF16 KV cache; not a state of the art 8 or 4 bit KV cache scheme.
Current "TurboQuant" implementations are about 3.8X-4.9X on compression (w/ the higher end taking some significant hits of GSM8K performance) and with about 80-100% baseline speed (no improvement, regression): https://github.com/vllm-project/vllm/pull/38479
For those not paying attention, it's probably worth sending this and ongoing discussion for vLLM https://github.com/vllm-project/vllm/issues/38171 and llama.cpp through your summarizer of choice - TurboQuant is fine, but not a magic bullet. Personally, I've been experimenting with DMS and I think it has a lot more promise and can be stacked with various quantization schemes.
The biggest savings in kvcache though is in improved model architecture. Gemma 4's SWA/global hybrid saves up to 10X kvcache, MLA/DSA (the latter that helps solve global attention compute) does as well, and using linear, SSM layers saves even more.
None of these reduce memory demand (Jevon's paradox, etc), though. Looking at my coding tools, I'm using about 10-15B cached tokens/mo currently (was 5-8B a couple months ago) and while I think I'm probably above average on the curve, I don't consider myself doing anything especially crazy and this year, between mainstream developers, and more and more agents, I don't think there's really any limit to the number of tokens that people will want to consume.
The work going into local models seems to be targeting lower RAM/VRAM which will definately help.
For example Gemma 4 32B, which you can run on an off-the-shelf laptop, is around the same or even higher intelligence level as the SOTA models from 2 years ago (e.g. gpt-4o). Probably by the time memory prices come down we will have something as smart as Opus 4.7 that can be run locally.
Bigger models of course have more embedded knowledge, but just knowing that they should make a tool call to do a web search can bypass a lot of that.
The net effect won’t be a memory use reduction to achieve the same thing. We’ll do more with the same amount of memory. Companies will increase the context windows of their offerings and people will use it.
I am not convinced that more context will be useful, practical use of current models at 1mil context window shows they get less effective as the window grows. Given model progress is slowing as well, perhaps we end up reaching a balance of context size and competency sooner than expected.
Stuff in more code. Stuff in more system prompt. Stuff in raw utf8 characters instead of tokens to fix strawberries. Stuff in WAY more reasoning steps.
Given the current tech, I also doubt there will be practical uses and I hope we’ll see the opposite of what I wrote. But given the current industry, I fully trust them so somehow fill their hardware.
Market history shows us than when the cost of something goes down, we do more with the same amount, not the same thing with less. But I deeply hope to be wrong here and the memory market will relax.
Your skepticism is well placed. Every time a new quantization or compression technique drops, the immediate response is to just scale up context length or run a bigger model to fill whatever headroom was freed up. It's Jevons paradox applied to VRAM - efficiency gains get eaten by increased usage almost immediately.
that will only increase the demand for RAM as models will now be usable in scenarios that weren't feasible prior, and the ceiling for model and context size is not even visible at this point
I hate to mention Jevons paradox as it has become cliche by now, but this is a textbook such scenario
Something I haven’t been able to reconcile: If AI makes software easier to create, that will drive the price down. How are software companies going to make enough revenue to pay for AI, when the amount of money being spent on AI is already multiples of the current total global expenditure on software? This demand for RAM is built on a foundation of sand, there will be a glut of capacity when it all shakes out.
This is probably the wrong denominator. There are more than a billion white-collar workers. Making them all just 10% more effective would possibly be worth more than $650B/year (~$650/worker).
> If AI makes software easier to create, that will drive the price down.
Supposedly AI drives down the cost of producing software,not the "price".
> How are software companies going to make enough revenue to pay for AI, when the amount of money being spent on AI is already multiples of the current total global expenditure on software?
Currently, the cost of AI is between $20/month and around $200/month per developer.
I think the huge billions you're seeing in the news are the investment cost on AI companies, who are burning through cash to invest in compute infrastructure to allow both training and serving users.
> This demand for RAM is built on a foundation of sand, there will be a glut of capacity when it all shakes out.
Who knows? What I know is that I need >64GB of RAM to run local models, and that means most people will need to upgrade from their 8Gb/16GB setup to do the same. Graphics cards follow mostly the same pattern.
You can run huge local models slowly with the weights stored on SSDs.
Nowadays there are many computers that can have e.g. 2 PCIe 5.0 SSDs, which allow a reading throughput of 20 to 30 gigabyte per second, depending on the SSDs (or 1 PCIe 5.0 + 1 PCIe 4.0, for a throughput in the range 15-20 GB/s).
There are still a lot of improvements that can be done to inference back-ends like llama.cpp to reach the inference speed limit determined by the SSD throughput.
It seems that it is possible to reach inference speed in the range from a few seconds per token to a few tokens per second.
That may be too slow for a chat, but it should be good enough for an AI coding assistant, especially if many tasks are batched, so that they can progress simultaneously during a single read pass over the SSD data.
You can do that, but you're going to have rather low throughput unless you have lots of PCIe lanes to attach storage to. That's going to require either a HEDT or some kind of compute cluster.
Batching inferences doesn't necessarily help that much since as models get sparser the individual inferences are going to share fewer experts. It does always help wrt. shared routing layers, of course.
> Who knows? What I know is that I need >64GB of RAM to run local models, and that means most people will need to upgrade from their 8Gb/16GB setup to do the same. Graphics cards follow mostly the same pattern.
Depends how big the models are, how fast you want them to run and how much context you need for your usage. If you're okay with running only smaller models (which are still very capable in general, their main limitation is world knowledge) making very simple inferences at low overall throughput, you can just repurpose the RAM, CPUs/iGPUs and storage in the average setup.
I got a 128 GB MBP, and the current models are fit enough to manage the calendar or do research on web (very slowly), not to be useful companions for coding as I hoped.
> This demand for RAM is built on a foundation of sand
Not exactly.
LLMs are already quite useful today if you use them as a tool, so they are there to stay. The remaining problem is scalability, a.k.a. how to make LLMs cheap to use.
But scalability is not really a requirement when you look the bigger picture. If smaller software company/projects can't afford to use AI, the bigger ones might just. Eventually they will discover variable use cases for such tech, even if it only serves big firms i.e. defense, resource extraction, war, finance etc.
To the other end, if scalability is achieved, the use of LLM products will be cheaper too, so smaller project can also use them. But of course, if LLM usage is too cheap, then many were-to-be-consumers will just create software projects by themselves at their homes.
The advanced workflows such as programming are not the only use cases for LLMs. There are much simpler work such as answering simple questions or arrange simple tasks. Both will improve productivity. And more advanced model should improve productivity even more.
The usage of LLMs is continuing to increase ~exponentially. I'm going to bet on that rather than some half-baked scenario analysis that only takes into account one scenario and assigns a 100% probability to it.
> The usage of LLMs is continuing to increase ~exponentially
I would like a source for that statement. Additionally, I want to know by who? Because it certainly isn't end users. Inflating token usage doesn't make it any more economically viable if your user base, b2b or not, hasn't increased with it. On the contrary, that is a worse scenario for providers.
1. As a consultant pretty much every company I have worked with in the last 2 years are doing some kind of in-house "AI Revolution", I'm talking making "AI Taskforce" teams, having weekly internal "AI meetings" and pushing AI everywhere and to everyone. Small companies, SMEs and huge companies. From my observation it is mainly due to C-level being obsessed by the idea that AI will replace/uplift people and revenue will grow by either replacing people or launching features 10x quicker.
2. Did you see software job-boards recently? 9/10 (real) job listings are to do with AI. Either it is fully AI company (99% thin wrapper over Anthropic/OpenAI APIs) or some other SME that needs some AI implementations done. It is truly a breath of fresh air to work for companies that have nothing to do with AI.
The biggest laugh/cry for me are those thin wrappers that go down overnight - think all the "create your website" companies that are now completely useless since Ahtropic cut the middleman and created their own version of exactly that.
Yeah, my only hope is that this is unsustainable, admittedly for selfish reasons.
I know plenty of engineers being forced to use these tools whether they want to or not. A lot of which are okay with using AI liberally, but don't particularly like generative AI and see it as pretty irresponsible (which feels more true by the week and it is clear from first hand experience). I don't know, there is a huge gradient of users, but I would argue that in previous revolutionary technologies, we didn't have to force people to use a good tool. I didn't have to be forced to use Google search or Google Maps, tech that is now ubiquitous with western society. It seems really suspect that suits have to enforce the use of something that is supposed to change the way we work and be a force multiplier.
From my limited experience in multiple companies, as stated before I see one very common pattern - The process from feature idea to development is just bad. PMs do not know what exactly they want. C-level interjects in the middle and changes requirements. QAs are unsure what to test because acceptance criteria is vague.
C-level strongly believes that AI will fix all these issues. They believe that AI will fix their broken processes.
I see strong resemblance with "Agile Development" ~15 years ago. Extremely hyped, noone asked if their org even is a fit for it or need it, and most importantly - the only way to fix agile is to do more agile. Same with AI right now.
Users are willingly paying for larger volumes of tokens. You are layering your own unproven interpretation onto that. I would have arrived at an opposite interpretation given the available facts. Models are becoming more token efficient for the same task, such as ChatGPT 5.3 versus 5.2 which halved the token count, and capabilities show a log relationship with the number of tokens since o1 preview was revealed in September 2024.
No, you have gone off in your own tangent. The person you're responding to is talking about money and my point is that you're using a misleading metric. Even if the current user base is paying more for the "exponential token usage", it does not add up to the industry's cost of maintaining and building on this technology, especially since we are not taking into account what that token usage costs the provider. First you said Anthropic as your source, but now you're talking about OpenAI's ChatGPT, who are floundering for a product and user base, which they themselves claim will be profitable through subscriptions at numbers never seen before in a subscription business model.
I’m a bit of an optimist. I think this will smack the hands of developers who don’t manage RAM well and future apps will necessarily be more memory-efficient.
The point is being able to write it once with web developers instead of writing it a minimum of twice (Windows and macOS) with much harder to hire native UI developers.
And HTML/CSS/JS are far more powerful for designing than any of SwiftUI/IB on Apple, Jetpack/XML on Android, or WPF/WinUI on Windows, leaving aside that this is what designers, design platforms and AI models already work best with. Even if all the major OSes converged on one solution, it still wouldn't compete on ergonomics or declarative power for designing.
Lol SwiftUI/Jetpack/WPF aren’t design tools, they’re for writing native UI code. They’re simply not the right tool for building mockups.
I don’t see how design workflows matter in the conversation about cross-platform vs native and RAM efficiency since designers can always write their mockups in HTML/CSS/JS in isolation whenever they like and with any tool of their choice. You could even use purely GUI-based approaches like Figma or Sketch or any photo/vector editor, just tapping buttons and not writing a single line of web frontend code.
Who said anything about mockups? Design goes all the way from concept to real-world. If a designer can specify declaratively how that will look, feel, and animate, that's far better than a developer taking a mockup and trying their hardest to approximate some storyboards. Even as a developer working against mockups, I can move much faster with HTML/CSS than I can with native, and I'm well experienced at both (yes, that includes every tech I mentioned). With native, I either have to compromise on the vision, or I have to spend a long time fighting the system to make it happen (...and even then)
Yikes. I spent 15 years developing native on both mobile and desktop. If you think that native has the same design flexibility as HTML/CSS, you're objectively wrong.
By design, each operation system limits you to their particular design language, and styling of components is hidden by the API making forward-compatible customisation impossible. There's no escaping that. And if you acknowledge that fact, you can't then claim native has the same design flexibility as HTML/CSS. If you don't acknowledge that fact, you're unhinged from reality.
There's pros and cons to the two approaches, of course. But that's not what's being debated here.
The real disconnect is that the user doesn't really care all that much. It's mostly the designers who care. And Qt for example but also WPF let you style components almost to unrecognizable and unusable results. So if everyone will need to make do with 8GB for the foreseeable future, designers might just be told "No.", which admittedly will be a big shock to some of them. Or maybe someone finally figures out how to do HTML+CSS in a couple of megabytes.
I recently switched from Spotify (well known Electron-based app) to Apple Music (well known native app). The move was mostly an ethical one, but I must say, the UI functionality and app features are basically poverty in comparison. One tiny example, navigating from playlist entry to artist requires multiple interactions. This is just one of many frustrations I've had with the app. But hey, it has beautiful liquid glass effects!
In short: iteration time matters. Times from design to implementation, to internal review, to real user feedback, and back to design from each phase should be as fast as possible. You don't get the same velocity as you do in native. Add to that you have to design and implement in quadruplicate, iOS design for iOS, Android for Android, MacOS for Mac, Windows design for windows. All that is why people use Electon.
The point is you can be lazy and write the app in html and js. Then you dont need to write c, even though c syntax is similar to js syntax and most gui apps wont require needing advanced c features if the gui framework is generous enough.
Now that everyone who cant be bothered, vibe codes, and electron apps are the overevangelized norm… People will probably not even worry about writing js and electron will be here to stay. The only way out is to evangelize something else.
Like how half the websites have giant in your face cookie banners and half have minimalist banners. The experience will still suck for the end user because the dev doesnt care and neither do the business leaders.
But the point isn’t that they’re more different than alike. The point is that learning c is not really that hard it’s just that corporations don’t want you building apps with a stack they don’t control.
If a js dev really wanted to it wouldn’t be a huge uphill climb to code a c app because the syntax and concepts are similar enough.
We're not doing Electron because some popular software also using it. We're doing Electron because the ability to create truly cross-platform interfaces with the web stack is more important to us than 300 MB of user memory.
> We're doing Electron because the ability to create truly cross-platform interfaces with the web stack is more important to us than 300 MB of user memory.
It's closer to 1GB but trust me, everyone is well aware of your priorities.
You should check the memory use of that browser tab. You’re not saving much either way running in a browser or in Electron, which is effectively a browser.
Who cares about 300Mb, where is that going to move the needle for you? And if the alternative is a memory-unsafe language then 300Mb is a price more than worth paying. Likewise if the alternative is the app never getting started, or being single-platform-only, because the available build systems suck too bad.
There ought to be a short one-liner that anyone can run to get easily installable "binaries" for their PyQt app for all major platforms. But there isn't, you have to dig up some blog post with 3 config files and a 10 argument incantation and follow it (and every blog post has a different one) when you just wanted to spend 10 minutes writing some code to solve your problem (which is how every good program gets started). So we're stuck with Electron.
There's a world of difference between using a memory safe language and shipping a web browser with your app. I'm pretty sure Avalonia, JavaFX, and Wails would all be much leaner than electron.
The people who hate Electron hate JavaFX just as much if not more, and I'm not sure it would even use less memory. And while the build experience isn't awful, it's still a significant amount of work to package up in "executable" form especially for a platform different from what you're building on, or was until a couple of years ago. And I'm pretty sure Avalonia is even worse.
It likely would use less, and doesn't use a browser for rendering.
> And I'm pretty sure Avalonia is even worse
Definitely not
> The people who hate Electron hate JavaFX just as much if not more
In my opinion, I only see this from people that seem to form all of their opinions on tech forums and think Java=Bad. These are the people that think .NET is still windows only and post FUD because they don't know how to just ask for help.
If the alternative is memory-safe and easy to build, then maybe people will switch. But until it is it's irresponsible to even try to get them to do so.
Like what? Where else (that's a name brand platform and not, like, some obscure blog post's cobbled-together thing) can I start a project, push one button, and get binaries for all major platforms? Until you solve that people will keep using Electron.
Then again, after many, many years of claims that the following year would be the year of the Linux Desktop, there seems to be more and more of a push into that direction. Or at least into a significant increase in market share. We can thank a current head of state for that.
In practice, you generally see the opposite. The "CPU" is in fact limited by memory throughput. (The exception is intense number crunching or similar compute-heavy code, where thermal and power limits come into play. But much of that code can be shifted to the GPU.)
RAM throughput and RAM footprint are only weakly related. The throughput is governed by the cache locality of access patterns. A program with a 50MB footprint could put more pressure on the RAM bus than one with a 5GB footprint.
Reducing your RAM consumption is not the best approach to reducing your RAM throughput is my point. It could be effective in some specific situations, but I would definitely not say that those situations are more common than the other ones.
I don't understand how this connects to your original claim, which was about trading ram usage for CPU cycles. Could you elaborate?
From what I understand, increasing cache locality is orthogonal to how much RAM an app is using. It just lets the CPU get cache hits more often, so it only relates to throughout.
That might technically offload work to the CPU, but that's work the CPU is actually good at. We want to offload that.
In the case of Electron apps, they use a lot of RAM and that's not to spare the CPU
> increasing cache locality is orthogonal to how much RAM an app is using. It just lets the CPU get cache hits more often, so it only relates to throughout.
Cache misses mean CPU stalls, which mean wasted CPU (i.e. the CPU accomplises less than it could have in some amount of time).
> In the case of Electron apps, they use a lot of RAM and that's not to spare the CPU
The question isn't why apps use a lot of RAM, but what the effects of reducing it are. Redcuing memory consumption by a little can be cheap, but if you want to do it by a lot, development and maintenance costs rise and/or CPU costs rise, and both are more expensive than RAM, even at inflated prices.
To get a sense for why you use more CPU when you want to reduce your RAM consumption by a lot, using much less RAM while allowing the program to use the same data means that you're reusing the same memory more frequently, and that takes computational work.
But I agree that on consumer devices you tend to see software that uses a significant portion of RAM and a tiny portion of CPU and that's not a good balance, just as the opposite isn't. The reason is that CPU and RAM are related, and your machine is "spent" when one of them runs out. If a program consumes a lot of CPU, few other programs can run on the machine no matter how much free RAM it has, and if a program consumes a lot of RAM, few other programs can run no matter how much free CPU you have. So programs need to aim for some reasonable balance of the RAM and CPU they're using. Some are inefficient by using too little RAM (compared to the CPU they're using), and some are inefficient by using too little CPU (compared to the RAM they're using).
> Cache misses mean CPU stalls, which mean wasted CPU (i.e. the CPU accomplises less than it could have in some amount of time).
Yeah, I was saying CPU cache hits would result in better performance. The creator of Zig has argued that the easiest way to improve cache locality is by having smaller working sets of memory to begin with. No, it's not a given this will always work in every case. You can reduce working memory and not have better cache locality. But in a general sense, I understand why he argues for it.
> So programs need to aim for some reasonable balance of the RAM and CPU they're using
I agree with this, but
> but if you want to do it by a lot, development and maintenance costs rise and/or CPU costs rise, and both are more expensive than RAM, even at inflated prices
I would like you to clarify further, because saying CPU costs are more expensive than RAM costs is a bit misleading. A CPU might literally cost more than RAM, but a CPU is remarkably faster, and for work done, much cheaper and more efficient, especially with cache hits.
You had originally said
> It could be effective in some specific situations, but I would definitely not say that those situations are more common than the other ones
This is what I'm confused on. Why do you think most cases wouldn't benefit from this? Almost every app I've used is way on one end of the spectrum with regards to memory consumption vs CPU cycles. Don't you think there are actually a lot of cases where we could reduce memory usage AND increase cache locality, fitting more data into cache lines, avoiding GC pressure, avoiding paging and allocations, and the software would 100% be faster?
> But in a general sense, I understand why he argues for it.
Andrew is not wrong, but he's talking about optimisations with relatively little impact compared to others and is addressing people who already write software that's otherwise optimised. More concretely, keeping data packed tighter and reducing RAM footprint are not the same. The former does help CPU utilisation but doesn't make as big of an impact on the latter as things that are detrimental to the CPU (such as switching from moving collectors to malloc/free).
> Why do you think most cases wouldn't benefit from this?
The context to which "this" is referring to was "Reducing your RAM consumption is not the best approach to reducing your RAM throughput is my point." For data-packing, Andy Kelley style, to reduce the RAM bandwidth, the access patterns must be very regular, such as processing some large data structure in bulk (where prefetching helps). This is something you could see in batch applications (such as compilers), but not in most programs, which are interactive. If your data access patterns are random, packing it more tightly will not significantly reduce your RAM bandwidth.
> Andrew is not wrong... and is addressing people who already write software that's otherwise optimised
I'm getting lost. What are we talking about if not that? Because if you're talking about unoptimized software, you can absolutely reduce RAM consumption without putting extra load on the CPU. Using a language that doesn't box every single value is going to reduce RAM consumption AND be easier on the CPU. Which is what most people are talking about on this post.
> The context to which "this" is referring to was "Reducing your RAM consumption is not the best approach to reducing your RAM throughput is my point."
I'm more interested in the original claim, which was
> Using a lot less RAM often implies using more CPU
There are a lot of apps using a lot of RAM, and it's not to save CPU. So where is "often" coming from here? I think there are WAY more apps that could stand to be debloated and would use less CPU.
It feels like you're coming at this from a JVM perspective. Yeah, tweaking my JVM to use less RAM would result in more CPU usage. But I don't think there's a single app out there as optimized as the JVM is. They use more RAM for other reasons.
> If your data access patterns are random, packing it more tightly will not significantly reduce your RAM bandwidth
Packing helps random access too. A smaller working set means more of your random accesses land in cache. Prefetching is one benefit of packing, but cache and TLB pressure reduction is the bigger one, and it applies regardless of access pattern
> Using a language that doesn't box every single value is going to reduce RAM consumption AND be easier on the CPU. Which is what most people are talking about on this post.
What popular language does that? I admit that rewriting the software in a different language could lead to better efficiencies on all fronts, but such massive work is hardly "an optimisation", and there are substantial costs involved.
But more importantly, I don't think it's right. Removing boxing can certainly have an impact on RAM footprint without an adverse effect on CPU, but I don't think it's a huge one. RAM footprint is dominated by what data is kept in memory and the language's memory management strategy (malloc/free vs non-moving tracing collectors vs moving collectors), and changing either one of these can very much have an adverse effect on CPU.
> There are a lot of apps using a lot of RAM, and it's not to save CPU. So where is "often" coming from here?
That the developers may not be conscious of the RAM/CPU tradeoff doesn't mean it's not there. Keeping less data in memory (and computing more of it on demand) can increase CPU utilisation as can switching from a language with a moving collector to one that relies on malloc/free.
> Packing helps random access too. A smaller working set means more of your random accesses land in cache.
Unless your entire live set fits in the cache, what matters much more is the temporal locality, not the size of the live set. If your cache size is 50MB, a program with a 1GB live set could have just as many or just as few cache misses as a program with a 100MB live set. In other words, you could reduce your live set by a factor of 10 and not see any improvement in your cache hit rate, and you can improve your cache hit rate without reducing your live set one iota.
For example, consider a server that caches some session data and evicts it after a while. Reducing the allowed session idle time can drastically reduce your live set, but it will barely have an effect on cache locality.
Tighter data layouts absolutely improve cache behaviour, but they don't have a huge effect on the footprint. Coversely, what data is stored in RAM and your memory management strategy have a large effect on footprint but they don't help your cache behaviour much. In other words, Andy Kelley's emphasis on layout is very important for program speed, but it's largely orthogonal to RAM footprint.
Only if the software is optimised for either in the first place.
Ton of software out there where optimisation of both memory and cpu has been pushed to the side because development hours is more costly than a bit of extra resource usage.
The tradeoff has almost exclusively been development time vs resource efficiency. Very few devs are graced with enough time to optimize something to the point of dealing with theoretical tradeoff balances of near optimal implementations.
That's fine, but I was responding to a comment that said that RAM prices would put pressure to optimise footprint. Optimising footprint could often lead to wasting more CPU, even if your starting point was optimising for neither.
My response was that I disagree with this conclusion that something like "pressure to optimize RAM implies another hardware tradeoff" is the primary thing which will give, not that I'm changing the premise.
Pressure to optimize can more often imply just setting aside work to make the program be nearer to being limited by algorithmic bounds rather than doing what was quickest to implement and not caring about any of it. Having the same amount of time, replacing bloated abstractions with something more lightweight overall usually nets more memory gains than trying to tune something heavy to use less RAM at the expense of more CPU.
Some of the algorithms are built deep into the runtime. E.g. languages that rely on malloc/free allocators (which require maintaining free lists) are making a pretty significnant tradoff of wasting CPU to save on RAM as opposed to languages using moving collectors.
GC burns far more CPU cycles. Meanwhile I'm not sure where you got this idea about the value of CPU cycles relative to RAM. Most tasks stall on IO. Those that don't typically stall on either memory bandwidth or latency. Meanwhile CPU bound tasks typically don't perform allocations and if forced avoid the heap like the plague.
Far less for moving collectors. That's why they're used: to reduce the overhead of malloc/free based memory management. The whole point of moving collectors is that they can make the CPU cost of memory management arbitrarily low, even lower than stack allocation. In practice it's more complicated, but the principle stands.
The reason some programs "avoid the heap like the plague" is because their memory management is CPU-inefficient (as in the case of malloc/free allocators).
> Meanwhile I'm not sure where you got this idea about the value of CPU cycles relative to RAM
There is a fundamental relationship between CPU and RAM. As we learn in basic complexity theory, the power of what can be computed depends on how much memory an algorithm can use. On the flip side, using memory and managing memory requires CPU.
To get the most basic intuition, let's look at an extreme example. Consider a machine with 1 GB of free RAM and two programs that compute the same thing and consume 100% CPU for their duration. One uses 80MB of RAM and runs for 100s; the other uses 800MB of RAM and runs for 99s (perhaps thanks to a moving collector). Which is more efficient? It may seem that we need to compare the value of 1% CPU reduction vs a 10x increase in RAM consumption, but that's not necessary. The second program is more efficient. Why? Because when a program consumes 100% of the CPU, no other program can make use of any RAM, and so both programs effectively capture all 1GB, only the second program captures it for one second less.
This scales even to cases when the CPU consumption is less than 100% CPU, as the important thing to realise is that the two resources are coupled. The thing that needs to be optimised isn't CPU and RAM separately, but the RAM/CPU ratio. A program can be less efficient by using too little RAM if using more RAM can reduce its CPU consumption to get the right ratio (e.g. by using a moving collector) and vice versa.
There are (at least) two glaring issues with your analysis. First, the vast majority of workloads don't block on CPU (as I previously pointed out) and when they do they almost never do heap allocations in the hot path (again, as I previously pointed out). Second, we don't use single core single thread machines these days. Most workloads block on IO or memory access; the CPU pipeline is out of order and we have SMT for precisely this reason.
Anyway I'm not at all inclined to blindly believe your claim that malloc/free is particularly expensive relative to various GC algorithms. At present I believe the opposite (that malloc/free is quite cheap) but I'm open to the possibility that I'm misinformed about that. You're going to need to link to reputable benchmarks if you expect me to accept the efficiency claim, but even then that wouldn't convince me that any extra CPU cycles were actually an issue for the reasons articulated in the preceding paragraph.
> There are (at least) two glaring issues with your analysis. First, the vast majority of workloads don't block on CPU (as I previously pointed out) and when they do they almost never do heap allocations in the hot path (again, as I previously pointed out). Second, we don't use single core single thread machines these days. Most workloads block on IO or memory access; the CPU pipeline is out of order and we have SMT for precisely this reason.
This doesn't matter because if you're running a single program on a machine, it might as well use all the CPU and all the RAM. As long as you're under 100% on both, you're good. But we want to utilise the hardware well because we typically want to run multiple programs (or VMs) on a single machine, and the machine is exhausted when the first of CPU or RAM is exhausted. So the question is how should your CPU and RAM usage be balanced to offer optimal utilisation given that the machine is spent when the first of CPU and RAM is spent. E.g. you can only run two programs, each using 50% of CPU; if they each use only 5% of RAM, you've saved nothing as no third program can run. So if you spend either one of these resources in an unbalanced way, you're not using your hardware optimally. Using 2% more CPU to save 200MB of RAM could be suboptimal.
I'm not saying that for every program that uses X% CPU should also use exactly X% of RAM or it must be wasting one or the other, but that's the general perspective of how to think about efficiency. Using a lot of one and little of the other is, broadly speaking, not very efficient.
> Anyway I'm not at all inclined to blindly believe your claim that malloc/free is particularly expensive relative to various GC algorithms. At present I believe the opposite (that malloc/free is quite cheap) but I'm open to the possibility that I'm misinformed about that.
You are.
> You're going to need to link to reputable benchmarks if you expect me to accept the efficiency claim, but even then that wouldn't convince me that any extra CPU cycles were actually an issue for the reasons articulated in the preceding paragraph.
I don't believe there are any reputable benchmarks of full applications (which is where memory-management matters) that are apples-to-apples. I'm speaking from over two decades of experience with C++ and Java.
The important property of moving collectors is that they give you a knob that allows you to turn RAM into CPU and vice-versa (to some extent), and that's what you want to achieve the efficient balance.
Moving collectors as generally used are a huge waste of memory throughput, and this shows up consistently in the performance measurements. Moving data is very expensive! The whole point of ownership tracking in programming languages is so that large chunks of "owned" data can just stay put until freed, and only the owning handle (which is tiny) needs to move around. Most GC programming languages do a terrible job of supporting that pattern.
That's just not true. To give you a few pieces of the picture, moving collectors move little memory and do so rarely (relative to the allocation rate):
In the young generation, few objects survive and so few are moved (the very few that survive longer are moved into the old gen); in the old generation, most objects survive, but the allocation rate is so low that moving them is rare (although the memory management technique in the old gen doesn't matter as much precisely because the allocation rate is so low, so whether you want a moving algorithm or not in the old gen is less about speed and more about other concerns).
On top of that, the general principle of moving collectors (and why in theory they're cheaper than stack allocation) is that the cost of the overall work of moving memory is roughly constant for a specific workload, but its frequency can be made as low as you want by using more RAM.
The reason moving collectors are used in the first place is to reduce the high overhead of malloc/free allocators.
Anyway, the general point I was making above is that a machine is exhausted not when both CPU and RAM are exhausted, but when one of them is. Efficient hardware utilisation is when the program strikes some good balance between them. There's not much point to reducing RAM footprint when CPU utilisation is high or reducing CPU consumption when RAM consumption is high. Using much of one and little of the other is wasteful when you can reduce the higher one by increasing the other. Moving collectors give you a convenient knob to do that: if a program consumes a lot of CPU and little RAM, you can increase the heap and turn some RAM into CPU and vice versa.
>CXMT still trails Samsung, SK Hynix, and Micron by approximately three years in advanced DRAM node development, and yield rates on new production lines remain the variable that determines whether capacity targets translate into reliable supply. Liu notes that lines launched in the second half of 2026 are unlikely to change the global supply-demand balance until 2027.
The Verge article talks about demand exceeding supply in 2028. Your article suggests it'll take until 2029 before Chinese production catches up to current technology.
It'll help drive prices down in five yearss, but the Chinese memory production won't be ready and efficient enough to prevent the shortages from continuing to grow.
As an aside, recently I wanted to refresh my gaming PC, but the price shock and general lack of availability of buying components individually made it seem hardly worth it, so I just kept deferring the project.
Then, mostly by chance, I saw that my local Microcenter had some pre-builts for sale, and I ended up picking one up for <$5k that had "best in slot" components across the board, including a 5090 and even a high-end power supply.
The last time I built a gaming PC was upwards of a decade ago, and at that time the prevailing wisdom was to never buy a pre-built unless you had a massive amount of disposable income and couldn't spare even just one weekend to dedicate to a hobby project that could benefit you for years. Now, it was absolutely a no-brainer.
I'm struggling to put this in context. For comparison, what was your budget for refreshing the pc you had? Were the planned upgrades going to exceed $5k at current prices? Or is the situation that a pre-build machine with far better components was now only marginally more?
Or is it that pre-built gaming PCs have stopped being a joke? I had the experience building a bicycle: I was certain I was taking the frugal path sourcing each component individually and putting it together myself. At the end I was horrified to realize I spent far more than a new bike with superior components. It was pointed out that bicycle makers are buying by the pallet and will beat diy every time — so long as they're building something I want to buy.
I did the exact same thing during Covid, the prebuilt ended up being ~20% cheaper than buying the individual components (I needed a full upgrade). Maybe a little less since I could have reused my case.
> and at that time the prevailing wisdom was to never buy a pre-built
That's still the case, and always will be — with a pre-built you're at the very least paying for someone to assemble it for you, so it's always going to be more expensive as a baseline.
Beyond that, the chance they've chosen good components and haven't tried to screw you over on less flashy ones like the motherboard and power supply is low.
That's not to say it's literally impossible to ever find a good deal. You very well might have. Doesn't change anything though.
> with a pre-built you're at the very least paying for someone to assemble it for you, so it's always going to be more expensive as a baseline
Except isn't it possible that pre-built companies actually get better deals on hardware bought in bulk, and therefore could offset the labor costs with cheaper materials?
I believe this is exactly what's going on -- they're buying parts in bulk, often months in advance, and locking in deals that a single consumer can't easily go get on the open market right now.
Hardware pricing and availabilty pre-COVID was pretty predictable and stable, which meant the consumer could extract a meaningful cost advantage if they were willing to do the relatively modest amount of work of sourcing components individually and personally assembling the build. Right now, though, some places like Microcenter appear to have a cost advantage that fundamentally relies on market and pricing instability and can only be achieved through deeper integration with the supply chain and bulk purchasing in advance -- something a retailer like Microcenter can do, but I personally cannot.
I'm skeptical - the apps I use either have a) enough lock-in that they don't have the institutional will to optimize or b) a lack of institutional resources to optimize.
Basically, the optimizing that can happen is that I ditch heavy tools in favour of lighter ones, and hopefully enough other people do the same to help lighter tools with finances/dev resources.
Yeah, I mean, if I look at the trashy apps I currently have running, they all do have deep lock-in that makes the reverse-engineered clones impractical or impossible: Teams, Discord, Slack, the Windows RDP app.
If I look at the Activity Manager in macOS, of apps that are less trashy but currently taking up a lot of memory, they mostly aren't apps that I'm willing/able to move away from to save on resource use: Firefox, Safari, 1Password. (For browsers, you can blame poorly optimized websites for a lot of it, but I just don't see anyone rushing to create lightweight clones of websites in order to save users' RAM.)
I said it for many years that OS developers need to focus on over optimisations. If it wasnt a chip sgortage it would be the ever slowing progress on chip scaling.
But software optimisation helps all hardware and that doesnt drive sales.
Linux however, they dont have to worry about that. Maybe it is finally the era of Haiku OS as the ghost of BeOS rises!
There is almost no way any hyper optimized OS could protect you from Windsurf re-layouting the whole UI 60 times per second because of badly implemented spinner, and taking 8 GB of RAM to show it.
I think RAM shortages would be the least of our problems…
Assuming China takes TSMC in one piece (unlikely without internal sabotage in the best case scenario), it would still probably take years before it produces another high end GPU or CPU.
We would probably be stuck with the existing inventory of equipment for a long time…
I am surprised we consider TSMC like a natural resource: isn't it really a combination of know-how and build-out according to that know-how? If smarts leave the country, perhaps this moves with them.
The risk with China taking over Taiwan is that they mostly expedite their own production research by a couple of years.
It kinda does resemble a natural resource though. The machines and technology in use at TSMC are so insanely complex, that there isn't a single person on earth who knows everything about how it works. TSMC functions only because of all of the pieces of the puzzle being together in the right place and arranged in just the right way. It's a very fragile balance that keeps it all running, and a major disruption could mean we get thrown back by a decade in chip-making technology.
What you say is absolutely true, and is a serious problem—but the way our system operates does not allow us to correct for it.
Anyone trying to spin up a competitor to TSMC would have to first overcome a significant financial hurdle: the capital investment to build all the industrial equipment needed for fabrication.
Then they'd have to convince institutions to choose them over TSMC when they're unproven, and likely objectively worse than TSMC, given that they would not have its decades of experience and process optimization.
This would be mitigated somewhat if our institutions had common-sense rules in place requiring multiple vendors for every part of their supply chain—note, not just "multiple bids, leading to picking a single vendor" but "multiple vendors actively supplying them at all times". But our system prioritizes efficiency over resiliency.
A wealthy nation-state with a sufficiently motivated voter base could certainly build up a meaningful competitor to TSMC over the course of, say, a decade or two (or three...). But it would require sustained investment at all levels—and not just investment in the simple financial sense; it requires people investing their time in education and research. Dedicating their lives to making the best chips in the world. And the only reason that would work is that it defies our system, and chooses to invest in plants that won't be finished for years, and then pay for chips that they know are inferior in quality, because they're our chips, and paying for them when they're lower quality is the only way to get them to be the best chips in the world.
> A wealthy nation-state with a sufficiently motivated voter base could certainly build up a meaningful competitor to TSMC over the course of, say, a decade or two (or three...).
> I am surprised we consider TSMC like a natural resource: isn't it really a combination of know-how and build-out according to that know-how?
Have you seen how many states and countries look enviously at Silicon Valley’s tech companies, China’s manufacturing dominance, or London’s financial sector and try to replicate them?
Turns out it’s way harder than you’d expect.
Hell, Intel can’t match TSMC despite decades of expertise, much greater fame, and regulators happy to change the law and hand out tens of billions in subsidies.
Does China need to take Taiwan? American protection of the island seems to be a waning concern. I could see that being a peaceful event now. https://www.bbc.com/news/articles/cj94y87k2ljo
They've been burned before. The DRAM industry has a long history of booms and busts.
Demand increased, everyone built new fabs, then prices dropped and they couldn't pay off their investments. Many went out of business. It happened in the 80s, it happened in the 90s, it happened in the 2000s.
Now there's only three manufacturers left, and they know very well that demand for their product tends to be cyclical.
What’s the lose scenario for them? They’re basically a cartel, and you need ram irregardless. If they make less it’s still a cost:demand, just not the most optimal for them. They’ve done that math, and figure this is the best risk and reward for them. Your goodwill or opinion doesn’t matter to them, because you need them more than they need you.
The same thing everyone who's paying attention to the real world (and not the financial fantasy world) does: that OpenAI's purchase commitments are wildly unrealistic and unsustainable.
I've been in the industry for 30 years and I've worked at companies with fabs were demand was high and customers would only get 30% of what they ordered. Then just 2 years later our fab was only running at 50% capacity and losing money. It takes about $20 billion and 3-4 years to make a modern new fab. If you think that AI is a bubble then do you want to be left with a shiny new factory and no products to sell because demand has collapsed?
I wonder if this might motivate to write more memory efficient software. I mean we have so much memory, but even some trivial programs eat hundreds of megabytes of ram.
I fear that the real reason we do have a shortage, I mean, the real reason for the demand, is AI companies scooping what they can so that their competitors, whether existing or incumbent, can’t get to it.
It will last forever. After covid, all manufacturers understood the value of limiting supply and extracting profits. Cars used to super cheap before covid, they will never go back to the same levels.
From now on, RAM will always be super costly for consumers, because they can't make massive deals like Apple/OpenAI/etc. We are the bagholders.
When lithium prices decreased over 80% from 2022 to 2025, it was because lithium miners felt altruistic. Car manufacturers were feeling greedy. This is how bad the thinking has gotten.
It won't. DRAM prices are cyclical. They were super high in 2020, then demand crashed in 2022 to the point that manufacturers couldn't sell all their inventory.
Now it's high again, but give it a couple years and it'll once again crash.
Covid inflation was because of supply chain disruptions, loose fiscal policy (like Biden's ARP which a Central Bank analysis said added a few % to annual inflation), and money supply expansions. There was less goods and more money. When you go and trade money for goods, it should be obvious what happens.
if a shortage lasts years, it's not a shortage. "The market clearing price of RAM in the face of expected sustained healthy demand should lead to a stable market for years."
even if gaming is and will remain very popular for years, it and the desire to upgrade gaming rigs is still a discretionary activity with more price elasticity of demand than corporate uses for RAM in the dawn of the AI age. gamers live on the margin of this market, where low prices will stimulate upgrades and high prices will lead to holding out. The complaints about price are real, but that segment of the market is some combination of less large and less important.
It’s not merely a “gaming vs data center“. There’s so many other places DRAM and NVM are needed - mobile, automotive, other consumer electronics,… the current situation is that _all_ of that is deprived of the memory that it needs. And much of this is critical to the real economy.
Why are you only talking about gamers? Apple, the most cautious planners in the whole industry have straight up cancelled their 512gb RAM Mac Studio. Don’t ask; they won’t sell you one.
I've read that the chip manufacturers are looking into high bandwidth flash for on package storage of ai models. That would solve some of the cost issue, flash is significantly cheaper than dram.
I'm personally hoping that one of the AI or data center companies is suddenly unable to pay for their bills and deflate the entire industry. Probably the only hope of things getting better before the 2030s.
Of course, alternatively, the AI companies could go bust before finding profitability. Then, there’d be a ton of supply, prices would crash, and one or two of the current memory suppliers would go out of business. After that, the new Chinese memory companies might be producing at volume, and Renesas could be up and running.
At the moment, nothing is certain. Could this last? Sure. Could it not last? Yup.
let the analyst and news say what they want - the entire situation is artificial and is up to the manufacturers.
the current relative spike in the prices misses the medium-term trend of the vast decrease in memory price post-covid that led to the recent surge. the cartel got another opportunity to make bank and they will use that lever to the max.
funnily enough i've been personally stuck with 16 gigs since 2015, across three memory generations! but i am used to the past when you would spend 80-100 on an 8gb stick (jdec timings, nothing fancy but from a major brand) without accounting for inflation.
Your instincts are likely right on this one OP. Memory prices surged 80–90% in Q1 2026 compared to Q4 2025, DRAM, NAND, and HBM all at record highs. 3 suppliers for the entire planet?
This is simple extrapolation from current demand, nothing more. And that's a borderline silly analysis because it assumes the AI bubble won't burst. The great misadventure in the Persian Gulf probably accelerates that because we're almost certainly going to be facing a recession.
Another thing I've been thinking about is what happens when the next generation of NVidia chips comes out? I suspect NVidia is going to delay this to milk the current demand but at some point you'll be able to buy something that's better than the H100 or B200 or whatever the current state-of-the-art for half the price. And what's that going to do to the trillions in AI DC investment?
I'm interested when the next bump in DRAM chip density is coming. That's going to change things although it seems like much of production has moved from consumer DRAM chips to HBM chips. So maybe that won't help at all.
I do think that companies will start seeing little ot no return from billions spent on AI and that's going to be aproblem. I also think that the hudnreds of billions of capital expenditure of OpenAI is going to come crashing down as there just isn't any even theoretical future revenue that can pay for all that.
I just checked my gaming PC I built a few years ago with 64GB of DDR5 RAM, its actually gone up in value, that is unheard of generally.
Think I will scrap my PC and sell its parts.
I wonder if there are any niche companies building decent rigs with DDR3 and 5/6th generation Intel CPUs out there, it is cheap and might be a business opportunity?
I work at an e-waste recycling company. I have several dozen trays of RAM in my inventory, ~90% of it DDR3. DDR3 was selling as of a month ago, but I haven't tried to sell any RAM since. I'm looking forward to doing a huge one this week.
Same. Core i7 2600K clocked to 4.4GHz with 32GB DDR3. It still does its job as my stationary DAW, and basically handles anything DAW-related I throw at it with ease. The only issue is its lack of AVX2 support, and since this is required by Ableton Live 12, I'll be stuck at Ableton 11 forever.
What do you mean, in 5 years? It's not like everyone just bought a new computer. My gut says it's exactly the other way around: most computers are old. They may fail as soon as today.
There is enough older hardware floating around to last us for decades. You don't need a gaming rig to do 99% of your computing (excluding gaming obviously). Also computers don't really just break. It's mostly the disks that wear out and PSUs that age.
When I personally use chatgpt and friends, I am not seeing any slowdowns or anything, meaning that their servers can handle the loads just fine. So then, why are these companies spending so much building new capacity if the current capacity is enough?
Frontier labs flagship models are ~2T params at the moment, but they intend to ship 10T models like Claude Mythos, which would require substantial datacenter expansion. Same thing for training.
There's a future where RAM makers tool up for this massively increased demand, then the AI companies go broke as the bubble bursts, so RAM is cheap as. So laptop manufacturers get on that and start making laptops with 1TB+ memory so we can run decent LLMs on the local machine. Everyone happy :)
RAM makers are not increasing their capacity. If AI bubble bursts we might see a momentary drop in RAM prices but it won't be dramatic. Return to "normal" is the best scenario I can imagine but my gut tells me we're probably never going back to early 2025 memory prices.
I fear the author and most commenters are not aware of the law of demand and supply. If there is demand for consumer RAM, there will be supply for consumer RAM. It just takes time and risk-assessment to scale up operations.
We have RAM shortage now, we will have very cheap RAM tomorrow. It’s not like production is bottlenecked by raw materials. Chip companies just need to assess if the demand by AI companies will last so it’s better to scale up, or perhaps they should wait it out instead of oversupplying and cutting into their profits.
We're talking about advanced semiconductor manufacture. It takes years and 100s millions to billions of dollars to scale up operations. That's something you don't do unless you know there's demand to sustain it in future.
>I fear the author and most commenters are not aware of the law of demand and supply
I cannot stand how you and people like you try to justify everything by supply and demand. Also you act like it's some natural law of nature. It's not a law of nature- if you took an economics class you would realize it's try to maximize PROFIT. It's not for the good of the people.
All of these things are a CHOICE that people are making to now completely screw the average person for, again, the needs of big corporations and the top 0.01%.
Ok so Samsung, SK Hynix and Micron do not have the capacity to meet demand. Also, what little capacity they do have they are allocating to HBM over DRAM. Based on my limited knowledge HBM can not be easily repurposed for consumer electronics. Translation: main street is cooked for the next 3-4 years.
It doesn't stop there though. OpenAI is currently mired in a capital crunch. Their last round just about sucked all the dry powder out of the private markets. Folks are now starting to ask difficult questions about their burn rate and revenue. It is increasingly looking like they might not commit to the purchase order they made which kick-started this whole panic over RAM.
Soo ... how sure are we that the memory makers themselves are not going to be the ones holding the bag?
The Radeon VII came out in 2019 as a $700 consumer GPU with an 1TB/s HBM2 memory subsystem which is more than any consumer GPU you can get today, including the high-end ones afaik. At that point in time, there was a whole lineup of AMD GPUs with HBM going down into the midrange.
If they could make this stuff and sell it to regular people a decade ago for very palatable prices, why do they come up with the idea that this is the technology of the gods, unaffordable by mere mortals?
I have been wondering this recently. It was the convention that if you wanted to keep costs down, try to keep the memory bus size down as low as possible. Still remember the awful Radeon 9200 SE - 64bit data bus that strangled an already slow GPU.
Heck, I have a phone with a 16bit memory bus for instance. The high(ish) clock rate only makes up the difference slightly.
But with general prices on all components going up, it might not be such a big factor any more.
HBM migght make sense for higher end products which can free up space for the lower end that will never use the tech.
Eh I feel like the memory bus width thing was more a case of binning memory controllers and the like.
Designing a part with a wide bus and putting the traces down on the board is what I would expect to be the easy part these days (surely).
But yield, yield comes for us all.
[dead]
> why do they come up with the idea that this is the technology of the gods, unaffordable by mere mortals?
because the gods want it all and are willing to pay top dollar.
Isn't this the case of money going from left pocket to the right, since these companies are owned by the same investment funds?
I wonder whether this is some kind of a racket.
"Owned"? You mean they invested
Investors are owners, yes.
> Isn't this the case of money going from left pocket to the right, since these companies are owned by the same investment funds?
No.
Yes, it's interesting that HBM was invented by a collaboration between AMD and SK Hynix. It seems, HBM is the way to go for GPUs, anyway.
The GB202 die that's in the GDDR7 based RTX 5090 and RTX 6000 Pro literally needed to be this big to support the 512bit memory bus. It's probably only getting worse with smaller node sizes. (see https://www.youtube.com/watch?v=rCwgAGG2sZQ&t=65s).
BTW: The 1TB/s is matched by RTX4090 and surpassed by the RTX5090 (1,79 TB/s).
> 1TB/s HBM2 memory subsystem which is more than any consumer GPU you can get today
5090 has 1.8 TB/s?
5090 is an overpriced outlier. A typical consumer GPU, like RTX 5070, has a 3-times lower memory throughput.
Even a RTX 5080 has a lower memory throughput than a Radeon VII from 2019, 7 years ago, while being much more expensive.
The memory throughput of GPUs per dollar has regressed greatly during the last 5 years, despite the fact that the widths of the GPU memory interfaces have been reduced, in order to decrease the production costs.
RTX 5080 has a 256-bit memory interface, while the much cheaper Radeon VII had an 1024-bit memory interface. RTX 5080 has almost 4-times faster memories than Radeon VII, but it has not used this to increase the memory throughput, but only to reduce the production costs, while simultaneously increasing the product price.
> Even a RTX 5080 has a lower memory throughput than a Radeon VII from 2019, 7 years ago, while being much more expensive.
And it's faster for gaming, I guess? Which is what matters for the typical user.
Anyway you can buy much faster GPUs now than in 2019. They are also much more expensive, yes.
Modern GPUs like RTX 5080 are much faster for the applications that are limited by computational capabilities, mainly because they have more execution units, whose clock frequencies have also increased.
I suppose that most games are limited by computation, so they are indeed much faster on modern GPUs.
However, there are applications that are limited by memory throughput, not by computation, including AI inference and many scientific/technical computing applications.
For such applications, old GPUs with higher memory throughput are still faster.
This is why I am still using an old Radeon VII and a couple of other ancient AMD GPUs with high memory throughput.
Last year I have bought an Intel GPU, which is still slower than my old GPUs, but it at least had very good performance per dollar, competitive with that of the old GPUs, because it was very cheap, while the current AMD and especially NVIDIA GPUs have poor performance per dollar.
then it must be the case you can't get one (for a fair price?)
Define "fair price"
5090s are certainly expensive compared to most other GPUs, but not expensive enough to be unobtanium for nearly any professional who could utilize one as part of their job
Hell, some of us utilize them just to play video games!
I was gonna say, I still use an AMD Vega that uses HBM2.
My main system still uses a Vega 64 and it plays all the games I'd care about. Undervolts like a champ! Will use it until it dies..
Mine is a Vega 56 that thinks it's a Vega 64. It plays Hitman pretty well, and thats really all I need it for.
Vega was a card with decent perf/$ for the consumer, but from a pure technical point of view (perf/mm2, perf/BW, perf/W) it was a major failure. Both Vega (and Fiji before it) showed that excess memory BW alone is not sufficient to win.
> Both Vega (and Fiji before it) showed that excess memory BW alone is not sufficient to win.
That's correct if you're targeting gamers, but local AI inference changes this picture substantially.
Oh I never said it was a great card. It does work for what I need it for though.
And the bottleneck at the time was HBM interposers, not actual ram dies.
The fury X was a beast for a consumer card, sadly limited by having a mere 4gb of (hbm) VRAM, and a non-refillable AIO. But back when games could fit in 4gb, it was incredible.
I'd absolutely buy another hbm consumer GPU if it had at least 8gb (and if I got the vibe/hope AMD will actually support for a couple years...)
Supply and demand. The prices are high because of high demand.
That card only had 16GB of memory; its memory bandwidth was 1TB/s.
The Pro variant had 32GB, I had one in a 2019 Mac Pro
You're saying this in a world where AMD's highest end consumer GPU in 2026 is also limited to 16 GB.
RX7900 XTX has 24GB
this card is 4 years old, it's not on store shelves anymore.
FWIW that depends on the stores you're looking at. There are three models from different manufacturers available here in a few shops. The prices are a bit ouchier than what i paid for mine around Christmas 2024 though (i got mine on a sale).
You can still get "new" ones on amazon in europe.
7900XT has 20GB and you can still get some unused ones.
R9700 has 32GB and is cheaper than most NVidia consumer GPUs, even though it's a "pro".
And I can still buy a new W7800 48GB for a relatively decent price.
It also does 64 bit floating point I think?
After NVIDIA essentially removed FP64 from consumer GPUs (their 1:64 performance ratio is worse than what you can obtain by software emulation, so it is useless, except for testing programs intended to run on datacenter GPUs), AMD persisted for a few years, but then they also followed NVIDIA.
AMD Hawaii GPUs still had 1:2 FP64:FP32, while the consumer variant of Radeon VII dropped to 1:4. The following AMD consumer GPUs dropped the FP64 performance to levels that are not competitive with CPUs.
Nowadays the only consumer GPUs with decent FP64 performance are the Intel Battlemage GPUs, which have a 1:8 performance ratio, which provides very good performance per dollar.
the hardware dump once we're through the bubble is gonna be wild
[flagged]
What do you mean exactly by “Epstein People”? I thoroughly hope it’s not the 4chan definition of it
A RAM discussion descends into cranks referring to "Epstein People" and you're the one downvoted - website's absolutely cooked.
Only indirectly. They have most of the money, so if they want something that’s in short supply, the price will rise to the point that it becomes unaffordable to everyone else.
Reason number 7,322 why US-style ultracapitalism is self-destructive, anti-social, and dystopian.
To add a more local hurdle as well, the Dutch power grid is at capacity and its managing company is now telling companies that planned to build a datacenter that they can't be connected to the grid until 2030, even though said companies already paid for and got guarantees about that connection.
That is, memory capacity is reserved for datacenters yet to be built, but this will do weird things if said datacenter construction is postponed or cancelled altogether.
That guarantee is not as much of a guarantee as stated in the media. You get a guarantee it will be planned at a certain time (as in looked at), not that it will be build. The cost of doing business is taking risks and mitigating them. There is a reason the nuclear plant in Borsele was build: an aluminium smelter. Maybe you should arrange for something similar as a datacenter (no politician will fall on a sword for that but you can try). The (original) power draw is about the same 80-100MW.
I'm not disagreeing with your post, but I did a quick check here: https://en.wikipedia.org/wiki/Electricity_sector_in_the_Neth...
It says that in 2025, Netherlands was a net exporter of electricity (~14,000 GWh). My guess: Where they want to build data centers, the grid cannot handle it, but the overall system has more than enough power to build data centers. Do you think that sounds like a resonable guess?
> the Dutch power grid is at capacity and its managing company is now telling companies that planned to build a datacenter that they can't be connected to the grid until 2030, even though said companies already paid for and got guarantees about that connection.
Are the Netherlands a large proportion of global datacenters?
Amsterdam hosts a major internet exchange. It's not a bad place to build a datacenter and there are many. Northern latitude brings free air cooling, but also additional distance to clients. Lots of peers in AMS-IX, but not a lot of oceananic cable landings (one with two paths to the US, but most of the submarine cables land nearby in Europe)
Whether it's generally a reasonable place to build them isn't the percentage. The number seems to be ~3%.
3% of the global total does seem surprisingly significant.
London is about 3X bigger than Amsterdam in terms of capacity. If you look at the core western europe market (so called FLAP-D) which I am rather familiar with London holds 35% of that market and Amsterdam about 12%. I'm not at all sold that the old rule of thumb that this market is about 25% of global capacity has been true since about 2023 because of the AI buildout in NA/ME.
Yes. Amsterdam has one of the largest IXPs (AMS-IX) in Europe and is also one of the largest European markets for Internet Infrastructure services (i.e. hosting, DNS provision, domain name registration, etc.)
And all of these are practically irrelevant for AI data centers.
Do AI data centers not need internet connectivity anymore?
The value of an IX isn't just in the IX itself, but also in the presence of hundreds of parties for direct peering, and excellent connectivity to the rest of the world.
It makes a lot of sense to build your DC near one - even if you have no intention of actually participating in the IX itself.
> Do AI data centers not need internet connectivity anymore?
They don't need entire IX worth of connectivity. You're sending mostly text back and forth and any media is in far lower volume than even normal far less dense DC would generate, all the major traffic is inside the AI DC.
All it needs is fiber to nearest IX
Is that relevant? The grid in every country is getting ridiculously stressed by datacenters.
What the grid looks like in different countries is very different. The Dutch power grid is already almost 50% renewables, which is an inconvenience for adding capacity because that's around where you have to start really dealing with storage in order to add more.
In most other places the percentage is significantly less than that and then you can easily add more of the cheap-but-intermittent stuff because a cloudy day only requires you to make up a 10% shortfall instead of a 50% one, which existing hydro or natural gas plants can handle without new storage when there are more of them to begin with.
I calculate about 43.5% was solar or wind. What is way crazier is the "bend in the curve" of production sources in the last 10 years. Look here at how fast solar and wind is growing! https://en.wikipedia.org/wiki/File:Netherlands_electricity_g...
I don’t think the source of the electricity is particularly relevant to whether or not you have the transport capacity to add tens of megawatts of demand to the grid. The problem is generally not the supply but whether your local transformers have capacity left.
When you're talking about something that draws megawatts existing transformers are pretty irrelevant because you're going to run high voltage lines directly to the site itself and install new dedicated transformers on site.
What's more common is that they don't have the transmission capacity itself, but that one's pretty easy in this case too, because what that means is that you have an existing transmission line which is already near capacity with generation on one end and customers on the other. So then you just build the data center on the end of the transmission line where the generation is rather than the end where the existing customers are, at which point you can add new generation anywhere you want -- and if you put it near the existing customers you've just freed up transmission capacity because you now have new customers closer to the existing generation and new generation closer to the existing customers.
High-level, I would agree with you. One thing that blows me away: I think I read that Northern Virginia, USA has the highest data center density in the world. Mostly it is due to demand from US gov't, military, and spy agencies (like NSA). How did they do it? In mainstream media, I don't see any news about a stressed power grid in this area. I guess the US gov't carefully coordinated with local power providers to continuously upgrade their power grid? This is a real question. It makes no sense to me. No shilling/trolling here.
> I think I read that Northern Virginia, USA has the highest data center density in the world. Mostly it is due to demand from US gov't, military, and spy agencies (like NSA).
That's where AWS us-east-1 is, i.e. the oldest AWS region where they got started to begin with. Google and Microsoft also have a large presence there. It's not just the US government, it's everybody, and it's not new.
> How did they do it?
Here's the US nuclear plant map, guess where a bunch of them are:
https://www.eia.gov/todayinenergy/detail.php?id=65104
The area around Virginia is also a major coal producer and when this was getting started it was a source of cheap electricity, but coal is quickly being replaced with natural gas via pipelines from the Gulf coast. Their current power mix is ~30% nuclear, ~12% renewables (solar) and almost all the rest natural gas.
> I don't see any news about a stressed power grid in this area.
That's because you don't live in Maryland.
Our energy bills are through the roof and our transmission company is talking about rolling blackouts in 2027.
https://www.thebanner.com/community/climate-environment/cont...
Well, they have done pretty well for 20 years of planning. Google tells me that AWS us-east-1 region (Northern Virginia) was started in 2006!
EDIT
The opening paragraph:
I bet this is pure NIMBYism. Just this phrase alone is a dead giveaway: "controversial power line". LOL: What is controversial about a power line? Hint: They aren't, but NIMBYism exists.Sure, just pointing to some news coverage in which grid operators allege that the grid is stressed in the DMV area.
The grid is constantly "stressed" everywhere, because they're designed to be. You don't build five times as much capacity as you need at huge expense for no reason, you design the thing to have only slightly more capacity than you expect to need.
If demand increases you have to build more generation and power lines etc. This is not a problem but for NIMBYs, it's just the logical consequence. If the local population increases and you don't have enough grocery stores you don't say "grocery stores are stressed" and regard it as an insurmountable problem, people just open more of them.
Chef's kiss for this reply. It is perfect. I like how you reframed the argument around grocery stores. It would be ridiculous to write a newspaper article about "grocery stores are stressed" and there is a newly proposed "controversial grocery store". NIMBYists will perform all kinds of mental gymnastics to defend their positions.
>The grid in every country is getting ridiculously stressed by datacenters.
In every country? Citation needed.
I think Denmark is in the same situation, and recently gave a timeline of 10 years for new projects to get connected to the grid.
That year 2030, man I'm feeling weird on that timeline in general...
I know that you comment is midly off topic, but I am going through the same out of body experience each time I see a major project announces an opening date of 2030 or later.
This just highlights what an utter failure and self-inflicted wound the green policies of Euro countries have been. Europe has already lost the AI race to the U.S. and China.
Renewables is the only realistic path to energy independence. Today's global situation should show the absolute necessity of that, even if you dont give a sh*t about the environment.
Had we done more 10 years ago we would have been better of. The second best time to start is now.
I mean nuclear provides that too.
(We used to build it at a fraction of the cost and less than half of the time that we do with our modern fuckups and fuel can come from just about anywhere if need be. It might be a lot more expensive than the stuff kazachstan and still be a fraction of the cost.)
I think ideally we would've done both to press the cost of nuclear down and given the fact that the renewables rollout turned out to be a lot lot more expensive than proponents claimed it would be whilst still tying us up into gass to cover winter.
Yes, we should definitely optimise for the most expensive form of electricity: https://ca.finance.yahoo.com/news/bjorn-lomborg-solar-wind-p...
When reading an article written by Bjorn Lomborg, you should also do the effort to read the cited sources. This is not an ad hominem attack, just an observation. Do it and you will see.
Europe still has coal, lots of it, not using it is a political choice and a self-inflicted wound.
It does not. That is not economically mined. Last big hard coal producer in EU - Poland, has extraction cost x2 or x3 of the mountain top removal mining in US. This sector is shrinking rapidly. Poland coal production came back to ~1915 levels (taking into account current PL territory). This sector would be closed already if not for massive subsidies.
Last year, China's coal use decreased, while China installed 300x more renewables than nuclear. Coal and nuclear aren't cost competitive with renewables, either in a free market or a technocratic top-down economy. Coal and gas still maintain a valid niche of firming intermittency. But that niche is temporary and shrinking.
Of all the coal consumed per year, China uses half of it. They are not a green economy.
They are greening fast, and enabling greening of others through cost competitive supplies.
> either in a free market o
Then why all the anti-coal mining diktats coming down from Brussels?
Large Combustion Plant Directive: coal incompatible with both traditional air quality measurements and CO2 emissions.
Renewables deployment is happening fast. Grid upgrades are not. Batteries .. it depends.
Even nuclear darling France has set solar records: https://www.pv-magazine.com/2026/04/15/france-germany-set-da...
The free market installs a tiny amount of coal, and a lot of renewable energy. Whether you believe this means "coal is/isn't cost competitive with renewables in a free market" is a debate about word definitions that I'm not terribly interested in.
Brussels is trying to reduce "tiny" to zero, because of this: https://en.wikipedia.org/wiki/Tragedy_of_the_commons
China, like Brussels, is trying to reduce coal for similar reasons. They don't like the air pollution health hazard (fully believable), and they say they don't like global warming (somewhat believable).
China and Texas are both installing silly amounts of renewables. They install very little new fossil fuels or nuclear. They both maintain cheap electricity prices through abundance.
The problem in the EU is not renewables, it's the same problem that Democratic states in the US face. Regulations and permitting hurdles that block private renewable energy developers.
China is building a lot of coal power plants
Their coal generation decreased last year. They're building on the order of 70GW of new coal while they decomission or underutilized more than 70GW of pre-existing coal. Meanwhile they installed 450GW of new renewables energy.
They also consume half of all coal consumed globally per year. They are in no way a green economy.
Not relevant to the question of which energy source makes sense to build in the year 2026. But sure China has many coal plants left over from 2003 when renewables was more expensive, nobody would dispute that this is a fact, however irrelevant.
There appears to be zero advantage to having the datacenter actually in your country apart from minor local property tax, in exchange for which it will put up the electricity bills of every single citizen, who already hate how much they're paying.
The only way to lose even harder would be to build a shitload of datacenters for them
Don’t the memory makers always get left holding the bag? I feel this has happened at least three times before.
DRAM and to a lesser degree storage are notorious for their feast and famine cycles
(Well that and collusion)
The problem in this case seems to have sprung from a lack of collusion. Altman reportedly approached Samsung and SK independently to strike deals for a large chunk of both companies' production. Neither party apparently knew he was negotiating with the other.
If they had actually been communicating or colluding with each other, they would have put the screws to him, making it harder for OpenAI to assert control over the vast majority of the DRAM market.
Failing that, you'd like to think a regulatory agency somewhere would step in to keep a single player from hosing everybody else, but...
> Failing that, you'd like to think a regulatory agency somewhere would step in to keep a single player from hosing everybody else, but...
Up until AI there weren't really players being able to gobble 40% of the market so nobody was looking.
> Neither party apparently knew he was negotiating with the other.
I don’t buy it that two of the largest manufacturers of DRAM in the world, from the same country, didn’t know this. Even of you ignore each company’s intelligence teams, that’s also the job of the country’s internal intelligence services, to make sure they know what all companies are doing and then make it so they have the best leverage to gain as much as possible. Both companies would have known “somehow” and played hardball.
Wut? How they [gov] would know that?
By spying?
How would the country’s internal intelligence services know what’s happening? Yes, by spying. That’s literally their job and they have assets in every critical area in a country. Every institution, every major industry player, they are monitored to a degree by the internal intelligence in every country in the world. There are more nefarious reasons to do this but the ostensible one is that if it’s of strategic importance the country needs to know everything there is to know.
The companies also do a lot of spying themselves, every bit of info could give them an edge.
There are a lot of cyclical businesses that make money every year. It requires careful management. Factories can produce less than full capacity - but you better design for that. you can make money in the worst years without laying anyone off even - but it requires careful attention to details and not over hiring in good times as if they will never end.
Factories working at (significantly) less than full capacity gets a bit harder when you've got one of the most expensive machines on earth working in them, and production lines that'll be out of date in a couple of years
the normal way to do that is by hiring/firing to meet demand, but in the fab business, you have 10s of billions of dollars of capex with relatively little opex. if you're running at <90% capacity, you're losing money.
That is the common way, but there are companies that manage without hiring/firing. (or they hire temp workers). There is a minimum level of capacity you need to run just to keep the lights on, and figuring out how to get that low without impacting your ability to serve the highs is hard. Memory manufactures have not gotten very low, probably for good reasons, but it is something they should work on.
Last year ai folks are all over wallstreet and articles, decrying how hardware folks are a roadblock to new frontiers in AI. They just couldn't print and pack the new chips faster.
I don’t know if they realize that collusion lends itself to feast/famine.
but if you don't collude during times of feast you will have famine, and during times of famine you will have famine, in an economy based on feast/famine you must sometimes feast or die.
All of the capital intensive businesses face this issue. Chemicals, Shipping, Semiconductors etc.
You get market signals that the demand is there, you acquire the necessary capital, you spend 5 years to build capacity, but guess what, 5 other market players did the same thing. So now you are doomed, because the market is flooded and you have low cash flow since you need to drop prices to compete for pennies.
Now you cannot find capital, you don't invest, but guess what, neither your competitors did. So now the demand is higher than the supply. Your price per unit sold skyrocketed, but you don't have enough capacity!
Rinse and repeat.
Capitalists claim that this is optimal.
The book Capital Returns: Investing Through the Capital Cycle details this phenomenon, including historical cases.
If anything, it shows it's possible for you to arbitrage this and in doing so help "smooth out the cycle."
Forecasting demand 5 years into the future is intrinsically highly unreliable. It doesn’t matter if it is capitalism or a command economy. The bet is always going to be risky and someone will have to pay for that risk.
At least with capitalism you have many different people with different perspectives on the risk making independent bets. That mitigates the more extreme negative outcomes.
To add to that, investors who do make the bet get punished for over-building, which is better than tax payers paying for it. And before someone says it, big corps do get bailed out by gov't, but that's definitely goes against capitalist ideas.
Also it is government job to regulate. Monopolies should be busted, and behaviour like that should be culled. But US govt is full on AI to mask them cratering their own economy.
Is the DRAM industry really capitalist? Focusing on just the Korean parties, it functions like a command economy. I would say the same about most high end semi-conductor manufacturing, TSMC, Intel, ASML are being commanded and driven by nation-state level decision making. Right now the command is to focus on high wattage centralized AI systems at the expense of everything else.
No one at high levels is capitalist, in ideology or action. An ideological capitalist would be in favor of competition, but these people disdain it and collude regularly. The only 'capitalist' actions they take are by accident, the real goal is as much power/money as possible as fast as possible.
We don't even expect companies to plan long-term anymore, it's just moving wealth as fast as possible.
That isn't really a change, very few people could ever have been said to be ideological capitalists. (capitalist is not a word with a hard definition, but I'm considering it a different thing than the more modern pure libertarian zero-regulation ideology)
I think the common use of the term capitalist is as a participant in capitalism.
This is distinct from someone who is a proponent of capitalism as a system, which appears to be the way you are using capitalist. For which I don't blame you.
Liberalism (in the traditional economic sens) likes competition. Capitalism is a mode of production, and capitalists notoriously don’t like competition when they are the incumbents
> Capitalists claim that this is optimal.
Because that does not happen exactly as you say for all players. The demand signals will be processed and long-term risk is balanced against short-term gain in a distributed fashion, so not everyone will do the same.
Sufficient diversity of response will only appear when there are enough competitors. When there are only a few, it doesn't necessarily work out.
It's not optimal, it's pathological. Definitely better than starving under communist dictatorships though.
>Capitalists claim that this is optimal.
It's more optimal than planned economies until we have AI planned economies with realtime feedback, I guess.
Consumers get cheap goods during oversupply and most inefficient companies get elliminated during bust while consolidation leads to economies of scale.
No this is literally a sign of an unstable system with too high of a gain K.
There is an alternative where legislation dampens this behavior but the short term profits will be lower. Hence the hawks don’t like it.
>legislation dampens this behavior
Potentially. Well meaning and thought out legislation still distorts the markets, possibly making things objectively worse.
This is a wild take.
Sophomoric take more precisely.
Why is the opposite of capitalist markets automatically assumed to be a command economy? Co-op style businesses aren't really capitalist orientated but are also not reliant on government action.
> Capitalists claim that this is optimal
That's not what capitalists claim. Capitalists claim communism is responsible for tens, if not hundreds, of millions of death due to famine. And overall miserable ways of life, as if humans were termites deprived of any individualism and any freedom.
Capitalists claim that France importing 500 000 people from the third world each year, out of which only 10% are ever going to work (and these are official numbers) and yet offering them all a safety net is unsustainable. And that that socialism is only going to lead to one thing: running out of taxpayers' money.
Capitalists don't claim they have the best system: what they claim is that they haven't seen a less worse one.
I don't know who these "capitalists" are, but generally speaking business regularizes as entrepreneurs learn to read new signals after unexpected market shifts.
It is unlike a socialist system in that there are signals to read in the first place. What, do socialists claim that failure is optimal when you can't even tell if you've failed?
The sheer fucking blazing ignorance of this comment
> Capitalists claim that this is optimal.
Compared to starving under communism coz someone at top got the number wrong, yes. And it only really happens when there are massive, unpredictable market movements and governments not doing their job. Govt should look at the whole thing and just say "no", blame them.
No market system self regulates well enough, and it's government job to file down the edge cases like this. But the revolution happened in country which has two utterly incompetent parties, both in pockets of billionaires, fighting for power, and the clowns from one that won last battle use AI to smokescreen the economic growth their actions cratered
I am betting the pendulum swings faster to the other side to excess capacity as all the construction lies of Altman fall through with financiers waking up the the fact they can't build the infrasctructure as fast nor make any profits on that infrastructure that will get built.
Those financiers can’t risk not being involved in a company with even just a slight potential for AGI.
> Those financiers can’t risk not being involved in a company with even just a slight potential for AGI.
Do recent actions of Open AI give you the impression of a company that believes it is about to attain AGI imminently?
All it matters that gullible investors do.
Hell, all it matters to investors is not being left holding the bag in the end so they don't even need to believe it
What if it takes 100 years to get to AGI or we never achieve it? All bets on AGI will just fail over and over again for decades in that case. It seems a bit like saying financiers can't risk not being involved with Faster-than-light travel technology. Yeah, it would change everything if we got it, but betting that we'll get it soon over and over again is probably not going to get you a lot of money.
We've been projecting both FTL and AGI as future possibilities for almost 100 years now. Do LLMs get us a lot closer to AGI? I think they get us a little closer and Moore's "law" making compute faster probably is a much bigger factor, but I think we're still a very very long ways away.
Who has been projecting FTL as a realistic technology ever? FTL is not possible according to the current laws of Physics, while AGI is at least not forbidden by them.
I think this should be something you can answer for yourself by looking at human media and news over the lasy 100 years. I find it hard to belive you haven't ever noticed anyone seriously saying we may possibly have FTL sometime in the future. Incidentially I think I read on HackerNews that Sam Altman has been talking about building Dyson Spheres in the future. I suppose they're not forbidden by the current laws of phyisics either, but I don't know if I would call them a realistic technology.
We don't even know what human consciousness is. We can't even answer if we have free will or not. And you are proposing that AGI..is what exactly ?
The IQ of the smartest human, the perfect memory storing and recollection of computers, the fact that it never tires. I don't know if it's AGI but it's already something greater than us.
Is the IQ measured by tests created for and answered by humans in the training data?
If it was greater than humans already it wouldn't need humans to help it work.
ok...there's no 'general' IQ afaik
> We don't even know what human consciousness is.
I think Douglas Hofstadter satisfactorily answered this question.
> We can't even answer if we have free will or not.
Sure we can, it's just that most people don't like the answer.
> I think Douglas Hofstadter satisfactorily answered this question.
He didn't prove anything. It's a theory, just like many others: https://plato.stanford.edu/entries/consciousness-higher/
> Sure we can, it's just that most people don't like the answer.
Again, not proven
There are also plenty of people who think you can't prove whether the Christian God exists. I don't take those people seriously.
Don't mistake continued debate amongst "experts" as being a signal that something is unsettled.
OpenAI does have projections for making money with ads that would make Google and Meta blush.
To show ads you need people to stay on your platform. This is especially true once ads become more intrusive or of lower quality, something the big players seem to gravitate towards to keep revenue up. Google and Meta have ways to lock in users (networking effects, the best search engine available, having your data stored there).
I am not sure if OpenAI has that. Their edge regarding models is small, their strategy currently seems to be "buy ALL the hardware so nobody else can". Users can quite easily switch to other models.
If only there were privacy laws and working antitrust laws. There could also be a law straight up banning chat uses of llms and only allow agentic uses with human review. Would solve a lot of problems for lawmakers worried about AI I think.
> Soo ... how sure are we that the memory makers themselves are not going to be the ones holding the bag?
The memory makers specifically did not scale up capacity to avoid being left holding the bag.
Memory makers did get themselves into this situation by selling all wafers for empty promises and alienating everyone but OpenAI tbh. I do hope they end up holding the bag once again, cause after covid and the cartel thing they don't seem to ever learn their lesson on how to have the tiniest amount of integrity.
> Memory makers did get themselves into this situation by selling all wafers for empty promises and alienating everyone but OpenAI tbh.
Wasn't the problem here that OpenAI was negotiating with Samsung and SK Hynix at the same time without the other one knowing about it? People only realized the implications when they announced both deals at once.
While we're giving away bags, I'd like HDD manufacturers to get some too.
That wouldn’t help if another one goes bankrupt that’ll only make things worse.
Sounds like they're too big to fail, maybe we should bail them out to reinforce that they will get rewarded for making bad decisions.
Permanent public ownership of (very large stakes in) these companies doesn't seem like such a bad idea anymore, does it? It's what we used to have for most of the 20th century at least in Europe.
Yeah and exactly zero of those companies are European as a consequence. I think we won’t be taking Europe’s lead on this, thanks.
How do you reconcile this capitalistic zeal with how opposed to honest capitalism these companies are?
I reconcile that with the fact that Europe doesn't make enough DRAM for me.
So, as long as you can have your desires met, you don't really care about however much harm happens to anyone?
Neither does anyone else, as is being demonstrated currently.
That's only sound is it not. If you take away the hype - nothing critical actually depends on LLMs. You can remove them all today, and nothing bad would happen.
> Soo ... how sure are we that the memory makers themselves are not going to be the ones holding the bag?
I hope they do, they did not have to agree to sell so much RAM to one customer. They’ve been caught colluding and price fixing more than once, I hope they take it in the shorts and new competitors arise or they go bankrupt and new management takes over the existing plants.
Don’t put all your eggs in the one basket is how the old saying goes.
> Soo ... how sure are we that the memory makers themselves are not going to be the ones holding the bag?
We aren't. The remaining memory manufacturers fear getting caught in a "pork cycle" yet again - that is why there's only the three large ones left anyway.
If they don't expand capacity much, the only negative consequences I foresee happening for them is that they might lose spending discipline, and that systems will be set up to make do with a little less memory. Apart from that, it's just very high profits followed by more or less regular profits.
They could wind up losing all their business to China though.
China has memory makers who are creeping up through the stages of production maturity, and once they hit then there's no going back.
If the existing makers can't meet supply such that Chinese exports get their foot in the door, they may find they never get ahead again due to volume - that domestic market is huge so they have scale, and the gaming market isn't going to care because they get anything at the moment, which is all you'll need for enterprise to say "are we really afraid of memory in this business?"
Good point, it's a risk but so far the Chinese competition isn't up to par and it's unclear whether they'll be able to exploit the current window of opportunity.
You think this window is short? We've been dealing with this for years and years, and to me it seems more like incumbent manufacturers are too comfortable milking cash cows.
This window of opportunity (very high RAM prices) opened about half a year ago.
That's only for normal RAM. I'm talking about broader aspect: HBM shortage and high prices have been around for longer, and Chinese manufacturers are also climbing up and expanding there.
> If the existing makers can't meet supply such that Chinese exports get their foot in the door, they may find they never get ahead again due to volume
The answer to that is government regulation. Ban anything Chinese or slap it with tariffs. That is what tariffs are intended for - not for the BS the current administration has done.
Good luck actually pulling that off, it'll just kneecap US/Western industry, just as the current tariff and war madness has helped drag the US economy into the gutter.
Surely this can be solved with financial engineering. The memory makers build more capacity, but they finance it with something like floating-rate notes linked to an index of memory prices, or even catastrophe bonds or AT1s. Or more crudely, set up special purpose vehicles to build the extra capacity, and issue convertible bonds from those; if the memory market collapses, investors don't get paid, but they do get a memory factory.
But wouldn’t you rather hbm prices come down first ? Memory makers will be fine. There is practically infinite demand. Unless you get china style rationing of compute per person world wide.
The real issue is everyone wanting to upgrade to hbm, ddr5, and nvme5 at the same time.
What kind of consumer electronics can you build with HBM? That's the startup you should be founding...
HBM is just normal DDR RAM that's been packaged with (much) wider-than-usual data buses. That's where the high bandwidth comes from, not from high clock rates or any other innovation or improvement in core specifications.
Very few applications other than GPUs need HBM.
AMD has built some consumer GPUs in the recent past with HBM - RX Vega and Radeon VII (although I assume not all "HBM" is created equal).
Isn't their APU also capable of doing HBM? There was an Intel AMD hybrid chip that used unified a while back too.
That was not unified. It was just on same package. Functionally it was like if you had a dedicated gpu.
My vega 56 still has 400gb/s of memory which is still insane for how old the card is.
AMD's Hawaii architecture had 320GB/s on a 512b GDDR5 bus in 2013.
The Fiji XT architecture after it had 512GB/S on a 4096b HBM bus in 2015.
The Vega architecture did have 400GB/s or so in 2017, which was a bit of a downgrade.
I think the datacenters will simple be repurposed for hardware as a service, specially cloud gaming, sense it will be to expensive to buy a new computer anyway.
Anything where there's _at worst_ solder and traces between the compute and the memory. That's why you see it on GPUs (and Apple hardware). DRAMs advantage is modularity.
At least as I understand it.
They can just switch back to normal DRAM if HBM demand drops, you ain't getting it cheap just coz AI flops
[dead]
I'm under the impression OAI wrote Letters of Intent, and are not actually on the hook for the RAM they requested.
The others...they did not. Memory makers won't be holding the bag because Apple/Google/Samsung are contractually obligated to purchase, after the panic OAI caused.
> Based on my limited knowledge HBM can not be easily repurposed for consumer electronics. Translation: main street is cooked for the next 3-4 years.
It's worse. HBM have lower yields so they are essentially making less GB per wafer too
Not a rec, but just my source: Atrioc (streamer, YouTuber) is good at gathering all the facts for the rest of us. There's many other things in play, like the Strait of Hormuz (helium, bromine). Ultimately it works out that the shortage, and shortage profits, will continue; the chip makers are probably going to continue to see record profits (as Samsung has).
The specific mix of factors could change at any time, but the supply chain is relatively inelastic, it will take some time to show up on price labels.
It's not going to last until 2028, it'll last until 'min(AI_bubble_burst, 2028)', which I expect will be a lot smaller than just '2028'. So the real question is, how long will it take to retool for non-HBM, and will there be a fire sale as they scramble to recover?
Which also explains why production is falling behind demand, companies aren't going to sink billions into creating product for a market that could dry up overnight.
> Folks are now starting to ask difficult questions about their burn rate and revenue.
this view isn't updated correctly post-claude code and codex. there will clearly be sufficient demand.
Seriously? One release is all it took to turn the whole ship around?
I think I’m missing something. Financially, what bag would the memory makers be holding here? I don’t think I’m well informed regarding how these deals were structured.
Memory makers make capital investements (build different factories, convert physical production lines, etc.) to meet orders that have been place for the next ~5 years.
OpenAI (or whoever) crashes and can't pay for the order leaving the memory makers in a tough spot.
> leaving the memory makers in a tough spot
Oh noes! Think of a poor memory makers!
The amount of money flowing both from the AI bubble and from quite literally scalping both the server and consumer market... They gambled on the opportunity and if they fail - it's their problem.
Exactly, that's why they are not building more capacity and that's why RAM prices will stay up for years.
And how is that a problem and more importantly how's that a problem of Average Joe?
Capitalists did their gamble things. If they fail in that gamble what forbids them to sell the regular RAM they made for AI bubbleists to the regular consumers? Besides HBM it's just the regular chips which are exactly the same for the consumer/server market, why it would be any different?
There's actually plenty of demand for LPDDR even in the AI datacenter, because HBM is quite wasteful of area for any given memory capacity.
Wafer area?
Yes, HBM requires more wafer area for the TSVs and maybe some other features.
The market is already stagnated. Even if OpenAI doesn’t buy what they reserved other players will do so. SK Hynix CEO said there is a 20% gap between supply and demand per year. And that doesn’t account the shock effect that will take place the moment prices normalize and everyone and their dog will go out and start buying inventory to avoid the next crisis. I for one would certainly buy more than I currently need just in case.
I think: s/stagnated/saturated/
Edit: also, that demand pressure is going to be applied constantly; there isn’t going to be a shock, it’s just going to keep prices high longer.
The FOMO is strong, but can also indicate a bubble. Demand is from circular deals and APIs are being locked down already.
Did OpenAI's order actually kickstart the "panic" over RAM?
Are they really such a big RAM buyer?
Apparently...
> OpenAI’s rapid growth, fueled by the success of ChatGPT and other AI products, led to a landmark agreement in October to purchase 900,000 DRAM wafers per month from Samsung and SK Hynix—amounting to roughly 40% of global supply. This surge in demand, coupled with limited manufacturing capacity, sent prices for memory kits skyrocketing. [0]
[0]: https://peq42.com/blog/openai-canceling-many-large-purchase-...
They ordered 40% of the global RAM production for 2025/26. It was a non-binding agreement that either side could easily withdraw from but they're essentially trying to buy about half of all the RAM.
Blows my mind that it's non-binding.
If I booked half a hotel's rooms then suddenly said "yeah never mind. Half my friends cancelled and we're not staying", basically any hotel would be coming at me for my money because there's no way they can fill their rooms now and they're losing revenue. But OpenAI can really get the whole world to pivot towards it then say "cool but we don't need your product anymore" and RAM makers are just going to let it go.
Whoever decided that was a good idea needs to be fired and publicly shamed.
Well if that hotel was then able to sell the other half of hotel rooms for 10x the old price. Then the hotel might actually be happy as they can now charge 10x for the other half or slowly lower prices back down over years.
it's used for price gouging by the cartel
if anything, OpenAi might be in on it
Do the memory makers not have a contract in place for an order this large? I assume that they aren't going to take "trust us bro" as good enough for several million dollars in orders, and even if there is a way to cancel the order it won't be free. I would assume so at least, but i would like if anyone knew for certain.
I was interning at a company that made networking gear that was put out of business when their largest customer canceled an order within a week of the delivery date.
The customer ran out of money. In terms of where you are in line of debtors when you haven't even delivered the product to a customer, it's so far back as to be assured you won't get your money.
If the memory makers got a deposit from OpenAI as part of this deal, that is likely to be the only money they will get for any undelivered memory, particularly if OpenAI runs out of capital.
You can’t squeeze blood from a stone as they say. If AI companies go bankrupt there’s no money to be had.
Instead of canceling the orders, OpenAI could take delivery and immediately flip the DRAM for a profit.
That would also be a pretty obvious admission of defeat for their original business model, though...
* several Billion dollars in orders.
Good point. I think both AI companies and hardware makers should pay for the damage they caused to us here.
They act as a de-facto monopoly and milk us. Why is this allowed?
It's a business with huge up-front capital expenses and typically very low margins. Supply is scaling up slowly because it's hard, and if you overshoot, you go out of business.
Nobody is "allowing" this. It's a natural property of being both advanced technology and a commodity at the same time.
The strange deals on the entire future output are what was allowed. Try to do the same thing with onions and the government understands you are a criminal.
https://en.wikipedia.org/wiki/Onion_Futures_Act
That is quite the amusing read but it seems like a poorly constructed law. It wasn't futures themselves that were the problem there. The duo engaged in blatant market manipulation and severely disrupted part of the food supply in the process.
And now OpenAI is engaged in blatant market manipulation and severely disrupted the entire world's DRAM supply.
Cornering the market with the intent to flip the goods is not quite the same as cornering the market because you actually want the goods and intend to use them yourself.
And it just so happens that many people will now have to use OpenAI’s products because they can’t get enough RAM to run a local LLM. What a coincidence.
Right, the second was a conspiracy to form a monopoly.
It has the makings of a natural monopoly, except its compounded by RAM cartels colluding to shut out the last of the competitors.
Recently they had a second price fixing lawsuit thrown out (in the US).
Now with the state of things I'm sure another lawsuit will arrive and be thrown out because the government will do anything to keep the AI bubble rolling and a price fixing suit will be a threat to national security, somehow. Obviously thats speculative and opinion but to be clear, people are allowing it. There are and more so were things that could be done.
Because for the last 60 years we've allowed big business to buy and hollow out our legal and education systems.
Allowed? We live in a neoliberal world where corporate monopolies / oligopolies aren’t even remotely regulated. If you try to do even the gentlest regulation of companies people scream about communism and totalitarianism. Unless the regulation serves the monopolies by making it harder to enter the market.
It started with raegan, and even parties on the “left” in the west believe in it with very few exceptions.
> We live in a neoliberal world where corporate monopolies / oligopolies aren’t even remotely regulated. If you try to do even the gentlest regulation of companies people scream about communism and totalitarianism. Unless the regulation serves the monopolies by making it harder to enter the market.
The thing that enables this is pretty obvious. The population is divided into two camps, the first of which holds the heuristic that regulations are "communism and totalitarianism" and this camp is used to prevent e.g. antitrust rules/enforcement. The second camp holds the heuristic that companies need to be aggressively "regulated" and this camp is used to create/sustain rules making it harder to enter the market.
The problem is that ordinary people don't have the resources to dive into the details of any given proposal but the companies do. So what we need is a simple heuristic for ordinary people to distinguish them: Make the majority of "regulations" apply only to companies with more than 20% market share. No one is allowed to dump industrial waste in the river but only dominant companies have bureaucratic reporting requirements etc. Allow private lawsuits against dominant companies for certain offenses but only government-initiated prosecutions against smaller ones, the latter preventing incumbents from miring new challengers in litigation and requiring proof beyond a reasonable doubt.
This even makes logical sense, because most of the rules are attempts to mitigate an uncompetitive market, so applying them to new entrants or markets with >5 competitors is more likely to be deleterious, i.e. drive further consolidation. Whereas if the market is already consolidated then the thicket of rules constrains the incumbents from abusing their dominance in the uncompetitive market while encouraging new entrants who are below the threshold.
Arguably a more efficient approach might just be to have a tax that adds on to corporate tax incrementally for every % of market share a company has above say 7-8%. Then dominant companies are incentivised to re-invest in improving their efficiencies rather than just buying/squeezing out competitors. A more evenly spread market would then, as a result, be against regulations that make smaller market participants less competitive, as they'd all be in relatively less table positions.
> Arguably a more efficient approach might just be to have a tax that adds on to corporate tax incrementally for every % of market share a company has above say 7-8%.
How is this more efficient? You'd still be applying all of the inefficient regulatory rules intended to mitigate a lack of competition to the smaller companies trying to sustain a competitive market, and those rules are much more deleterious for smaller entities than higher tax rates.
If you have $100M in fixed regulatory overhead for a larger company with $10B in profit, it's only equivalent to a 1% tax. The same $100M for a smaller company with $50M in profit is a 200% tax. There is no tax rate you can impose on the larger company to make up for it because the overhead destroys the smaller company regardless of what you do to the larger one.
First, on the surface, this sounds like a terrible idea. Almost all ideas that I see on HN about economics fail with even the tiniest amount of common sense.
As a counterpoint: Look at very high value goods, like jet engines and MRI machines. I went for an MRI the other day and wondered to myself (then asked an LLM) what the international MRI market looks like. They are vanishingly small number of manufacturers and are usually dominated by a few international players. How are you going to apply this tax to non-domiciled (international) companies? Also, companies like General Electric, Mitsubishi Heavy, and Seimens are enormous and incredibly diverse. This idea falls apart quickly.
reagan was not remotely neoliberal (though he was a disillusioned former socialist and union leader) but he did knock liberalism off its perch.
it was clinton who delivered the democrats to wall street and vice versa
Reagan is canonically neoliberal: https://en.wikipedia.org/wiki/Reaganomics (it’s literally the first descriptive word used when defining his economic policies).
Maybe it’s time for a refresher on what neoliberal means? It’s not simply “new liberalism”. Reaganomics was the start of neoliberalism in the US, tho of course it shifted and developed its character further over time into the monster that drives 99% of our problems today
not all DRAM capacity can switch to HBM quickly. That lag is where volatility comes from
This will result in demand destruction which will starve the enterprise which will starve the hyperscaler. theres no situation where people not being able to afford hardware for 4 years results in the bubble not popping
I'd expect unaffordable hardware to drive demand for thin clients connected to cloud services which is something that had already been happening gradually prior to this.
The people who fucked over consumers are left holding the back that they sold us out over?
Oh no!
They won't be, prices are high because they are refusing to build capacity for demand that may evaporated by the time they are done. They are holding back and building only enough so when the bubble pops they will be fine.
You can't build capacity overnight, and even with that in mind, it's hard to say if it is sensible to increase capacity now that we are in an AI bubble. For all we know, the bubble might burst.
So the ML hate is weaponized in the form of memory demand collapse FUD, and the public at large has to pay through their nose for it... thanks party poopers!
I don't think its from the ML collapse FUD, its most likely from the multiple time's in the past when they overbuilt and it resulted in a memory oversupply and price collapses. The 1985–1988, 1993–1994, 1998–2002 and the post pandemic oversupply. These were all cases where shortages followed by over corrections caused oversupply, financial losses due to low prices and fewer surviving companies. I think they're taking their time and are cautiously adding more capacity in such a way that prices won't end up collapsing again. Regardless, the result is still that we the consumers have to pay more.
At this point the remaining memory companies are… the ones that didn’t die during an over-supply collapse, right? I guess there’s been a strong evolutionary pressure against giving consumers what we want, haha.
its not like all the RAM is passing the same machine, they can gradually increase machines and observe the change in demand, and smoothly match it.
If they gradually increase production capacity then prices stay high for 10+ years (or for as long as it takes for demand to crash) because a gradual increase in production takes that long for them to add enough capacity for current demand.
If they add enough capacity to meet current demand quickly then if demand crashes they still have billions of dollars in loans used to build capacity for demand that no longer exists and then they go bankrupt.
The biggest problem is predicting future demand, because it often declines quickly rather than gradually.
do we have evidence of RAM manufacturers going bankrupt? do we have evidence that the increased capacities after the mentioned past shortages went unused or were operated at a loss?
There used to be a dozen DRAM manufacturers and now there are five. I don't know if the others went bankrupt but they got out of the market somehow.
as a starting point, asianometry has some good videos on this
Machines take up space in buildings (factories); both of which are discrete rather than continuous functions. If your factory is already full of memory-making machines, and want to add one more, it will cost you billions and many months to build another factory.
If you suppose you have cracked the smooth-ramping problem, perhaps you should throw your hat in the ring and soak up all the pent-up demand that SK Hynix, Samsung and Micron are neglecting.
Think of the factory problem from physics first principals instead, as Elon would say. Musk says he will outcompete earth fabs by building them on the moon in just a few years, deploy radiation harden versions of the chips into space, and beat out TCO vs doing this on earth.
If he can do all that that fast, the RAM makers should be able to at least 1000X their fab capacity on earth in one year. One year for scaling up existing tech is an eternity compared to Elon's timeframe for moon-fabs given the relative complexity of the challenge.
The most well known thing about Musk besides being an asshat, is that his timelines are almost always imaginary. He is not building fabs on the Moon in "just a few years".
He doesn't have to build them to sell them. FSD has been sold for almost a decade, more than half the depreciation time of the cars. With first principles sales techniques it's possible X could sell these moon chips now and deliver them made on earth 6 or 7 Moore's law cycles later by a different company, like with the solar roof or hydraulic brick machine from alibaba, way cheaper.
But how is that at all related to the DRAM manufacturers' short term production limitations? Unless I'm being dumb and you're saying that these manufacturers could also just kick the can down the road for a few fab generations?
I would expect that OpenAI gets as much money as they ask for for the next 10 years.
There’s virtually infinite capital: if needed, more can be reallocated from the federal government (funded with debt), from public companies (funded with people’s retirement funds), from people’s pockets via wealth redistribution upwards, from offshore investment.
They will be allowed to strangle any part of the supply chain they want.
China already has a well developed DRAM industry, as DRAM is somewhat easier than logic, and can tolerate a much higher defect rate. The industry will figure this out.
Another point is I often see the money argument - like country X has more money, so they can afford to do more and better R&D, make more stuff.
This stuff comes out of factories, that need to be built, the machinery procured, engineers trained and hired.
If China capitalises on the big three focusing on data centre team, the big three might have a very hard time post bubble
I think the article has a giant blind spot as far as China is concerned , considering they have already a mature enough memory ecosystem via YMTC that Apple was considering sourcing from them. As well as continued expansion in the DRAM and HBM Fabs [1]. It feels like the memory cartel once again trying to incentivise their various govt to cough up some more tax breaks/funding to cushion the AI buildout bet that they made and the bubble seeming about to pop. In any case if they leave the consumer market underserved it should be no surprise if before that 2030 prediction we are all on cheaper YMTC memory modules.
[1]https://www.tomshardware.com/tech-industry/semiconductors/ym...
I'm guessing they become pets.com within the year. At least I hope.
Maybe if they had no competitors...
I think you're massively overestimating how much money is really accessible here. The parent comment's right that all of the easily available VC & private equity investment is basically used up. OpenAI was struggling to sell $600M of private equity, the big multi-billion dollar investment packages had lots of conditions and non-cash in it.
> more can be reallocated from the federal government (funded with debt)
While this is the most reliable funding, it's still not very accessible. OpenAI is a money pit, and their demands are growing quickly. The US government has started a bunch of very expensive spending. If OpenAI were to require yearly bundles of it's recent "$120B" deal, that's 6% of the US' discretionary budget. 12.5% of the non-military discretionary budget. (And the military is going to ask for a lot more money this year) Even the idea of just issuing more debt is dubious because they're going to want to do that to pay for the wars that are rapidly spiralling out of control.
None of this is saying that the US government can't or wouldn't pay for it, but it's non trivial and it's unclear how much Altman can threaten the US government "give me a trillion dollars or the economy explodes" without consequences.
Further deficit-spending isn't without it's risks for the US government either. Interests rates are already creeping up, and a careless explosion of deficit may well trigger a debt crisis.
> from public companies (funded with people’s retirement funds)
This would be at great cost. OpenAI would need to open up about it's financial performance to go public itself. With it's CFO being put on what is effectively Administrative Leave for pushing against going public, we can assume the financials are so catastrophic an IPO might bomb and take the company down with it. Nobody's going to be investing privately in a company that has no public takers.
Getting money through other companies is also running into limits. Big Tech has deep pockets but they've already started slowing down, switching to debt to finance AI investment, and similarly are increasingly pressured by their own shareholders to show results.
> from people’s pockets via wealth redistribution upwards
The practical mechanism of this is "AI companies raise their prices". That might also just crash the bubble if demand evaporates. For all the hype, the productivity benefit hasn't really shown up in economy-wide aggregates. The moment AI becomes "expensive", all the casual users will drop it. And the non-casual users are likely to follow. The idea of "AI tokens" as a job perk is cute, but exceedingly few are going to accept lower salary in order to use AI at their job.
There's simply not much money to take out of people's pockets these days, with how high cost of living has gotten.
> from offshore investment.
This is a pretty good source of money. The wealthy Arabian oil states have very deep slush funds, extensively investing in AI to get ties to US businesses and in the hope of diversifying their resource economies.
...
...
"Was". Was a good source of money.
I'm genuinely curious to find out how many billions they get every year from now.
love that theres virtually infinite capital there. meanwhile in the rest of the world there is virtually no food.
The "no food in other countries" is because of failed/corrupt governments, not because people use AI to generate cat pictures in the West. The economy is not a "fixed pie" that needs to be allocated among people of the world.
Just look at Cuba, which could be a very rich country and one of the prime tourist destinations of the world.
Both of those problems exist largely or primarily due to the US, destabilizing countries with bombs and coups, and the cuban blockade.
are you kidding? if spent all that money on food you guys would just use it to bullshit all day and make funny pictures, while if we spend it on AI..
I'm a bit surprised the article makes no mention of Google's TurboQuant[0] introduced 26 days prior.
Given that TurboQuant results in a 6x reduction in memory usage for KV caches and up to 8x boost in speed, this optimization is already showing up in llama.cpp, enabling significantly bigger contexts without having to run a smaller model to fit it all in memory.
Some people thought it might significantly improve the RAM situation, though I remain a bit skeptical - the demand is probably still larger than the reduction turboquant brings.
[0] https://news.ycombinator.com/item?id=47513475
TurboQuant is known across the industry to not be state of the art. There are superior schemes for KV quant at every bitrate. Eg, SpectralQuant: https://github.com/Dynamis-Labs/spectralquant among many, many papers.
> Given that TurboQuant results in a 6x reduction in memory usage for KV caches
All depends on baseline. The "6x" is by stylistic comparison to a BF16 KV cache; not a state of the art 8 or 4 bit KV cache scheme.
BTW, a number of corrections. The TurboQuant paper was submitted to Arxiv back in April 2025: https://arxiv.org/abs/2504.19874
Current "TurboQuant" implementations are about 3.8X-4.9X on compression (w/ the higher end taking some significant hits of GSM8K performance) and with about 80-100% baseline speed (no improvement, regression): https://github.com/vllm-project/vllm/pull/38479
For those not paying attention, it's probably worth sending this and ongoing discussion for vLLM https://github.com/vllm-project/vllm/issues/38171 and llama.cpp through your summarizer of choice - TurboQuant is fine, but not a magic bullet. Personally, I've been experimenting with DMS and I think it has a lot more promise and can be stacked with various quantization schemes.
The biggest savings in kvcache though is in improved model architecture. Gemma 4's SWA/global hybrid saves up to 10X kvcache, MLA/DSA (the latter that helps solve global attention compute) does as well, and using linear, SSM layers saves even more.
None of these reduce memory demand (Jevon's paradox, etc), though. Looking at my coding tools, I'm using about 10-15B cached tokens/mo currently (was 5-8B a couple months ago) and while I think I'm probably above average on the curve, I don't consider myself doing anything especially crazy and this year, between mainstream developers, and more and more agents, I don't think there's really any limit to the number of tokens that people will want to consume.
The work going into local models seems to be targeting lower RAM/VRAM which will definately help.
For example Gemma 4 32B, which you can run on an off-the-shelf laptop, is around the same or even higher intelligence level as the SOTA models from 2 years ago (e.g. gpt-4o). Probably by the time memory prices come down we will have something as smart as Opus 4.7 that can be run locally.
Bigger models of course have more embedded knowledge, but just knowing that they should make a tool call to do a web search can bypass a lot of that.
The net effect won’t be a memory use reduction to achieve the same thing. We’ll do more with the same amount of memory. Companies will increase the context windows of their offerings and people will use it.
That is the sad reality of the future of memory.
I am not convinced that more context will be useful, practical use of current models at 1mil context window shows they get less effective as the window grows. Given model progress is slowing as well, perhaps we end up reaching a balance of context size and competency sooner than expected.
Stuff in more code. Stuff in more system prompt. Stuff in raw utf8 characters instead of tokens to fix strawberries. Stuff in WAY more reasoning steps.
Given the current tech, I also doubt there will be practical uses and I hope we’ll see the opposite of what I wrote. But given the current industry, I fully trust them so somehow fill their hardware.
Market history shows us than when the cost of something goes down, we do more with the same amount, not the same thing with less. But I deeply hope to be wrong here and the memory market will relax.
You can still use as much memory, but fit more things into it, so I don’t think the current market hogs will let go easily.
> 6x reduction in memory usage for KV caches and up to 8x boost in speed
mind that you're quoting marketing material that's largely based on unfair baseline testing (like comparing 4 bit vs 32 bit to get "8x speed")
https://www.youtube.com/watch?v=haoAI2lIZ74
Your skepticism is well placed. Every time a new quantization or compression technique drops, the immediate response is to just scale up context length or run a bigger model to fill whatever headroom was freed up. It's Jevons paradox applied to VRAM - efficiency gains get eaten by increased usage almost immediately.
You still need to hold the model in memory. If you have for example 16 GB ram, the gains aren't that much
That's not what consumes the most memory at scale. The KV caches are per-user.
that will only increase the demand for RAM as models will now be usable in scenarios that weren't feasible prior, and the ceiling for model and context size is not even visible at this point
I hate to mention Jevons paradox as it has become cliche by now, but this is a textbook such scenario
[dead]
[dead]
Something I haven’t been able to reconcile: If AI makes software easier to create, that will drive the price down. How are software companies going to make enough revenue to pay for AI, when the amount of money being spent on AI is already multiples of the current total global expenditure on software? This demand for RAM is built on a foundation of sand, there will be a glut of capacity when it all shakes out.
> This demand for RAM is built on a foundation of sand
RAM is built on a foundation of sand.
I laughed.
Analyst says it would require a new $35/month subscription for every iPhone user, or a new $180/month for every Netflix subscriber.
Claude Max subscriptions have gone up, but do you think every Netflix user will pay for one?..
https://www.tomshardware.com/tech-industry/artificial-intell...
This is probably the wrong denominator. There are more than a billion white-collar workers. Making them all just 10% more effective would possibly be worth more than $650B/year (~$650/worker).
Everyone's betting on Jevons paradox
the hope is that Ai is "the next semiconductor" and "the next internet"
> If AI makes software easier to create, that will drive the price down.
Supposedly AI drives down the cost of producing software,not the "price".
> How are software companies going to make enough revenue to pay for AI, when the amount of money being spent on AI is already multiples of the current total global expenditure on software?
Currently, the cost of AI is between $20/month and around $200/month per developer.
I think the huge billions you're seeing in the news are the investment cost on AI companies, who are burning through cash to invest in compute infrastructure to allow both training and serving users.
> This demand for RAM is built on a foundation of sand, there will be a glut of capacity when it all shakes out.
Who knows? What I know is that I need >64GB of RAM to run local models, and that means most people will need to upgrade from their 8Gb/16GB setup to do the same. Graphics cards follow mostly the same pattern.
You need >64 GB of DRAM to run local models fast.
You can run huge local models slowly with the weights stored on SSDs.
Nowadays there are many computers that can have e.g. 2 PCIe 5.0 SSDs, which allow a reading throughput of 20 to 30 gigabyte per second, depending on the SSDs (or 1 PCIe 5.0 + 1 PCIe 4.0, for a throughput in the range 15-20 GB/s).
There are still a lot of improvements that can be done to inference back-ends like llama.cpp to reach the inference speed limit determined by the SSD throughput.
It seems that it is possible to reach inference speed in the range from a few seconds per token to a few tokens per second.
That may be too slow for a chat, but it should be good enough for an AI coding assistant, especially if many tasks are batched, so that they can progress simultaneously during a single read pass over the SSD data.
You can do that, but you're going to have rather low throughput unless you have lots of PCIe lanes to attach storage to. That's going to require either a HEDT or some kind of compute cluster.
Batching inferences doesn't necessarily help that much since as models get sparser the individual inferences are going to share fewer experts. It does always help wrt. shared routing layers, of course.
> Who knows? What I know is that I need >64GB of RAM to run local models, and that means most people will need to upgrade from their 8Gb/16GB setup to do the same. Graphics cards follow mostly the same pattern.
Depends how big the models are, how fast you want them to run and how much context you need for your usage. If you're okay with running only smaller models (which are still very capable in general, their main limitation is world knowledge) making very simple inferences at low overall throughput, you can just repurpose the RAM, CPUs/iGPUs and storage in the average setup.
I got a 128 GB MBP, and the current models are fit enough to manage the calendar or do research on web (very slowly), not to be useful companions for coding as I hoped.
> This demand for RAM is built on a foundation of sand
Not exactly.
LLMs are already quite useful today if you use them as a tool, so they are there to stay. The remaining problem is scalability, a.k.a. how to make LLMs cheap to use.
But scalability is not really a requirement when you look the bigger picture. If smaller software company/projects can't afford to use AI, the bigger ones might just. Eventually they will discover variable use cases for such tech, even if it only serves big firms i.e. defense, resource extraction, war, finance etc.
To the other end, if scalability is achieved, the use of LLM products will be cheaper too, so smaller project can also use them. But of course, if LLM usage is too cheap, then many were-to-be-consumers will just create software projects by themselves at their homes.
Would they be considered as useful if users are required to pay non-subsidized price though.
They are.
The advanced workflows such as programming are not the only use cases for LLMs. There are much simpler work such as answering simple questions or arrange simple tasks. Both will improve productivity. And more advanced model should improve productivity even more.
It's a trojan horse. They expect the world will get hooked on it.
The usage of LLMs is continuing to increase ~exponentially. I'm going to bet on that rather than some half-baked scenario analysis that only takes into account one scenario and assigns a 100% probability to it.
> The usage of LLMs is continuing to increase ~exponentially
I would like a source for that statement. Additionally, I want to know by who? Because it certainly isn't end users. Inflating token usage doesn't make it any more economically viable if your user base, b2b or not, hasn't increased with it. On the contrary, that is a worse scenario for providers.
> Additionally, I want to know by who?
1. As a consultant pretty much every company I have worked with in the last 2 years are doing some kind of in-house "AI Revolution", I'm talking making "AI Taskforce" teams, having weekly internal "AI meetings" and pushing AI everywhere and to everyone. Small companies, SMEs and huge companies. From my observation it is mainly due to C-level being obsessed by the idea that AI will replace/uplift people and revenue will grow by either replacing people or launching features 10x quicker.
2. Did you see software job-boards recently? 9/10 (real) job listings are to do with AI. Either it is fully AI company (99% thin wrapper over Anthropic/OpenAI APIs) or some other SME that needs some AI implementations done. It is truly a breath of fresh air to work for companies that have nothing to do with AI.
The biggest laugh/cry for me are those thin wrappers that go down overnight - think all the "create your website" companies that are now completely useless since Ahtropic cut the middleman and created their own version of exactly that.
Yeah, my only hope is that this is unsustainable, admittedly for selfish reasons.
I know plenty of engineers being forced to use these tools whether they want to or not. A lot of which are okay with using AI liberally, but don't particularly like generative AI and see it as pretty irresponsible (which feels more true by the week and it is clear from first hand experience). I don't know, there is a huge gradient of users, but I would argue that in previous revolutionary technologies, we didn't have to force people to use a good tool. I didn't have to be forced to use Google search or Google Maps, tech that is now ubiquitous with western society. It seems really suspect that suits have to enforce the use of something that is supposed to change the way we work and be a force multiplier.
From my limited experience in multiple companies, as stated before I see one very common pattern - The process from feature idea to development is just bad. PMs do not know what exactly they want. C-level interjects in the middle and changes requirements. QAs are unsure what to test because acceptance criteria is vague.
C-level strongly believes that AI will fix all these issues. They believe that AI will fix their broken processes.
I see strong resemblance with "Agile Development" ~15 years ago. Extremely hyped, noone asked if their org even is a fit for it or need it, and most importantly - the only way to fix agile is to do more agile. Same with AI right now.
> I would like a source for that statement
The recent enterprise revenue numbers of Anthropic
So as I said, a self interested metric who also controls how many tokens it takes to get a desirable result from their models.
Users are willingly paying for larger volumes of tokens. You are layering your own unproven interpretation onto that. I would have arrived at an opposite interpretation given the available facts. Models are becoming more token efficient for the same task, such as ChatGPT 5.3 versus 5.2 which halved the token count, and capabilities show a log relationship with the number of tokens since o1 preview was revealed in September 2024.
No, you have gone off in your own tangent. The person you're responding to is talking about money and my point is that you're using a misleading metric. Even if the current user base is paying more for the "exponential token usage", it does not add up to the industry's cost of maintaining and building on this technology, especially since we are not taking into account what that token usage costs the provider. First you said Anthropic as your source, but now you're talking about OpenAI's ChatGPT, who are floundering for a product and user base, which they themselves claim will be profitable through subscriptions at numbers never seen before in a subscription business model.
I’m a bit of an optimist. I think this will smack the hands of developers who don’t manage RAM well and future apps will necessarily be more memory-efficient.
> I think this will smack the hands of developers who don’t manage RAM well
And hopefully kill Electron.
I have never seen the point of spinning up a 300+Mb app just to display something that ought to need only 500Kb to paint onto the screen.
The point is being able to write it once with web developers instead of writing it a minimum of twice (Windows and macOS) with much harder to hire native UI developers.
And HTML/CSS/JS are far more powerful for designing than any of SwiftUI/IB on Apple, Jetpack/XML on Android, or WPF/WinUI on Windows, leaving aside that this is what designers, design platforms and AI models already work best with. Even if all the major OSes converged on one solution, it still wouldn't compete on ergonomics or declarative power for designing.
Lol SwiftUI/Jetpack/WPF aren’t design tools, they’re for writing native UI code. They’re simply not the right tool for building mockups.
I don’t see how design workflows matter in the conversation about cross-platform vs native and RAM efficiency since designers can always write their mockups in HTML/CSS/JS in isolation whenever they like and with any tool of their choice. You could even use purely GUI-based approaches like Figma or Sketch or any photo/vector editor, just tapping buttons and not writing a single line of web frontend code.
Who said anything about mockups? Design goes all the way from concept to real-world. If a designer can specify declaratively how that will look, feel, and animate, that's far better than a developer taking a mockup and trying their hardest to approximate some storyboards. Even as a developer working against mockups, I can move much faster with HTML/CSS than I can with native, and I'm well experienced at both (yes, that includes every tech I mentioned). With native, I either have to compromise on the vision, or I have to spend a long time fighting the system to make it happen (...and even then)
well, then you are really bad at native and should not be comparing those technologies despite your claims otherwise (which make little sense).
> really bad at native
Yikes. I spent 15 years developing native on both mobile and desktop. If you think that native has the same design flexibility as HTML/CSS, you're objectively wrong.
By design, each operation system limits you to their particular design language, and styling of components is hidden by the API making forward-compatible customisation impossible. There's no escaping that. And if you acknowledge that fact, you can't then claim native has the same design flexibility as HTML/CSS. If you don't acknowledge that fact, you're unhinged from reality.
There's pros and cons to the two approaches, of course. But that's not what's being debated here.
The real disconnect is that the user doesn't really care all that much. It's mostly the designers who care. And Qt for example but also WPF let you style components almost to unrecognizable and unusable results. So if everyone will need to make do with 8GB for the foreseeable future, designers might just be told "No.", which admittedly will be a big shock to some of them. Or maybe someone finally figures out how to do HTML+CSS in a couple of megabytes.
> the user doesn't really care all that much
They do. But not in the way that you think.
I recently switched from Spotify (well known Electron-based app) to Apple Music (well known native app). The move was mostly an ethical one, but I must say, the UI functionality and app features are basically poverty in comparison. One tiny example, navigating from playlist entry to artist requires multiple interactions. This is just one of many frustrations I've had with the app. But hey, it has beautiful liquid glass effects!
In short: iteration time matters. Times from design to implementation, to internal review, to real user feedback, and back to design from each phase should be as fast as possible. You don't get the same velocity as you do in native. Add to that you have to design and implement in quadruplicate, iOS design for iOS, Android for Android, MacOS for Mac, Windows design for windows. All that is why people use Electon.
There is native to the OS and there's native to the machine.
Anyways, I'm both cases you don't really have to write it twice.
Native to the OS: write only the UI twice, but implement the Core in Rust.
Native to the machine: Write it only once, e.g. in iced, and compile it for every Plattform.
You mean the point is to dump it all on the end user's machine, hogging its resources.
It's bad enough having to run one boated browser, now we have to run multiples?
This is not the right path.
As the kids say: skill issue!
The point is you can be lazy and write the app in html and js. Then you dont need to write c, even though c syntax is similar to js syntax and most gui apps wont require needing advanced c features if the gui framework is generous enough.
Now that everyone who cant be bothered, vibe codes, and electron apps are the overevangelized norm… People will probably not even worry about writing js and electron will be here to stay. The only way out is to evangelize something else.
Like how half the websites have giant in your face cookie banners and half have minimalist banners. The experience will still suck for the end user because the dev doesnt care and neither do the business leaders.
Syntax ain't the problem. The semantics of C and JS could not be more different.
But the point isn’t that they’re more different than alike. The point is that learning c is not really that hard it’s just that corporations don’t want you building apps with a stack they don’t control.
If a js dev really wanted to it wouldn’t be a huge uphill climb to code a c app because the syntax and concepts are similar enough.
Honestly C and JavaScript could hardly be more different, as languages.
About the only thing they share is curly braces.
Yeah JS is closer to lisp/scheme than C (I say this as someone who writes JS, Clojure and the occasional C).
What "advanced features" are there to speak of in C? What does the syntax of C being similar to JS matter?
This comment makes no sense.
Well theres the whole c89 vs c99. I’ll let you figure the rest out since it’s a puzzle in your perspective.
It's happening. Cursor 3 moved to rust. A lot of people are using Zed (rust) instead of vscode.
It won't be "happening" until Slack, Teams, and Discord leave Electron behind. They are the apps that need to be open 24/7.
It's not entirely clear what the connection is.
We're not doing Electron because some popular software also using it. We're doing Electron because the ability to create truly cross-platform interfaces with the web stack is more important to us than 300 MB of user memory.
> web stack is more important to us than 300 MB of user memory.
May I never have to use or work on your project's software.
> We're doing Electron because the ability to create truly cross-platform interfaces with the web stack is more important to us than 300 MB of user memory.
It's closer to 1GB but trust me, everyone is well aware of your priorities.
"I would rather spend the user's money than my engineer's time"
Teams works similarly in browser tab and "natively". Slack was similar if I remember correctly.
You should check the memory use of that browser tab. You’re not saving much either way running in a browser or in Electron, which is effectively a browser.
I only ever use Discord in a browser window.
"cursor 3" is just a landing page. The editor is still the old vscode fork...
Are you sure about Cursor? I haven't seen anything about that, I think it's still based on VSCode/electron.
No. I saw they rebuilt it with some rust involved. It's no longer a vscode fork.
As if native apps are any better. Books app on my mac takes 400MB without even having a single book open.
I find that an exaggerated claim, have you really checked they aren't using a webview or some other non-native runtime?
I did't but that's exactly my point.
Native apps are so poorly optimized that they don't offer any advantage over Electron apps.
Won't happen. People are ok with swapping to their SSDs, Macbook Neo confirms that
Hopefully just kill off the javascript for everything mindset to be honest.
You do need a couple framebuffers, but for the most part yeah...
Who cares about 300Mb, where is that going to move the needle for you? And if the alternative is a memory-unsafe language then 300Mb is a price more than worth paying. Likewise if the alternative is the app never getting started, or being single-platform-only, because the available build systems suck too bad.
There ought to be a short one-liner that anyone can run to get easily installable "binaries" for their PyQt app for all major platforms. But there isn't, you have to dig up some blog post with 3 config files and a 10 argument incantation and follow it (and every blog post has a different one) when you just wanted to spend 10 minutes writing some code to solve your problem (which is how every good program gets started). So we're stuck with Electron.
There's a world of difference between using a memory safe language and shipping a web browser with your app. I'm pretty sure Avalonia, JavaFX, and Wails would all be much leaner than electron.
The people who hate Electron hate JavaFX just as much if not more, and I'm not sure it would even use less memory. And while the build experience isn't awful, it's still a significant amount of work to package up in "executable" form especially for a platform different from what you're building on, or was until a couple of years ago. And I'm pretty sure Avalonia is even worse.
> and I'm not sure it would even use less memory
It likely would use less, and doesn't use a browser for rendering.
> And I'm pretty sure Avalonia is even worse
Definitely not
> The people who hate Electron hate JavaFX just as much if not more
In my opinion, I only see this from people that seem to form all of their opinions on tech forums and think Java=Bad. These are the people that think .NET is still windows only and post FUD because they don't know how to just ask for help.
> And if the alternative is a memory-unsafe language
and if not?
> and if not?
If the alternative is memory-safe and easy to build, then maybe people will switch. But until it is it's irresponsible to even try to get them to do so.
Until? Just take what's out there - it's so easy to improve on Electron
Like what? Where else (that's a name brand platform and not, like, some obscure blog post's cobbled-together thing) can I start a project, push one button, and get binaries for all major platforms? Until you solve that people will keep using Electron.
There are quite a few options. Many of them look dated though. I think that's the usp of electron.
Hope dies last, as they say.
Then again, after many, many years of claims that the following year would be the year of the Linux Desktop, there seems to be more and more of a push into that direction. Or at least into a significant increase in market share. We can thank a current head of state for that.
like 1973 oil crisis? End of V8 engines (pun intended)
Yeah, like that. Modern engines eclipsed pre-1970s engines in performance, efficiency, and even (yes I said it) in reliability.
At a cost of simplicity and beauty. And two lost decades of mediocre performance. Sigh
The demand is being driven by inference though. I really don't think there will be much motivation.
The large models are incredibly inefficient. We'll be squeezing them down for generations.
Right, that's where the major push is right now. Not with shrinking down some code libraries.
oh that would be a dream
Using a lot less RAM often implies using more CPU, so even with inflated RAM prices, it's not a good tradeoff (at least not in general).
In practice, you generally see the opposite. The "CPU" is in fact limited by memory throughput. (The exception is intense number crunching or similar compute-heavy code, where thermal and power limits come into play. But much of that code can be shifted to the GPU.)
RAM throughput and RAM footprint are only weakly related. The throughput is governed by the cache locality of access patterns. A program with a 50MB footprint could put more pressure on the RAM bus than one with a 5GB footprint.
You're absolutely right? I don't really disagree with anything you're saying there, that's why I said "generally" and "in practice".
Reducing your RAM consumption is not the best approach to reducing your RAM throughput is my point. It could be effective in some specific situations, but I would definitely not say that those situations are more common than the other ones.
I don't understand how this connects to your original claim, which was about trading ram usage for CPU cycles. Could you elaborate?
From what I understand, increasing cache locality is orthogonal to how much RAM an app is using. It just lets the CPU get cache hits more often, so it only relates to throughout.
That might technically offload work to the CPU, but that's work the CPU is actually good at. We want to offload that.
In the case of Electron apps, they use a lot of RAM and that's not to spare the CPU
> increasing cache locality is orthogonal to how much RAM an app is using. It just lets the CPU get cache hits more often, so it only relates to throughout.
Cache misses mean CPU stalls, which mean wasted CPU (i.e. the CPU accomplises less than it could have in some amount of time).
> In the case of Electron apps, they use a lot of RAM and that's not to spare the CPU
The question isn't why apps use a lot of RAM, but what the effects of reducing it are. Redcuing memory consumption by a little can be cheap, but if you want to do it by a lot, development and maintenance costs rise and/or CPU costs rise, and both are more expensive than RAM, even at inflated prices.
To get a sense for why you use more CPU when you want to reduce your RAM consumption by a lot, using much less RAM while allowing the program to use the same data means that you're reusing the same memory more frequently, and that takes computational work.
But I agree that on consumer devices you tend to see software that uses a significant portion of RAM and a tiny portion of CPU and that's not a good balance, just as the opposite isn't. The reason is that CPU and RAM are related, and your machine is "spent" when one of them runs out. If a program consumes a lot of CPU, few other programs can run on the machine no matter how much free RAM it has, and if a program consumes a lot of RAM, few other programs can run no matter how much free CPU you have. So programs need to aim for some reasonable balance of the RAM and CPU they're using. Some are inefficient by using too little RAM (compared to the CPU they're using), and some are inefficient by using too little CPU (compared to the RAM they're using).
> Cache misses mean CPU stalls, which mean wasted CPU (i.e. the CPU accomplises less than it could have in some amount of time).
Yeah, I was saying CPU cache hits would result in better performance. The creator of Zig has argued that the easiest way to improve cache locality is by having smaller working sets of memory to begin with. No, it's not a given this will always work in every case. You can reduce working memory and not have better cache locality. But in a general sense, I understand why he argues for it.
> So programs need to aim for some reasonable balance of the RAM and CPU they're using
I agree with this, but
> but if you want to do it by a lot, development and maintenance costs rise and/or CPU costs rise, and both are more expensive than RAM, even at inflated prices
I would like you to clarify further, because saying CPU costs are more expensive than RAM costs is a bit misleading. A CPU might literally cost more than RAM, but a CPU is remarkably faster, and for work done, much cheaper and more efficient, especially with cache hits.
You had originally said
> It could be effective in some specific situations, but I would definitely not say that those situations are more common than the other ones
This is what I'm confused on. Why do you think most cases wouldn't benefit from this? Almost every app I've used is way on one end of the spectrum with regards to memory consumption vs CPU cycles. Don't you think there are actually a lot of cases where we could reduce memory usage AND increase cache locality, fitting more data into cache lines, avoiding GC pressure, avoiding paging and allocations, and the software would 100% be faster?
> But in a general sense, I understand why he argues for it.
Andrew is not wrong, but he's talking about optimisations with relatively little impact compared to others and is addressing people who already write software that's otherwise optimised. More concretely, keeping data packed tighter and reducing RAM footprint are not the same. The former does help CPU utilisation but doesn't make as big of an impact on the latter as things that are detrimental to the CPU (such as switching from moving collectors to malloc/free).
> Why do you think most cases wouldn't benefit from this?
The context to which "this" is referring to was "Reducing your RAM consumption is not the best approach to reducing your RAM throughput is my point." For data-packing, Andy Kelley style, to reduce the RAM bandwidth, the access patterns must be very regular, such as processing some large data structure in bulk (where prefetching helps). This is something you could see in batch applications (such as compilers), but not in most programs, which are interactive. If your data access patterns are random, packing it more tightly will not significantly reduce your RAM bandwidth.
> Andrew is not wrong... and is addressing people who already write software that's otherwise optimised
I'm getting lost. What are we talking about if not that? Because if you're talking about unoptimized software, you can absolutely reduce RAM consumption without putting extra load on the CPU. Using a language that doesn't box every single value is going to reduce RAM consumption AND be easier on the CPU. Which is what most people are talking about on this post.
> The context to which "this" is referring to was "Reducing your RAM consumption is not the best approach to reducing your RAM throughput is my point."
I'm more interested in the original claim, which was
> Using a lot less RAM often implies using more CPU
There are a lot of apps using a lot of RAM, and it's not to save CPU. So where is "often" coming from here? I think there are WAY more apps that could stand to be debloated and would use less CPU.
It feels like you're coming at this from a JVM perspective. Yeah, tweaking my JVM to use less RAM would result in more CPU usage. But I don't think there's a single app out there as optimized as the JVM is. They use more RAM for other reasons.
> If your data access patterns are random, packing it more tightly will not significantly reduce your RAM bandwidth
Packing helps random access too. A smaller working set means more of your random accesses land in cache. Prefetching is one benefit of packing, but cache and TLB pressure reduction is the bigger one, and it applies regardless of access pattern
> Using a language that doesn't box every single value is going to reduce RAM consumption AND be easier on the CPU. Which is what most people are talking about on this post.
What popular language does that? I admit that rewriting the software in a different language could lead to better efficiencies on all fronts, but such massive work is hardly "an optimisation", and there are substantial costs involved.
But more importantly, I don't think it's right. Removing boxing can certainly have an impact on RAM footprint without an adverse effect on CPU, but I don't think it's a huge one. RAM footprint is dominated by what data is kept in memory and the language's memory management strategy (malloc/free vs non-moving tracing collectors vs moving collectors), and changing either one of these can very much have an adverse effect on CPU.
> There are a lot of apps using a lot of RAM, and it's not to save CPU. So where is "often" coming from here?
That the developers may not be conscious of the RAM/CPU tradeoff doesn't mean it's not there. Keeping less data in memory (and computing more of it on demand) can increase CPU utilisation as can switching from a language with a moving collector to one that relies on malloc/free.
> Packing helps random access too. A smaller working set means more of your random accesses land in cache.
Unless your entire live set fits in the cache, what matters much more is the temporal locality, not the size of the live set. If your cache size is 50MB, a program with a 1GB live set could have just as many or just as few cache misses as a program with a 100MB live set. In other words, you could reduce your live set by a factor of 10 and not see any improvement in your cache hit rate, and you can improve your cache hit rate without reducing your live set one iota.
For example, consider a server that caches some session data and evicts it after a while. Reducing the allowed session idle time can drastically reduce your live set, but it will barely have an effect on cache locality.
Tighter data layouts absolutely improve cache behaviour, but they don't have a huge effect on the footprint. Coversely, what data is stored in RAM and your memory management strategy have a large effect on footprint but they don't help your cache behaviour much. In other words, Andy Kelley's emphasis on layout is very important for program speed, but it's largely orthogonal to RAM footprint.
Only if the software is optimised for either in the first place.
Ton of software out there where optimisation of both memory and cpu has been pushed to the side because development hours is more costly than a bit of extra resource usage.
That'll stay true for consumer software, because the cost for extra resource usage is not borne by the development house.
The tradeoff has almost exclusively been development time vs resource efficiency. Very few devs are graced with enough time to optimize something to the point of dealing with theoretical tradeoff balances of near optimal implementations.
That's fine, but I was responding to a comment that said that RAM prices would put pressure to optimise footprint. Optimising footprint could often lead to wasting more CPU, even if your starting point was optimising for neither.
My response was that I disagree with this conclusion that something like "pressure to optimize RAM implies another hardware tradeoff" is the primary thing which will give, not that I'm changing the premise.
Pressure to optimize can more often imply just setting aside work to make the program be nearer to being limited by algorithmic bounds rather than doing what was quickest to implement and not caring about any of it. Having the same amount of time, replacing bloated abstractions with something more lightweight overall usually nets more memory gains than trying to tune something heavy to use less RAM at the expense of more CPU.
You're thinking an algorithmic tradeoff, but this is an abstraction tradeoff.
Some of the algorithms are built deep into the runtime. E.g. languages that rely on malloc/free allocators (which require maintaining free lists) are making a pretty significnant tradoff of wasting CPU to save on RAM as opposed to languages using moving collectors.
Free lists aren't expensive for most usage patterns. For cases where they are we've got stuff like arena allocators. Meanwhile GC is hardly cheap.
Of course memory safety has a quality all its own.
hopefully not implying needing a gc for memory safety...
Yeah, there's always Fil-C (Rust isn't memory safe in practice).
> Free lists aren't expensive for most usage patterns.
Whatever little CPU they waste is often worth more than the RAM they save.
> For cases where they are we've got stuff like arena allocators.
... that work by using more RAM to save on CPU.
GC burns far more CPU cycles. Meanwhile I'm not sure where you got this idea about the value of CPU cycles relative to RAM. Most tasks stall on IO. Those that don't typically stall on either memory bandwidth or latency. Meanwhile CPU bound tasks typically don't perform allocations and if forced avoid the heap like the plague.
> GC burns far more CPU cycles
Far less for moving collectors. That's why they're used: to reduce the overhead of malloc/free based memory management. The whole point of moving collectors is that they can make the CPU cost of memory management arbitrarily low, even lower than stack allocation. In practice it's more complicated, but the principle stands.
The reason some programs "avoid the heap like the plague" is because their memory management is CPU-inefficient (as in the case of malloc/free allocators).
> Meanwhile I'm not sure where you got this idea about the value of CPU cycles relative to RAM
There is a fundamental relationship between CPU and RAM. As we learn in basic complexity theory, the power of what can be computed depends on how much memory an algorithm can use. On the flip side, using memory and managing memory requires CPU.
To get the most basic intuition, let's look at an extreme example. Consider a machine with 1 GB of free RAM and two programs that compute the same thing and consume 100% CPU for their duration. One uses 80MB of RAM and runs for 100s; the other uses 800MB of RAM and runs for 99s (perhaps thanks to a moving collector). Which is more efficient? It may seem that we need to compare the value of 1% CPU reduction vs a 10x increase in RAM consumption, but that's not necessary. The second program is more efficient. Why? Because when a program consumes 100% of the CPU, no other program can make use of any RAM, and so both programs effectively capture all 1GB, only the second program captures it for one second less.
This scales even to cases when the CPU consumption is less than 100% CPU, as the important thing to realise is that the two resources are coupled. The thing that needs to be optimised isn't CPU and RAM separately, but the RAM/CPU ratio. A program can be less efficient by using too little RAM if using more RAM can reduce its CPU consumption to get the right ratio (e.g. by using a moving collector) and vice versa.
There are (at least) two glaring issues with your analysis. First, the vast majority of workloads don't block on CPU (as I previously pointed out) and when they do they almost never do heap allocations in the hot path (again, as I previously pointed out). Second, we don't use single core single thread machines these days. Most workloads block on IO or memory access; the CPU pipeline is out of order and we have SMT for precisely this reason.
Anyway I'm not at all inclined to blindly believe your claim that malloc/free is particularly expensive relative to various GC algorithms. At present I believe the opposite (that malloc/free is quite cheap) but I'm open to the possibility that I'm misinformed about that. You're going to need to link to reputable benchmarks if you expect me to accept the efficiency claim, but even then that wouldn't convince me that any extra CPU cycles were actually an issue for the reasons articulated in the preceding paragraph.
> There are (at least) two glaring issues with your analysis. First, the vast majority of workloads don't block on CPU (as I previously pointed out) and when they do they almost never do heap allocations in the hot path (again, as I previously pointed out). Second, we don't use single core single thread machines these days. Most workloads block on IO or memory access; the CPU pipeline is out of order and we have SMT for precisely this reason.
This doesn't matter because if you're running a single program on a machine, it might as well use all the CPU and all the RAM. As long as you're under 100% on both, you're good. But we want to utilise the hardware well because we typically want to run multiple programs (or VMs) on a single machine, and the machine is exhausted when the first of CPU or RAM is exhausted. So the question is how should your CPU and RAM usage be balanced to offer optimal utilisation given that the machine is spent when the first of CPU and RAM is spent. E.g. you can only run two programs, each using 50% of CPU; if they each use only 5% of RAM, you've saved nothing as no third program can run. So if you spend either one of these resources in an unbalanced way, you're not using your hardware optimally. Using 2% more CPU to save 200MB of RAM could be suboptimal.
I'm not saying that for every program that uses X% CPU should also use exactly X% of RAM or it must be wasting one or the other, but that's the general perspective of how to think about efficiency. Using a lot of one and little of the other is, broadly speaking, not very efficient.
> Anyway I'm not at all inclined to blindly believe your claim that malloc/free is particularly expensive relative to various GC algorithms. At present I believe the opposite (that malloc/free is quite cheap) but I'm open to the possibility that I'm misinformed about that.
You are.
> You're going to need to link to reputable benchmarks if you expect me to accept the efficiency claim, but even then that wouldn't convince me that any extra CPU cycles were actually an issue for the reasons articulated in the preceding paragraph.
I don't believe there are any reputable benchmarks of full applications (which is where memory-management matters) that are apples-to-apples. I'm speaking from over two decades of experience with C++ and Java.
The important property of moving collectors is that they give you a knob that allows you to turn RAM into CPU and vice-versa (to some extent), and that's what you want to achieve the efficient balance.
Moving collectors as generally used are a huge waste of memory throughput, and this shows up consistently in the performance measurements. Moving data is very expensive! The whole point of ownership tracking in programming languages is so that large chunks of "owned" data can just stay put until freed, and only the owning handle (which is tiny) needs to move around. Most GC programming languages do a terrible job of supporting that pattern.
That's just not true. To give you a few pieces of the picture, moving collectors move little memory and do so rarely (relative to the allocation rate):
In the young generation, few objects survive and so few are moved (the very few that survive longer are moved into the old gen); in the old generation, most objects survive, but the allocation rate is so low that moving them is rare (although the memory management technique in the old gen doesn't matter as much precisely because the allocation rate is so low, so whether you want a moving algorithm or not in the old gen is less about speed and more about other concerns).
On top of that, the general principle of moving collectors (and why in theory they're cheaper than stack allocation) is that the cost of the overall work of moving memory is roughly constant for a specific workload, but its frequency can be made as low as you want by using more RAM.
The reason moving collectors are used in the first place is to reduce the high overhead of malloc/free allocators.
Anyway, the general point I was making above is that a machine is exhausted not when both CPU and RAM are exhausted, but when one of them is. Efficient hardware utilisation is when the program strikes some good balance between them. There's not much point to reducing RAM footprint when CPU utilisation is high or reducing CPU consumption when RAM consumption is high. Using much of one and little of the other is wasteful when you can reduce the higher one by increasing the other. Moving collectors give you a convenient knob to do that: if a program consumes a lot of CPU and little RAM, you can increase the heap and turn some RAM into CPU and vice versa.
Or just using less electron and writing less shit code.
I'm a bit surprised the article makes no mention of China's new memory companies.
[0] https://techwireasia.com/2026/04/chinese-memory-chips-ymtc-c...
As the article states:
>CXMT still trails Samsung, SK Hynix, and Micron by approximately three years in advanced DRAM node development, and yield rates on new production lines remain the variable that determines whether capacity targets translate into reliable supply. Liu notes that lines launched in the second half of 2026 are unlikely to change the global supply-demand balance until 2027.
The Verge article talks about demand exceeding supply in 2028. Your article suggests it'll take until 2029 before Chinese production catches up to current technology.
It'll help drive prices down in five yearss, but the Chinese memory production won't be ready and efficient enough to prevent the shortages from continuing to grow.
Most people don't need current tech. Ddr4 is good enough
You could buy CXMT DDR5 modules like, right now.
As an aside, recently I wanted to refresh my gaming PC, but the price shock and general lack of availability of buying components individually made it seem hardly worth it, so I just kept deferring the project.
Then, mostly by chance, I saw that my local Microcenter had some pre-builts for sale, and I ended up picking one up for <$5k that had "best in slot" components across the board, including a 5090 and even a high-end power supply.
The last time I built a gaming PC was upwards of a decade ago, and at that time the prevailing wisdom was to never buy a pre-built unless you had a massive amount of disposable income and couldn't spare even just one weekend to dedicate to a hobby project that could benefit you for years. Now, it was absolutely a no-brainer.
> I ended up picking one up for <$5k
I'm struggling to put this in context. For comparison, what was your budget for refreshing the pc you had? Were the planned upgrades going to exceed $5k at current prices? Or is the situation that a pre-build machine with far better components was now only marginally more?
Or is it that pre-built gaming PCs have stopped being a joke? I had the experience building a bicycle: I was certain I was taking the frugal path sourcing each component individually and putting it together myself. At the end I was horrified to realize I spent far more than a new bike with superior components. It was pointed out that bicycle makers are buying by the pallet and will beat diy every time — so long as they're building something I want to buy.
I did the exact same thing during Covid, the prebuilt ended up being ~20% cheaper than buying the individual components (I needed a full upgrade). Maybe a little less since I could have reused my case.
> and at that time the prevailing wisdom was to never buy a pre-built
That's still the case, and always will be — with a pre-built you're at the very least paying for someone to assemble it for you, so it's always going to be more expensive as a baseline.
Beyond that, the chance they've chosen good components and haven't tried to screw you over on less flashy ones like the motherboard and power supply is low.
That's not to say it's literally impossible to ever find a good deal. You very well might have. Doesn't change anything though.
> with a pre-built you're at the very least paying for someone to assemble it for you, so it's always going to be more expensive as a baseline
Except isn't it possible that pre-built companies actually get better deals on hardware bought in bulk, and therefore could offset the labor costs with cheaper materials?
I believe this is exactly what's going on -- they're buying parts in bulk, often months in advance, and locking in deals that a single consumer can't easily go get on the open market right now.
Hardware pricing and availabilty pre-COVID was pretty predictable and stable, which meant the consumer could extract a meaningful cost advantage if they were willing to do the relatively modest amount of work of sourcing components individually and personally assembling the build. Right now, though, some places like Microcenter appear to have a cost advantage that fundamentally relies on market and pricing instability and can only be achieved through deeper integration with the supply chain and bulk purchasing in advance -- something a retailer like Microcenter can do, but I personally cannot.
The era of optimisation is finally here. I'm excited.
I'm skeptical - the apps I use either have a) enough lock-in that they don't have the institutional will to optimize or b) a lack of institutional resources to optimize.
Basically, the optimizing that can happen is that I ditch heavy tools in favour of lighter ones, and hopefully enough other people do the same to help lighter tools with finances/dev resources.
That lock-in is going to have to be very deep to fend off reverse engineered clones.
Yeah, I mean, if I look at the trashy apps I currently have running, they all do have deep lock-in that makes the reverse-engineered clones impractical or impossible: Teams, Discord, Slack, the Windows RDP app.
If I look at the Activity Manager in macOS, of apps that are less trashy but currently taking up a lot of memory, they mostly aren't apps that I'm willing/able to move away from to save on resource use: Firefox, Safari, 1Password. (For browsers, you can blame poorly optimized websites for a lot of it, but I just don't see anyone rushing to create lightweight clones of websites in order to save users' RAM.)
I said it for many years that OS developers need to focus on over optimisations. If it wasnt a chip sgortage it would be the ever slowing progress on chip scaling.
But software optimisation helps all hardware and that doesnt drive sales.
Linux however, they dont have to worry about that. Maybe it is finally the era of Haiku OS as the ghost of BeOS rises!
There is almost no way any hyper optimized OS could protect you from Windsurf re-layouting the whole UI 60 times per second because of badly implemented spinner, and taking 8 GB of RAM to show it.
Wait until China invades Taiwan.. (ok, it's not too likely, but what if?)
I think RAM shortages would be the least of our problems…
Assuming China takes TSMC in one piece (unlikely without internal sabotage in the best case scenario), it would still probably take years before it produces another high end GPU or CPU.
We would probably be stuck with the existing inventory of equipment for a long time…
I am surprised we consider TSMC like a natural resource: isn't it really a combination of know-how and build-out according to that know-how? If smarts leave the country, perhaps this moves with them.
The risk with China taking over Taiwan is that they mostly expedite their own production research by a couple of years.
It kinda does resemble a natural resource though. The machines and technology in use at TSMC are so insanely complex, that there isn't a single person on earth who knows everything about how it works. TSMC functions only because of all of the pieces of the puzzle being together in the right place and arranged in just the right way. It's a very fragile balance that keeps it all running, and a major disruption could mean we get thrown back by a decade in chip-making technology.
What you say is absolutely true, and is a serious problem—but the way our system operates does not allow us to correct for it.
Anyone trying to spin up a competitor to TSMC would have to first overcome a significant financial hurdle: the capital investment to build all the industrial equipment needed for fabrication.
Then they'd have to convince institutions to choose them over TSMC when they're unproven, and likely objectively worse than TSMC, given that they would not have its decades of experience and process optimization.
This would be mitigated somewhat if our institutions had common-sense rules in place requiring multiple vendors for every part of their supply chain—note, not just "multiple bids, leading to picking a single vendor" but "multiple vendors actively supplying them at all times". But our system prioritizes efficiency over resiliency.
A wealthy nation-state with a sufficiently motivated voter base could certainly build up a meaningful competitor to TSMC over the course of, say, a decade or two (or three...). But it would require sustained investment at all levels—and not just investment in the simple financial sense; it requires people investing their time in education and research. Dedicating their lives to making the best chips in the world. And the only reason that would work is that it defies our system, and chooses to invest in plants that won't be finished for years, and then pay for chips that they know are inferior in quality, because they're our chips, and paying for them when they're lower quality is the only way to get them to be the best chips in the world.
China is 10 years into what you describe, no?
> the way our system operates
They have the other system.
This bit, I mean:
> A wealthy nation-state with a sufficiently motivated voter base could certainly build up a meaningful competitor to TSMC over the course of, say, a decade or two (or three...).
Yes. And then taking down TSMC will be a nuclear bomb that wipes everyone else’s economy at once.
They don't even need to nuke TSMC to wipe everyone else out.
They just need to blockade Taiwan.
> I am surprised we consider TSMC like a natural resource: isn't it really a combination of know-how and build-out according to that know-how?
Have you seen how many states and countries look enviously at Silicon Valley’s tech companies, China’s manufacturing dominance, or London’s financial sector and try to replicate them?
Turns out it’s way harder than you’d expect.
Hell, Intel can’t match TSMC despite decades of expertise, much greater fame, and regulators happy to change the law and hand out tens of billions in subsidies.
With you on the first two, but I haven't heard of London's financial sector being a big deal, what's going on there?
You really haven't heard? It's like the only thing the British economy has going for it.
They are world experts at legalizing the corruption profits from both first world and third world countries, just research HSBC.
London has been the capitol of capital since before capitalism.
the scientists will switch sides with minimal issues, like they did after WWII
Does China need to take Taiwan? American protection of the island seems to be a waning concern. I could see that being a peaceful event now. https://www.bbc.com/news/articles/cj94y87k2ljo
The chance of significant disruption is higher than the chances of a full blown invasion. China has hybrid options like a quarantine of chip exports.
It seems that RAM manufacturers are still reluctant to increase production. They know something that investors don't about long term RAM demands?
They've been burned before. The DRAM industry has a long history of booms and busts.
Demand increased, everyone built new fabs, then prices dropped and they couldn't pay off their investments. Many went out of business. It happened in the 80s, it happened in the 90s, it happened in the 2000s.
Now there's only three manufacturers left, and they know very well that demand for their product tends to be cyclical.
What’s the lose scenario for them? They’re basically a cartel, and you need ram irregardless. If they make less it’s still a cost:demand, just not the most optimal for them. They’ve done that math, and figure this is the best risk and reward for them. Your goodwill or opinion doesn’t matter to them, because you need them more than they need you.
> They’re basically a cartel,
The lawsuits in the past prove that statement to not be basically but actually.
The same thing everyone who's paying attention to the real world (and not the financial fantasy world) does: that OpenAI's purchase commitments are wildly unrealistic and unsustainable.
The semiconductor industry has been a boom and bust industry for over 50 years.
https://imgur.com/a/cDLoeZm
I've been in the industry for 30 years and I've worked at companies with fabs were demand was high and customers would only get 30% of what they ordered. Then just 2 years later our fab was only running at 50% capacity and losing money. It takes about $20 billion and 3-4 years to make a modern new fab. If you think that AI is a bubble then do you want to be left with a shiny new factory and no products to sell because demand has collapsed?
They are not happy to invest tens of billions for new capacities for Sam Altman’s “I will surely buy once I have money, pinky promise”.
They're doing this but it takes time and if they overshoot even a little they're screwed.
I wonder if this might motivate to write more memory efficient software. I mean we have so much memory, but even some trivial programs eat hundreds of megabytes of ram.
I've definitely done some vibe-coding with the explicit intent to reduce memory usage.
How efficient is AI at reducing RAM consuption?
This feels like an oxymoron
I fear that the real reason we do have a shortage, I mean, the real reason for the demand, is AI companies scooping what they can so that their competitors, whether existing or incumbent, can’t get to it.
This was one of the theories behind the wafer buyout by OpenAI indeed. Pretty efficient way to make everyone panic and cut off of new hardware.
Was it debubked in any way (e.g. by OpenAI actually showing what they do with the wafers?)
It will last forever. After covid, all manufacturers understood the value of limiting supply and extracting profits. Cars used to super cheap before covid, they will never go back to the same levels.
From now on, RAM will always be super costly for consumers, because they can't make massive deals like Apple/OpenAI/etc. We are the bagholders.
Of course, clearly businesses were just stupid before and only learned how to make revenue in 2021.
When lithium prices decreased over 80% from 2022 to 2025, it was because lithium miners felt altruistic. Car manufacturers were feeling greedy. This is how bad the thinking has gotten.
>> Cars used to super cheap before covid
Have they really ever been cheap? Also Tesla 3 is cheaper now, Yaris is still cheap as well.
It won't. DRAM prices are cyclical. They were super high in 2020, then demand crashed in 2022 to the point that manufacturers couldn't sell all their inventory.
Now it's high again, but give it a couple years and it'll once again crash.
Covid inflation was because of supply chain disruptions, loose fiscal policy (like Biden's ARP which a Central Bank analysis said added a few % to annual inflation), and money supply expansions. There was less goods and more money. When you go and trade money for goods, it should be obvious what happens.
If only we have not allowed oligopolies to exist. Meanwhile, EU is not in the race at all and US has very few fabs.
People don't want to pay more every day so they can pay less in an emergency.
if a shortage lasts years, it's not a shortage. "The market clearing price of RAM in the face of expected sustained healthy demand should lead to a stable market for years."
even if gaming is and will remain very popular for years, it and the desire to upgrade gaming rigs is still a discretionary activity with more price elasticity of demand than corporate uses for RAM in the dawn of the AI age. gamers live on the margin of this market, where low prices will stimulate upgrades and high prices will lead to holding out. The complaints about price are real, but that segment of the market is some combination of less large and less important.
It’s not merely a “gaming vs data center“. There’s so many other places DRAM and NVM are needed - mobile, automotive, other consumer electronics,… the current situation is that _all_ of that is deprived of the memory that it needs. And much of this is critical to the real economy.
i could have used lemons and lemonade stands to explain supply and demand, the lesson is still the same.
letting the market set prices ensures that the chips go to the critical markets and uses. less critical uses will not allocate funds for purchases.
letting the market set prices ensures that the chips go to the critical markets and uses.
Can you please elaborate what you mean by "critical market"?
Edit: formatting
Why are you only talking about gamers? Apple, the most cautious planners in the whole industry have straight up cancelled their 512gb RAM Mac Studio. Don’t ask; they won’t sell you one.
Everybody’s getting pinched, not just the gamers.
The issue is supply is inelastic so even as prices soar they can only make more so fast and that won’t get fixed until 2028.
I've read that the chip manufacturers are looking into high bandwidth flash for on package storage of ai models. That would solve some of the cost issue, flash is significantly cheaper than dram.
I'm personally hoping that one of the AI or data center companies is suddenly unable to pay for their bills and deflate the entire industry. Probably the only hope of things getting better before the 2030s.
That’s likely to happen if all the talks about OpenAI pulling out of their wafer deals are true.
Of course, alternatively, the AI companies could go bust before finding profitability. Then, there’d be a ton of supply, prices would crash, and one or two of the current memory suppliers would go out of business. After that, the new Chinese memory companies might be producing at volume, and Renesas could be up and running.
At the moment, nothing is certain. Could this last? Sure. Could it not last? Yup.
Hilarious, The ram in my PC i built 5 years ago is will soon be worth more than i spent on building the whole PC.
I was about to give away my old PC, but I think it could be worth my hassle to sell it for the RAM now (64GB DDR4).
let the analyst and news say what they want - the entire situation is artificial and is up to the manufacturers.
the current relative spike in the prices misses the medium-term trend of the vast decrease in memory price post-covid that led to the recent surge. the cartel got another opportunity to make bank and they will use that lever to the max.
funnily enough i've been personally stuck with 16 gigs since 2015, across three memory generations! but i am used to the past when you would spend 80-100 on an 8gb stick (jdec timings, nothing fancy but from a major brand) without accounting for inflation.
The RAM market is a square wave
We have the saying in my country: The days of things being cheap are over.
Your instincts are likely right on this one OP. Memory prices surged 80–90% in Q1 2026 compared to Q4 2025, DRAM, NAND, and HBM all at record highs. 3 suppliers for the entire planet?
This is simple extrapolation from current demand, nothing more. And that's a borderline silly analysis because it assumes the AI bubble won't burst. The great misadventure in the Persian Gulf probably accelerates that because we're almost certainly going to be facing a recession.
Another thing I've been thinking about is what happens when the next generation of NVidia chips comes out? I suspect NVidia is going to delay this to milk the current demand but at some point you'll be able to buy something that's better than the H100 or B200 or whatever the current state-of-the-art for half the price. And what's that going to do to the trillions in AI DC investment?
I'm interested when the next bump in DRAM chip density is coming. That's going to change things although it seems like much of production has moved from consumer DRAM chips to HBM chips. So maybe that won't help at all.
I do think that companies will start seeing little ot no return from billions spent on AI and that's going to be aproblem. I also think that the hudnreds of billions of capital expenditure of OpenAI is going to come crashing down as there just isn't any even theoretical future revenue that can pay for all that.
something that's better than the current state-of-the-art for half the price. And what's that going to do to the trillions in AI DC investment?
They'll just spend whatever they were planning to spend and get more performance.
It sounds to me like an incentive for new companies to make RAM.
I want those AI companies that drove the prices up, to pay an immediate back-tax to all of us.
I don't want to pay more because of AI companies driving the price up. That is milking.
Are we entering the Reverse-Moore's Law era.
I just checked my gaming PC I built a few years ago with 64GB of DDR5 RAM, its actually gone up in value, that is unheard of generally.
Think I will scrap my PC and sell its parts.
I wonder if there are any niche companies building decent rigs with DDR3 and 5/6th generation Intel CPUs out there, it is cheap and might be a business opportunity?
I work at an e-waste recycling company. I have several dozen trays of RAM in my inventory, ~90% of it DDR3. DDR3 was selling as of a month ago, but I haven't tried to sell any RAM since. I'm looking forward to doing a huge one this week.
do you have an online storefront?
We have an Ebay store: https://www.ebay.com/str/evolutionecycling
I am still running a DDR3 2nd Gen i7. 32GB RAM, it is surprisingly comfortable but I also dont push it too hard.
Same. Core i7 2600K clocked to 4.4GHz with 32GB DDR3. It still does its job as my stationary DAW, and basically handles anything DAW-related I throw at it with ease. The only issue is its lack of AVX2 support, and since this is required by Ableton Live 12, I'll be stuck at Ableton 11 forever.
Im thankfull for buying 16gb of RAM, but what is gonna happen in 5 years when users PCs start to fail?
What do you mean, in 5 years? It's not like everyone just bought a new computer. My gut says it's exactly the other way around: most computers are old. They may fail as soon as today.
All computers in my household are 8+ years old.
There is enough older hardware floating around to last us for decades. You don't need a gaming rig to do 99% of your computing (excluding gaming obviously). Also computers don't really just break. It's mostly the disks that wear out and PSUs that age.
Usually, right after articles like this, things come crashing down.
I bought a workstation with 3 TB of ram for FDTD simulations last year. Glad I got it then ...
When I personally use chatgpt and friends, I am not seeing any slowdowns or anything, meaning that their servers can handle the loads just fine. So then, why are these companies spending so much building new capacity if the current capacity is enough?
Frontier labs flagship models are ~2T params at the moment, but they intend to ship 10T models like Claude Mythos, which would require substantial datacenter expansion. Same thing for training.
Where did you get the 10T figure from? I thought it was a big secret.
Rumors and extrapolated from the token price.
Oh, we have estimated token pricing for Mythos? Man, I'm missing out on all the juicy gossip.
so it's 5x as expensive as Opus then.
Fabricated shortage to fasten US Chip Act and US Chip Security Act
Sad news, I didn't buy enough RAM before....
Can we stop advertising paid Blog/News websites we can't read without a subscription.
Thank god they shut down 3D XPoint.
But thank god we were all able to generate some SVGs of pelicans, right guys?
This could be great.
There's a future where RAM makers tool up for this massively increased demand, then the AI companies go broke as the bubble bursts, so RAM is cheap as. So laptop manufacturers get on that and start making laptops with 1TB+ memory so we can run decent LLMs on the local machine. Everyone happy :)
RAM makers are not increasing their capacity. If AI bubble bursts we might see a momentary drop in RAM prices but it won't be dramatic. Return to "normal" is the best scenario I can imagine but my gut tells me we're probably never going back to early 2025 memory prices.
can't read the article due to a paywall.
https://archive.is/QGU89
here is the source: https://www.reuters.com/world/asia-pacific/south-koreas-sk-g...
[dead]
[dead]
Expect shortages across the board. RAM? That's the tip of the iceberg, think food and gas.
I fear the author and most commenters are not aware of the law of demand and supply. If there is demand for consumer RAM, there will be supply for consumer RAM. It just takes time and risk-assessment to scale up operations.
We have RAM shortage now, we will have very cheap RAM tomorrow. It’s not like production is bottlenecked by raw materials. Chip companies just need to assess if the demand by AI companies will last so it’s better to scale up, or perhaps they should wait it out instead of oversupplying and cutting into their profits.
We're talking about advanced semiconductor manufacture. It takes years and 100s millions to billions of dollars to scale up operations. That's something you don't do unless you know there's demand to sustain it in future.
The law of supply and demand works in a perfect competition market.
There are two RAM suppliers...
>I fear the author and most commenters are not aware of the law of demand and supply
I cannot stand how you and people like you try to justify everything by supply and demand. Also you act like it's some natural law of nature. It's not a law of nature- if you took an economics class you would realize it's try to maximize PROFIT. It's not for the good of the people.
All of these things are a CHOICE that people are making to now completely screw the average person for, again, the needs of big corporations and the top 0.01%.