Data warehousing is quickly becoming a commodity through open-source.
I know a company who had 2PBs+ of data in Cloudera. But instead of moving to the cloud (and Databricks), they saved 5X costs by building their own analytics platform with Iceberg, Trino and Superset. The k8s operators are enterprise quality now. On-premises S3 is good, too. You can have great hardware (servers with 128 cpus and 1 TB) and networking.
It's not just Trino. StarRocks and Clickhouse have enterprise grade k8s helm charts/operators. That 60bn valuation is an albtross on Databrick's neck - their pricing will have to justify it, and their core business is commoditizing.
Neon filled their product gap of not having an operational (row-oriented) DB.
Not commoditising for enterprise. My last gig wouldn’t allow open source software or any company that might not be there in a decade, or which kept data anywhere but our own tenant. We’d look for the “call us” pricing rather than hate it, which I normally do. We added databricks and it was considered one of my top three achievements, because they don’t have to think about data platforms again, just focus on using it. It’s SO expensive for an enterprise to rejig for a new platform that you can’t rely on (insert open source project here).
I managed to add one startup and so far it’s done very well, but it was an exceptional case and the global CEO wanted the functionality. But it used MongoDB and ops team didn’t have any skills, so rather than learn one tiny thing for an irrelevant data store they added cash to use Atlas with all the support and RBAC etc etc. They couldn’t use the default Azure firewall because they only know one firewall, so added one of those too. Also loaded with contracts. Kept hiring load down, one number to call, job done. Startups cost is $5-10k per year. Support BS about $40k. (I forget the exact numbers but it dwarfed the startup costs.)
Startups are from Venus, enterprise are from Jupiter.
I'm curious - as someone who hasn't been paying super close attention to the space, what does it mean to be a full data platform? Is it just having all the different flavors of DB you might need all from one vendor, or is there also tighter integration than if you cobbled it together from multiple vendors?
Essentially, yes. Different DB’s, federated queries (aka delta sharing, zero copy), definition/semantic layer tools, data engineering/pipelines, model training and notebooks, governance, data lineage, row/column/whatever access control.
It’s basically a luxury minivan. It’s may not be the fastest or prettiest or cheapest, but it’s a safe way for a large family of “data and AI people” to traverse a large organisation.
More seriously, I like to call it an “analytics workbench” in a professional setting.
Are there even any "plain data warehouse" vendors left? Even the oldest of the old-school providers seem to have crawled up the modern data stack since the data warehouse glory days, refocused on lakehouse + Open Table Formats either in their core platform or complementary products.
Yup. And OpenShift. And Red Hat for Linux. And SAP. And IBM. But you know what? Tiny group of people relative to the impact, revenue and competitors. If we needed skills we clicked “buy”, 100 consultants would arrive who are experts, sort it and we’d move on. Not scratching around looking for people who know what we use and needing to learn 50 different open source tools. Coming from a much looser universe I learned to appreciate the principles for that context.
I've worked at a Swedish streaming company, we bought a filesystem from IBM called "GPFS" or something before it was renamed. Well we had shit performance, reliability and everything else that could be bad. It was running on FC switches recommended by IBM, config from IBM, everything from IBM and it was an absolute fucking joke, metadata timed out regularly and the solution was "deal with it". I think my superiors should've been harder claiming we haven't received what we bought but we're Swedish so we'd rather just fall over and die.
> Not commoditising for enterprise. My last gig wouldn’t allow open source software or any company that might not be there in a decade, or which kept data anywhere but our own tenant.
Totally agree. Happy open source StarRocks user here using the k8s operator for customer-facing analytics on terabytes of data. There's very little need for Databricks in my world.
Looking at StarRocks site (https://www.starrocks.io/), they compare against Clickhouse, Druid and Trino. Don't even compare against Spark/Databricks! Guess Spark is just not competitive.
On-premise open-source S3 is a problem though. MinIO is not something we're touching and other than that it looks a bit empty with enterprise ready solutions.
Great to see cost-effective alternatives to Cloudera and Databricks! We’ve spent three years building IOMETE, a self-hosted data lakehouse that combines Apache Iceberg and Spark, designed to run natively on Kubernetes. We’re focused on on-premises deployments to address the growing need for data sovereignty and low TCO, with a streamlined setup for large-scale analytics. Early adopters are seeing strong results. Curious about your experience with Trino and Superset—any tips for optimizing performance at scale?
Ceph would be a theoretical option, but a) we don't have a lot of experience with it and b) it's relatively complex to operate. We'd really love to add a lighter option to our stack that's under the stewardship of a foundation.
Try expanding a cluster, or changing erasure coding configuration, or using anything that needs random access within a file (parquet), or any day 2 operation.
Even some basic s3 storage patterns weren’t considered when the core storage scheme was designed. Lacks an index and depends on filesystem to organize objects and then crumbles to lock contention when too many versions are stored or under walkdir calls when anything is listed. It also can’t even support writing to the same set of keys as S3 should allow since it implicitly depends on underlying filesystem paths.
They might have added an index by now but gatekept it to their enterprise AIStor offering since they’ve abandoned any investment in open source at this point or appearance that they care about that. Their initial inclination in response to this issue says everything - https://github.com/minio/minio/issues/20845#issuecomment-259...
Guessing you’re referring to minio not ceph? Have they still not figured out how to do day 2? I mainly avoid them because of their license and the way they interpret it
They are not efficient; they have a one-time static hash to create a cluster. After that, it is all duct tape and glue. Want to expand? Add another cluster (pool) and then look for the cluster that contains the object. They don't know which cluster has the object, and performance does not scale as well with additional clusters. Want to decommission a single node, drain the cluster. They refer to multiple pools as a single cluster, but it is essentially a set of static hashes that lack the intelligence to locate objects. Got the initial EC configuration not quite right.. sorry need to redo the entire cluster.
MinIO is a good fit if you want a small cluster that doesn't require day 2 operational complexity, as you only store a few TBs.
I have not looked into them recently, but I doubt the core has changed.
Being VC-funded and looking for an exit makes them overinvest in marketing and story telling.
It's been a commodity for decades now. Metrics like price-performance have a long history, but the SnowBricks products fail at them quite dramatically. The difference is hard-sell vs. soft or no-sell.
Not having to buy an appliance and pay for it up front is quite a valuable option. Also the split between processing and storage allows for better archival and scaling strategies.
I have been happily using ClickHouse for the past couple of years without any issue. Rock solid database with wide variety of features and fulfills all my needs. My favourite is the "external dictionary" feature which easily allows me to integrate it with other datastores like Postgres and Redis.
In addition to the AI use cases, sometimes you wanna share the data warehouse data in oltp way for fast lookups and high concurrency. Not sure whether Neon will do that but I hope so.
One example from Snowflake is hybrid tables which adds rowstore next to columnar.
I wonder why Singlestore has been so unpopular (at least I never hear about it). Quick guess is that HTAP itself isn’t a significant feature requirement, maybe just a cherry on top of other major db features.
it was a couple things.
1. It's really really hard to replace anyone's OLTP.
2. OLTP and OLAP are owned by such different teams. Who do you make your champion?
3. The modern HTAP dream is possible without something like SingleStore. You need a columnstore that can keep up with your OLTP tables and provide transactional correctness. Who cares if it's all within one system
ETL to bring all your data into Databricks/Snowflake is a lot of effort. Much better if your OLTP data already exists in Databricks and you directly access it from your OLAP layer.
With the push towards open table formats (Iceberg) from both Snowflake and Databricks, it's even harder to get your Postgres OLTP tables ready for OLAP.
The problem isn't in the CDC / replication tools in the market.
The problem is that columnar stores (especially Iceberg) are not designed for the write /upserts patterns of OLTP systems.
They just can't keep up...
This is a big problem we're hoping to solve at Mooncake [0]. Turn Iceberg into an operational columnstore. So that it can be keep up (<s freshness) with your Postgres.
DataFile(parquet) is not enough for table with update/delete, (they are part of iceberg "metadata").
for CDC from OLTP use-cases, the pattern involves rapidly marking rows as deleted/ insert new rows and optimizing small files. This is required for minutes-latency replication.
And for second latency replication, it is more involving, you actually need to build layer on top of iceberg to track pk/ apply deletion.
if Databricks just wanted a row DB they couldve done postgres themselves. paying this much for Neon i think is a sign that Neon has something special they want (which, knowing their marketing line, is "independently scalable storage and compute for postgres")
I applied to neon last week and then the news broke about the acquisition. They rejected it this morning — I have never been happier to receive a rejection to an application.
This would’ve been three acquisitions straight for me and… I’m okay, they’re awful. I just want stability.
Congrats to the neon team! I use and love neon. Really hope this doesn’t change them too much.
I got hired at Kenna Security a month before they were acquired by Cisco and it was such a miserable experience that I won't work for any company the Kenna leadership are involved with, nor would I ever consider working at Cisco.
I've been through two now, and for one of them nothing much changed, and the other one I was basically lost in a stack of papers for a year. Can I ask what made the experience miserable for you?
The first acquisition I was apart of wasn’t too bad! But we were still culturally very different. So after 2 years and properly transitioning things, I bounced to another start up.
Walking into something like that is tough because the two teams sort of don’t like each other and you’re really “neither”. I’d want to make sure I was interviewed by both teams
I've been part of an acquisition as a first-year engineering manager, during which I had to navigate subsequent two rounds of layoffs. I was also a part of the group to help restructure teams and help make calls on who to keep. Morale was terrible, and the cultures also did not gel at all.
It led to some serious burnout and I took several months off. I'm now happily working as an IC again.
Yes, that is what i expect, too. They have been paying DynamoDB and CosmosDB for a few years now. However, Neon is not competitive latency/throughput-wise for real-time workloads, needed for high end AI (like personalized recommendations). There are a few others I would have expected like Cockroach, Aerospike, or RonDB.
I was a very early employee at the other two start ups that were acquired and even with equity it was not worth it. After all the class A shares were paid out, the rest of us got little.
I mean, hindsight 20/20 here, but I would have loved the theoretical money @ 1 billion. But those are so rare and my experience in the past 15 years hasn’t matched those unicorns.
Basically I’ve come to the conclusion unless you have serious equity or you’re a founder, acquisition suck. You’re the one doing the work making these two companies come together, while the founders usually bounce or are stripped of any real power to change things.
Databricks started in 2013 when Spark sucked (it still does) and they aimed to make it better / faster (which they do).
The product is still centered Spark, but most companies don't want or need Spark and a combination of Iceberg and DuckDB will work for 95% of companies. It's cheaper, just as fast or faster and way easier to reason about.
We're building a data platform around that premise at Definite[0]. It includes everything you need to get started with data (ETL, BI, datalake).
Aren't the alternatives you mentioned - icerberg and duckdb - both storage solutions while spark is a way to express distributed compute? I'm a bit out of touch with this space, is there a newer way to express distributed compute?
duckdb is primarily a query engine. It does have a storage format, but one of it's strengths is querying data where it already resides (e.g. a parquet file sitting in S3).
There are some examples[0] of enabling DuckDB to manage distributed workloads, but these are pretty experimental.
I think what many people are finding out is they don’t really need distributed processing. DuckDB on a single node can get you really far, and it’s much simpler.
DuckDB is not only a storage solution. It can directly query a variety of file formats at rest, without having to re-store anything. That's one of its selling points: you can query across archival/log data stored in S3 (or wherever) without needing to "ingest" anything or double-pay to duplicate the data you've already stored.
I’m just getting into DuckDB lately and finding this feature so exciting. It’s a totally new paradigm. Such a great tool for scientists, and probably many other people. I wish I took it seriously sooner.
Flink is designed around streaming first, while Spark is built around batch first and you're likely best off selecting accordingly. Though any streaming application likely needs batch processing to some degree. Latency vs throughput.
Databricks is the Jira of dealing with data. No one wants to use it, it sucks, there are too many features to try to appease all possible users but none of them particularly good, and there are substantially better options now than there were not long ago. I would never, ever use it by choice.
Eh you don’t even need to go through all the trouble building a startup. imo Neon was interesting and filled a niche while open source solutions were really gaining maturity and adoption. Now they have, lots and lots of recommendations in this comment section, so my sense is that building a startup would be like reinventing the Neon wheel, just too late. Perhaps, depending on licensing, running OSS as a software is viable.
If you come up with the answers to some of these questions, I'd definitely read those blog articles on how you came to those conclusions. Keep asking interesting questions! Cheers
But these days just use trino or whatever. There are lots of new ways to work on data that are all bigger steps up - ergonomically, performance and price - over spark as spark was over hadoop.
The nice thing about spark is the scala/python/R APIs. That helps to avoid lots of the irritating things about SQL (the same transformation applied to multiple columns is a big one).
I really can't speak highly enough of Trino (though I used it as AWS Athena, and this was back when Trino was called Presto). It's impressive how well it took "ever growing pile of CSV/JSON/Excel/Parquet/whatever" and let you query it via SQL as-is without transforming it and putting it into some other system.
Hadoop was fundamentally a batch processing system for large data files that was never intended for the sort of online reporting and analytics workloads for which the DW concept addressed. No amount of Pig and Hive and HBase and subsequent tools layered on top of it could ever change that basic fact.
I used to be a big fan of the platform because back in 2020 / 2021 it really was the only reasonable choice compared to AWS / Azure / Snowflake for building data platforms.
Today it suffers from feature creep and too many pivots & acquisitions. That they are insanely bad at naming features doesn't help either.
But if you're inclined to use it, databricks' setup of spark just saves you an incredible amount of time that you'd normally waste on configuration and wiring infrastructure (storage, compute, pipelines, unified access, VPNs etc). It's expensive and opinionated, but the data engineers you need to deal with spark OOM errors constantly is greater.
Also databricks' default configs give you MUCH better performance out of the box than anything DIY and you don't have to fiddle with partitions and super niche config options to get even medium workloads stable
* No persist(). Not being able to cache dataframes is a nightmare when it's a workflow that involves taking a massive source of data, doing some rough filtering on it that gets it down to a tiny subset, and then doing more complex stuff with that.
* No good way to get usage info programmatically that I've found. For things like monitoring for periodic queries that get out of hand.
* Can't set Spark config. There are often ways to get around this, like when I recently had to set S3A credentials and needed a way that wasn't OS environment variables (this doesn't work for worker nodes). Eventually, through much documentation browsing and finally an exasperated hail mary question to ChatGPT that solved it (told me the things to pass into options() ) I got it working. But all the documentation and online QA resources just say to use Spark config.
* This is more of a Unity Catalog problem, but kind of applies because Serverless and UC often go very hand in hand (particularly when dealing with things that used to be stored in a cluster like credentials), but it drives me insane that I can only mount external volumes with the same block storage as my workspace provider. So I can't mount an external volume to an AWS bucket on an Azure UC. That means if I want to write stuff that can run the same regardless of what my customer is running their Databricks workspace with, I need to use less sophisticated approaches.
It's still nowhere near the pain that Databrick's attempt at copying Snowflake's VARIANT data type has caused me, but there are many times when I find myself having to work around serverless limitations. Especially when these limitations aren't really mentioned much upfront when Databricks pushes serverless aggressively.
TBH it's really quite boring. You just have to go back in time to the late 2010s. They had an excellent Spark-as-a-Service product, at a time when you'd have better luck finding a leprechaun than a reliable self-hosted Spark instance in an enterprise environment. That was simply beyond the capabilities of most enterprise IT teams at the time. The first-party offerings from the hyperscalars were relatively spartan.
Databricks' proprietary notebook format that introduced subtle incompatibilities with Jupyter was infuriating embrace-extend-extinguish style bullshit, but on-prem cluster instability causing jobs to crash on a daily basis was way more infuriating, and at that time, enterprises were more than happy to pay a premium to accelerate analytics teams.
In the 2010s, Databricks had a solid billion-dollar business. But Spark-as-a-Service by itself was never going to be a unicorn idea. AWS EMR was the giant tortoise lurking in the background, slowly but surely closing the gap. The status quo couldn't hold, and who doesn't want to be a unicorn? So, they bloated the hell out of the product, drank that off-brand growth-hacker Kool-Aid, and started spewing some of the most incoherent buzz-word salad to ever come out of the Left Coast. Just slapping data, lake, and house onto the ends of everything, like it was baby oil at a Diddy Party.
Now, here we are in 2025, deep into the terminal decline of enshittification, and they're just rotting away, waiting for One Real Asshole Called Larry Ellison to scoop them up and take them straight to Hell. The State of Florida, but for Big Data companies.
It would be a mystery to me too, why anyone would pick Databricks today for a greenfield project, but those enterprises from 5+ years ago are locked in hard now. They'll squeeze those whales and they'll shit money like a golden goose for a few more years, but their market share will steadily decrease over the next few years.
It's the cycle of life. Entropy always wins. Eventually the Grim Reaper Larry comes for us all. I wouldn't hate on them too hard. They had a pretty solid run.
Databricks is Oracle-level bad. They will definitely ruin Neon or make it expensive. In the medium to long term, I will start looking for Neon alternatives.
Definitely agree, their M&A strategy is setup to strangle whoever they buy and they don't even know it. They're struggling in the face of Iceberg, DuckDB and the other tectonic shifts happening in the open source world. They are trying to innovate through acquisition, but can't quite make it because their culture kills the companies they buy.
I'm biased, I'm a big-data-tech refugee (ex-Snowflake) and am working on https://tower.dev right now, but we're definitely seeing the open source trend supported by Iceberg. It'll be really interesting to see how this plays out.
>As Neon became GA last year, they noticed an interesting stat: 30% of the databases were created by AI agents, not humans. When they looked at their stats again recently, the number went from 30% to over 80%. That is, AI agents were creating 4 times more databases versus humans.
For me this has alarm bells all over it. Databricks is trying to pump postgres as some sort of AI solution. We do live in weird times.
Congratz to neon team (i like what they built), but i don’t see the value or relation to databricks. I hope neon will continue as a standalone product, otherwise we lose a solid postgres provider from the market.
Its pretty heavy in Azure, so I would be surprised if it went away. This is DBX play to move into the transactional database space in addition to the analytical database.
I remember the first post by the Neon team here on HN. I think I commented at the time that I thought it was a great idea. I’ve never had a need to use them yet, but thought I always would.
Cynically, am I the only one who takes pause because of an acquisition like this? It worries me that they will need to be more focused on the needs of their new owners, rather than their users. In theory, the needs should align — but I’m not sure it usually works out that way in practice.
> I remember the first post by the Neon team here on HN. I think I commented at the time that I thought it was a great idea.
Same! I remember it too. I found it quite fascinating. Separation of storage and compute was something new to me, and I was asking them about Pageserver [0]. I also asked for career advice on how to get into database development [1].
Two years later, I ended up working on very similar disaggregated storage at Turso database.
Taking a pause also... I don't believe serving IA can be aligned to serving devs. I hope that the part of the work related to the core of PostgreSQL will help the community.
Congrats to the Neon team. They make an awesome product. Obviously it’s sad to see this, but it’s inevitable when you’re VC funded. Let’s hope Nikita and co remain strong and don’t let Databricks bit.io them.
To be honest this is a little sad for me. I'd hoped that Neon would be able to fill the vacuum left by CockroachDB going "business source"
Being bought by DataBricks makes Neon far less interesting to me. I simply don't trust such a large organisation that has previously had issues acquiring companies, to really care about what is pretty much the most important infrastructure I've got.
There certainly is enough demand for a more "modern" postgresql, but pretty much all of the direct alternatives are straying far from its roots. Whether it be pricing, compatibility, source available etc.
Back when I was looking at alternatives to postgres these were considered:
1. AWS RDS: We were already on AWS RDS, but it is expensive, and has scaling and operations issues
2. AWS Aurora: The one that ended up being recommended, solved some operations issues, but came with other niche downsides. Pretty much the same downsides as other wire compatible postgresql alternatives
3. CockroachDB: Was very interesting, wire compatible, but had deeper compatibility issues, was open source at the time, it didn't fit with our tooling
4. Neon: Was considered to be too immature at the time, but certainly interesting, looked to be able to solve most of our challenges, maybe except for some of the operations problems with postgresql, I didn't look deeper into it at the time
5. Yugabyte: interesting technology, had some of the same compatibility issues, but less that the others, as they're also using the query engine from postgresql as far as I can tell.
There are also various self hosting utilities for PostgreSQL I looked at, specifically CloudPG, but we didn't have the resources to maintain a stateful deployment of kubernetes and postgres ourselves. It would fulfill most of our requirements, but with extra maintenance burden, both for Kubernetes and PostgreSQL.
Hosting PostgreSQL by itself, didn't have mature enough replication and operations features by itself at that point. It is steadily maturing, but as we'd got many databases manual upgrades and patches would be very time consuming, as PostgreSQL has some not so nice upgrade quirks. You basically have to unload and reload all data during major upgrades. Unless you use extensions and other services to circumvent this issue.
> 5. Yugabyte: interesting technology, had some of the same compatibility issues, but less that the others, as they're also using the query engine from postgresql as far as I can tell.
In my brief experience as an engineer (2014->), I've learned that the best "modern" alternative to PostgreSQL at year X has been PostgreSQL at year X+5. :)
Mainly in relation to notify/listen and advisory locks. Most of our code bases use advisory lock based migration tools. It would be a large lift moving to an alternative or building a migration scheduler out of process
Hey everyone, I'm an engineer at Neon and I wanted to share this FAQ which covers a lot of the questions that are being brought up in the comments here:
For what it's worth the questions can't really be answered by a simple FAQ, because history has shown that those answers aren't worth the page they're written on. Many companies that get bought talk all about the fact that nothing is going to change.
Something is always going to change, almost always in a way that impacts customers. In the best case it's something simple like a different name on the bill, other times it will leave customers scrambling for an alternative before a ridiculous deadline. It could happen within weeks, after a month, or it might take a year. The answers at the time of the announcement are the same regardless.
That’s a nice FAQ and all but after what happened to bit.io [0] you have to understand why people (like me) are extremely worried about this.
We’ve all read glowing blog posts and reassuring FAQs enough times after an acquisition only to see a
complete about-face a few months or a year later.
I quite enjoyed using Neon but as a solo founder running my business on Neon I can’t help but think it’s insanity to not be looking for alternatives.
Databricks is _not_ a company I trust at all.
[0] if you don’t know, databricks acquired bit.io and shut down all databases within 30 days. Production databases had <30 days to migrate.
Love Neon, but that FAQ is worthless. Every company that gets acquired reassures customers that "nothing will change"... then it does, once the new company is in the acquirer's belly and gets digested.
This is really, really exciting. I see it as the “right” way OLTP and OLAP will converge.
The OP and I built an HTAP system at SingleStore. A single database with one copy of data for both OLTP and OLAP workloads. HTAP never took off [0].
What we learned was that OLTP (Postgres) should handle OLTP, while OLAP (data warehouses/lakes) should handle OLAP, with replication between them.
Designing the 'up-to-date' replication between these systems is hard.... columnar stores just aren’t built for OLTP‑style writes, and can't keep up with your OLTP tables.
Let’s see if Databricks and Neon can pull this off
“give me up‑to‑date Postgres tables in Unity Catalog", no debezium --> kafka --> flink --> Iceberg. With Spark jobs in the back ensuring that Iceberg is an optimal state.
As it happens, we've just launched our new Xata platform (https://xata.io/) which has some of the key Neon features: instant copy-on-write branching and separation of storage and compute. As an extra twist, we also can do anonymization (PII masking) between your production database and developer branches.
The way we do copy-on-write branches is a bit different. We haven't done any modifications to Postgres but do it completely at the storage layer, which is a distributed system in itself. This also brings some I/O performance opportunities.
While Xata has been around for a while, we're just launching this new platform, and it is in Private Beta. But we are happy to work with you if you are interested.
Several components are open source as their own projects (see below) which will allow you to reproduce most of the features on top of regular Postgres. But the storage part is not open source. We are considering a simpler implementation of it that would be realistic to self-host and still do copy-on-write branching.
These are the open source components:
* pgstream for the anonymization from the production branch
I think when people look at Neon, the Aurora-style disaggregated compute/data architecture allowing highly scalable read replicas on cloud storage is the defining feature, and it's the only such project that offers it for Postgres. So the storage part is the point.
The PII masking aspect is very interesting and something we couldn't get when we decided on DBLab a month ago. What does the deployment model within AWS look like?
If you want to deploy the whole platform inside your own AWS account, we have a Bring Your Own Cloud model: https://xata.io/byoc
If you want to get anonymization from your RDS/Aurora instance and into Xata branches, then you run only a CLI command (`xata clone`) which does something similar to pg_dump/pg_restore but with masking. It is based on our pgstream open source project.
1. Would you sign BAA (for HIPAA) for the Pay As You Go plan? Can't find that anywhere on your site except for that Lite is HIPAA compliant (https://lite.xata.io/security).
2. FYI, couldn't request access via the BYOC form so I sent an email as per the error: There was an error, please try again or contact us at info@xata.io.
Hi @tudorg - do the Xata copy-on-write branches work like Neon in that you effectively get an isolated Postgres cluster, allowing you to test roles, extensions, creating/dropping DBs, altering schema & data of existing DBs, etc? I looked in the docs but it wasn’t clear to me.
Yes, exactly, you get a new isolated Postgres database, just started from essentially a snapshot at the time of the fork. You can test any types of changes.
Do you support http or websocket connections like https://github.com/neondatabase/serverless? In my experience neon is ultra fast that way in serverless environments like 1-5ms per query with network roundtrip.
(Disclaimer: I work at Xata.)
Just wanted to mention that we also support anonymization, in case that’s something you're looking into:
https://xata.io/postgres-data-masking
If all you care about is the forking aspect we use DBLab Engine pretty effectively: https://postgres.ai/products/dblab_engine. Gets deployed within your own infrastructure.
I really do hope that their OSS strategy does not change due to this, as it's really friendly to people who want to learn their product and run smaller deployments. It's (intentionally or not) really hard to run at a big scale as the control plane is not open-source, which makes the model actually work.
How are Neon employees doing? Heard Neon laid off a few teams this week. That's fun. Anyone hear if their shares are worth anything in the acquisition?
It's my understanding that Neon had some tech to basically "wake up" the DB when a request came out -- so you could "scale down to zero," if you will. I was hoping to explore this for small personal projects: I by far prefer Postgres and would love an isolated database per project.
Is there an alternative for that? Scale-to-zero postgres, basically?
For small personal projects, coolify (featured recently here on HN) lets you quickly stand up postgres with SSL, etc. and get a connection string in seconds. You can deploy in the same project or expose pg to the world like neon does.
One click turns it off, or you can just leave it on. A $5 VM will run a lot of small postgres.
I use both neon and coolify, and could live with either, though apples and oranges when it comes to the data branching feature. But a quick pg_dump/restore which could even be scripted solves my problem. Disclaimer: I like devops in addition to just dev.
I'm not afraid of running servers, that was not the point. The point was exactly that I wanted a serverless postgres.
If I can throw together a random project, completely isolated, that costs $0.10 per month, that enables me to do many orders more random projects than something that costs me $5 per month.
AWS Aurora is way too expensive and their "serverless" offerings are overly complicated and not worth it IMHO.
I used Serverless v1 and then they doubled the prices for v2 while removing features so I moved to PlanetScale. They were great but as I grew and wanted multiple smaller DBs they didn't really have a good way to do that and I moved to Neon. Now, with this news, I guess I'll be looking for an alternative.
> AWS Aurora Postgres Serverless v2 has that capability
Was just about to react to someone being wrong on the internet and say that this is not true. Instead, TIL that this is, in fact, the case. Since 2024Q4.
Does anyone have insight into Neon's financials - specifically their revenue, COGS, and gross margins? I'm trying to understand what made Databricks value them at $1B. Was it strong unit economics, rapid growth, or mostly strategic/tech value?
If you're someone who researched the company, enjoyed the interview and accepted an offer, you're probably not going to be in the same group as the people who hate Databricks. Databricks is a 10k people enterprise software company that just raised $10bn and is using their deep pockets to hoover up smaller companies. If that doesn't scare you, you'll be fine. For many of us, the thought of working with or using the product of a company like that strikes fear into our hearts because we have different values to you.
Databricks is the antithesis of Neon. Neon is driven by product, Databricks is driven by sales. Opinions of Databricks in a thread about Neon are going to be on the negative side (but not necessarily representative).
It's big, enterprise, and competes aggressively on marketing and hype. Also there have been a string of acquisitions where databricks has kind of just absorbed the team and product and then not done a great job for customers of the old company.
It's fine. Probably actually a good place to work.
I've been an SA at Databricks for the past two years and love it here. The people you get to work with here are world-class and our customers legitimately love our product.
I too am a little confused about comments in threads on HN about Databricks, they seriously don't reflect what I see internally and what my customers say. I don't think I'd be working here if they did.
Neon (open-source alternative to Aurora) is 73.6% Rust. Databend (open-source Snowflake alternative) is even more Rust-heavy at 97.2%.
Interesting trend - modern serverless databases choosing Rust for its memory safety, performance predictability. Makes sense for systems where reliability and efficiency are non-negotiable.
We've been running OpenRaft in production for several years now and have found it to be quite stable. It's designed as a generic, feature-complete Raft library that handles the complexities of distributed consensus well. If you're looking for a mature Rust Raft implementation, it's definitely worth considering.
Not too familiar with Neon other than the basics - its premise is that you use S3 as bottomless storage for Postgres and it’s otherwise the same as standard Postgres right? And this is all open source? Why are people paying? Can’t you use a cloud provider and have them host this for you?
> you use S3 as bottomless storage for Postgres [...] Why are people paying?
It's vastly more complicated to do this efficiently than you might imagine. Postgres' internal architecture is built around a very different set of assumptions (pages, WAL, local disk etc.) than what the S3 API offers.
It's not clear to me that the _entire_ Neon stack is OSS and available to self-host (though they do share a lot of OSS code, which is great), and in any case, it's not currently supported/documented beyond some "local development" instructions, e.g. "We do not officially support use of autoscaling externally" [0]
> Can’t you use a cloud provider and have them host this for you?
If it really is all OSS, then I guess the moat is the impressive execution of this team.
There are some sections of the code that is close-source. Like the Console, but the devs are very active in their Discord.
There is a small community trying to self-host, and from what I have seen gotten most of it working. There isn't an easy way to start it up for a more junior developer, but if you want to self-host it you are able to since all "components" are available
Hosting and operating the autoscaling of the various services (compute, pageserver, safekeeper, storage broker) that it takes to make all that work is complex enough that most folks would rather not. Same as any other "managed X" service.
Neon is only scale-to-zero from the point of view of the tenants; whoever runs the software, must keep nodes running, including triple-replicated WAL storage.
I believe that future data platforms will adopt an all-in-one approach, offering OLTP, OLAP, as well as support for other hybrid workloads such as vector, graph, and time series. This will lower user costs and be more friendly to applications in the AI era.
>I believe that future data platforms will adopt an all-in-one approach, offering OLTP, OLAP, as well as support for other hybrid workloads such as vector, graph, and time series
It's already available and open source by the nice folks from MIT and they even wrote a book on it [1],[2].
You can use it to develop modern datahub for data engineering and analytics [3],[4].
[1] D4M: Dynamic Distributed Dimensional Data Model:
I really like databricks at the beginning, Spark clusters magically appear whenever I needed them. No more thoughts were given on how to store the data, etc.. After a few years, I've grown, so did my skill set and my employer. Databricks simply couldn't grow with us at the rate that we needed. No, not because of scaling, but the toolings such as AI/ML, what the market is demanding us to do. Faster iterations and experiments, less cost.
Databricks now reminds me of Oracle. Still a great product, but it's a melting delicious ice cream.
Neon is still early‑stage and, AFAIK, not profitable. It’s a perfect snapshot of 2025: anything that’s (1) serverless, and (2) even vaguely AI‑adjacent is trading at a multiple nobody would have believed two years ago. Also supports my hypothesis that the next 12 months will be filled with cash acquisitions.
> Databricks will ruin Neon;
I certainly hope not. Focus on DX, friendly free tier, and community support is what made it special. If that vanishes behind Databricks’ enterprise guardrails, the goodwill will vanish with it.
No, it doesn't matter when you're paying $1B. Why would it? Tech companies don't care about profits. It's easy to become profitable - tech margins are obnoxiously high. They're bought and valued for their ability to scale and rapidly absorb market share.
Fair point. I submit that I'm probably overestimating how much profitability matters here.
I've been hearing that Neon is burning through cash pretty aggressively, which raised eyebrows for me. But you're right: high margins and scalability mean profits can be deferred.
I don't get this metric - 80% of databases are being created by AI agents. Is this because of tools like Lovable. Are they just creating databases when creating a website?
Guess this is the beginning of the end of a great service, not holding my breath. Sounds like from the WSJ article that they’ll just become some AI agent backend service for Replit, and from the previous conversation on HN that Databricks ruins and shutters their acquisitions. Congrats on the big payout for the employees, though.
Most likely a holding state for a bit before databricks ruins it or shuts it down. I started looking around when the news broke last week or so for alternatives.
Supabase is one that I'll consider, Xata [0] is another one that is interesting. Thankfully I just need "postgres", I don't need branching/PII-clearing/etc. That's all nice to have but I don't need it for my app.
I really would prefer a managed DB for multiple reasons but I might need to look at just self-hosting. I might have spent less time futzing with my DB if I had done that from the start instead of going Aurora Serverless v1 -> Planetscale -> Neon.
I believe that, it's a cool concept. But I was too nervous to build on top of that feature, I wanted to maintain my ability to leave Neon easily. After Planetscale (and using their version of schema branching) I didn't want to get pinched again when I went to switch (PS vs Neon branching was/is very different).
I think one of the coolest features of neon is being able to quickly spin up new DBs via the API (single tenant DBs) and while that is cool, my client list is small so manually creating the DBs is not a problem (B2B).
Someone deeper into the Databricks world correct me if this sounds like nonsense, but after thinking about this more my theory is that this is a defensive move for Databricks.
Smaller companies don't usually need Databricks until they grow and become larger with more complex needs, and enough data that queries start to take a long time to run.
As bare metal gets so much faster the point where you hit scaling issues with a DB becomes further and further away.
As this trend continues more companies might decide they don't need Databricks they just need a few Postgres replicas and nothing else.
Neon is kind of like an insurance policy against this outcome.
Crazy how big the data ecosystem has grown. Congrats to the Neon team on a good outcome, but good luck integrating into DBX culture and surviving.
I'm seeing a lot of DBX hate in this thread overall. I think it's warranted. At Tower[0], we're trying to provide a decent open solution. It stars with owning your own data, and Iceberg helps you break free.
I’m incredibly disappointed by this news. I really enjoyed Neon but I seriously doubt I’m going to like Databricks’ stewardship if it. And that’s if they even still care about catering to people like me and don’t jack the prices us.
I guess it’s time to go back to the well of managed/serverless Postgres options…
Data warehousing is quickly becoming a commodity through open-source. I know a company who had 2PBs+ of data in Cloudera. But instead of moving to the cloud (and Databricks), they saved 5X costs by building their own analytics platform with Iceberg, Trino and Superset. The k8s operators are enterprise quality now. On-premises S3 is good, too. You can have great hardware (servers with 128 cpus and 1 TB) and networking. It's not just Trino. StarRocks and Clickhouse have enterprise grade k8s helm charts/operators. That 60bn valuation is an albtross on Databrick's neck - their pricing will have to justify it, and their core business is commoditizing.
Neon filled their product gap of not having an operational (row-oriented) DB.
Not commoditising for enterprise. My last gig wouldn’t allow open source software or any company that might not be there in a decade, or which kept data anywhere but our own tenant. We’d look for the “call us” pricing rather than hate it, which I normally do. We added databricks and it was considered one of my top three achievements, because they don’t have to think about data platforms again, just focus on using it. It’s SO expensive for an enterprise to rejig for a new platform that you can’t rely on (insert open source project here).
I managed to add one startup and so far it’s done very well, but it was an exceptional case and the global CEO wanted the functionality. But it used MongoDB and ops team didn’t have any skills, so rather than learn one tiny thing for an irrelevant data store they added cash to use Atlas with all the support and RBAC etc etc. They couldn’t use the default Azure firewall because they only know one firewall, so added one of those too. Also loaded with contracts. Kept hiring load down, one number to call, job done. Startups cost is $5-10k per year. Support BS about $40k. (I forget the exact numbers but it dwarfed the startup costs.)
Startups are from Venus, enterprise are from Jupiter.
Enterpise also often wants a full data platform (like Databricks), not a plain data warehouse.
I'm curious - as someone who hasn't been paying super close attention to the space, what does it mean to be a full data platform? Is it just having all the different flavors of DB you might need all from one vendor, or is there also tighter integration than if you cobbled it together from multiple vendors?
Essentially, yes. Different DB’s, federated queries (aka delta sharing, zero copy), definition/semantic layer tools, data engineering/pipelines, model training and notebooks, governance, data lineage, row/column/whatever access control.
It’s basically a luxury minivan. It’s may not be the fastest or prettiest or cheapest, but it’s a safe way for a large family of “data and AI people” to traverse a large organisation.
More seriously, I like to call it an “analytics workbench” in a professional setting.
Are there even any "plain data warehouse" vendors left? Even the oldest of the old-school providers seem to have crawled up the modern data stack since the data warehouse glory days, refocused on lakehouse + Open Table Formats either in their core platform or complementary products.
Exactly, look at what GCP are adding to BigQuery.
> My last gig wouldn’t allow open source software or any company that might not be there in a decade
I bet they had VMware all over the place.
Yup. And OpenShift. And Red Hat for Linux. And SAP. And IBM. But you know what? Tiny group of people relative to the impact, revenue and competitors. If we needed skills we clicked “buy”, 100 consultants would arrive who are experts, sort it and we’d move on. Not scratching around looking for people who know what we use and needing to learn 50 different open source tools. Coming from a much looser universe I learned to appreciate the principles for that context.
I've worked at a Swedish streaming company, we bought a filesystem from IBM called "GPFS" or something before it was renamed. Well we had shit performance, reliability and everything else that could be bad. It was running on FC switches recommended by IBM, config from IBM, everything from IBM and it was an absolute fucking joke, metadata timed out regularly and the solution was "deal with it". I think my superiors should've been harder claiming we haven't received what we bought but we're Swedish so we'd rather just fall over and die.
We only used MQ. Against my recommendation, but it was fine if imperfect, and a known entity to all concerned. Did not use that file system.
Battered woman syndrome
> Not commoditising for enterprise. My last gig wouldn’t allow open source software or any company that might not be there in a decade, or which kept data anywhere but our own tenant.
Hence IBM talking up Iceberg: https://www.ibm.com/think/topics/apache-iceberg
Totally agree. Happy open source StarRocks user here using the k8s operator for customer-facing analytics on terabytes of data. There's very little need for Databricks in my world.
Looking at StarRocks site (https://www.starrocks.io/), they compare against Clickhouse, Druid and Trino. Don't even compare against Spark/Databricks! Guess Spark is just not competitive.
Don't fall for benchmarketing. Do your own benchmarks for your use case.
Anyone looking for an open-source Cloudera alternative based on Kubernetes operators. We're building one (~5 years old now): https://stackable.tech/ & https://github.com/stackabletech/
On-premise open-source S3 is a problem though. MinIO is not something we're touching and other than that it looks a bit empty with enterprise ready solutions.
Don’t SeaweedFS and ceph/rook also offer this? Ceph/rook is definitely enterprise ready
> On-premise open-source S3 is a problem though
Rook/ceph with object storage is pretty bulletproof: https://www.rook.io/docs/rook/v1.17/Storage-Configuration/Ob...
I do wish more systems had high quality operators out there. A lot of operators I have looked into are half baked, not reliable, or not supported.
Great to see cost-effective alternatives to Cloudera and Databricks! We’ve spent three years building IOMETE, a self-hosted data lakehouse that combines Apache Iceberg and Spark, designed to run natively on Kubernetes. We’re focused on on-premises deployments to address the growing need for data sovereignty and low TCO, with a streamlined setup for large-scale analytics. Early adopters are seeing strong results. Curious about your experience with Trino and Superset—any tips for optimizing performance at scale?
Wouldn't Rook be a good solution? It's definitely proven in much larger settings than Minio, as it's just Ceph.
What's wrong with minio out of curiosity? Ceph an option?
This is at least partially subjective.
https://news.ycombinator.com/item?id=32148007
https://news.ycombinator.com/item?id=35299665
Ceph would be a theoretical option, but a) we don't have a lot of experience with it and b) it's relatively complex to operate. We'd really love to add a lighter option to our stack that's under the stewardship of a foundation.
Try expanding a cluster, or changing erasure coding configuration, or using anything that needs random access within a file (parquet), or any day 2 operation.
Even some basic s3 storage patterns weren’t considered when the core storage scheme was designed. Lacks an index and depends on filesystem to organize objects and then crumbles to lock contention when too many versions are stored or under walkdir calls when anything is listed. It also can’t even support writing to the same set of keys as S3 should allow since it implicitly depends on underlying filesystem paths.
They might have added an index by now but gatekept it to their enterprise AIStor offering since they’ve abandoned any investment in open source at this point or appearance that they care about that. Their initial inclination in response to this issue says everything - https://github.com/minio/minio/issues/20845#issuecomment-259...
on what?
Look under the hood, the limitations are based on the core, sticking a UI on it does not hide what needs to happen at scale.
Guessing you’re referring to minio not ceph? Have they still not figured out how to do day 2? I mainly avoid them because of their license and the way they interpret it
They are not efficient; they have a one-time static hash to create a cluster. After that, it is all duct tape and glue. Want to expand? Add another cluster (pool) and then look for the cluster that contains the object. They don't know which cluster has the object, and performance does not scale as well with additional clusters. Want to decommission a single node, drain the cluster. They refer to multiple pools as a single cluster, but it is essentially a set of static hashes that lack the intelligence to locate objects. Got the initial EC configuration not quite right.. sorry need to redo the entire cluster.
MinIO is a good fit if you want a small cluster that doesn't require day 2 operational complexity, as you only store a few TBs.
I have not looked into them recently, but I doubt the core has changed. Being VC-funded and looking for an exit makes them overinvest in marketing and story telling.
That tracks with my past analysis as well, thanks
It's been a commodity for decades now. Metrics like price-performance have a long history, but the SnowBricks products fail at them quite dramatically. The difference is hard-sell vs. soft or no-sell.
Not having to buy an appliance and pay for it up front is quite a valuable option. Also the split between processing and storage allows for better archival and scaling strategies.
I have been happily using ClickHouse for the past couple of years without any issue. Rock solid database with wide variety of features and fulfills all my needs. My favourite is the "external dictionary" feature which easily allows me to integrate it with other datastores like Postgres and Redis.
But why would you buy an operational DB from Databricks? The only thing that makes sense is Databricks flailing to maintain market cap.
In addition to the AI use cases, sometimes you wanna share the data warehouse data in oltp way for fast lookups and high concurrency. Not sure whether Neon will do that but I hope so.
One example from Snowflake is hybrid tables which adds rowstore next to columnar.
OLAP + OLTP = HTAP
SingleStore been doing that for years . Unistore been strugglin
I wonder why Singlestore has been so unpopular (at least I never hear about it). Quick guess is that HTAP itself isn’t a significant feature requirement, maybe just a cherry on top of other major db features.
it was a couple things. 1. It's really really hard to replace anyone's OLTP. 2. OLTP and OLAP are owned by such different teams. Who do you make your champion? 3. The modern HTAP dream is possible without something like SingleStore. You need a columnstore that can keep up with your OLTP tables and provide transactional correctness. Who cares if it's all within one system
ps: I worked at SingleStore. https://www.mooncake.dev/blog/htap-is-dead
SingleStore (nee MemSQL) was/is niche -- great fit inside banks in particular.
ETL to bring all your data into Databricks/Snowflake is a lot of effort. Much better if your OLTP data already exists in Databricks and you directly access it from your OLAP layer.
With the push towards open table formats (Iceberg) from both Snowflake and Databricks, it's even harder to get your Postgres OLTP tables ready for OLAP.
The problem isn't in the CDC / replication tools in the market.
The problem is that columnar stores (especially Iceberg) are not designed for the write /upserts patterns of OLTP systems.
They just can't keep up...
This is a big problem we're hoping to solve at Mooncake [0]. Turn Iceberg into an operational columnstore. So that it can be keep up (<s freshness) with your Postgres.
https://www.mooncake.dev/
Is Iceberg involved in every read/write? I thought it was mostly metadata?
DataFile(parquet) is not enough for table with update/delete, (they are part of iceberg "metadata"). for CDC from OLTP use-cases, the pattern involves rapidly marking rows as deleted/ insert new rows and optimizing small files. This is required for minutes-latency replication.
And for second latency replication, it is more involving, you actually need to build layer on top of iceberg to track pk/ apply deletion.
if Databricks just wanted a row DB they couldve done postgres themselves. paying this much for Neon i think is a sign that Neon has something special they want (which, knowing their marketing line, is "independently scalable storage and compute for postgres")
Easy quick cheap forks of database state for AI agents to muck with.
That sounds like AWS Aurora?
mashed w a lil bit of snowflake arch
"Time is the denominator"
what do they use for ETL?
I applied to neon last week and then the news broke about the acquisition. They rejected it this morning — I have never been happier to receive a rejection to an application.
This would’ve been three acquisitions straight for me and… I’m okay, they’re awful. I just want stability.
Congrats to the neon team! I use and love neon. Really hope this doesn’t change them too much.
I got hired at Kenna Security a month before they were acquired by Cisco and it was such a miserable experience that I won't work for any company the Kenna leadership are involved with, nor would I ever consider working at Cisco.
I've been through two now, and for one of them nothing much changed, and the other one I was basically lost in a stack of papers for a year. Can I ask what made the experience miserable for you?
Had personally the opposite experience. Acquisitions being one of the most interesting times to be hired into.
In a couple cases I’ve been recruited because I have a history of scaling and integrating acquisitions into companies successfully
The first acquisition I was apart of wasn’t too bad! But we were still culturally very different. So after 2 years and properly transitioning things, I bounced to another start up.
Walking into something like that is tough because the two teams sort of don’t like each other and you’re really “neither”. I’d want to make sure I was interviewed by both teams
>you’re really “neither”.
IMO, this is where the power of being hired into the situation is. No existing bias for either company and all the baggage that comes with that.
Allows a person to see the pros and cons of how things get done on both sides of the fence, and act accordingly
I've been part of an acquisition as a first-year engineering manager, during which I had to navigate subsequent two rounds of layoffs. I was also a part of the group to help restructure teams and help make calls on who to keep. Morale was terrible, and the cultures also did not gel at all.
It led to some serious burnout and I took several months off. I'm now happily working as an IC again.
> Really hope this doesn’t change them too much.
My guess is that this team gets rolled into Online Tables tech, which would make product sense.
https://docs.databricks.com/aws/en/machine-learning/feature-...
Yes, that is what i expect, too. They have been paying DynamoDB and CosmosDB for a few years now. However, Neon is not competitive latency/throughput-wise for real-time workloads, needed for high end AI (like personalized recommendations). There are a few others I would have expected like Cockroach, Aerospike, or RonDB.
what if you had joined at neon's previous valuation (whatever it was) and got a sudden payday (assuming you had juuuust enough vesting)
I was a very early employee at the other two start ups that were acquired and even with equity it was not worth it. After all the class A shares were paid out, the rest of us got little.
I mean, hindsight 20/20 here, but I would have loved the theoretical money @ 1 billion. But those are so rare and my experience in the past 15 years hasn’t matched those unicorns.
Basically I’ve come to the conclusion unless you have serious equity or you’re a founder, acquisition suck. You’re the one doing the work making these two companies come together, while the founders usually bounce or are stripped of any real power to change things.
The 1b is very likely not all cash. Probably significant portion id db illiquid equity
Maybe unrelated but Databricks is the most annoying garbage I have ever had to use. It fascinates me how anyone uses it by choice.
Databricks started in 2013 when Spark sucked (it still does) and they aimed to make it better / faster (which they do).
The product is still centered Spark, but most companies don't want or need Spark and a combination of Iceberg and DuckDB will work for 95% of companies. It's cheaper, just as fast or faster and way easier to reason about.
We're building a data platform around that premise at Definite[0]. It includes everything you need to get started with data (ETL, BI, datalake).
0 - https://www.definite.app/
Aren't the alternatives you mentioned - icerberg and duckdb - both storage solutions while spark is a way to express distributed compute? I'm a bit out of touch with this space, is there a newer way to express distributed compute?
duckdb is primarily a query engine. It does have a storage format, but one of it's strengths is querying data where it already resides (e.g. a parquet file sitting in S3).
There are some examples[0] of enabling DuckDB to manage distributed workloads, but these are pretty experimental.
0 - https://www.definite.app/blog/smallpond
Thanks for the pointers!
I think what many people are finding out is they don’t really need distributed processing. DuckDB on a single node can get you really far, and it’s much simpler.
DuckDB is not only a storage solution. It can directly query a variety of file formats at rest, without having to re-store anything. That's one of its selling points: you can query across archival/log data stored in S3 (or wherever) without needing to "ingest" anything or double-pay to duplicate the data you've already stored.
I’m just getting into DuckDB lately and finding this feature so exciting. It’s a totally new paradigm. Such a great tool for scientists, and probably many other people. I wish I took it seriously sooner.
Not a new way like Ray, but a new way to express Spark super-efficiently (GPU-acceleration): https://news.ycombinator.com/item?id=43964505
Flink. It has more momentum than Spark right now.
"momentum" is a tricky word. Zig has more momentum than C++, but will it ever overtake the language? I'd bet not.
Well its not a tricky word it just wrong. Velocity maybe. Or more probably acceleration.
Flink is designed around streaming first, while Spark is built around batch first and you're likely best off selecting accordingly. Though any streaming application likely needs batch processing to some degree. Latency vs throughput.
Databricks is the Jira of dealing with data. No one wants to use it, it sucks, there are too many features to try to appease all possible users but none of them particularly good, and there are substantially better options now than there were not long ago. I would never, ever use it by choice.
What options do you use? I don't work for Databricks but I am building my own data infra startup, so I'd like to hear what "good" looks like!
Eh you don’t even need to go through all the trouble building a startup. imo Neon was interesting and filled a niche while open source solutions were really gaining maturity and adoption. Now they have, lots and lots of recommendations in this comment section, so my sense is that building a startup would be like reinventing the Neon wheel, just too late. Perhaps, depending on licensing, running OSS as a software is viable.
Oh, my startup isn't about Postgres, but rather a GPU-accelerated Spark: https://news.ycombinator.com/item?id=43964505
What are some bad UX choices you generally dislike in data products?
If you come up with the answers to some of these questions, I'd definitely read those blog articles on how you came to those conclusions. Keep asking interesting questions! Cheers
Which questions are you referring to specifically?
Really hard disagree. Coming from hadoop, databricks is utopia. It's stable, fast, scales really well if you have massive datasets.
The biggest gripe in have is how crazy expensive it is.
Spark was a really big step up from hadoop.
But these days just use trino or whatever. There are lots of new ways to work on data that are all bigger steps up - ergonomically, performance and price - over spark as spark was over hadoop.
The nice thing about spark is the scala/python/R APIs. That helps to avoid lots of the irritating things about SQL (the same transformation applied to multiple columns is a big one).
I really can't speak highly enough of Trino (though I used it as AWS Athena, and this was back when Trino was called Presto). It's impressive how well it took "ever growing pile of CSV/JSON/Excel/Parquet/whatever" and let you query it via SQL as-is without transforming it and putting it into some other system.
What an impressive feat of engineering.
Hadoop was fundamentally a batch processing system for large data files that was never intended for the sort of online reporting and analytics workloads for which the DW concept addressed. No amount of Pig and Hive and HBase and subsequent tools layered on top of it could ever change that basic fact.
If cost (or perf) is the issue, we're building a super-efficient, GPU-accelerated, easy-to-use Spark: https://news.ycombinator.com/item?id=43964505
I used to be a big fan of the platform because back in 2020 / 2021 it really was the only reasonable choice compared to AWS / Azure / Snowflake for building data platforms.
Today it suffers from feature creep and too many pivots & acquisitions. That they are insanely bad at naming features doesn't help either.
I’d settle for only one bad name per feature from them. Alas, they don’t feel so limited
I'm building another Spark-based choice now with ParaQuery (GPU-accelerated Spark): https://news.ycombinator.com/item?id=43964505
Is hosting spark really that groundbreaking ? Also isn't spark kind of too complicated for 90% of enterprisey data-processing .
I really don't understand the valuation for this company. Why is it so high.
Yes, spark is too complicated for most cases;
But if you're inclined to use it, databricks' setup of spark just saves you an incredible amount of time that you'd normally waste on configuration and wiring infrastructure (storage, compute, pipelines, unified access, VPNs etc). It's expensive and opinionated, but the data engineers you need to deal with spark OOM errors constantly is greater. Also databricks' default configs give you MUCH better performance out of the box than anything DIY and you don't have to fiddle with partitions and super niche config options to get even medium workloads stable
The market for IBM-like software and platforms (everyone else uses this! It must be good!) apparently wasn't saturated yet
They push Serverless so hard but there are SO MANY limitations and surprise gotchas. It's driving me absolutely insane.
And it tends to be notably more expensive! 4-5x the price for less features...
the new cost-optimized mode is very promising, though
Hey, what are the most painful limitations/gotchas you're hitting? I'm on this team and would like to hear about painpoints.
To list off a few:
* No persist(). Not being able to cache dataframes is a nightmare when it's a workflow that involves taking a massive source of data, doing some rough filtering on it that gets it down to a tiny subset, and then doing more complex stuff with that.
* No good way to get usage info programmatically that I've found. For things like monitoring for periodic queries that get out of hand.
* Can't set Spark config. There are often ways to get around this, like when I recently had to set S3A credentials and needed a way that wasn't OS environment variables (this doesn't work for worker nodes). Eventually, through much documentation browsing and finally an exasperated hail mary question to ChatGPT that solved it (told me the things to pass into options() ) I got it working. But all the documentation and online QA resources just say to use Spark config.
* This is more of a Unity Catalog problem, but kind of applies because Serverless and UC often go very hand in hand (particularly when dealing with things that used to be stored in a cluster like credentials), but it drives me insane that I can only mount external volumes with the same block storage as my workspace provider. So I can't mount an external volume to an AWS bucket on an Azure UC. That means if I want to write stuff that can run the same regardless of what my customer is running their Databricks workspace with, I need to use less sophisticated approaches.
It's still nowhere near the pain that Databrick's attempt at copying Snowflake's VARIANT data type has caused me, but there are many times when I find myself having to work around serverless limitations. Especially when these limitations aren't really mentioned much upfront when Databricks pushes serverless aggressively.
TBH it's really quite boring. You just have to go back in time to the late 2010s. They had an excellent Spark-as-a-Service product, at a time when you'd have better luck finding a leprechaun than a reliable self-hosted Spark instance in an enterprise environment. That was simply beyond the capabilities of most enterprise IT teams at the time. The first-party offerings from the hyperscalars were relatively spartan.
Databricks' proprietary notebook format that introduced subtle incompatibilities with Jupyter was infuriating embrace-extend-extinguish style bullshit, but on-prem cluster instability causing jobs to crash on a daily basis was way more infuriating, and at that time, enterprises were more than happy to pay a premium to accelerate analytics teams.
In the 2010s, Databricks had a solid billion-dollar business. But Spark-as-a-Service by itself was never going to be a unicorn idea. AWS EMR was the giant tortoise lurking in the background, slowly but surely closing the gap. The status quo couldn't hold, and who doesn't want to be a unicorn? So, they bloated the hell out of the product, drank that off-brand growth-hacker Kool-Aid, and started spewing some of the most incoherent buzz-word salad to ever come out of the Left Coast. Just slapping data, lake, and house onto the ends of everything, like it was baby oil at a Diddy Party.
Now, here we are in 2025, deep into the terminal decline of enshittification, and they're just rotting away, waiting for One Real Asshole Called Larry Ellison to scoop them up and take them straight to Hell. The State of Florida, but for Big Data companies.
It would be a mystery to me too, why anyone would pick Databricks today for a greenfield project, but those enterprises from 5+ years ago are locked in hard now. They'll squeeze those whales and they'll shit money like a golden goose for a few more years, but their market share will steadily decrease over the next few years.
It's the cycle of life. Entropy always wins. Eventually the Grim Reaper Larry comes for us all. I wouldn't hate on them too hard. They had a pretty solid run.
With cookies disabled I get a blank website, which is a massive red flag and an immediate nope from me.
Can't imagine someone incapable of building a website would deliver a good (digital) product.
They did build a website though. It even looks pretty nice. The restriction you've placed on yourself just prevents you from viewing it.
But.. but.... we MUST track you! That's the whole purpose of our site /s
Databricks is Oracle-level bad. They will definitely ruin Neon or make it expensive. In the medium to long term, I will start looking for Neon alternatives.
Definitely agree, their M&A strategy is setup to strangle whoever they buy and they don't even know it. They're struggling in the face of Iceberg, DuckDB and the other tectonic shifts happening in the open source world. They are trying to innovate through acquisition, but can't quite make it because their culture kills the companies they buy.
I'm biased, I'm a big-data-tech refugee (ex-Snowflake) and am working on https://tower.dev right now, but we're definitely seeing the open source trend supported by Iceberg. It'll be really interesting to see how this plays out.
From the actual article
>As Neon became GA last year, they noticed an interesting stat: 30% of the databases were created by AI agents, not humans. When they looked at their stats again recently, the number went from 30% to over 80%. That is, AI agents were creating 4 times more databases versus humans.
For me this has alarm bells all over it. Databricks is trying to pump postgres as some sort of AI solution. We do live in weird times.
and how many of those DBs are still being actively used...
Congratz to neon team (i like what they built), but i don’t see the value or relation to databricks. I hope neon will continue as a standalone product, otherwise we lose a solid postgres provider from the market.
Its pretty heavy in Azure, so I would be surprised if it went away. This is DBX play to move into the transactional database space in addition to the analytical database.
They claim they will in the FAQ… but we know how this usually goes
If only companies were held liable for breaking promises they made when acquiring other companies
https://ourincrediblejourney.tumblr.com/
I remember the first post by the Neon team here on HN. I think I commented at the time that I thought it was a great idea. I’ve never had a need to use them yet, but thought I always would.
Cynically, am I the only one who takes pause because of an acquisition like this? It worries me that they will need to be more focused on the needs of their new owners, rather than their users. In theory, the needs should align — but I’m not sure it usually works out that way in practice.
> I remember the first post by the Neon team here on HN. I think I commented at the time that I thought it was a great idea.
Same! I remember it too. I found it quite fascinating. Separation of storage and compute was something new to me, and I was asking them about Pageserver [0]. I also asked for career advice on how to get into database development [1].
Two years later, I ended up working on very similar disaggregated storage at Turso database.
Congrats to the Neon team!
[0] - https://news.ycombinator.com/item?id=31756671
[1] - https://news.ycombinator.com/item?id=31756510
Taking a pause also... I don't believe serving IA can be aligned to serving devs. I hope that the part of the work related to the core of PostgreSQL will help the community.
Congrats to the Neon team. They make an awesome product. Obviously it’s sad to see this, but it’s inevitable when you’re VC funded. Let’s hope Nikita and co remain strong and don’t let Databricks bit.io them.
Congratulations to the Neon team.
To be honest this is a little sad for me. I'd hoped that Neon would be able to fill the vacuum left by CockroachDB going "business source"
Being bought by DataBricks makes Neon far less interesting to me. I simply don't trust such a large organisation that has previously had issues acquiring companies, to really care about what is pretty much the most important infrastructure I've got.
There certainly is enough demand for a more "modern" postgresql, but pretty much all of the direct alternatives are straying far from its roots. Whether it be pricing, compatibility, source available etc.
Back when I was looking at alternatives to postgres these were considered:
1. AWS RDS: We were already on AWS RDS, but it is expensive, and has scaling and operations issues
2. AWS Aurora: The one that ended up being recommended, solved some operations issues, but came with other niche downsides. Pretty much the same downsides as other wire compatible postgresql alternatives
3. CockroachDB: Was very interesting, wire compatible, but had deeper compatibility issues, was open source at the time, it didn't fit with our tooling
4. Neon: Was considered to be too immature at the time, but certainly interesting, looked to be able to solve most of our challenges, maybe except for some of the operations problems with postgresql, I didn't look deeper into it at the time
5. Yugabyte: interesting technology, had some of the same compatibility issues, but less that the others, as they're also using the query engine from postgresql as far as I can tell.
There are also various self hosting utilities for PostgreSQL I looked at, specifically CloudPG, but we didn't have the resources to maintain a stateful deployment of kubernetes and postgres ourselves. It would fulfill most of our requirements, but with extra maintenance burden, both for Kubernetes and PostgreSQL.
Hosting PostgreSQL by itself, didn't have mature enough replication and operations features by itself at that point. It is steadily maturing, but as we'd got many databases manual upgrades and patches would be very time consuming, as PostgreSQL has some not so nice upgrade quirks. You basically have to unload and reload all data during major upgrades. Unless you use extensions and other services to circumvent this issue.
> 5. Yugabyte: interesting technology, had some of the same compatibility issues, but less that the others, as they're also using the query engine from postgresql as far as I can tell.
Neon is Postgres.
That is why I was hopeful for Neon unlike a lot of the other ones. Yugabyte however isn't just postgres.
In my brief experience as an engineer (2014->), I've learned that the best "modern" alternative to PostgreSQL at year X has been PostgreSQL at year X+5. :)
> same downsides as other wire compatible postgresql alternatives
I'm interested if you'd care to elaborate.
Mainly in relation to notify/listen and advisory locks. Most of our code bases use advisory lock based migration tools. It would be a large lift moving to an alternative or building a migration scheduler out of process
Neon's blogpost: https://neon.tech/blog/neon-and-databricks
WSJ article: https://www.wsj.com/articles/databricks-to-buy-startup-neon-...
Hey everyone, I'm an engineer at Neon and I wanted to share this FAQ which covers a lot of the questions that are being brought up in the comments here:
https://neon.tech/databricks-faq
We're really excited about this, and will try to respond to some of the questions people have here later.
For what it's worth the questions can't really be answered by a simple FAQ, because history has shown that those answers aren't worth the page they're written on. Many companies that get bought talk all about the fact that nothing is going to change.
Something is always going to change, almost always in a way that impacts customers. In the best case it's something simple like a different name on the bill, other times it will leave customers scrambling for an alternative before a ridiculous deadline. It could happen within weeks, after a month, or it might take a year. The answers at the time of the announcement are the same regardless.
That’s a nice FAQ and all but after what happened to bit.io [0] you have to understand why people (like me) are extremely worried about this.
We’ve all read glowing blog posts and reassuring FAQs enough times after an acquisition only to see a complete about-face a few months or a year later.
I quite enjoyed using Neon but as a solo founder running my business on Neon I can’t help but think it’s insanity to not be looking for alternatives.
Databricks is _not_ a company I trust at all.
[0] if you don’t know, databricks acquired bit.io and shut down all databases within 30 days. Production databases had <30 days to migrate.
The FAQ, as meaningless as history as shown it is, is missing one key question: why?
That's the easiest to answer. Money.
Love Neon, but that FAQ is worthless. Every company that gets acquired reassures customers that "nothing will change"... then it does, once the new company is in the acquirer's belly and gets digested.
Will there be a statement about the OSS nature of Neon?
I'm also an engineer at Neon. The plan is to continue developing Neon as an Apache-2.0 licensed software.
This is really, really exciting. I see it as the “right” way OLTP and OLAP will converge.
The OP and I built an HTAP system at SingleStore. A single database with one copy of data for both OLTP and OLAP workloads. HTAP never took off [0].
What we learned was that OLTP (Postgres) should handle OLTP, while OLAP (data warehouses/lakes) should handle OLAP, with replication between them.
Designing the 'up-to-date' replication between these systems is hard.... columnar stores just aren’t built for OLTP‑style writes, and can't keep up with your OLTP tables.
Let’s see if Databricks and Neon can pull this off
“give me up‑to‑date Postgres tables in Unity Catalog", no debezium --> kafka --> flink --> Iceberg. With Spark jobs in the back ensuring that Iceberg is an optimal state.
https://www.mooncake.dev/blog/htap-is-dead
By OP you mean Nikita Shamgunov the founder of Neon who also founded MemSQL (SingleStore) earlier, right?
I’ve loved Neon and now I’m a little worried. Are there any alternatives?
[Disclaimer: I work for Xata]
As it happens, we've just launched our new Xata platform (https://xata.io/) which has some of the key Neon features: instant copy-on-write branching and separation of storage and compute. As an extra twist, we also can do anonymization (PII masking) between your production database and developer branches.
The way we do copy-on-write branches is a bit different. We haven't done any modifications to Postgres but do it completely at the storage layer, which is a distributed system in itself. This also brings some I/O performance opportunities.
While Xata has been around for a while, we're just launching this new platform, and it is in Private Beta. But we are happy to work with you if you are interested.
Btw, congrats to the Neon team!
Is this open source? A major point of Neon is that it's open source and self-hostable.
Several components are open source as their own projects (see below) which will allow you to reproduce most of the features on top of regular Postgres. But the storage part is not open source. We are considering a simpler implementation of it that would be realistic to self-host and still do copy-on-write branching.
These are the open source components:
* pgstream for the anonymization from the production branch
* pgroll for schema changes
* Xata Agent for the LLM-powered optimizations
I think when people look at Neon, the Aurora-style disaggregated compute/data architecture allowing highly scalable read replicas on cloud storage is the defining feature, and it's the only such project that offers it for Postgres. So the storage part is the point.
The PII masking aspect is very interesting and something we couldn't get when we decided on DBLab a month ago. What does the deployment model within AWS look like?
If you want to deploy the whole platform inside your own AWS account, we have a Bring Your Own Cloud model: https://xata.io/byoc
If you want to get anonymization from your RDS/Aurora instance and into Xata branches, then you run only a CLI command (`xata clone`) which does something similar to pg_dump/pg_restore but with masking. It is based on our pgstream open source project.
Happy to organize a demo any time.
1. Would you sign BAA (for HIPAA) for the Pay As You Go plan? Can't find that anywhere on your site except for that Lite is HIPAA compliant (https://lite.xata.io/security).
2. FYI, couldn't request access via the BYOC form so I sent an email as per the error: There was an error, please try again or contact us at info@xata.io.
1. Yes, we will sign BAA for Pay As You Go.
2. Thanks, I see you sent the email already, not sure why it failed. Will reach out over email.
Hi @tudorg - do the Xata copy-on-write branches work like Neon in that you effectively get an isolated Postgres cluster, allowing you to test roles, extensions, creating/dropping DBs, altering schema & data of existing DBs, etc? I looked in the docs but it wasn’t clear to me.
Yes, exactly, you get a new isolated Postgres database, just started from essentially a snapshot at the time of the fork. You can test any types of changes.
Great, thanks for confirming
Do you support http or websocket connections like https://github.com/neondatabase/serverless? In my experience neon is ultra fast that way in serverless environments like 1-5ms per query with network roundtrip.
We have support for SQL over HTTP in Xata Lite: https://lite.xata.io/docs/sdk/sql/overview
Any plans for websockets? Neon supports transactions over them.
Neon also supports anonymization: https://neon.tech/docs/extensions/postgresql-anonymizer as well as schema only branching
Hey!
(Disclaimer: I work at Xata.) Just wanted to mention that we also support anonymization, in case that’s something you're looking into: https://xata.io/postgres-data-masking
I’ve had good experiences with Supabase.
If all you care about is the forking aspect we use DBLab Engine pretty effectively: https://postgres.ai/products/dblab_engine. Gets deployed within your own infrastructure.
https://www.koyeb.com/docs/databases
Give Prisma Postgres a shot? https://prisma.io/postgres (I work for Prisma)
Supabase is your best bet.
geldata.com
Big congratulations!
I really do hope that their OSS strategy does not change due to this, as it's really friendly to people who want to learn their product and run smaller deployments. It's (intentionally or not) really hard to run at a big scale as the control plane is not open-source, which makes the model actually work.
I am excited to see Databricks turn into the next Oracle. This type of acquisition was inevitable. The king is dead! Long live the king!
And yes, congratulations to the Neon team! (Nikita is, after all, YC)
How are Neon employees doing? Heard Neon laid off a few teams this week. That's fun. Anyone hear if their shares are worth anything in the acquisition?
It's my understanding that Neon had some tech to basically "wake up" the DB when a request came out -- so you could "scale down to zero," if you will. I was hoping to explore this for small personal projects: I by far prefer Postgres and would love an isolated database per project.
Is there an alternative for that? Scale-to-zero postgres, basically?
For small personal projects, coolify (featured recently here on HN) lets you quickly stand up postgres with SSL, etc. and get a connection string in seconds. You can deploy in the same project or expose pg to the world like neon does.
One click turns it off, or you can just leave it on. A $5 VM will run a lot of small postgres.
I use both neon and coolify, and could live with either, though apples and oranges when it comes to the data branching feature. But a quick pg_dump/restore which could even be scripted solves my problem. Disclaimer: I like devops in addition to just dev.
I'm not afraid of running servers, that was not the point. The point was exactly that I wanted a serverless postgres.
If I can throw together a random project, completely isolated, that costs $0.10 per month, that enables me to do many orders more random projects than something that costs me $5 per month.
AWS Aurora Postgres Serverless v2 has that capability, though it takes multiple seconds.
AWS Aurora is way too expensive and their "serverless" offerings are overly complicated and not worth it IMHO.
I used Serverless v1 and then they doubled the prices for v2 while removing features so I moved to PlanetScale. They were great but as I grew and wanted multiple smaller DBs they didn't really have a good way to do that and I moved to Neon. Now, with this news, I guess I'll be looking for an alternative.
[Neon employee] p99 for Neon compute start is 500ms
Yikes. No real-time ML with that.
If your project database is suspending for lack of requests I doubt a 500ms wake up delay is an issue.
> AWS Aurora Postgres Serverless v2 has that capability
Was just about to react to someone being wrong on the internet and say that this is not true. Instead, TIL that this is, in fact, the case. Since 2024Q4.
Thanks for invalidating my stale cache.
Congrats folks at Neon! Been following the team and product since the very beginning. Well done, good DX and good education content too :).
This seems like quite the pivot though
Does anyone have insight into Neon's financials - specifically their revenue, COGS, and gross margins? I'm trying to understand what made Databricks value them at $1B. Was it strong unit economics, rapid growth, or mostly strategic/tech value?
Previous discussion a few days ago:
https://news.ycombinator.com/item?id=43899016
Databricks in talks to acquire startup Neon for about $1B (174 comments)
So... As someone who's joining databricks in a few weeks, what's with the hate in the comments?
If you're someone who researched the company, enjoyed the interview and accepted an offer, you're probably not going to be in the same group as the people who hate Databricks. Databricks is a 10k people enterprise software company that just raised $10bn and is using their deep pockets to hoover up smaller companies. If that doesn't scare you, you'll be fine. For many of us, the thought of working with or using the product of a company like that strikes fear into our hearts because we have different values to you.
Databricks is the antithesis of Neon. Neon is driven by product, Databricks is driven by sales. Opinions of Databricks in a thread about Neon are going to be on the negative side (but not necessarily representative).
It's big, enterprise, and competes aggressively on marketing and hype. Also there have been a string of acquisitions where databricks has kind of just absorbed the team and product and then not done a great job for customers of the old company.
It's fine. Probably actually a good place to work.
Welcome to Databricks!
I've been an SA at Databricks for the past two years and love it here. The people you get to work with here are world-class and our customers legitimately love our product.
I too am a little confused about comments in threads on HN about Databricks, they seriously don't reflect what I see internally and what my customers say. I don't think I'd be working here if they did.
Hopefully you weren't one of the SAs working on the bit.io migration after databricks acquired them.
Thank you, I'm excited!
In my experience, one factor is databricks releasing features fast but unpolished.
I like how they’re innovating, but it can be rough around the edges sometimes.
Every company gets a ton of hate on Hacker News. Don't let it bother you too much. But the specific concerns may be a directional signal.
The Databricks vs. Snowflake bidding war is probably an insanely good time to be a database startup.
Neon (open-source alternative to Aurora) is 73.6% Rust. Databend (open-source Snowflake alternative) is even more Rust-heavy at 97.2%.
Interesting trend - modern serverless databases choosing Rust for its memory safety, performance predictability. Makes sense for systems where reliability and efficiency are non-negotiable.
Wow, go them! https://github.com/databendlabs/jepsen.meta#jepsenmetasrv
Yes, Databend's metadata service (https://github.com/databendlabs/databend/tree/main/src/meta) also uses a Raft implementation written in Rust: https://github.com/databendlabs/openraft.
We've been running OpenRaft in production for several years now and have found it to be quite stable. It's designed as a generic, feature-complete Raft library that handles the complexities of distributed consensus well. If you're looking for a mature Rust Raft implementation, it's definitely worth considering.
How do they know 80% of Neon databases are created by AI agents?
We can see which database creations are coming from products such as Replit, v0, Same.new, Create.xyz, and a few more etc.
Surely, there might be other agents creating Neon databases so we might be under-counting.
Not too familiar with Neon other than the basics - its premise is that you use S3 as bottomless storage for Postgres and it’s otherwise the same as standard Postgres right? And this is all open source? Why are people paying? Can’t you use a cloud provider and have them host this for you?
> you use S3 as bottomless storage for Postgres [...] Why are people paying?
It's vastly more complicated to do this efficiently than you might imagine. Postgres' internal architecture is built around a very different set of assumptions (pages, WAL, local disk etc.) than what the S3 API offers.
Would "essential internal abstraction, with the introduction of read stream APIs" evolve into full storage abstraction?
https://pganalyze.com/blog/5mins-postgres-17-streaming-io
I get that, but my understanding is that they opened sourced this itself, no?
It's not clear to me that the _entire_ Neon stack is OSS and available to self-host (though they do share a lot of OSS code, which is great), and in any case, it's not currently supported/documented beyond some "local development" instructions, e.g. "We do not officially support use of autoscaling externally" [0]
> Can’t you use a cloud provider and have them host this for you?
If it really is all OSS, then I guess the moat is the impressive execution of this team.
[0] https://github.com/neondatabase/autoscaling
There are some sections of the code that is close-source. Like the Console, but the devs are very active in their Discord. There is a small community trying to self-host, and from what I have seen gotten most of it working. There isn't an easy way to start it up for a more junior developer, but if you want to self-host it you are able to since all "components" are available
The only component which is not currently open source is the control plane.
Hosting and operating the autoscaling of the various services (compute, pageserver, safekeeper, storage broker) that it takes to make all that work is complex enough that most folks would rather not. Same as any other "managed X" service.
Neon is only scale-to-zero from the point of view of the tenants; whoever runs the software, must keep nodes running, including triple-replicated WAL storage.
Congratulations to the Neon.
I believe that future data platforms will adopt an all-in-one approach, offering OLTP, OLAP, as well as support for other hybrid workloads such as vector, graph, and time series. This will lower user costs and be more friendly to applications in the AI era.
>I believe that future data platforms will adopt an all-in-one approach, offering OLTP, OLAP, as well as support for other hybrid workloads such as vector, graph, and time series
It's already available and open source by the nice folks from MIT and they even wrote a book on it [1],[2].
You can use it to develop modern datahub for data engineering and analytics [3],[4].
[1] D4M: Dynamic Distributed Dimensional Data Model:
https://d4m.mit.edu/
[2] Mathematics of Big Data: Spreadsheets, Databases, Matrices, and Graphs:
https://mitpress.mit.edu/9780262038393/mathematics-of-big-da...
[3] Technical Report: Developing a Working Data Hub:
https://arxiv.org/abs/2004.00190
[4] Collaborative data analytics with DataHub:
https://dspace.mit.edu/handle/1721.1/100937
Interesting! Thanks for sharing.
I really like databricks at the beginning, Spark clusters magically appear whenever I needed them. No more thoughts were given on how to store the data, etc.. After a few years, I've grown, so did my skill set and my employer. Databricks simply couldn't grow with us at the rate that we needed. No, not because of scaling, but the toolings such as AI/ML, what the market is demanding us to do. Faster iterations and experiments, less cost.
Databricks now reminds me of Oracle. Still a great product, but it's a melting delicious ice cream.
> Neon is valued at $1B;
Neon is still early‑stage and, AFAIK, not profitable. It’s a perfect snapshot of 2025: anything that’s (1) serverless, and (2) even vaguely AI‑adjacent is trading at a multiple nobody would have believed two years ago. Also supports my hypothesis that the next 12 months will be filled with cash acquisitions.
> Databricks will ruin Neon;
I certainly hope not. Focus on DX, friendly free tier, and community support is what made it special. If that vanishes behind Databricks’ enterprise guardrails, the goodwill will vanish with it.
Are people still making comments like these in 2025?
What the hell do profits have to do with valuing tech startups?
Profitability might not be as relevant as it used to be in M&A discussions, but it matters when you’re paying $1B.
Valuations like this only make sense if there’s a clear path to significant strategic leverage or future cash flow.
No, it doesn't matter when you're paying $1B. Why would it? Tech companies don't care about profits. It's easy to become profitable - tech margins are obnoxiously high. They're bought and valued for their ability to scale and rapidly absorb market share.
Fair point. I submit that I'm probably overestimating how much profitability matters here.
I've been hearing that Neon is burning through cash pretty aggressively, which raised eyebrows for me. But you're right: high margins and scalability mean profits can be deferred.
I don't get this metric - 80% of databases are being created by AI agents. Is this because of tools like Lovable. Are they just creating databases when creating a website?
When data bricks contain neon, do they glow in the dark?
Guess this is the beginning of the end of a great service, not holding my breath. Sounds like from the WSJ article that they’ll just become some AI agent backend service for Replit, and from the previous conversation on HN that Databricks ruins and shutters their acquisitions. Congrats on the big payout for the employees, though.
https://www.youtube.com/watch?v=QM3VCYA1e-Q
What's the relationship to replit?
I misread the WSJ article, it was just some generic comment from Replit; but still, I am not very optimistic about this.
At first I thought it had something do to with arm64 SIMD instructions.
Anyone considering Turso?
What happens to existing customers of Neon?
Ask the bit.io customers….
Most likely a holding state for a bit before databricks ruins it or shuts it down. I started looking around when the news broke last week or so for alternatives.
Any alternatives that you are aware of ? Most search results show me Supabase.
Supabase is one that I'll consider, Xata [0] is another one that is interesting. Thankfully I just need "postgres", I don't need branching/PII-clearing/etc. That's all nice to have but I don't need it for my app.
I really would prefer a managed DB for multiple reasons but I might need to look at just self-hosting. I might have spent less time futzing with my DB if I had done that from the start instead of going Aurora Serverless v1 -> Planetscale -> Neon.
[0] https://xata.io/
Same here...I too just need Postgres... Will check out Xata, My workload isn't super critical.
Prisma Postgres is also an option to dig into: https://prisma.io/postgres
Branching is one of the most useful features of Neon.
I believe that, it's a cool concept. But I was too nervous to build on top of that feature, I wanted to maintain my ability to leave Neon easily. After Planetscale (and using their version of schema branching) I didn't want to get pinched again when I went to switch (PS vs Neon branching was/is very different).
I think one of the coolest features of neon is being able to quickly spin up new DBs via the API (single tenant DBs) and while that is cool, my client list is small so manually creating the DBs is not a problem (B2B).
https://neon.tech/databricks-faq
Someone deeper into the Databricks world correct me if this sounds like nonsense, but after thinking about this more my theory is that this is a defensive move for Databricks.
Smaller companies don't usually need Databricks until they grow and become larger with more complex needs, and enough data that queries start to take a long time to run.
As bare metal gets so much faster the point where you hit scaling issues with a DB becomes further and further away.
As this trend continues more companies might decide they don't need Databricks they just need a few Postgres replicas and nothing else.
Neon is kind of like an insurance policy against this outcome.
Crazy how big the data ecosystem has grown. Congrats to the Neon team on a good outcome, but good luck integrating into DBX culture and surviving.
I'm seeing a lot of DBX hate in this thread overall. I think it's warranted. At Tower[0], we're trying to provide a decent open solution. It stars with owning your own data, and Iceberg helps you break free.
[0] - https://tower.dev
congrats to Nikita and all the wonderful folks at Neon!
I’m incredibly disappointed by this news. I really enjoyed Neon but I seriously doubt I’m going to like Databricks’ stewardship if it. And that’s if they even still care about catering to people like me and don’t jack the prices us.
I guess it’s time to go back to the well of managed/serverless Postgres options…
Supabase!
A VC funded company that has never been profitable spending a billion on another startup…
Databricks is profitable afaik.
https://www.wing.vc/content/comparing-the-financials-of-data...
[dead]