for anyone else looking for tl;dr on WTF KDB actually is, it's buried in the AI slop marketing text in the blog announcement: "KDB-X is […] both a programming language and a database."
Kx released a 32bit, free-for-commercial-use version in 2014 and then reversed course around a year later after banks and hedge funds surprised them by flocking to it for a large subset of their developers / dev machines.
Hopefully they stick to it this time around. It's an incredible system. It's the only thing I've ever used, including Pandas, dplyr & Matlab, where someone could stand over my shoulder asking data analysis questions and I could answer them, on the fly.
LLM's, though notoriously bad (so far) at KDB+/Q compared to other languages, are still a godsend for folks getting started. I recently returned to writing Q after being away from it for 8 years and I've been amazed how good even Google's AI suggestions have been at helping with functions & queries.
Getting started tip: try using Q strictly as a query language, avoid K. Do everything else (data shoveling, devops,...) with a different language.
Could you give an example of where it shines? I have had to answer many a SQL/pandas question with someone over my shoulder, so curious where you see the sharpest benefit.
Admittedly, I am unlikely to learn this proprietary DSL, but always good to know what is the best tool for a job.
Q is not just a query language or a database. It's an array programming language with database capability. You can build an entire framework with just Q, from real-time streaming, to in memory database to on disk database. You can build all APIs and business logic around it. Because it's vector oriented and in memory it's faster than pretty much everything else, no loops required. I have seen a team of 15 KDB developers build what would require an entire etrading department of 200+ developers
I just tried to install the community edition and it fails claiming the license key generated by the site is invalid. Not a great onboarding experience.
do you have a previous community edition installed?
I ran into this than scanned up and saw some warnings about clearing your environment vars & re-sourcing your profile so it no longer points to the old QLIC/QHOME
Summary of the KX Community Edition License Agreement:
Key Points:
What you get:
- Free license to use KX software for personal or internal business purposes only
- No support or maintenance services included
- Software provided "as is" without warranties
Major restrictions:
- NO commercial use
- Cannot sell, distribute, or monetize any product that uses or depends on this software
- Cannot bundle it with commercial products
- Cannot reverse engineer, modify, or create derivative works
- Cannot remove copyright notices or trademarks
- Software may phone home to verify valid license
Important limitations:
- KX's liability capped at $100
- They disclaim all warranties including fitness for purpose
- You must delete software if agreement terminates
- Subject to export control laws
- KX can audit your compliance
Legal terms:
- Governed by New York law
- You retain no IP rights in the software
- Confidentiality obligations for 5 years
- KX can terminate at any time
Bottom line:
This is a restrictive free license meant for evaluation/personal use only. Any commercial use or distribution is strictly prohibited. If you need commercial rights, you'll need a different license.
Just adding, this is completely the opposite to what the original blogspam article says. I think the references to the "Poisoned Chalice", should be updated to replace Macbeth with KX. The promise, (and delivery) of a hyper efficient way to analyse data that kdb/q gives you will always lead to final regret.
What a confusing website. The website is named 'defcon', like the famous defense conference. What does 'defcon' have to do with Q? What even is Q?
The landing page is uninformative. No explanation of what Q is. References ticks, doesn't mention what these are.
The About page is filled with generic words and no substance - so much fluff.
I couldn't tell if KDB-X is an exploit database for Defcon participants, or some other kind of specialized database, and at this point I'm so disappointed by this presentation that I lost interest in whatever this piece of software is or does.
Luckily someone else on HN figured it out and commented with a TLDR, but I'd use this site as an example of how not to design websites.
> The website is named 'defcon', like the famous defense conference.
Both the defense conference and this are references to the military term DEFCON, that stands for "defense readiness condition". Q here is referring to the Q programming language that is built on top of KDB's K language.
The website is a blog and learning resource for the Q language.
If your site requires people to explain your thought process, your presentation has failed, unless your goal is to confuse as many visitors as possible, in which case I wouldn't change a thing.
The topic is clearly of interest to the community, and isn’t half the fun of HN learning about things other people are interested in, from the sources they deem acceptable? Why would you only want to read things that are immediately and totally legible to you, specifically?
KDB code is as proprietary as java, python, C++ code found at other financial institutions. What's proprietary is the Q language. not the code you write with it
QuestDB. It is built for finance workloads (can be also used for other timeseries data, like energy, or aerospace, but has heavy optimizations for common finance data patterns), it is very performant, it has been in used for years at large finance entities, and it is Apache 2.0.
Full disclosure: I am a Developer Advocate at QuestDB.
The Open Source edition does not limit any commercial use or the size of the machine you can install, as per the Apache 2.0 license terms.
If you want more enterprise-y options, like single sign on, or RBAC, there is an Enterprise edition. But Open Source is as performant as the Enterprise version. Enterprise offers also things like replication and TLS on all endpoints, which can be somehow replicated in Open Source with manual sharding or proxies.
QuestDB is just a DB. What people get wrong about KDB is that it's so much more. It's a programming language with DB capabilities. You can use for real time streaming, in memory DB and on disk DB. You can build your entire analytics on top of that. There's nothing else out there that let's you build and entire framework/platform with just one single tech stack.
Context for people lacking it:
- Blog announcement: https://kx.com/blog/introducing-kdb-x-public-preview/
- Product landing page: https://kx.com/products/introducing-kdb-x-public-preview/
Link should be changed to this from the current blogslop
for anyone else looking for tl;dr on WTF KDB actually is, it's buried in the AI slop marketing text in the blog announcement: "KDB-X is […] both a programming language and a database."
The overview here is better than the marketing:
https://code.kx.com/q4m3/0_Overview/
Kx released a 32bit, free-for-commercial-use version in 2014 and then reversed course around a year later after banks and hedge funds surprised them by flocking to it for a large subset of their developers / dev machines.
Hopefully they stick to it this time around. It's an incredible system. It's the only thing I've ever used, including Pandas, dplyr & Matlab, where someone could stand over my shoulder asking data analysis questions and I could answer them, on the fly.
LLM's, though notoriously bad (so far) at KDB+/Q compared to other languages, are still a godsend for folks getting started. I recently returned to writing Q after being away from it for 8 years and I've been amazed how good even Google's AI suggestions have been at helping with functions & queries.
Getting started tip: try using Q strictly as a query language, avoid K. Do everything else (data shoveling, devops,...) with a different language.
Could you give an example of where it shines? I have had to answer many a SQL/pandas question with someone over my shoulder, so curious where you see the sharpest benefit.
Admittedly, I am unlikely to learn this proprietary DSL, but always good to know what is the best tool for a job.
Q is not just a query language or a database. It's an array programming language with database capability. You can build an entire framework with just Q, from real-time streaming, to in memory database to on disk database. You can build all APIs and business logic around it. Because it's vector oriented and in memory it's faster than pretty much everything else, no loops required. I have seen a team of 15 KDB developers build what would require an entire etrading department of 200+ developers
Given Wes McKinney created Pandas for quantitative analysis, it's possible that Pandas wouldn't exist if AQR were paying for a q license.
I just tried to install the community edition and it fails claiming the license key generated by the site is invalid. Not a great onboarding experience.
do you have a previous community edition installed?
I ran into this than scanned up and saw some warnings about clearing your environment vars & re-sourcing your profile so it no longer points to the old QLIC/QHOME
Nope - fresh install.
If you email preview@kx.com with the details, the team can help resolve the issue and get you up and running.
> A quick look at the product licensing page reveals that KDB-X gives you up to 16GB of RAM, 4 secondary threads per process, and 8 IPC connections.
It also says it is not for production use.
I see 'KDB-X is not yet intended for production use' in multiple places, presumably due to the preview nature of the software.
Yes it says the preview is not intended for production. Once it exits preview there's nothing saying it can't be used in prod
TL;DR TOS
Summary of the KX Community Edition License Agreement:
Key Points:
What you get:
- Free license to use KX software for personal or internal business purposes only
- No support or maintenance services included
- Software provided "as is" without warranties
Major restrictions:
- NO commercial use
- Cannot sell, distribute, or monetize any product that uses or depends on this software
- Cannot bundle it with commercial products
- Cannot reverse engineer, modify, or create derivative works
- Cannot remove copyright notices or trademarks
- Software may phone home to verify valid license
Important limitations:
- KX's liability capped at $100
- They disclaim all warranties including fitness for purpose
- You must delete software if agreement terminates
- Subject to export control laws
- KX can audit your compliance
Legal terms:
- Governed by New York law
- You retain no IP rights in the software
- Confidentiality obligations for 5 years
- KX can terminate at any time
Bottom line:
This is a restrictive free license meant for evaluation/personal use only. Any commercial use or distribution is strictly prohibited. If you need commercial rights, you'll need a different license.
Just adding, this is completely the opposite to what the original blogspam article says. I think the references to the "Poisoned Chalice", should be updated to replace Macbeth with KX. The promise, (and delivery) of a hyper efficient way to analyse data that kdb/q gives you will always lead to final regret.
What a confusing website. The website is named 'defcon', like the famous defense conference. What does 'defcon' have to do with Q? What even is Q?
The landing page is uninformative. No explanation of what Q is. References ticks, doesn't mention what these are.
The About page is filled with generic words and no substance - so much fluff.
I couldn't tell if KDB-X is an exploit database for Defcon participants, or some other kind of specialized database, and at this point I'm so disappointed by this presentation that I lost interest in whatever this piece of software is or does.
Luckily someone else on HN figured it out and commented with a TLDR, but I'd use this site as an example of how not to design websites.
> The website is named 'defcon', like the famous defense conference.
Both the defense conference and this are references to the military term DEFCON, that stands for "defense readiness condition". Q here is referring to the Q programming language that is built on top of KDB's K language.
The website is a blog and learning resource for the Q language.
Thanks. Looks like at least someone could follow my thought process
If your site requires people to explain your thought process, your presentation has failed, unless your goal is to confuse as many visitors as possible, in which case I wouldn't change a thing.
It’s a pretty terrible secondary source, BUT
The topic is clearly of interest to the community, and isn’t half the fun of HN learning about things other people are interested in, from the sources they deem acceptable? Why would you only want to read things that are immediately and totally legible to you, specifically?
With the rise of LLM-coding do these specialized/niche languages lose their edge? (i.e: prototyping speed, job security, etc)
No LLM can write KDB yet
As most KDB code is proprietary, one wonders if there's even enough available code to train/steal from.
KDB code is as proprietary as java, python, C++ code found at other financial institutions. What's proprietary is the Q language. not the code you write with it
I think what they mean is that there is less publicly available kdb code for llms to be trained on
No "public" LLM can write KDB yet.
In this case it is a little beyond specialized and niche. It is right to left too.
https://news.ycombinator.com/item?id=44498766
What is the threshold before having to buy a commercial license?
Free as in freedoom? Still not.
So, klong for prototyping, and a libre build of 'j' for everything else.
When does this cost money now?
It's not about money. It's about being able to run it everywhere for any purpose, even commercial ones.
Yeah it sounds promising. I will have to take a look.
What would be the closest open-source equivalent for this? Influx?
QuestDB. It is built for finance workloads (can be also used for other timeseries data, like energy, or aerospace, but has heavy optimizations for common finance data patterns), it is very performant, it has been in used for years at large finance entities, and it is Apache 2.0.
Full disclosure: I am a Developer Advocate at QuestDB.
The Open Source edition does not limit any commercial use or the size of the machine you can install, as per the Apache 2.0 license terms.
If you want more enterprise-y options, like single sign on, or RBAC, there is an Enterprise edition. But Open Source is as performant as the Enterprise version. Enterprise offers also things like replication and TLS on all endpoints, which can be somehow replicated in Open Source with manual sharding or proxies.
QuestDB is just a DB. What people get wrong about KDB is that it's so much more. It's a programming language with DB capabilities. You can use for real time streaming, in memory DB and on disk DB. You can build your entire analytics on top of that. There's nothing else out there that let's you build and entire framework/platform with just one single tech stack.
QuestDB does not even support nanosecond, not quite suitable for financial funds
OP cleverly avoided your question by hallucinating one about what could be done with KBD, but yeah.
Influx is basically the same niche.
There's nothing that gets close to KDB. You can build an entire framework with just KDB
klongpy
[dead]