Top comment on LWN is a very interesting read (although neither the commenter nor myself claim any such trickery was involved in this case).
> Trail of Bits were able to craft an input that beats Google's circuit and prove it... by virtue of a bug in the verifier: https://blog.trailofbits.com/2026/04/17/we-beat-googles-zero...
Google patched the vuln and the original proof still stands, but this is a pretty strange path we seem to be walking down [...]
This has been used for centuries. It is not a new invention.
Hundreds of years ago, it was not unusual to publish an encrypted solution of some mathematical problem, in order to establish priority without disclosing the algorithm that was used.
Of course, at that time very simple encryption methods were used, for instance an anagram of the solution was published (i.e. encryption by letter transposition).
If only AI safety research had a mechanism this clear. "We have proof that building the machine will kill everybody, so get to work making a provably safe version."
Except that you have the logic backwards. It's an argument that something ("safe" general purpose AI) can't exist rather than that it has to.
People want AI to be able to do every good thing but no bad thing, which is impossible twice. First because false positives and false negatives trade against each other, so a general purpose AI which can do anything approximating all the good things is going to have the bias leaning heavily towards being able to do things in general and therefore being able to do many things that are bad. And second because "good" and "bad" aren't things that anybody can agree on and then some people will demand that it must do X while others demand that it not do X (e.g. "help the rebels win the war"), which means someone is inherently going to be unsatisfied and it's not a thing that can be sensibly regarded as everyone working towards a common goal.
Only that doesn't work either, because what people want is for themselves to have it but not their opponents, and you not building it while your opponents do is the opposite of that.
It's like calling for a general halt to the production of military equipment. How do you expect that to actually happen?
The first one is a difficult balance but not really impossible. The second is basically utilitarianism: Of course you can't maximize all wishes because they often contradict each other, but there can be a reasonable trade-off. Some tradeoffs are clearly better than others.
> The first one is a difficult balance but not really impossible.
It's a direct trade off. If you want it to do more "good" things you make it able to do more "bad" things.
> Of course you can't maximize all wishes because they often contradict each other, but there can be a reasonable trade-off. Some tradeoffs are clearly better than others.
The easy tradeoffs are the ones nobody disputes and everybody is already trying to do. There is no lobby for having it hallucinate more or give you ingredients that will combine to make poison when you ask for a tasty recipe.
But the algorithm still isn't practical on existing quantum computers, or ones that are going to be around any time soon, so there's no reason not to publish in full.
> See, some of the most reputable people in quantum hardware and quantum error-correction—people whose judgment I trust more than my own on those topics—are now telling me that a fault-tolerant quantum computer able to break deployed cryptosystems ought to be possible by around 2029.
Evidence that a hard problem is solvable, and information on solution characteristics, are a big help to others.
Even non-disclosure is just science-neutral, not anti-science.
Partial disclosures are common where disclosures involve risky things, or where a problem was solved as part of an economic concern. But there are non-conflicting opportunities to partially inform others.
That's the whole point. And it's not "build on their work", it's "question their work", because so far every time someone's announced some magic quantum thing it's been followed up shortly afterwards by people poking holes on it, a famous recent example being the "quantum computer" that was replaced by /dev/random and it produced the same results. So the magic here isn't the quantum, it's coming up with a way to publish a claim in a way that it can't be refuted.
>We contend that the amount of time remaining before the arrival of CRQCs still exceeds the amount of time needed to migrate public blockchains to PQC, though the margin for error is increasingly narrow. Therefore, we have offered updated resource estimates for quantum attacks on blockchain cryptography together with an analysis of vulnerabilities and mitigations in order to urge all vulnerable cryptocurrency communities to begin PQC transition immediately while its timely completion is still the likely prospect.
they really couldn't be shouting "mitigate now or never" any louder. I'm curious how they arrived at the efficiency improvements, but perhaps any mention of that would be similar to releasing the circuit.
Wait, the article mentions that Shor's algorithm is factoring (which is what I understood), but then it's talking about elliptic curve cryptography? I thought ECC didn't use the same mathematical foundations of RSA, and RSA has been slowly phased out anyways...
Quite the contrary. Shor's algorithm actually works better for the shorter keys of ECC. The rule of thumb is 2n qbits for RSA keys and 6n qbits for ecc. I believe it has something to do with hownit applies to the hidden subgroup problem of finite abelian groups rather than factorisation, but I am really not a cryptographer not especially mathsy. I just asked the same question you did, and someone in the know pointed me to that.
It doesn't, but it's much harder to cheat with the DLP than it is with the IFP, which is what RSA is, which is why everyone announces records for RSA and ignores the fact that the actual problem to solve is the DLP. An example of how to cheat with the IFP is the "compiled Shor's algorithm" which produces the answer by non-quantum means and then throws in a quantum of quantum to make it look like magic happened.
How is it possible to provide a zero knowledge proof that their circuit works for large problem instances if there is no efficient way to run or simulate the circuit with the required instance size?
The dominant cost in Shor's algorithm is the elliptic curve point addition subroutine. That subroutine can be implemented using reversible classical gates. For that kind of implementation, approximate correctness can be verified by fuzz testing classical trajectories through the subroutine.
Note you could ask the same question about Shor's original paper: how did he show the algorithm works without running it? Running X just isn't the only way to analyze X.
>On superconducting architectures with 10−3 physical error rates...
So still 1-2 orders of magnitude better than what we can achieve.
This is against a 256 bit elliptic curve. For some reason most people are stating the difficulty of using Shor's against 2048 bit RSA. Elliptic curves are easier to break with Shor's. I wonder how much of the optimization came from that fact alone...
That's because it's easier to cheat with the IFP (the underlying problem that RSA is built on) than the DLP, so everyone generates RSA "records" and ignores the actual problem that needs to be solved.
> If the paper's authors had chosen to release their circuit, they would certainly have been recognized for the important progress they made in the science of quantum computing. Other researchers would have gone on to build on their work, and the entire scientific community would be richer for it.
... and the world could well have been unsafer. There is pretty strong reason not to release insights which could be used as an attack on public key cryptography. We already know the fix anyway, post quantum cryptography algorithms.
Sometimes scientific curiosity has to step back when it comes to potentially dangerous research. Scott Aaronson recently [1] compared this case to when scientists stopped publishing on nuclear fission research because the possibility of developing an atomic bomb became concrete:
> When I got an early heads-up about these results—especially the Google team’s choice to “publish” via a zero-knowledge proof—I thought of Frisch and Peierls, calculating how much U-235 was needed for a chain reaction in 1940, but not publishing it, even though the latest results on nuclear fission had been openly published just the year prior.
Top comment on LWN is a very interesting read (although neither the commenter nor myself claim any such trickery was involved in this case).
> Trail of Bits were able to craft an input that beats Google's circuit and prove it... by virtue of a bug in the verifier: https://blog.trailofbits.com/2026/04/17/we-beat-googles-zero... Google patched the vuln and the original proof still stands, but this is a pretty strange path we seem to be walking down [...]
Publishing a zero knowledge proof rather than the solution is pretty clever.
This has been used for centuries. It is not a new invention.
Hundreds of years ago, it was not unusual to publish an encrypted solution of some mathematical problem, in order to establish priority without disclosing the algorithm that was used.
Of course, at that time very simple encryption methods were used, for instance an anagram of the solution was published (i.e. encryption by letter transposition).
Seems like a wonderful use case for a one time pad! Every time you offer up the encryption key, it shows you were right!
> Every time you offer up the encryption key, it shows you were right!
Only if you have a pre-commitment.
Is it? Nobody else can really build on their work.
AIU the intent of this publication is not to further research but to make it clear to anyone that we need to move to post quantum cryptography ASAP.
If only AI safety research had a mechanism this clear. "We have proof that building the machine will kill everybody, so get to work making a provably safe version."
"AI safety" is essentially incoherent. It's like trying to build an all-purpose chemistry lab that can't produce explosives.
Neat, an ontological argument against AI safety. Similar argument:
"God doesn't exist" is essentially incoherent. God is the perfect being, and if he didn't exist, he wouldn't be perfect.
I think the logical mistake is obvious.
Except that you have the logic backwards. It's an argument that something ("safe" general purpose AI) can't exist rather than that it has to.
People want AI to be able to do every good thing but no bad thing, which is impossible twice. First because false positives and false negatives trade against each other, so a general purpose AI which can do anything approximating all the good things is going to have the bias leaning heavily towards being able to do things in general and therefore being able to do many things that are bad. And second because "good" and "bad" aren't things that anybody can agree on and then some people will demand that it must do X while others demand that it not do X (e.g. "help the rebels win the war"), which means someone is inherently going to be unsatisfied and it's not a thing that can be sensibly regarded as everyone working towards a common goal.
You've made a great argument for calling a general halt to AI development, but I'm not sure that was your intent.
Only that doesn't work either, because what people want is for themselves to have it but not their opponents, and you not building it while your opponents do is the opposite of that.
It's like calling for a general halt to the production of military equipment. How do you expect that to actually happen?
The first one is a difficult balance but not really impossible. The second is basically utilitarianism: Of course you can't maximize all wishes because they often contradict each other, but there can be a reasonable trade-off. Some tradeoffs are clearly better than others.
> The first one is a difficult balance but not really impossible.
It's a direct trade off. If you want it to do more "good" things you make it able to do more "bad" things.
> Of course you can't maximize all wishes because they often contradict each other, but there can be a reasonable trade-off. Some tradeoffs are clearly better than others.
The easy tradeoffs are the ones nobody disputes and everybody is already trying to do. There is no lobby for having it hallucinate more or give you ingredients that will combine to make poison when you ask for a tasty recipe.
But the algorithm still isn't practical on existing quantum computers, or ones that are going to be around any time soon, so there's no reason not to publish in full.
> See, some of the most reputable people in quantum hardware and quantum error-correction—people whose judgment I trust more than my own on those topics—are now telling me that a fault-tolerant quantum computer able to break deployed cryptosystems ought to be possible by around 2029.
https://scottaaronson.blog/?p=9718
Could be one of the intents, but the main intent is reputation building.
Wake me up when there's an actual working machine.
That may be the intent, but it is very anti-science.
Partial disclosures are actively pro-science.
Evidence that a hard problem is solvable, and information on solution characteristics, are a big help to others.
Even non-disclosure is just science-neutral, not anti-science.
Partial disclosures are common where disclosures involve risky things, or where a problem was solved as part of an economic concern. But there are non-conflicting opportunities to partially inform others.
That's the whole point. And it's not "build on their work", it's "question their work", because so far every time someone's announced some magic quantum thing it's been followed up shortly afterwards by people poking holes on it, a famous recent example being the "quantum computer" that was replaced by /dev/random and it produced the same results. So the magic here isn't the quantum, it's coming up with a way to publish a claim in a way that it can't be refuted.
>We contend that the amount of time remaining before the arrival of CRQCs still exceeds the amount of time needed to migrate public blockchains to PQC, though the margin for error is increasingly narrow. Therefore, we have offered updated resource estimates for quantum attacks on blockchain cryptography together with an analysis of vulnerabilities and mitigations in order to urge all vulnerable cryptocurrency communities to begin PQC transition immediately while its timely completion is still the likely prospect.
they really couldn't be shouting "mitigate now or never" any louder. I'm curious how they arrived at the efficiency improvements, but perhaps any mention of that would be similar to releasing the circuit.
Wait, the article mentions that Shor's algorithm is factoring (which is what I understood), but then it's talking about elliptic curve cryptography? I thought ECC didn't use the same mathematical foundations of RSA, and RSA has been slowly phased out anyways...
Shor published multiple quantum algorithms, including one for discrete logarithms. The term is sometimes used interchangeably.
They're closely related, ECC and RSA are both instances of the hidden subgroup problem.
Quite the contrary. Shor's algorithm actually works better for the shorter keys of ECC. The rule of thumb is 2n qbits for RSA keys and 6n qbits for ecc. I believe it has something to do with hownit applies to the hidden subgroup problem of finite abelian groups rather than factorisation, but I am really not a cryptographer not especially mathsy. I just asked the same question you did, and someone in the know pointed me to that.
> I thought ECC didn't use the same mathematical foundations of RSA
It kinda does, it just uses them differently
The basis here is the discrete inverse logarithm in a specific group (elliptic curves over rationals or multiplicative group module n)
It doesn't, but it's much harder to cheat with the DLP than it is with the IFP, which is what RSA is, which is why everyone announces records for RSA and ignores the fact that the actual problem to solve is the DLP. An example of how to cheat with the IFP is the "compiled Shor's algorithm" which produces the answer by non-quantum means and then throws in a quantum of quantum to make it look like magic happened.
How is it possible to provide a zero knowledge proof that their circuit works for large problem instances if there is no efficient way to run or simulate the circuit with the required instance size?
The dominant cost in Shor's algorithm is the elliptic curve point addition subroutine. That subroutine can be implemented using reversible classical gates. For that kind of implementation, approximate correctness can be verified by fuzz testing classical trajectories through the subroutine.
Note you could ask the same question about Shor's original paper: how did he show the algorithm works without running it? Running X just isn't the only way to analyze X.
Checking the required hardware noise performance:
>On superconducting architectures with 10−3 physical error rates...
So still 1-2 orders of magnitude better than what we can achieve.
This is against a 256 bit elliptic curve. For some reason most people are stating the difficulty of using Shor's against 2048 bit RSA. Elliptic curves are easier to break with Shor's. I wonder how much of the optimization came from that fact alone...
That's because it's easier to cheat with the IFP (the underlying problem that RSA is built on) than the DLP, so everyone generates RSA "records" and ignores the actual problem that needs to be solved.
> If the paper's authors had chosen to release their circuit, they would certainly have been recognized for the important progress they made in the science of quantum computing. Other researchers would have gone on to build on their work, and the entire scientific community would be richer for it.
... and the world could well have been unsafer. There is pretty strong reason not to release insights which could be used as an attack on public key cryptography. We already know the fix anyway, post quantum cryptography algorithms.
Sometimes scientific curiosity has to step back when it comes to potentially dangerous research. Scott Aaronson recently [1] compared this case to when scientists stopped publishing on nuclear fission research because the possibility of developing an atomic bomb became concrete:
> When I got an early heads-up about these results—especially the Google team’s choice to “publish” via a zero-knowledge proof—I thought of Frisch and Peierls, calculating how much U-235 was needed for a chain reaction in 1940, but not publishing it, even though the latest results on nuclear fission had been openly published just the year prior.
1: https://scottaaronson.blog/?p=9665
They are not doing Science, they are just bragging.
Oh please, the government could easily force them to hand over their research. This is not a serious argument.
Would it be really so straightforward for the government to do that?
i doubt they don't have access already
Plenty of people doubt evolution.
Doubt without evidence is just noise.
[dead]
However, the author managed to squeeze the word "however" eleven times in this article, however.
Never twice in one sentence like you did, and there are 13 “but”s. Is something wrong with using ‘however’? If so, what exactly?
However, there was only 57 however's in the paper it self.
> A new paper provides a major step in that direction, however.
It may have gone unnoticed if used only used once in the article, however.
[flagged]