"GitHub's own security guidance recommends pinning actions to full commit SHAs as the only truly immutable way to consume an action"
Why doesn't GitHub just enforce immutable versioning for actions? If you don't want immutable releases, you don't get to publish an Action. They could decide to enforce this and mitigate this class of issue.
> Why doesn't GitHub just enforce immutable versioning for actions?
I always wish these arguments came with a requirement to include a response to "well, what about the other side of the coin?", otherwise, you've now forced me to ask: well?
The two sides of the coin: Security wants pinned versions, like you have, so that compromises aren't pulled in. Security does not want¹ pinned versions, so that security updates are pulled in.
The trick, of course, is some solution that allows the latter without the former, that doesn't just destroy dev productivity. And remember, …there is no evil bit.
(… I need to name this Law. "The Paradox of Pinning"?)
(¹it might not be so explicitly state, but a desire to have constant updated-ness w/ security patches amounts to an argument against pinning.)
> it might not be so explicitly state, but a desire to have constant updated-ness w/ security patches amounts to an argument against pinning
When you want to update, you update the hashes too. This isn’t an issue in any other packaging ecosystem, where locking (including hashing) is a baseline expectation. The main issue is developer ergonomics, which comes back to GitHub Actions providing very poor package management primitives out of the box.
(This is the key distinction between updating and passively being updated because you have mutable pointers to package state. The latter gets confused for the former, but you almost always want the former.)
Their question isn't about pinned versions, it's about immutable versions. The question is why it is possible to change what commit "v5" refers to, not "why would you want to write v5".
You already don't get updates pulled in with the system unless they swap the version out from under you, which is not a normal way to deploy.
Version tags should obviously be immutable, and if you want to be automatically updated you can select 1.0.*, if you don't you just pick the version tag.
It amounts to an argument against pinning in a (IMO) weird world view where the package maintainer is responsible for the security of users' systems. That feels wrong. The user should be responsible for the security of their system, and for setting their own update policy. I don't want a volunteer making decisions about when I get updates on my machine, and I'm pretty security minded. Sure, make the update available, but I'll decide when to actually install it.
In a more broad sense I think computing needs to move away from these centralised models where 'random person in Nebraska'[0] is silently doing a bunch of work for everyone, even with good intentions. Decisions should be deferred to the user as much as possible.
I assume this is because it is modeled after git tags, and at this point it would be a major change to move away from this. But it should probably get started at some point.
This is a good wake-up call (or reminder) that many “supply chain security” products are no more secure or responsibly engineered than the stacks they’re intended to protect. This is a characteristic of security software in general, but the rise of these kinds of “run us everywhere” tools/products invite new and exciting ways for an attacker to compromise large numbers of users in a single campaign.
My initial thought is that if this isn't a new compromise, Trivy must not have rotated the old credentials. They claim, however,
> We rotated secrets and tokens, but the process wasn't atomic and attackers may have been privy to refreshed tokens
… does anyone know what exactly they're talking about, here? To my knowledge, GH does not divulge new tokens after they're issued, but it depends on the exact auth type we're talking about, and GH has an absurd number of different types of tokens/keys one can use.
OpenClaw creator made some related claims, that as soon as he created a GitHub organization with a new name, somehow it was stolen from him, and he had to ask Github people to do it for him atomically.
So the first incident was on March 19th and the second incident is March 22nd —- evidently the attackers maintained persistence through maybe two separate credential rotation efforts.
Well, not my best 2 weeks at work, now I have to fill out a dozen forms and sit trough a shitload of meeting, just because they got pwned (twice, or once, but really badly :D )
Friendly reminder that just because someone is building security software it doesn't mean they are competent and won't cause more harm than good.
Every month the security team wants me to give full code or cloud access to some new scanner they want to trial. They love the fancy dashboards and lengthy reports but if I allowed just 10% of what they wanted we would be pwned on the regular...
I audited Trivy's GitHub Actions a while back and found some worrying things, the most worrying bit was in the setup-trivy Action where it was doing a clone of main of the trivy repo and executing a shell script in there. There was no ref pinning until somebody raised a PR a few months ago. So a security company gave themselves arbitrary code execution in everyone's CI workflows.
Aqua were breached earlier this month, failed to contain it, got breached again last week, failed to contain it again, and now the attackers have breached their Docker Hub account. Shit happens but they're clearly not capable of handling this and should be enlisting outside help.
Granting broad access to "security" tools so some vendor can take another shot at your prod keys is not risk reduction.
Most of these things are just report printers that makes more noise than a legacy SIEM, and once an attacker is inside they don't do much besides dump findings into a dashboard nobody will read.
If you want less self-inflicted damage, stick new scanners in a tight sandbox, feed them read-only miror data, and keep them away from prod perms until they have earned trust with a boring review of exactly what they touch and where the data goes.
Otherwise you may as well wire your secrets to a public pastebin and call it testing.
Yet many of these tools have setup like: create a service account, give it about thousand permissions (if not outright full ownership) and send us the JSON private key.
Most of corporate security nowadays involves "endpoint security solutions" installed on all devices, servers and VMs, piping everything into an AI-powered dashboard so we can move fast and break everything.
My hypothesis is that generally, there's no quality floor at which security departments are "allowed" to say "actually, none of the options on the market in this category are good enough; we're not going to use any of them". The norm is to reflexively accept extreme invasiveness and always say yes to adding more software to the pile. When these norms run deeply enough in a department, it's effectively institutionally incapable of avoiding shitty security software.
Fwiw w/r/t Trivy in particular,I don't think Trivy is bad software and I use it at work. We're unaffected by this breach because we use Nix to provide our code scanning tools and we write our own Actions workflows. Our Trivy version is pinned by Nix and periodically updated manually, so we've skipped these bad releases.
> This allowed the threat actor to perform authenticated operations, including force-updating tags
Hey look, infrastructure underpinning the security of thousands of products, being compromised in a way a simple setting could have prevented (Do not allow overriding tags is an old GH setting). Yet another reason we need a Software Building Code. I wonder how many more of these reasons we'll find in 2026.
Wasn't this discovered already last week, on Friday, that the threat actor had replaced the legit images with malware images? And republished 75 out of 76 tags?
No, the actor reappeared. This article is not fully updated. On March 22nd, the actor compromised their DockerHub account and published new Docker images.
The sandbox will need internet access (to update data) and you will need to send code to test into it; so compromise already equals leaking all your code, without even breaking the sandboxing
> The sandbox will need internet access (to update data) and you will need to send code to test into it; so compromise already equals leaking all your code, without even breaking the sandboxing
Compromising all code in one directory is bad.
Compromising all my data in all other directories, including mounted cloud drives, is worse.
I restrict most dev tools to access only the current directory.
Whenever possible, credentials shouldn't be inside the sandbox either. Credential proxying, or transparent credential injection, for example with Sandcat: https://github.com/VirtusLab/sandcat
second breach in a month from the same initial credential compromise. the first rotation didn't fully revoke access. the attacker walked right back in. no persistence needed.
"GitHub's own security guidance recommends pinning actions to full commit SHAs as the only truly immutable way to consume an action"
Why doesn't GitHub just enforce immutable versioning for actions? If you don't want immutable releases, you don't get to publish an Action. They could decide to enforce this and mitigate this class of issue.
> Why doesn't GitHub just enforce immutable versioning for actions?
I always wish these arguments came with a requirement to include a response to "well, what about the other side of the coin?", otherwise, you've now forced me to ask: well?
The two sides of the coin: Security wants pinned versions, like you have, so that compromises aren't pulled in. Security does not want¹ pinned versions, so that security updates are pulled in.
The trick, of course, is some solution that allows the latter without the former, that doesn't just destroy dev productivity. And remember, …there is no evil bit.
(… I need to name this Law. "The Paradox of Pinning"?)
(¹it might not be so explicitly state, but a desire to have constant updated-ness w/ security patches amounts to an argument against pinning.)
> it might not be so explicitly state, but a desire to have constant updated-ness w/ security patches amounts to an argument against pinning
When you want to update, you update the hashes too. This isn’t an issue in any other packaging ecosystem, where locking (including hashing) is a baseline expectation. The main issue is developer ergonomics, which comes back to GitHub Actions providing very poor package management primitives out of the box.
(This is the key distinction between updating and passively being updated because you have mutable pointers to package state. The latter gets confused for the former, but you almost always want the former.)
Their question isn't about pinned versions, it's about immutable versions. The question is why it is possible to change what commit "v5" refers to, not "why would you want to write v5".
You already don't get updates pulled in with the system unless they swap the version out from under you, which is not a normal way to deploy.
One of the only useful things my previous employer did was disallow moving tags via hg hooks.
Version tags should obviously be immutable, and if you want to be automatically updated you can select 1.0.*, if you don't you just pick the version tag.
It amounts to an argument against pinning in a (IMO) weird world view where the package maintainer is responsible for the security of users' systems. That feels wrong. The user should be responsible for the security of their system, and for setting their own update policy. I don't want a volunteer making decisions about when I get updates on my machine, and I'm pretty security minded. Sure, make the update available, but I'll decide when to actually install it.
In a more broad sense I think computing needs to move away from these centralised models where 'random person in Nebraska'[0] is silently doing a bunch of work for everyone, even with good intentions. Decisions should be deferred to the user as much as possible.
[0]: https://xkcd.com/2347/
I assume this is because it is modeled after git tags, and at this point it would be a major change to move away from this. But it should probably get started at some point.
GitHub has an opt-in option to enforce immutable tags as part of immutable releases.
https://docs.github.com/en/code-security/concepts/supply-cha...
Nice, yeah I think they should start to migrate to that behavior by default.
what if you pin it to a version that is compromised for years before finding out?
Allowing it to be updated can also fix security problems.
It’s basically all the same arguments as static vs dynamic linking.
Plus, I believe I saw that the one action was getting the latest version of trivy anyway.
This is a good wake-up call (or reminder) that many “supply chain security” products are no more secure or responsibly engineered than the stacks they’re intended to protect. This is a characteristic of security software in general, but the rise of these kinds of “run us everywhere” tools/products invite new and exciting ways for an attacker to compromise large numbers of users in a single campaign.
My initial thought is that if this isn't a new compromise, Trivy must not have rotated the old credentials. They claim, however,
> We rotated secrets and tokens, but the process wasn't atomic and attackers may have been privy to refreshed tokens
… does anyone know what exactly they're talking about, here? To my knowledge, GH does not divulge new tokens after they're issued, but it depends on the exact auth type we're talking about, and GH has an absurd number of different types of tokens/keys one can use.
OpenClaw creator made some related claims, that as soon as he created a GitHub organization with a new name, somehow it was stolen from him, and he had to ask Github people to do it for him atomically.
It is a bit different. What happened to openclaw:
He created a new org “openclaw” to reserve the name. Then he wanted to swap it with “moltbot” org.
So he opened two browser windows, one with “moltbot” repo settings another with “openclaw” repo settings.
Then he renamed “openclaw” to whatever, and quickly tried to rename “moltbot” to now available “openclaw”.
But in a second when “openclaw” was available, a bot snatched the repo.
You're supposed to scan for vulnerabilities, not become one!
When you stare into the vuln, the vuln stares back at you.
> On March 22, 2026, a threat actor used compromised credentials to publish a malicious Trivy v0.69.5 and v0.69.6 DockerHub images. (https://github.com/aquasecurity/trivy/security/advisories/GH...)
So the first incident was on March 19th and the second incident is March 22nd —- evidently the attackers maintained persistence through maybe two separate credential rotation efforts.
As far as I understood it, their entire repo got pwnd in February, and this now is the third successful attack by the same actor.
Recent and related:
Trivy ecosystem supply chain temporarily compromised - https://news.ycombinator.com/item?id=47450142 - March 2026 (35 comments)
temporarily might be a bit of an euphemism here
temporally
Well, not my best 2 weeks at work, now I have to fill out a dozen forms and sit trough a shitload of meeting, just because they got pwned (twice, or once, but really badly :D )
Friendly reminder that just because someone is building security software it doesn't mean they are competent and won't cause more harm than good.
Every month the security team wants me to give full code or cloud access to some new scanner they want to trial. They love the fancy dashboards and lengthy reports but if I allowed just 10% of what they wanted we would be pwned on the regular...
I audited Trivy's GitHub Actions a while back and found some worrying things, the most worrying bit was in the setup-trivy Action where it was doing a clone of main of the trivy repo and executing a shell script in there. There was no ref pinning until somebody raised a PR a few months ago. So a security company gave themselves arbitrary code execution in everyone's CI workflows.
Aqua were breached earlier this month, failed to contain it, got breached again last week, failed to contain it again, and now the attackers have breached their Docker Hub account. Shit happens but they're clearly not capable of handling this and should be enlisting outside help.
It seems they did end up contracting with Sygnia
Granting broad access to "security" tools so some vendor can take another shot at your prod keys is not risk reduction. Most of these things are just report printers that makes more noise than a legacy SIEM, and once an attacker is inside they don't do much besides dump findings into a dashboard nobody will read.
If you want less self-inflicted damage, stick new scanners in a tight sandbox, feed them read-only miror data, and keep them away from prod perms until they have earned trust with a boring review of exactly what they touch and where the data goes. Otherwise you may as well wire your secrets to a public pastebin and call it testing.
Couldn't agree more.
Yet many of these tools have setup like: create a service account, give it about thousand permissions (if not outright full ownership) and send us the JSON private key.
At least they make the red flag nice and obvious.
Most of corporate security nowadays involves "endpoint security solutions" installed on all devices, servers and VMs, piping everything into an AI-powered dashboard so we can move fast and break everything.
My hypothesis is that generally, there's no quality floor at which security departments are "allowed" to say "actually, none of the options on the market in this category are good enough; we're not going to use any of them". The norm is to reflexively accept extreme invasiveness and always say yes to adding more software to the pile. When these norms run deeply enough in a department, it's effectively institutionally incapable of avoiding shitty security software.
Fwiw w/r/t Trivy in particular,I don't think Trivy is bad software and I use it at work. We're unaffected by this breach because we use Nix to provide our code scanning tools and we write our own Actions workflows. Our Trivy version is pinned by Nix and periodically updated manually, so we've skipped these bad releases.
> This allowed the threat actor to perform authenticated operations, including force-updating tags
Hey look, infrastructure underpinning the security of thousands of products, being compromised in a way a simple setting could have prevented (Do not allow overriding tags is an old GH setting). Yet another reason we need a Software Building Code. I wonder how many more of these reasons we'll find in 2026.
Wasn't this discovered already last week, on Friday, that the threat actor had replaced the legit images with malware images? And republished 75 out of 76 tags?
No, the actor reappeared. This article is not fully updated. On March 22nd, the actor compromised their DockerHub account and published new Docker images.
How the heck are credential compromises still a thing with 2FA and refresh tokens???
Since there is no requirement for anyone to use them... people don't use them. If people aren't forced to do the right thing, they do the lazy thing.
How bugs are still possible now when we all write everything in Rust?
I always run such tools inside sandboxes to limit the blast radius.
The sandbox will need internet access (to update data) and you will need to send code to test into it; so compromise already equals leaking all your code, without even breaking the sandboxing
> The sandbox will need internet access (to update data) and you will need to send code to test into it; so compromise already equals leaking all your code, without even breaking the sandboxing
Compromising all code in one directory is bad. Compromising all my data in all other directories, including mounted cloud drives, is worse.
I restrict most dev tools to access only the current directory.
You only need internet access to grab the image, I don't think trivy requires internet access itself. All of my image scanning tools run in isolation.
I don't think it would help here, they were stealing credentials
Whenever possible, credentials shouldn't be inside the sandbox either. Credential proxying, or transparent credential injection, for example with Sandcat: https://github.com/VirtusLab/sandcat
> I don't think it would help here, they were stealing credentials
So, stealing credentials in the current directory and in all other directories are the same thing?
second breach in a month from the same initial credential compromise. the first rotation didn't fully revoke access. the attacker walked right back in. no persistence needed.
/s But I thought npm was the issue, and all of this couldn't happen anywhere else?!
What if we just rebuild everything from scratch with AI? No more supply chain attacks!
Just use OpenClaw. Oh wait, I think Microslop already did...
No one has ever said that supply chain attacks are limited to npm.
Don't underestimate the prowess of Microslop to fuck up. I'm just glad I saw all of this coming and abandoned this hellscape long ago.
fatiguing
GG
People have been warning about giant security holes in GitHub Actions dependency but MS did nothing.