It's amazing how often product decisions head this way for the purposes of 'removing friction'.
In security, friction is a GOOD thing. You want to have some ceremony to do things. It's pretty much inevitable that people will give up control for convenience.
I think this is another oppertunity for folks to think of security, especially technical folks. (And maybe even another market pricing differentiator): User account management systems should more widely allow users to make permission-restricted 'subaccounts'. In this bold new era where folks are delegating their machines to take actions on their behalf, we need a new way that is far more fine grained than "all, ask every time, or nothing".
Choosing how to sub-divide abilities, and allowing the user to manage that, will be an art unto itself, and depend largely on the system. Thinking a banking API with a rule like "Allow sub user to send payment to water utility once per month, not to exceed $200". The user only needs to intervene if something goes wrong and the amount goes over.
Sure, but in work, friction is not a good thing. So maybe security, if it involves friction, is not a good thing in work? Popularity of npm might be and evidence of that. Distribution system (along in its integration in a myriad of pojects) that's pretty insane from the security stanpoint contributed to so much great stuff and cooperation.
Which is in turn why for convenience, security is a bad thing, making people who do not think about threat models in their sleep opt out.
Problem is striking the right balance, and auto-forwarding your agent to foreign machines is not the right balance. At least not without being able to filter and approve individual signing requests.
It's an inherent tension, and is also why so many things are insecure. People hate friction, and so they don't use secure stuff and turn security features off. Security abominations like "curl | bash" and package managers that traverse trees and download stuff and run scripts are popular for a reason.
It's true in the real world too which is why even in secure buildings a person can often slip in by coat-tailing someone. Stopping every single person to check ID is cumbersome and slows everyone down. People have stuff to do.
Truly high security environments are zero-tolerance, which makes them annoying and expensive to work in. I've talked to people who have worked in US military and CIA/NSA places and a very meaningful amount of time is spent complying with security and jumping through hoops. Even with all that, stuff gets leaked and stolen.
> It's an inherent tension, and is also why so many things are insecure. People hate friction...
It's not just in computer security, it's in almost everything. People hate friction, but they need it make good decisions.
What we really need is smart people creating well-designed eusocial frictions, rather than trying to mindlessly eliminate friction from all areas. But guess which thing leads to products that can be sold more successfully on the market?
This is a great point. If something is overly complex and arduous often times people will find a way to go "under the table" and do something in a way that accomplishes the goal but undermines the process.
The classic example is password rotation. It used to be standard to force users to rotate their passwords every 30/90 days, but that led to users having a sticky note on their monitor with their password on it, so security authorities no longer recommend this practice.
My work still forces password rotations. Everyone I know jokes about just changing the last character of their password each time, eg horse-staple-battery-1 to -2 to -3 etc.
It's about balance between security and getting things done. When I give an AI a task, I say: do this and this, but your boundaries are here and here. It should only do what it's allowed to. Even in simple cases, like running separate sessions on different tasks, they shouldn't be able to touch each other's work.
One more thing on balancing security and speed: I run regular audits and vulnerability checks, so I know the gaps are actually closed.
The Bitwarden desktop app has a built in SSH agent [1], which lets you store SSH keys inside Bitwarden. I have set the app to always ask me for confirmation when the key is requested.
you can make your SSH agent do that, no need to use bitwarden.
If you're using say yubikey for hardware ssh key (via gpg ssh agent/smartcard interface) you can also make it require physical touch of the key to confirm every operation
I'd suggest to use ssh keys inside the secure enclave on macOS that require your fingerprint for every new session [0]. That way agents cannot initiate anything, and you will always be aware when a new ssh connection is being made.
1Password also supports this BUT that does not prevent your agent from being forwarded and you still have to inspect each command being used if you don't trust your agent.
Halfway better would be a separate key for each concern, then check that the key you expect to be used is being used before authorizing.
Secretive is amazing! I know that it's not its purpose, but it would be great if it could store more than ssh keys, like providing a safe place for my .envs
I know it's not open-source, but this is how I use 1Password with FIFO .envs. I'm unsure if 1P stores its SSH in the enclave, but it works the same way - whenever SSH key is accessed, it prompts for TouchID or password.
Besides the open source, any reason why I should use secretive over 1P?
- someone trying to be better about security no matter how annoyed.
At this point, should cases like this even be news? Isn't that a fundamental security design issue? Local agent security guardrails appear to always be built around blacklists, which will always be unreliable and leaky. It is left as an exercise to the user to properly sandbox them.
There may be some limited form of whitelisting, but often it is just a facade, as it is very coarse-grained. So, with tools with very broad capabilities (ssh, kubectl, gcloud, etc) you end up with the agent performing unwanted actions.
Same for [devpod](https://devpod.sh/) for sshing to devcontainers. They are particularly annoying because they just dump it into your config. Use an include statement to your own file for goodness sake.
It was often not ideal in the old world. Now agents are doing everything in those boxes, it is kind of mad.
I have my own utilities now for managing these sockets and files now to deal with this stuff.
I find a lot of people take a very backward approach to agents: They adopt desktop versions with full access first, and only later discover sandboxes and cloud environments.
Claude and ChatGPT both come with free cloud sandboxes that work great! If they are just writing code there's little reason to hand over your laptop.
At work we have rather strict rules for how to run AI agents, and for good reasons apparently. The problem is that once these rules are in place and enforced, using an agent is just more hassle that I feel it's worth. The agents work in complete isolation in a VM with access to pretty much nothing. Some colleagues are making good use of them, but just looking at the policies and tooling we have put in place makes me give up in advance.
This sucks, a lot, but you've gotta never run these tools as your ordinary daily-driver account. These tools are rather unpredictable and -because of the nature of the incredibly reckless development methodology- it's nearly impossible to understand every behavioral change in new versions of the ones that are largely or entirely vibecoded. [0]
As others have said, it looks like the primary way the Major LLM Manufacturers have chosen to control these things is with a combination of great big blacklists and voluminous pleading, begging, and scolding. Most of us know the problems both with enumerating badness and with attempting to appeal to something that's incapable of care, sympathy, or shame.
[0] For another example of this principle, go read the recent Jepsen report on ParticleDB.
Another example of OpenAI giving little thought to security because they want "iteration speed" or whatever.
It took decades of work just to bring Internet security up to the state it is (was?) in at this point, and it is hardly where it should be. AI adds new security concerns (e.g. prompt injection) that need to be addressed and we do not have great answers to that. Bringing us back to the state of security we had in the 90s while adding new problems might be the most shortsighted decision imaginable...
Nah, if they asked LLM whether that's a good idea it would say no
They just don't care, they did it to make as convenient as possible for the users that don't care either, as they are gonna click accept without looking anyway
I think the agent "eagerness" tuning leaves much to be desired. They are very zealous when performing random tasks, resorting to all kinds of dodgy actions just to get it done. Then you ask them to help you with the security of your own repo, the one you've been adding code to day after day for months, and they go full Gandalf on you: "Thou shall not pass!"
Of course it does. That's what the codex <> chat merge is actually doing.
But codex <> chat merge is what they're giving as the "real reason" for what actually is raising prices (a LOT, I might add) for all the subscriptions. Nobody wants it (certainly not applied globally to everything like it is), but they're still going to do it. I even know someone who works at OpenAI and there's apparently a few people trying to fight it at least internally.
Given Sam Altman's speech about how he wants OpenAI pricing to work, I don't think this can be stopped. Btw: that is that you type something into ChatGPT and instead of subscription or per-token cost or anything like that, as soon as you type something in, anything, the way he sees it, ChatGPT would give you a cost for that specific answer and you'd agree or not.
I just feel like this guy is an even worse example of not seeing the viewpoint of a normal (you know non-millionaire) person than even Google's leadership. Can I just say "I think Google is going to win from Altman", you know, despite OpenAI having the far superior technology. Because if there's one guy in the whole world that can destroy the advantages of OpenAI's engineering accomplishments for everyone else, it's this guy ...
It's amazing how often product decisions head this way for the purposes of 'removing friction'.
In security, friction is a GOOD thing. You want to have some ceremony to do things. It's pretty much inevitable that people will give up control for convenience.
Look at Artifacts for example.
I think this is another oppertunity for folks to think of security, especially technical folks. (And maybe even another market pricing differentiator): User account management systems should more widely allow users to make permission-restricted 'subaccounts'. In this bold new era where folks are delegating their machines to take actions on their behalf, we need a new way that is far more fine grained than "all, ask every time, or nothing".
Choosing how to sub-divide abilities, and allowing the user to manage that, will be an art unto itself, and depend largely on the system. Thinking a banking API with a rule like "Allow sub user to send payment to water utility once per month, not to exceed $200". The user only needs to intervene if something goes wrong and the amount goes over.
> In security, friction is a GOOD thing.
Sure, but in work, friction is not a good thing. So maybe security, if it involves friction, is not a good thing in work? Popularity of npm might be and evidence of that. Distribution system (along in its integration in a myriad of pojects) that's pretty insane from the security stanpoint contributed to so much great stuff and cooperation.
Which is in turn why for convenience, security is a bad thing, making people who do not think about threat models in their sleep opt out.
Problem is striking the right balance, and auto-forwarding your agent to foreign machines is not the right balance. At least not without being able to filter and approve individual signing requests.
It's an inherent tension, and is also why so many things are insecure. People hate friction, and so they don't use secure stuff and turn security features off. Security abominations like "curl | bash" and package managers that traverse trees and download stuff and run scripts are popular for a reason.
It's true in the real world too which is why even in secure buildings a person can often slip in by coat-tailing someone. Stopping every single person to check ID is cumbersome and slows everyone down. People have stuff to do.
Truly high security environments are zero-tolerance, which makes them annoying and expensive to work in. I've talked to people who have worked in US military and CIA/NSA places and a very meaningful amount of time is spent complying with security and jumping through hoops. Even with all that, stuff gets leaked and stolen.
> It's an inherent tension, and is also why so many things are insecure. People hate friction...
It's not just in computer security, it's in almost everything. People hate friction, but they need it make good decisions.
What we really need is smart people creating well-designed eusocial frictions, rather than trying to mindlessly eliminate friction from all areas. But guess which thing leads to products that can be sold more successfully on the market?
> In security, friction is a GOOD thing.
Oh boy, it isn't. Friction leads to frustration and therefore more likelihood to ignore the importance altogether.
This is a great point. If something is overly complex and arduous often times people will find a way to go "under the table" and do something in a way that accomplishes the goal but undermines the process.
The classic example is password rotation. It used to be standard to force users to rotate their passwords every 30/90 days, but that led to users having a sticky note on their monitor with their password on it, so security authorities no longer recommend this practice.
My work still forces password rotations. Everyone I know jokes about just changing the last character of their password each time, eg horse-staple-battery-1 to -2 to -3 etc.
It's about balance between security and getting things done. When I give an AI a task, I say: do this and this, but your boundaries are here and here. It should only do what it's allowed to. Even in simple cases, like running separate sessions on different tasks, they shouldn't be able to touch each other's work.
One more thing on balancing security and speed: I run regular audits and vulnerability checks, so I know the gaps are actually closed.
The Bitwarden desktop app has a built in SSH agent [1], which lets you store SSH keys inside Bitwarden. I have set the app to always ask me for confirmation when the key is requested.
[1] https://bitwarden.com/help/ssh-agent/
you can make your SSH agent do that, no need to use bitwarden.
If you're using say yubikey for hardware ssh key (via gpg ssh agent/smartcard interface) you can also make it require physical touch of the key to confirm every operation
If your machine has a TPM chip you can also use that and require a PIN to unlock it, with a mandatory lock out period enforced by the chip
How do I do it with touchid and a MacBook Air?
1Password too, FWIW
I'd suggest to use ssh keys inside the secure enclave on macOS that require your fingerprint for every new session [0]. That way agents cannot initiate anything, and you will always be aware when a new ssh connection is being made.
[0]: https://gist.github.com/arianvp/5f59f1783e3eaf1a2d4cd8e952bb...
Or in a Yubikey and set the touch-required mechanism (light in the key will blink when it needs a touch to continue the auth).
1Password also supports this BUT that does not prevent your agent from being forwarded and you still have to inspect each command being used if you don't trust your agent.
Halfway better would be a separate key for each concern, then check that the key you expect to be used is being used before authorizing.
I've been using secretive on the mac for exactly this.
Secretive is amazing! I know that it's not its purpose, but it would be great if it could store more than ssh keys, like providing a safe place for my .envs
I know it's not open-source, but this is how I use 1Password with FIFO .envs. I'm unsure if 1P stores its SSH in the enclave, but it works the same way - whenever SSH key is accessed, it prompts for TouchID or password.
Besides the open source, any reason why I should use secretive over 1P?
- someone trying to be better about security no matter how annoyed.
> I'm unsure if 1P stores its SSH in the enclave
I am reasonably sure the answer is no, as evidenced by it being exportable
I don't understand what the linked pull request is supposed to prove, considering it was closed and not merged.
this is a normal ssh feature meant to allow keys to be used while keeping keys local …
its actually a keystone of secure infrastructure setups, aka jump hosts.
granted its meant for human or scripted uses, as is everything else
> keystone of secure infrastructure
But only when YOU choose where your SSH agent is accessible.
very true :)
At this point, should cases like this even be news? Isn't that a fundamental security design issue? Local agent security guardrails appear to always be built around blacklists, which will always be unreliable and leaky. It is left as an exercise to the user to properly sandbox them.
There may be some limited form of whitelisting, but often it is just a facade, as it is very coarse-grained. So, with tools with very broad capabilities (ssh, kubectl, gcloud, etc) you end up with the agent performing unwanted actions.
Things are awful for this at the moment. [Was the same for docker desktop at some point.](https://rupertrutledge.com/posts/docker-sbx-s-ssh-security-i...)
Same for [devpod](https://devpod.sh/) for sshing to devcontainers. They are particularly annoying because they just dump it into your config. Use an include statement to your own file for goodness sake.
It was often not ideal in the old world. Now agents are doing everything in those boxes, it is kind of mad.
I have my own utilities now for managing these sockets and files now to deal with this stuff.
Regular reminder to never download/install AI chatbot apps - they all work in a browser where they can be isolated.
Do not let these things loose on your computer.
I find a lot of people take a very backward approach to agents: They adopt desktop versions with full access first, and only later discover sandboxes and cloud environments.
Claude and ChatGPT both come with free cloud sandboxes that work great! If they are just writing code there's little reason to hand over your laptop.
For e2e testing?
At work we have rather strict rules for how to run AI agents, and for good reasons apparently. The problem is that once these rules are in place and enforced, using an agent is just more hassle that I feel it's worth. The agents work in complete isolation in a VM with access to pretty much nothing. Some colleagues are making good use of them, but just looking at the policies and tooling we have put in place makes me give up in advance.
This sucks, a lot, but you've gotta never run these tools as your ordinary daily-driver account. These tools are rather unpredictable and -because of the nature of the incredibly reckless development methodology- it's nearly impossible to understand every behavioral change in new versions of the ones that are largely or entirely vibecoded. [0]
As others have said, it looks like the primary way the Major LLM Manufacturers have chosen to control these things is with a combination of great big blacklists and voluminous pleading, begging, and scolding. Most of us know the problems both with enumerating badness and with attempting to appeal to something that's incapable of care, sympathy, or shame.
[0] For another example of this principle, go read the recent Jepsen report on ParticleDB.
What exactly does this five-month-old never-merged PR prove besides that 90% of people on HN are just reading the headline?
Another example of OpenAI giving little thought to security because they want "iteration speed" or whatever.
It took decades of work just to bring Internet security up to the state it is (was?) in at this point, and it is hardly where it should be. AI adds new security concerns (e.g. prompt injection) that need to be addressed and we do not have great answers to that. Bringing us back to the state of security we had in the 90s while adding new problems might be the most shortsighted decision imaginable...
Doesn't matter how many 0-days chatgpt can find in the linux kernel when it also pulls things like this.
Whoever told you AI has solved security was hallucinating.
They might have solved coding but they did not solve common sense.
Nah, if they asked LLM whether that's a good idea it would say no
They just don't care, they did it to make as convenient as possible for the users that don't care either, as they are gonna click accept without looking anyway
I think the agent "eagerness" tuning leaves much to be desired. They are very zealous when performing random tasks, resorting to all kinds of dodgy actions just to get it done. Then you ask them to help you with the security of your own repo, the one you've been adding code to day after day for months, and they go full Gandalf on you: "Thou shall not pass!"
or "users running a program whose purpose is to eval commands received from a remote service, shocked that readable credentials are being read"
anyone who thinks THIS is the security problem is mental.
Of course it does. That's what the codex <> chat merge is actually doing.
But codex <> chat merge is what they're giving as the "real reason" for what actually is raising prices (a LOT, I might add) for all the subscriptions. Nobody wants it (certainly not applied globally to everything like it is), but they're still going to do it. I even know someone who works at OpenAI and there's apparently a few people trying to fight it at least internally.
Given Sam Altman's speech about how he wants OpenAI pricing to work, I don't think this can be stopped. Btw: that is that you type something into ChatGPT and instead of subscription or per-token cost or anything like that, as soon as you type something in, anything, the way he sees it, ChatGPT would give you a cost for that specific answer and you'd agree or not.
I just feel like this guy is an even worse example of not seeing the viewpoint of a normal (you know non-millionaire) person than even Google's leadership. Can I just say "I think Google is going to win from Altman", you know, despite OpenAI having the far superior technology. Because if there's one guy in the whole world that can destroy the advantages of OpenAI's engineering accomplishments for everyone else, it's this guy ...