Tokens can get expended very quickly driving costs up substantially since there is no clarity in its usage If your service can cut the use substantially, for a large company with a huge AI bill, the savings could make up for the cost of the service which I think is pricey in my opinion. But I can see how it could work out.
This is an interesting approach. I think people still underestimate how quickly token limits and context bloat become the bottleneck when you start running agents in production loops. Seems very relevant topic also given the environment. Will check it out
We prevent way more than that from being added to the cloud bill by showing engineers cost estimates that enables them to make better decisions pre-deploy - e.g. when an engineer knows the IOPS option on their EC2 instance is costing them a lot, they're more likely to reduce that or not use that in dev envs vs just copy/paste what's on production. There's an ROI report on infracost.io that shows how we measure the cost prevention between the first and last commit on merged PRs.
Seems to be targeted at quickly reducing infa cost for small-human teams with high-compute costs. I can see some value, but it's something I'd want to review quarterly instead of per-commit. I might feel different if I was really trying to stretch some runway.
I can see why YC is interested in this issue, as I'm sure lots of startups are trying to stretch that runway.
When we started, we thought everyone could use it from startups to medium sized companies. What we learnt is that the most value comes for the enterprises. The reason is they have used Terraform to decentralize the infra provisioning, so now instead of a central platform team making all the IaC changes, you have hundreds or thousands of engineers making changes every month.
Each of them are making a lot of decisions on the infra. and that combines with the crazy pricing models from the cloud providers was saving companies a lot of money.
Then, we saw how much time is saved when you catch it at this point vs after the fact. Basically avoiding a bunch of tech debt
If you're primarily serving enterprise then the $250/mo foot in the door price makes sense. No reason to make too aggressive of a play for the small market/mid market.
OpenRouter is great for keeping your LLM API bill down, Infracost is about the AWS/Azure/GCP bill your IaC creates. When an agent writes IaC that creates a NAT gateway or an RDS instance, that's $50-5000/mo in cloud spend, so the agent knowing that estimate and the best practices as it's generating the code can optimize it pre-deploy.
The interesting bit is making cloud cost a first-class constraint for the agent loop, not just a post-hoc report. I'd be curious how you handle confidence/uncertainty in estimates, since a wrong cheap-looking recommendation can be worse than no estimate in infra PRs.
We've a lot of experience doing this! Also while this feeds into and supports LLMs and non-deterministic systems, our recommendations are entirely deterministic. So it's pretty rare to have a "wrong" recommendation given they've essentially been implemented + reviewed by actual people.
What can definitely happen though is you get one that is inappropriate in a given context. An example here might be a recommendation from an m5.2xlarge to an m6g.2xlarge instance. Same vCPUs and memory, lower cost, but... also a switch from Intel -> ARM architectures. For a lot of companies their build pipelines make it easy enough to make that change. For others there may be some specific dependency on Intel for that workload which means changing the architecture isn't viable. In that case you can simply dismiss the recommendation and we'll stop suggesting it.
The useful split here seems to be: let the CLI do price lookup and validation, and let the agent decide which diff to make. The thing I’d watch is how visible the source of the estimate is in review — if a PR says “saved $X”, reviewers need to see which prices/rules produced that number.
The 79% / 67% reduction generalizes broader than IaC. Any CLI agents shell out to (curl, jq, grep, kubectl, gh, psql) burns the same token tax — verbose JSON, free-form text output, agent-composed pipelines. A predicate-flag + compact-output redesign would land on all of those.
Direct answer to your question: agents-writing-IaC-in-prod is rare today but not zero. I see more "agent reviews the IaC PR a human wrote", which Cost.dev sounds well-suited to since verification runs locally and the agent only consumes the result. Even if the prod-IaC path takes another year, the design pattern earns its keep on every agent-shellout you already do. One question: does the CLI surface its cache state to the agent, or does each invocation start fresh Repeated price-fetches across a single agent run would be the obvious next-tier savings.
We do cache the results locally so that we're not repeatedly hitting our pricing API. The LLM doesn't access that cache directly though as it'd suffer the token tax you mention. Instead we optimised our CLI to return agent optimised results. We're constantly iterating and improving on it, but it already reduces the tokens usage very significantly. I wrote about it here: https://www.infracost.io/resources/blog/we-cut-claude-s-toke...
We've found even more improvements since that post so those will be shipping soon too.
I'm not sure I follow, which profile do you mean? My profile on HN?
I don't know if we'll keep dissecting every incremental improvement we make as (so far) the general approach is the same as documented in the existing blog post: document common use cases -> benchmark them -> identify bottlenecks/expensive hot spots -> fix them -> repeat
The main thing changing right now is observing new more frequent use cases (either because we're adding new capabilities, or users are doing things we didn't entirely predict) and adding them to the test cases.
Tokens can get expended very quickly driving costs up substantially since there is no clarity in its usage If your service can cut the use substantially, for a large company with a huge AI bill, the savings could make up for the cost of the service which I think is pricey in my opinion. But I can see how it could work out.
yep, the ROI for cloud and AI spend tools is fairly easy to measure: it has to save multiples of what it costs.
This is an interesting approach. I think people still underestimate how quickly token limits and context bloat become the bottleneck when you start running agents in production loops. Seems very relevant topic also given the environment. Will check it out
Thanks - indeed, it's also difficult to estimate how much the agents will cost in advance, and how much changes could reduce or increase costs.
I don't know how they can justify 250 USD / month bill. let alone 1000 USD / month.
We prevent way more than that from being added to the cloud bill by showing engineers cost estimates that enables them to make better decisions pre-deploy - e.g. when an engineer knows the IOPS option on their EC2 instance is costing them a lot, they're more likely to reduce that or not use that in dev envs vs just copy/paste what's on production. There's an ROI report on infracost.io that shows how we measure the cost prevention between the first and last commit on merged PRs.
Seems to be targeted at quickly reducing infa cost for small-human teams with high-compute costs. I can see some value, but it's something I'd want to review quarterly instead of per-commit. I might feel different if I was really trying to stretch some runway.
I can see why YC is interested in this issue, as I'm sure lots of startups are trying to stretch that runway.
When we started, we thought everyone could use it from startups to medium sized companies. What we learnt is that the most value comes for the enterprises. The reason is they have used Terraform to decentralize the infra provisioning, so now instead of a central platform team making all the IaC changes, you have hundreds or thousands of engineers making changes every month.
Each of them are making a lot of decisions on the infra. and that combines with the crazy pricing models from the cloud providers was saving companies a lot of money.
Then, we saw how much time is saved when you catch it at this point vs after the fact. Basically avoiding a bunch of tech debt
If you're primarily serving enterprise then the $250/mo foot in the door price makes sense. No reason to make too aggressive of a play for the small market/mid market.
why would anyone need 10,000 runs a month? do people modify their infrastructure 10,000 times a month?
CI/CD pipelines needs 1 CLI run per commit (like any other code scanning tool), we regularly see enterprises with 100K+ runs/month.
Ephemeral environments
Not really seeing the point I just use openrouter if I'm penny pinching
OpenRouter is great for keeping your LLM API bill down, Infracost is about the AWS/Azure/GCP bill your IaC creates. When an agent writes IaC that creates a NAT gateway or an RDS instance, that's $50-5000/mo in cloud spend, so the agent knowing that estimate and the best practices as it's generating the code can optimize it pre-deploy.
The interesting bit is making cloud cost a first-class constraint for the agent loop, not just a post-hoc report. I'd be curious how you handle confidence/uncertainty in estimates, since a wrong cheap-looking recommendation can be worse than no estimate in infra PRs.
We've a lot of experience doing this! Also while this feeds into and supports LLMs and non-deterministic systems, our recommendations are entirely deterministic. So it's pretty rare to have a "wrong" recommendation given they've essentially been implemented + reviewed by actual people.
What can definitely happen though is you get one that is inappropriate in a given context. An example here might be a recommendation from an m5.2xlarge to an m6g.2xlarge instance. Same vCPUs and memory, lower cost, but... also a switch from Intel -> ARM architectures. For a lot of companies their build pipelines make it easy enough to make that change. For others there may be some specific dependency on Intel for that workload which means changing the architecture isn't viable. In that case you can simply dismiss the recommendation and we'll stop suggesting it.
The useful split here seems to be: let the CLI do price lookup and validation, and let the agent decide which diff to make. The thing I’d watch is how visible the source of the estimate is in review — if a PR says “saved $X”, reviewers need to see which prices/rules produced that number.
Exactly! The estimates + cost diffs are expandable in the PR so you can see the working.
The 79% / 67% reduction generalizes broader than IaC. Any CLI agents shell out to (curl, jq, grep, kubectl, gh, psql) burns the same token tax — verbose JSON, free-form text output, agent-composed pipelines. A predicate-flag + compact-output redesign would land on all of those.
We do cache the results locally so that we're not repeatedly hitting our pricing API. The LLM doesn't access that cache directly though as it'd suffer the token tax you mention. Instead we optimised our CLI to return agent optimised results. We're constantly iterating and improving on it, but it already reduces the tokens usage very significantly. I wrote about it here: https://www.infracost.io/resources/blog/we-cut-claude-s-toke...
We've found even more improvements since that post so those will be shipping soon too.
Great, will it be possible to see it in your profile?
I'm not sure I follow, which profile do you mean? My profile on HN?
I don't know if we'll keep dissecting every incremental improvement we make as (so far) the general approach is the same as documented in the existing blog post: document common use cases -> benchmark them -> identify bottlenecks/expensive hot spots -> fix them -> repeat
The main thing changing right now is observing new more frequent use cases (either because we're adding new capabilities, or users are doing things we didn't entirely predict) and adding them to the test cases.