With open router, no matter what happens you won't spend more than you deposited or owe more. It's much safer.
> This is about scalability and performance. Billing for as many requests per second as a cloud provider gets can't be done live, without significant performance and reliability degradation.
I don't buy this, for LLMs specifically. For lots of things a cloud provider gives, things might be aggregated and batched before showing up, but there is no reason your LLM spend should take nearly as long as bank system clearing to show. Especially an estimate, which it already gets disclaimed as.
There are companies that will monitor your cloud spend much faster pretty cheaply, and they are essentially having to reimplement the whole thing from the outside and keep up with Google's pricing changes through a shadow recreation of the billing system.
And open router is able to reflect your spend to you immediately, or couldn't implement their cap. If they can do it why can't Google, at least for the broad number of customers without custom price agreements.
> This is about scalability and performance. Billing for as many requests per second as a cloud provider gets can't be done live, without significant performance and reliability degradation.
I don't buy this, for LLMs specifically. For lots of things a cloud provider gives, things might be aggregated and batched before showing up, but there is no reason your LLM spend should take nearly as long as bank system clearing to show. Especially an estimate, which it already gets disclaimed as.
There are companies that will monitor your cloud spend much faster pretty cheaply, and they are essentially having to reimplement the whole thing from the outside and keep up with Google's pricing changes through a shadow recreation of the billing system.
And open router is able to reflect your spend to you immediately, or couldn't implement their cap. If they can do it why can't Google, at least for the broad number of customers without custom price agreements.