If I can get Sol level capabilities on a $20k machine, then it is well worth it for my employer to buy me that machine for work as a workstation. When you start paying in tokens vs subscription costs due to enterprise agreements, you really start to see how much cash utilizing frontier models at the frontier costs (and I'm efficiently using luna and other models where possible!)
Even using multiple windows in parallel for as many as 5-10 hours per day, I find that I am not fully using my claude max (20x) and chatgpt pro (20x) accounts. I can for sure use up the claude max account, but chatgpt either gives me a free reset before I run out of tokens or I just fail to use the full quota. The quota for Sol seems like 10x that of Claude Opus at the same level, and forget Fable, you can use a 5 hour quota in 20 minutes.
But lets do the math:
Lets say a 20k workstation can run 1 inference at a time at the same speed you get with Sol hosted by openai (big assumption) and run an equally capable model (big assumption).
Each month this gives you about 100-170 inference hours on a Sol 20x Pro account, and 720 hours (if you utilize 24/7) on the workstation.
Assuming a 36 month amortization before the workstation has to be replaced due to no longer being able to run frontier models or is too inefficient due to electrical costs or what have you:
The monthly workstation cost is about $550 capex and $150 electricity -> $700/month
You would need about 6 Pro accounts to reach that capacity, which would cost you $1200 a month.
But this fails because:
- You most likely can't utilize the workstation 24/7. Your work hours will be concentrated into 6-10 hours per day.
- During work hours you are capable of utilizing more than 1 concurrent session. 6 Sol accounts would support as many as 20-30 during working hours, not all the time but if you could burst to that many (don't forget sub-agents and agent directed parallel agent workloads).
- In 1 year the cost of Sol level models is likely to cost a fraction of what it does now.
this leads to:
Workstation 1 Sol Pro 2 Sol Pro
Monthly cost $700 $200 $400
Raw capacity (hrs) 720 120 240
Usable capacity (hrs) 100-130 120 240
Concurrent sessions 1 3-5 6-10
$ per usable hour ~$6.00 $1.67 $1.67
Usable hours per $700 ~115 ~420 ~420
One of the advantages of LLM's is that you can set up a task list and tell it to burn through those tasks overnight. It's a rare night that Claude isn't busy for me, and I do burn through my 20x subscription, sufficiently that I downgrade from fable around about ... now in the week...
"You most likely can't utilize the workstation 24/7. Your work hours will be concentrated into 6-10 hours per day."
I have agents running 24/7 doing research, in fact I would argue this how they will be used for most programming tasks in the near future. For chatting, I agree local inference makes no sense. But for tasks that run continually, I'm not so sure. Personal computers took a while, local inference will too, but I think it will happen.
Sorry for the snark, but are you trying to cure cancer? What could possibly need 24/7 research in our domain, that doesn’t need your input every 30 minutes?
Normal boring CS scientific work. Just running running my experiments, reproducing other papers, etc. A lot of it does involve the agent waiting for some computation, but the fact that it resumes independently when I'm sleeping is kind of the point (+ usually I have several running in parallel).
I'm not trying to cure cancer, although I do hope people who are use LLMs. ;)
> - You most likely can't utilize the workstation 24/7. Your work hours will be concentrated into 6-10 hours per day.
isn't the whole point of all this ..... agents? isn't that what literally everyone is always clammering about in these threads? in which case the workstation is useful 720 hours out of 720 hours.
Subscriptions are, and will likely remain, the best deal in town. Unfortunately, larger companies aren't able to do that. When your monthly token costs are in the $5-10k range, the local inference starts to look a lot more attractive
In the case where you pay for tokens without a subscription, the analysis is still very much not in favor of buying hardware.
The assumption previously used was that you can run a Sol level model on an M6 or whatever hardware $20k gives you. That is not true, it was an assumption made to show that even giving your own hardware every reasonable advantage it still loses.
Lets compare buying tokens of the best model you might run on your own hardware (still being unrealistic in favor of your own hardware) vs that same class of model on the market. I think one of the best models you might be able to run is GLM 5.4, but lets just look at chinese models generally:
$20k workstation, best case: $15k M5 Ultra 512GB, 36-month amortization, ~$440/mo. Runs a GLM-5.3-class model at ~30 tok/s. Saturated 24/7 it produces roughly 58M output tokens/month.
Buying those tokens:
DeepSeek V4 Pro @ $0.87/M $50
Kimi K2.6 @ $4.00/M $232
GLM-5.3 @ $4.40/M $255
Kimi K3 @ $15.00/M $870 (does not fit on the box)
The economics can never work in your favor for buying your own hardware here, unless you can utilize it or sell excess capacity and you have access to nearly free electricity. The reason is someone else can buy the same hardware at scale (or realistically more efficient hardware), park it somewhere with very cheap electricity, and sell tokens. They can get very high utilization that you are not likely to get.
And keep in mind I am giving 'your own hardware' no overhead or maintenance cost, despite your condition that it's in a large corporate environment. In reality corporate IT would make it almost impossible to set up and your would need huge lead times to buy the hardware and get it installed.
> $20k workstation, best case: $15k M5 Ultra 512GB, 36-month amortization, ~$440/mo. Runs a GLM-5.3-class model at ~30 tok/s. Saturated 24/7 it produces roughly 58M output tokens/month.
For agentic coding, ~90% of the cost comes from cached input tokens. This cost increases quadratically with the session length. If sessions go near 1M context, the number of cached input tokens can easily exceed 1B in a day.
So yes, at that speed for sure. But if the speed goes up? or the ability to batch at the same speed goes up? The economics start to shift. The gap is much closer, and you'd end up with a box you can still use or sell later.
As speed goes up the cost / Mtoken will necessarily go down at roughly the same ratio so it will wash out. The still use hardware or sell hardware value is factored in to the amortized monthly cost, it assumes a 3 year markdown, and does not factor in the cost of money which should almost cancel the resale value in the end, which I think is quite accurate (any residual cost on a graphics card after 3 years is so small compared to the current price it should be discounted and in included there).
Where you might win by owning your own hardware:
- Hardware costs go up, and thus api costs go up. You've locked in your pricing.
- Chinese/Open models become illegal/hard to access the way we do now. OpenAI and Anthropic are trying very hard to build a regulatory capture scheme to do this. I think they will be unsuccessful because China just won't participate.
Yeah, I'm not sure people realize how expensive ZDR/Zero Data Retention is, and how important it is to a lot of businesses, this kind of thing starts looking really cheap really fast if it's a reasonable substitute.
