AI Coding at Home Without Going Broke

I haven't found anything that requires running all night. I could tell it to one-shot a big plan but given how often I realize I want an intermediary thing to be slightly different it seems like a waste of effort.

I'm guessing the next thing I should probably look into is some sort of machine vm I can tunnel my codex-gui requests to so I don't have to deal with the sandbox approvals (I don't want to give it "dangerous" access to my entire mac).

I don't understand what people are doing with their side projects that is leading them to churn through tokens so quickly, to the point of requiring two $200/month subscriptions and a bunch of token charges besides.

Oh, so this is not a post about AI coding at home. It's about vibe coding at home.

There's a lot I disagree with in this post, but I'm posting this from a home computer with 64 GB of RAM and no GPU. I do lots of AI coding while spending very little money. I run Gemma 4 26b (mixture of experts) and Qwen 3 coder with Ollama. I use Github Copilot code completions. I use the Gemini and Mistral API free tiers. I have a Gemini paid API account. It's now prepaid, so you don't have to worry about an accidental $1000 bill. You can do a lot of things with Gemini Flash Lite 3.1.

None of this is burning through tokens to create an expensive blob of spaghetti code, but it does qualify as AI coding.

Power is not free.

What I’ve found is that you’re basically paying a premium for privacy, and that’s worth it for me.

I did explore self-hosting models but hardware right now is just too expensive.

3090s and 7900s are going well so far.

Next year an Arc Pro B70 won't produce you less tokens than today.

They aren't fast but if you have flows where you can make money with them - they are a bargain in terms of price per Gb.

I learned coding nearly 24 years ago and still learning new stuff all the time. At no point in time I had to rely on a subscription model to learn and do new stuff.

If LLM and agents are the default tools for coding and building software, at least for next few years, it seems like a no-brainer to invest $2000-3000 on hardware, like a Halo Strix PC.

What does this look like after 6-12 months? Like, how much code are you trying to write total?

Maybe it just doesn’t click in my mind, but sometimes I wonder about how much work people are trying to do and how they actually have enough to get done so quickly in such a short amount of time.

My baseline is sonnet 4.6. I think it's good enough for most tasks sincerly. So, from what I see, we are already at a point where we don't need frontier models for serious coding and debuging. Give it a couple of years and that level will fit 120B models.

At the same time, we saw the rise of direct acess memory systems like DGX or Stryx Halo that will allow to run models of this size for "cheap" in the medium term.

That's what I'm betting in. That in 2 years I can buy a system for about $2500 that will run a model that's similar to Sonnet 4.6 locally.

I might be spectacularly wrong though. But I'm willing to wait and use subscriptions/API calls for now.

Depending on what one builds, comprehensive documentation and applicable skills and memory tools often allow for a substantial reduction of tokens previously used by the agent to comprehend and remember what is being built

1: https://news.ycombinator.com/item?id=48519181

I realize this text is just slop but it never stops being a "real bargain" at any point.

And it's more like $200/mo for $4000+/mo in tokens. You can also buy additional subscriptions.

There's no sense in running local models or doing anything else as long as VCs (and soon the public markets) are willing to pay your bill.

If you still need more tokens, odds that you're vibecoding unmaintainable throwaway trash.

As usual, an extraordinary claim without an extraordinary evidence: https://stephen.bochinski.dev/apps/

No clue what y'all are doing, perhaps because I'm hobbying, and also I'm old and can perhaps do more of this by hand.

But I'm basically just doing what I did before, plus ollama self hosted and sometimes gemini and I feel like I'm going lightspeed beyond what I've ever done.

And I suppose this is still very fine-grained. I have it make a draft, then just have them fix/change it step by step?

I tried one of the bigger boys that can one-shot apps, which I guess is cool, but I'm finding it's just as hard to modify as if I just grabbed someone elses repo on github.

In the good ol' days, we bought machines not only to run stuff, but to experiment.

I understand today experiments are limited. Inference is reasonable, fine-tuning is either niche or a stretch, and base training is impossible.

*That is bound to change*, and when it does, there will be an avalanche of hobbysts and amateurs poking at base training. They'll find optimizations no one found before, synthetize data no one ever imagined to synthetize, and when that happens we'll start getting libre models.

So, yeah. Right now, buying the machine doesn't pay off that well, unless you want to pioneer this stuff in severe adverse conditions (hardware prices inflated, etc). Eventually, it will.

I don't think its feasible to have something comparable to these frontier models when they are increasing usage and lowering token costs