Apple has new Mac app coming to make running local AI models easy

Apple, to me, is the winner in the end of this AI chapter.

Distillation works: we can get models leaner, have better performance with lesser and lesser parameters, more efficient, running locally. Apple gets this and we'll see the result with M7 Max/ultra chips and I reckon this will be one of the titular apps when M7 line of chips are introduced.

I guess it explains the haste Anthropic and OpenAI are in to IPO, cash out and leave the the retail investor holding the bag.

"Harbor is currently in a private preview under Apple’s Claris brand"

There's a name I haven't heard in a long time! Claris (always an Apple subsidiary, since being founded in 1987) renamed themselves to FileMaker in 1998, but apparently they rebranded back to Claris in 2019.

Looks like they've mostly just released annual upgrades to FileMaker for the past decade, but they did add MCP to it last year: https://www.claris.com/blog/2025/why-mcp-matters-for-claris-...

I have been using llama.app, which is the desktop interface built on top of llama.cpp by the same people. It is an alternative to Ollama that might interest people who want to switch given Ollama's reputation and previous decisions. Llama.app has a really clean design, an elegant and intuitive interface. But it's good to get alternatives as well, if Apple also gets into the game.

https://llama.app/

As models improve and hardware becomes more capable Apple are definitely well positioned.

I am super interested to see what the Apple silicon team can deliver when their full attention is turned to local LLM acceleration.

If you ask who would pay for Apple local models, I would. A Mac Studio high end 512GB costs ~$1000/month for 3 years versus my current run rate of ~$1000/month for five devs for Claude Ultra (or ~$10k/week for Claude Sonnet if I paid API rates rather than Ultra).

If/when Anthropic changes to pure API pricing, or for API-heavy projects, that's when I'd switch to more local models. Of course there are tradeoffs for speed, quality, upgradeability, etc.

I currently run a Mac with 128GB and qwen3.8:27b-mlx. It hits the heat limit of the Mac when working on a 200k LOC codebase for security auditing and hardening for many hours at a time. This isn't as strong/fast as Sonnet, to be sure, yet the cost is only $6K one-time plus the usual electricity, maintenance time, etc.

Calculate the total cost of two new Mac Studios with max memory and cables. For that budget you could burn tokens on cloud APIs for a long time or build a dedicated server on open hardware with clear server buses

That will be really cool. It is already possible though through Apple Intelligence. There is an open source app that already leverages that.

https://github.com/pdparchitect/noodle

I've been running local models under ollama, then lm-studio, and now mtplx (command-line, from supervisord). I'd welcome a vendor-supported solution, if it doesn't turn into bloatware (like lm-studio did).

> Cluster for scale: “Connect two or more Thunderbolt 5-capable Mac computers and Harbor pools their unified memory into a single inference resource. Run models that exceed any single computer’s capacity, or serve more concurrent requests across your cluster.”

This seems like the main way it would differentiate itself over existing solutions like LMStudio etc.

This is what I've wanted for the last few years. I have played around with local models and had some success, but I don't want to deal with the hassle of setting up, updating, etc.

If all of this were easy, I would happily invest in new dedicated hardware to run it. This could be a full-blown Mac or it could be a souped up HomePod-type device, loaded with RAM. Better yet, make it a revitalized Matter-enabled Airport!

What I’d like Apple Intelligence to be is this plus a good local app to chat with models plus an iCloud version of OpenRouter with privacy screening features.

Even better would be a peer to peer model hosting hardware pooling system but that’s probably too much to ask and would expose Apple to liability issues in some jurisdictions.

If Apple were willing to eat some margin on RAM they could use the present RAM bubble to gain ground in the hardware market too. Financially you could call it buying market share.

I really do not understand the value of running large models locally. I am absolutely fond of local models but for specific tasks. The agents need to do a lot of work in this case: orchestration, big or small model split of planning and work, split into small tasks, merging, etc.

LLMs with <30B parameter models seem like the sweet spot. I would rather focus on multiple tiny LLMs that are tuned to specific tasks (SQL queries, finance/accounting, backend/frontend dev, etc.).

For larger models, a $10-40 ish Neuralwatt/Opencode Go or any similar subscription goes so far for the average user that it is ridiculous. We are still in the "youth" era of agents. There are many patterns that work well with smaller LLMs. Or with a big-small LLM combination.

My context:

I own the tiny 16GB M4 Mac Mini and I use LLMs (coding agents) to build software and a few of my products use LLMs directly. Full-time user of LLMs, do not read or write code manually. I run the tiny Qwen series (<10B params) sometimes but a couple $20 subscriptions (Claude, Neuralwatt) take care of my full-time agent usage.

As a recent M6 Mac mini owner, HELL TO THE YEAHHH BABYYYYYYY

I use oMLX btw, on 32GB URAM, running Hermes with Qwen 3.6 @ 128k context window.

I realize I am a smalltimer but for general day to day sensitive agentic stuff, this is the entry point.

I hope it will be optional. This is too much to ask for on Apple platform. I recently had to install a 3rd party script to get rid of Apple's laregely useless AI stuff from my 5 year old mac and save more than a few GBs which I badly needed.

Running local models is already extremely easy.

I have a suspicion this app will be a way to add a too many gigabytes to the mac and of course like the Chess.app there will be no way to get rid of it. There are excellent cli apps/GUI apps like this https://github.com/ddalcu/mlx-serve - they are both cli and gui – and even its gui surprisingly nimble (I use this, but I am not related to the project). I am yet to try this https://github.com/ggml-org/Llama-macOS but I've seen only good reviews of it.

I also do not think Apple is doing it for the users who won't explore around. I can't wait to get off this operating system. Next time a friend travels to the continent I will explore getting a Framework laptop via them.

they should add a sandbox for the agentic part as well.

so you can control and monitor what the whole setup is doing inside and outside the box.

I really want capable models local but it’s impossible for me to afford the hardware. If there is a magical way to run glm 5.3 without quantization at 80-100tps with a total spend of less than 10K I’d be happy but that’s impossible now.

I love that they are bringing Claris back for this. Moof!

But you’re still in for a ton of pain if you want to run them headless right?

They desperately need to fix that.

After I think about it, this makes sense that Claris would tackle such a project under the auspices of Apple. They're a subsidiary so they can monkey around with this stuff without having to adhere to the draconian everyday operating restrictions of "Apple proper". Good on them if they can pivot into something that hews to AI.

AMD needs Apple to kick it in the pants and make it realise they need to ACTUALLY compete instead of trying to just copy Nvidia.

The Mac Pro was discontinued at the wrong time. Or maybe just introduced at the wrong time.

All this talk about Apple being the long-term winner in AI is putting an awful lot of faith in Apple. Siri is still about 1/10th as useful as ChatGPT on a good day.

It’s the “year of the Linux desktop” meme all over again. Any day now.

- From a happy iPhone Duo user

(1) All the headaches of maintaining FileMaker Server on Apple hardware, (2) multiplied with the additional requirement to cluster devices, (3) but without users accustomed to the silliness of misusing consumer devices running a consumer operating system for a service that must be reliable, and (4) an Apple subsidiary unable to bring about meaningful change.

Sherlocking Exo, then--although that will work in a regular LAN as well. I should have finished https://rcarmo.github.io/projects/go-exotic...

Developers! Developers! Developers! Developers!

"Harbor is currently in a private preview under Apple’s Claris brand, but it seems likely that we’ll see it launch to the public in the weeks and months ahead."

Claris?? Ok, this sounds like it's just someone's personal project. Something like this that was actually going to ship would be in a different org.

https://getlocalcode.app/ runs local AI models (larger ones of course, DeepSeek 4.1 Flash 1017) easy. You can download and try it now. The problem is elsewhere: you cannot find M5 Ultra that you can purchase and set it up today.

So many projects, start-ups and frameworks called Harbor already

OMG after 25 years, Claris gets something beyond FileMaker (among the most legacy software still alive)? Claris would not have the engineering talent, so why would Apple use them as a vector?

what is the difference to ollama, lama.cpp ... ?

You can chat with the local model that ships with macos on your M1+ mac right now. There's a few servers on github that make it easy.

What would make running local models easy is being able to buy a Mac Studio without a 6-month delay :-(

Great, except with current DRAM prices nobody can afford it.

Deepseek has found safe harbor!

An Apple M5 Ultra running an app like that? Because Apple hardware is a monolith, unlike the PC space, "Harbor" will be configured out of the box with the ideal settings for many well-known open-weight models.

... Looks like I picked the wrong week to buy parts for a linux-based inference box.

[dead]

[dead]

[dead]

MacOS is a bad server.

Why so much investment in local models when the remote ones are so cheap ? Everyone was fine with remote compute & storage before AI, now they balk?

Hopefully it won't be another preinstalled bloatware app like Image Playground. macOS enshittification has been greatly accelerating, even giving Win 11 Home a run for its money.

[flagged]