Llama.cpp on a jailbroken PS5: run any GGUF model natively on the console

Even though it only runs qwen 3.8 27b at a 2 bit quant, it's still highly viable, especially for things where you just need a constant token stream of slightly below average intelligence, ie this would be great for things like:

- monitoring server error rates

- reviewing security footage

- watching plant growth / controlling watering

etc. Any long running task you can now just slap a PS5 on it. 70 cents a day in power cost means you now get ~2M output tokens per day (~$3.50 worth of usage). It's not the worst thing in the world if you have one lying around. 2bit isn't even that terrible - it's ~80% the accuracy of 27b at full quant, which should put it at around GPT-5 levels of intelligence (which honestly is crazy in its own right).

I've never even considered getting a PS5, but hackers converting a locked-down appliance into a general purpose computer has me really considering.

Conceivably this should also work on the BC250 cards, which are ex-cryptocurrency mining motherboard built around AMD's "Cyan Skillfish" APU which is a cut-down version of the PlayStation 5's custom silicon. You can still pick them up on ebay for $150-$200.

The project's primary site should not be GitHub, and I am serious. Playstation 5 is not some historical curiosity and Sony might strike the page down any hour now.

Those projects should default to hosts less geared toward corporate governancy; custom internet domain being the first requirement.

With this in mind, having LLM-powered NPCs would be a beautiful thing. There's been limited testing on PC games (PUBG Ally), but I can imagine it will be the standard.

Kinda cool but also entirely useless given the limited VRAM.

The readme oversells it by pretending it would be something actually usable, when instead it is a cool achievement to make it happen.

Not long until Anthropic's dreams of self-replicating models starts taking over PS5s!

Relevant: https://en.wikipedia.org/wiki/PlayStation_3_cluster

For somebody who hasn't kept up with all the different open source models and what token rates mean, how would using this compare to something like codex or claude? I assume it would be much slower and less capable..?

Don't give Sony ideas...

11.4G of shared memory. Could do something interesting if you don't have a GPU.

The toaster is next!