Nördnytt! 🤓Qwen 3.8 27B
- hypfer - 4815 sekunder sedanSince it might be helpful to some, here's my current commandline for llama.cpp running on an RTX 4090 with my monitor moved to the iGPU to free up all of its VRAM.
llama-server -m Qwen3.8-27B-IQ4_NL.gguf --mmproj mmproj-BF16.gguf -c 170000 --parallel 1 -ngl -1 --cache-type-k q8_0 --cache-type-v q8_0 -b 1024 -ub 512 --flash-attn on --no-context-shift --no-mmproj-offload --spec-type draft-mtp --spec-draft-n-max 5 --spec-default --cache-type-k-draft q4_0 --cache-type-v-draft q4_0 --threads 24 --jinja --reasoning on -fit off
Identical to the qwen3.6 config. With a prompt like "svg owl" (which can reuse quite a lot compared with creative writing or similar, so ngram-mod shines), I get about 70-80t/s like this, with a memory overclock of about 1.5GHz
- KronisLV - 9619 sekunder sedanI hope really badly that we'll get a new 35B A3B or similar MoE model!
I also miss the Qwen 3 Coder Next, which was 80B A3B, there are quite a few use cases where a non-dense model <100B would be the sweet spot (when you have the VRAM but not the TDP or compute power). Heck, I'd gladly take A5B or A8B or even A10B as a sort of middle ground.
Also alternate link for viewing the images without signing in: https://xcancel.com/Alibaba_Qwen/status/2088280182356611304
- scrlk - 9673 sekunder sedanBeats Opus 4.7 Max (w/ Claude Code) on DeepSWE (42.2 vs 40). Looks like Qwen's 27B models continue to pack some punch.
Unsloth's GGUF quants are up: https://huggingface.co/unsloth/Qwen3.8-27B-GGUF
- ramon156 - 7828 sekunder sedanPeople will claim it's not comparable to Opus despite it beating the score. I'm not sure I disagree, but I'm also unsure whether I care. Most new models nowadays are "good enough". I cannot complain because I'd rather spend that time improving my prompts and docs. Opus might be a _slight bit better_ at picking up vague hints, but it's also extremely expensive, and I hit the 5 hour limit way too quick.
I care a lot about speed and efficiency right now. For my setup I would like to have 2-3 different model families. I've settled on GLM-5.3 (formerly Deepseek v4 pro 0813) for architecting, Deepseek V4 Pro 0813 for developing, and Gemini flash lite (any recent cheap model) for repo scouting. I'll add another one in the mix for reviewing (in this case Gemini 3.7) and that's all I need.
I've tried most models except Grok.
Qwen is too expensive IMO (Alibaba Cloud subscriptions are hard to come by and I'm not spending 50 euros a month for a tool, so 18 euros it is). If it ever becomes efficient enough to run locally I will definitely look back.
Claude is slow and expensive (the cache hit prices are absurd).
OAI is pretty good, I might add it to my arsenal seeing how cheap it is.
These opinions change every day. Last week I would've never picked Deepseek until I read about the pricing. even post aug 16 it's worth it (although it's getting close to gemini pricing).
Right now my costs are 12 euros a month (z.ai) + whatever deepseek consumes. This typically isn't more than 8 euros a week. 44 euros a month and I have a setup that is doing pretty well.
- satvikpendem - 2848 sekunder sedanAs usual, the Jinja templates are messed up so use this [0] to reduce or turn off thinking, fix tool calling, keep a 100% KV cache hit rate, etc.
[0] https://huggingface.co/froggeric/Qwen-Fixed-Chat-Templates
- onlyrealcuzzo - 9406 sekunder sedanIf the benchmarks don't lie, this is getting very close to Opus 4.6 capability - which was the turning point for me for when AI was "good enough" that it became very hard to justify not using it.
I'm sure there's some benchmaxxing going on, and some things you get only with a a larger model.
But I'm feeling pretty confident if not by Gemma 5 than by mid 2028 we'll have local models that are almost always as good as Opus 4.6 was and in many cases far better.
- LeBit - 8943 sekunder sedan
- Casteil - 6496 sekunder sedanOne thing a lot of people don't seem to factor when hyping Qwen is how much models like this tend to 'overthink' with seemingly endless 'second guessing'. 3.8 seems no different from what I've tried thus far.
As capable as it is, it's hard to justify using it when a competing model (e.g. Gemma4:26b-a3b) can consistently achieve the same or similar response with only 1/10th as many 'thinking' tokens, achieve much higher tokens/second, and take a small fraction of the time. I suppose 'YMMV' depending on your use case.
Also, I haven't used it enough yet to see if it's prone to infinite looping, but its predecessors sure were.
- literoldolphin - 1149 sekunder sedanWhy is anyone even using video cards these days? You may as well be burning cash.
This is the perfect candidate for just splattering it on your nvme and then reading it off there and into memory. All of these run perfectly fine on simple m4 silicone:
https://github.com/drumih/turbo-fieldfare
- monkmartinez - 759 sekunder sedanQwen3.6-27B has been the main LLM powering my little agentic stack. I have adopted the test and verify approach to any models allowed to run on my machine. When the "heretic" version drops, I will fire up the harness and test. Super excited to see how it stacks up against Qwen3.6!!!
- jedbrooke - 9125 sekunder sedanI hope the bonsai team makes another 1bit quant of this model (or releases code/instructions on how to do it), using the Qwen3.6 27B on my 16GB mac mini has been wild . The 1bit quant feels like opus level… for the first couple turns. Then it has trouble eg switching from plan mode to act mode. This is mostly mitigated by starting a new session. (tbf this limitation is called out on the hf page)
I saw unsloth has 1bit quants too so I might check that out, anybody have experience with those?
- Almondsetat - 3660 sekunder sedanThe $1500 Intel B70 with 32GB of VRAM can run this model at max context with good performance, btw. If you don't want to drop $5-10k for running DeepSeek this is your best budget option for local refactor/small scale dev help
- T0mSIlver - 7499 sekunder sedanUnsloth Q4_K_M on a single 3090, llama.cpp "Generate an SVG of a pelican riding a bicycle" first try https://www.reddit.com/r/LocalLLaMA/comments/1voa3ch/comment...
- TomGarden - 9369 sekunder sedanAny tips on the best approach at running this at an M4 Max 128GB? Token throughput was a bit slow with the last 27B one (MLX), ended up using the A3B variant but if I could get this one to reasonable speed I'd much prefer it.
- NorwegianDude - 9412 sekunder sedanIf the benchmarks are a real indication, we now have a local model that is runnable on a high-end personal PC that trades blows with the leading model Claude Opus 4.6 Max from half a year ago.
Insane if that is the case. Downloading now!
- xlayn - 8607 sekunder sedanThe file "Just loads" on llama.cpp, the Unsloth https://huggingface.co/unsloth/Qwen3.8-27B-GGUF is an MTP file, I see mostly the same speed on pp and generation. There has to be something wrong with those benchmarks, I find extremely hard to believe a 27B model can work similar or exceed opus 4.6.
- erdaltoprak - 10065 sekunder sedanThis is one of the most important model releases since most use cases don't need SOTA/Frontier
If you want Qwen3.8-27B Serving Configs for the DGX Spark vLLM NVFP4 and RTX 4090 llama.cpp GGUF I added the setups here https://x.com/ErdalToprak/status/2088299678085308761?s=20
- piyh - 8044 sekunder sedanQwen 3.6 is ~$2/m tok, 3.8 should be drop in replacement. Gemma 31B is $0.34/m tok. The price differential on these models is massive on openrouter.
- tosh - 9695 sekunder sedan27b dense model at Opus 4.6 level
Opus at home
I hope there also will be a new ~10b variant
- syntaxing - 1608 sekunder sedanWould I be surprised there’s bench maxing happening? Yes. But some users also use Q4 quantized and complain how dumb local models are.
- mraza007 - 4769 sekunder sedanMan what a week, We just had GLM 5.3 that came out and then we had smaller local model Qwen3.8-27B from Qwen
Just tried using Pi Agent and looks very promising
- chvid - 9892 sekunder sedanThese are massive improvements - and something you can actually run on a laptop.
- - 3861 sekunder sedan
- mickeyp - 8483 sekunder sedanModel benchmarks are useful, to a point, but it is the long tail of things you do with the model that determines if it's good at a wide range of activities. Ant/OAI, to their credit, build their models -- even the small ones -- so they follow instructions and do tool calling well, without the system prompts confusing them. This is especially important for long-horizon tool calling.
So one open weight model might "meet" Opus or whatever on benchmarks, but then fail to follow a simple answer format and also tool call correctly. The models are whipped to within an inch of their lives to strictly adhere to their post training quality gates.
- theanonymousone - 7897 sekunder sedanI'm wondering whether any provider can offer this for cheaper $/token than the new DSv4 Flash, which is both cheaper and smarter :/
Completely local use is a different story, of course.
- minimaltom - 6133 sekunder sedanArchitecture thread! Afaict they continue to use gated attention + delta net, which was also adopted+adapted by K3, but im surprised theres no improvements to the residual stream (deepseek are using manifold hyper-connections, kimi have attention residuals) ?
Perf improvements seem to all come from training?
- bertili - 5566 sekunder sedanWow. Speed improved as well. 200t/s on a RTX 5090!
- arjie - 5798 sekunder sedanI use the Qwens as a vision model for my DeepSeek V4 Flashes to handle. But the Qwens run on old RTX A6000 Ampere. Does anyone know if there's any news about INT4/AWQ quants for the RTX A6000?
- synergy20 - 8045 sekunder sedanI wish this can run directly on my RTX 4090, seems like 30B is the sweet spot for dense model to run locally, sadly RTX 5090 is very expensive and I need a new PC and new power supply(and UPS) to run that, adding a second RTX 4090 is another option, but not sure if my PC can do that yet.
- TomGarden - 10219 sekunder sedanReally excited to see what people do with this. 3.7 27B was probably the best compromise between size and intelligence to run on consumer hardware
- irthomasthomas - 5877 sekunder sedanWhy don't qwen/alibaba host the model themselves? I was looking forward to trying it on their coding plan. Google are the same way with their Gemma models.
- ThouYS - 9325 sekunder sedanI am so happy right now, qwen3.6-27b was an absolute game changer. To see another one in the same league.. phew
- - 9781 sekunder sedan
- kunver - 9384 sekunder sedanLooks like a pretty significant improvement on the DeepSWE benchmark compared to the previous 27B model.
- jlkivey - 7359 sekunder sedanNote: on the model card the comparison to Opus is Opus 4.6 Max, not 4.7
- kristopolous - 9623 sekunder sedanq4km is about 48 tps on a 4090. my llama.cpp params are --flash-attn on --parallel 1 --load-mode mmap
- yassa9 - 8255 sekunder sedanCan anyone who has that specific personal test he tries on different models , and tries this model , to tell us here if possible , how good or bad is this new model ? compared to others ?
I only trust those users genuine personal tests
- ThouYS - 8563 sekunder sedan3.6-27B on little-coder was already mind blowing. looking forward to this guy!
- anana_ - 9440 sekunder sedanMonstrous benchmarks! Hoping it is not benchmaxxed.
- tosh - 9104 sekunder sedanalso cool: Qwen 3.8 27b is multi modal!
- davidw - 2021 sekunder sedanI don't know much about the production of these models. How hard would it be to 'fork' something like this and have it not be full of CCP indoctrination?
- pu_pe - 8856 sekunder sedanSeems to be SOTA for its size. Hopefully independent benchmarks will come soon.
- kunver - 9439 sekunder sedanWelcome deepseek flash flash!
- - 9434 sekunder sedan
- expedited123 - 9509 sekunder sedanKinda was expecting to see Gemma 4 26B in benchmark comparisons :(
- lossolo - 2620 sekunder sedanWhy weren't the points merged again from the "dupe" thread that had 289 points?
https://news.ycombinator.com/item?id=49299684
What a weird mechanism. If someone is judging a thread/topic/event impact by the number of points it got, then doing this unfairly degrades that thread.
It should have deduped by user and combined the 168(at the time of writing this comment) + 289 points. Just add the twitter link from the previous thread as an additional link in the description, like you normally do, move all the points over, and remove the old thread.
- naasking - 2717 sekunder sedanCan anyone confirm whether this new Qwen release is any more concise when thinking? Overthinking was the biggest (only?) downside of the Qwen models.
- tristor - 3242 sekunder sedanI'm hoping to see folks distill this with current generation Opus / Fable reasoning traces. I have had my best results locally so far from Qwopus (Qwen 3.6-27B w/ Opus 4.6 reasoning distilled). This looks GREAT and I am definitely setting this up later today.
- altruios - 9524 sekunder sedanremember to let llama.cpp catch up to anything new in this model. Save your judgment until about 2 weeks of use.
- filup - 7556 sekunder sedanhttps://news.ycombinator.com/item?id=48403639
my prediction was way too far out. 4.6 at home! Woo.
- brcmthrowaway - 8865 sekunder sedanThis with ddg mcp to fill in world knowledge. Are local models the future when computer architectures catch up?
- alpha_trion - 9380 sekunder sedanNICE, i've been waiting for this drop, thanks for posting this
- Mr_Eri_Atlov - 8064 sekunder sedanThis is the homelab model hands down
- brcmthrowaway - 9728 sekunder sedanMy Strix Halo is about to go overdrive!
- cmrdporcupine - 3985 sekunder sedanI found this kind of amusing while running it (using Pi as the harness). Don't know if this is evidence of intense fine tuning from Claude but it smells like it...
" The user wants me to explore the repository at XXXX and report back. Let me start by understanding the project structure, reading the CLAUDE.md file, and getting a general overview of what this repository is.
Let me start by reading the main project documentation and exploring the directory structure.
I'll take a look around this repo. Let me start by getting a lay of the land.
read resource CLAUDE.md (ctrl+o to expand)
ENOENT: no such file or directory, access 'XXXX/CLAUDE.md'"
- ramon156 - 8674 sekunder sedanneed another fable uncensored merge with 3.8, really curious what it can deliver
- ggerganov - 518 sekunder sedan[dead]
- RobertasTa - 5293 sekunder sedan[flagged]
- fintuner - 6003 sekunder sedan[flagged]
- steffi_oliver - 3748 sekunder sedan[flagged]
- - 8625 sekunder sedan