GLM 5.2 vs. Opus

by definition, a single prompt wont' constitute the complexity of a software project. ergo, what you'll get is a series of assumptions made by the model based on preexisting code in its training corpus.

I'd rather see a coding agent that can follow steps in a plan file to a T while following guardrails and adhering to the proper coding conventions in the human reviewed spec.

Id rather see performance in agent loops against human defined objectives where it can be verified to stick to defined guardrails and continue without drift till its objectives are complete.

I'd also like to see it identify bugs and potential performance increases by identifying existing code and suggesting refactors based on context it can pickup about the particular use case you are trying to create.

These are way more valuable metrics than "hey build X"

Capability per dollar is something I care about:

    Opus API    $5/$25
    Sonnet API  $5/$15
    Haiku API   $1/$5

    GLM 5.2 API $1.4/$4.4

Running a single one-shot prompt is not a benchmark, not is it representative of any sort of real-world usage.

Most agent usage is collaborative so you need to test things like reliability (when I delegate a task, does it complete it without making up test results for e.g.) and steerability (does it obey my instructions or does it just do what it thinks is best).

- it takes it sweet time to get code rolling, not the fastest model by any means

- it strays a lot during discovery/planning but then corrects

- it's not steering friendly, as it hallucinates things that it doesn't follow later on

- its output is quite good

A sample use case: I was optimizing rendering on Swift+Zig codebase. It chocked on 5k data entries.

GLM 5.2 spent 20 minutes building the benchmarks and getting data out, which made me frustrated so I blocked non-editing tool access and went AFK, after approx. 30 minutes I found that it used already-made benchmarks and some "conclusions" to optimize 3 choke points. Output pointed that it couldn't validate suspicions and asked for more data.

Implementation worked well, it was idiomatic and non-intrusive. I would even say that it was more idiomatic than GPT 5.5 effects on same repo.

I would opt in in using it more BUT GPT usually completes same requests 5x faster.

GLM 5.2 was spark for preparing and running inside isolated containers with JJ workspaces (so that multiple can be ran in parallel).

I'm not sure what exactly triggers it, but it seems to happen when it has to look at lists of countries. I suspect there must be at least one country name that triggers the safety guardrail.

You'd expect GLM to balk at something like Taiwan, but so far, it hasn't.

I haven't been keeping up on hardware costs for state of the art LLM inference, but this remark made me ask myself how many readers of the article would actually be able to run this model on hardware they own. How much would it cost to acquire such a setup?

This implies Opus was potentially much (?) better value.

GLM cost a quarter but Opus was twice as fast. So we are already at GLM actually costing half when you compare on time, without even considering the extra effort and time it would take to get Opus-par results.

It's good to have cheaper options and very impressive to see the Chinese continue to set open standards in this field, but the article is maybe a little over-generous.

Can someone explain to me where that time usage is coming from if not from the model operation itself?

Are the individual tool calls more complex and take more time to complete? Or is the rate of tok/s lower because the model does more compute per token?

A better way would be to use https://github.com/openbmb/MiniCPM-V

https://aibenchy.com/compare/anthropic-claude-opus-4-8-mediu...

But, it produces solid results for a fraction of the price. Worth checking out if you have the time.

One of my goto "tests" of a new frontier models is having it rebuild a programming language from scratch. For GLM 5.2 I had it rebuild the old Rebol language in Rust:

https://github.com/mhs/rebol-clone-glm-5.2

It did a fairly good job roughing in the language for a low token cost.

We have come a long way, and very clearly have a long way yet to go.

Off topic, but does anyone else instantly pick up on LLMisms like this? It seems like all the models have converged on this style of writing, and improvements aren't really changing it.

Yes, in terms of API pricing, GLM 5.2 outperforms the competition. But the only people that use API billing for their coding work are large corporations, where these highly subsidized subscriptions are being fazed out.

At the same time, none of these companies will use a Chinese API for their employees.

For individuals and smaller teams, Z.ai's coding subscription is outperformed by Anthropic and OpenAI. You probably get around the same usage with Claude, but Codex definitely offers more usage for the amount you pay.

We can have a debate how much Z.ai closed the gap to GPT5.5 and Opus 4.8, but if I can freely decide between them in a world where they all cost the same, I simply wouldn't choose GLM.

So the important question becomes: How good will the offering from Z.ai get with GLM 5.3 or 6 and how much will OpenAI and Anthropic cripple their current offering in the near future.

Glm game was completely broken Opus game was at first glance ok but also with bugs

Different models with different cost produced different non perfect results . How is it “close” ? :)

Also on costs : glm burns more tokens on average vs opus . Gpt5.5 burns less surprisingly

Opus is most expensive model in pay as you go model, but IMO fair comparison should include subscription price as well. For example when one has $100 Claude Max and use it up through the month, it might not be more expensive than GLM, or at least not 5x.

https://github.com/ggml-org/llama.cpp/issues/24730

"Build a 3D platformer game from scratch, in raw WebGL, with no game engine or 3D library" would be a zero-shot prompt.

I like it, but the lite plan ate 22% usage of my 5h reset window in a single session after 2 prompts on xhigh of GLM 5.2 [1m]

Result was satisfactory, I think stuff is decent, I'm happy to use either, wish there was a combined subscription plan where I could get both

So, 8000$, plus it's unavailable. 3 years of Codex/Opus subscription.

> API prices

Which are irrelevant for 200$ Codex/Opus plans that are times cheaper.

I would like to give them a try but I certainly not have the money to get a system able to run them, and I don't really want to pay more than the state of the art

My only, I guess feedback, is that it's not really clear about the price.

Would the 21.92 be the API pricing I guess?

Cost $5.39 (real billed) ~$21.92 (estimate, list pricing)

The real time 3d fluid dynamics appear to be the tricky part, I wish I still had opus access, would love to see if it can do it.

If it builds a UI and can't look at it, it's askin ls whether the app looks right.

I am not sure where this is going to lead us but it is fun to watch.

I used it with Cerebras inference at a time when it had a good coding plan at a low price, and delivered tons of stuff using it.

Tried with 2 harnesses and it seems bad + slow

At the end of the day, the time earned is more important then the cost for big players.

The ability to spawn 10 claude agents and rush a project to outcompete someone is more important for big businesses in my imo. Also the small details that GLM missed would take significant more time to iron out, considering it already took double the time.

I do hope other (open weight) models catch up, but to act like they are anywhere close for me is a bit disingenuous.