Artificial Analysis Intelligence Index v4.2

I have no idea how artificial analysis got to be something anyone took seriously. This is their new benchmark set?

A glance at their new index shows that whatever they're measuring, it isn't useful.

Spend an hour with gemini 3.8 and tell me that model belongs in 2026. It feels like the model has Alzheimer's. It gets confused about whether what it reads is what it did. Just crazy bad.

I haven't tried muse spark 1.3. But it must have been a miracle since 1.2 to hit that rank.

Video game journalism vibes all over this.

They realized Astra having the same score as Sol was silly so they rushed to update the index so it fits what people expect.

The old index was clearly bad (Astra is way better than Sol) but it's also unscientific to tweak it like this.

Imo the omniscience index they have has the highest correlation to actual usefulness of the models.

https://artificialanalysis.ai/evaluations/omniscience

> measures knowledge reliability and hallucination. It rewards correct answers, penalizes hallucinations, and has no penalty for refusing to answer.

This is so useful because it makes you actually trust a models output. A high score on benchmarks is not as useful because a model overtrained to always answer will give confidently wrong responses. But this index measures how often it is correct while penalizing wrong responses so that a high score means you can trust this model more and when it doesn't know it is more likely to tell you that it really doesn't know rather than making shit up.

Fable also performs a lot better than opus 5 here which correlates very strongly with perceived strength despite the models performing similarly on e.g. DeepSWE

Astra is a big jump from sol and performs the same or slightly better than fable here.

It is very unfortunate they upweighted SciCode from 8% to 10%. SciCode is a broken benchmark: see https://arxiv.org/abs/2608.04975.

This is really a great achievement: "Astra dominates the output token frontier"

Many labs used increased thinking to boost benchmark scores and performance. Most of the Chinese models were doing that for a while. Google and Anthropic as well.

Not OpenAI. 5.6 already was much more token efficient than other models and Astra beats Sol in token efficiency by a wide margin.

Edit: Just to make the point: Astra (max) has the 2nd highest score and the third lowest output tokens (among the models shown by AA).

In terms of intelligence per token per cost, OpenAI is really killing it.

This update really gives OpenAI a boost. Not saying there's anything inaccurate or untoward about that, but the timing is unfortunate. It would have looked better had it been done prior to the Fable 5.1 and GPT 6 releases. I guess AA would say that there's no perfect time to do these updates, given the rapid fire pace of releases!

What version the intelligence vs cost graph is using? they didn't ran v4.2 to all models.

how did you check the prev version to compare?

How to view the previous version to compare?

Why have they not included ARC-Agi-3 on their index?

Clearly that would move things around.