Verification Is Not Knowledge: Science needs more than answers

Ahhh, epistemology... Does any academic researcher even care about these things anymore?

If you're in academia, you've probably noticed a major shift recently. Research production at an industrial scale through AI.

Grants have "always" been awarded to professors based on quantity multiplied by quality. Now, some professors are churning out 1–3 papers a month with the same group of co-authors. All the grant money goes to them. Grant givers are just only counting the numbers of papers being published.

The Industrial Revolution has hit academia, and some academics are busy arguing for the artisanal value of research while some are creating factories for research.

The article's perspective reminds me of math class when I was a kid. The teacher didn't want the right answer alone, she wanted to see how I got there. In an absolute sense, the right answer is the right answer with or without my understanding, but that's not how the class was graded. We're approaching a point where AI is to science as my old calculator is to elementary school math -- it's capable of leapfrogging to answers faster and more accurately than we can.

Thing is, though, verification is knowledge itself. And if we can get to a point where the important questions have answers, we can surely have the verifiers explain those to us in ways we can understand them.

We all know Terry Tao, do you know Peter Scholz?

Fields medalist and as Tao listed as one of the most important mathematicians of our age.

Scholz is known to find „precisely the correct and cleanest way to incorporate all the previously done work and find an elegant formulation for that — and then, because he found really the correct framework, go way beyond the known results“

In a way that man is the complement of an AI, he generates understanding rather than just proofs.

Maybe he is the right template for the mathematicians of the future.

https://www.quantamagazine.org/peter-scholze-and-the-future-...

Arriving at truth values has limited value in science, obviously (with full understanding being more useful). For example, knowing a chemical reaction will produce heat is useful (cavemen discovered that one), but knowing _why_ unlocks an entire world of useful tools (the combustion engine, space heaters, etc).

However, even getting truth answers will soon be beyond AI. It is of course making impacts in Math and in publishing in "data heavy" subjects where you can just formally verify, program, and/or munge data.

But in "real science" where we actually need to build a lab, buy supplies, hire personnel, get approval, form physical test harnesses, install sensors in proper locations, calibrate everything, then finally make one measurement, question the whole thing, then finally make a conclusion, AI is of limited benefit.

Yes, it can help speed up coming up with some formulas, spitballing, or checking existing research. It can even help write the jank code for sensor calibration, measurement taking, etc. That's not nothing, but we are _very far_ from meaningful contributions to real experimental hard sciences.

As for math, theoretical physics, computer science, and other "logic based" studies, the question becomes one of benefit to cost of true understanding versus simply knowing something is true.

I'd argue that it paints a starkly negative picture of most science. Once answers are easy to obtain, it seems most people are concluding that actually understanding why its true isn't worth the effort, which means that the question maybe wasn't worth asking.

Its the science equivalent to the recent realization in tech: most ideas suck, your product didn't need more bells and whistles. Actually the engineers pushing back on every shower thought from product was a good thing. The friction that stops you from putting every random-ass feature into your already bloated useless SaaS product was good, and now the sales guy is pushing features to prod for a customer demo. We thought "too many ideas, not enough engineers", we now know "not enough meaningful quality ideas".

In the same way we thought "too many questions, not enough scientists", we now know "not enough real study on meaningful topics".

Surely the way forwards is to take the results obtained by brute force with AI, for example that certain conjectures are true or false, then to use that knowledge to further human understanding.

Knowing that something is true or false will at least stop humans wasting time trying to prove or disprove it, and allow them to find a more elegant proof, or investigate the implications of the true/false result.

> “The other side argues… human understanding is paramount, because it drives integration of new findings into the canon […] But the first side, those who emphasize correct outputs, dismisses this as gatekeeping or nostalgia.”

Is this description AKA. theory vs praxis? Academia vs Engineering?

Science has been industrialized, it has to change.

I propose a two-part approach.

First, autonomous or human-directed AI agents submit their work to a public repository.

Second, the scientific community draws from this pool of results, identifies the important ones and asks the next set of important questions. This work of interpretation, synthesis and question formation should be recognised and rewarded in peer-reviewed academic journals.

"Is science about humans achieving insight, or the archiving of verifiable solutions?"

I wonder if this dichotomy fully captures the value of scientific discoveries. One could argue that their practical impact can be independent of whether humans fully understand how they were reached.

You are conflating science with philosophy

> Verification Is Not Knowledge

A bold stand on the P = NP question.

[dead]

[dead]

I would rephrase it as: "Verification is not human knowledge", but that doesn't really matter. We are going to be several steps behind AI as it reaches a level of intelligence far beyond human capabilities, and we are not going to be able to catch up with it. It's like trying to beat AI at chess: we are bound to be left behind as AI continues to expand its knowledge in mathematics, physics, computer science, and other fields. Eventually, no human being will be able to reach that level of knowledge within their lifetime.