AI boosted homework scores, then exam scores dropped: study

The kids who used AI and studied for a similar amount of time as the non-AI high performers apparently had similar (slightly higher) performance. The kids who used AI and did poorly used the AI to do the homework for them (this is what the article says).

I believe AI is basically an amplifier of bad and good. I’m cynical about the world and assume it will be used more for bad than good, but I don’t doubt some of the best people in every field will be using AI to amplify their work in good ways.

All AI did here is exposed an old problem in education. Kids are expected to get everything perfectly for the first time but a lot of them don't so the class gets easier next year to keep up pass rates.

We need to restructure the system to treat failure as a signal instead of a disaster. Grades should come from hard randomized exams with unlimited retakes so one bad day won't hurt you. Homework should be optional material for self study, evaluated by teachers if you choose to do it but never forced.

Hold back students for individual classes instead of a whole grade so failing one can't ruin your social life and teachers are more willing to do it. F students will realize they have to study, start actually learning and then pass on the second time. No big deal. It happened to my friends in college, no reason they can't do it in high schools.

Discipline is a skill and it's one you have to get from experience. If you try and force kids to study when they don't want to "for their own good" you're not actually helping them. Everyone needs to find their own path. Let people fail.

Moneyshot: "The results are eye-opening. After six months, pupils using ai saw their average homework score rise by 18% across all subjects. The time they took to complete each assignment fell from an average of 64 minutes to 45. But come exam time, the same students scored 20% below their classmates who had not called on ai’s help. Homework scores once predicted exam performance; now those who score highest are, perversely, more likely to do worse in exams."

Study design: "David Stromberg of Stockholm University and Victor Lei and Wu Yanhui of the University of Hong Kong set out to fill the gap. They tracked 27,000 pupils aged 12-18 in China, where ai adoption has been fast. Around 80% reported using models such as Doubao and DeepSeek; the other 20% formed the control group."

I am waiting for AI Viva Voce (“AVV”). I am really surprised we aren’t hearing more about this idea?

Try it for yourself by making a prompt like this (adapt as needed):

Pretend I am an undergraduate student of Computer Science. I am learning about early microprocessors from the 1970s. I want you to ask me an examination question as if you were doing a viva voce exam with me, to test my understanding of concepts. I want you to receive my answer and then based on what I said I want you to ask me a more specific question to probe my understanding. Repeat this interaction up to 5 times. Then grade my understanding so far, by giving me a pass, merit, credit, or distinction. Can you explain how you arrive at the grade based on my answers and your expectation of undergraduate knowledge of microprocessor theory?

For some of the classes my kids were in, the lessons and homework time were reversed.

Learning the material (using the text book and videos) was what they had to do at night and class time was spent working through problems or applying the material in some way.

Education sustem is not about the actual knowledge, but about training various aspects of the brain during kids development. So even memorizing literature have benefits, even though it doesn't bring any career use.

If you skip this, well your brain won't develop as much at period of life it's able to do so.

Taking a forklift to the gym increases the number on the bar, but mysteriously your gains go down.

If you structure learning in such a way that makes learning just a means to some end, and overindex on that end being the ultimate goal, that's what you get.

This is a pedagogical problem that AI merely exposed. Educators need to figure out How to make students choose the scenic route instead of having them optimize for the most efficient completion of a task.

Glad to have scientific results on this, though in my view you could get this from first principles. Homework is onerous but it forces you to learn the material and get it into a configuration that works in your head, which you then validate with the exam.

Students seem to optimize for local maxima/short-term rewards (homework scores) at the expense of the global maximum/long-term rewards (learning).

In my professional career, my employers give me a lot more assignments than exams.

It is stopping some of the adults I work with from learning.

In the case of some management, is causing them to unlearn, forgetting about proper review and maintenance practices.

I know this study is focused in China but one thing I'd like to understand is the nuance in what kind of student is likely to always reach for ai for homework help as I think (especially in the US, can't speak for other countries) the varying powers that be that can determine the quality and type of education you can receive (does your country have means to buy textbooks per student or shared, do you have funds to give students ipads to take home or no, what sets the bar on how far curriculums can go, are teacher and faculty pay and incentives simply tied to student pass rates, etc.) and those do more in shaping what "use AI responsibly" will look like and why it's so different.

Honestly I wouldn't demonize AI in education. When we were students we were forbidden from using calculators, then the internet and now it's become the norm. It's the same with AI, I think. AI has already become a part of our lives. The important thing is just teaching children how to use it correctly.

The homework helps the teacher track their performance/progress.

When the signal is removed, this is what you get.

Use AI for homework, you will get it done faster and get a higher grade, and then get crushed on the exam.

Are all of our LeetCode scores down, too? We should probably code the old fashioned way periodically to ensure we still "got it".

The article does not relate AI usage for homework to exam score drops.

It tries to link them together with no obvious cause and effect.

The score could have dropped due to worse environment at schools or kids who were stuck at home studying online and now in physical locations could not adapt fast enough (meaning, possibly just temporary drops). We do not yet have enough metrics gathered yet.

This type of article bashing AI for the root of any problems, I find it appalling without any evidence.

It's nothing but a conjecture.

Teachers have become proctors and not educators.

i’m pretty sure there’s already a correlation between technology in the classroom and worse learning outcomes

I used to skip homework in high school because it was boring and cut into my personal life outside of school (including my job) but I would always do very well on tests.

Then I stopped caring about tests too.

I did the same thing through college. The only reason they passed me and I got a degree is that I built the school’s website and I built personal ecommerce sites for the head of art and his wife to sell their paintings.

I haven’t done shit since like 7th grade.

Worked at Facebook, Apple, Microsoft, same behavior there - did basically nothing for them while making thousands off my games on App Store.

Fuck authority

abolish homework

[flagged]

[flagged]

[dead]

[flagged]

I think we need to accept that AI causes cognitive decline, like Alzheimer's. For an Adult this may be manageable, but in a young brain, this is detrimental.

This feels like the calculator debate all over again.

Yes, using a calculator leads to worse outcomes in your learning. But also, yes, everyone always carries a calculator on their person 24/7 these days and you'll never be as fast and as accurate as a calculator.

I think it would have been kinda cool if you could have unlocked the privilege of using a calculator in school through good grades in math class. Prove you can do what the tool can, then you get to use it. Would have also provided at least one incentive for students to get good grades in at least one class.

Yeah, obviously. We also don't know how to use flint and steel to start fires any more. You wouldn't outlift a crane, why would you want to outhomework an AI? And you can't even argue that this toil is going to help in the workplace, because the workplace is now AI-native, and you can offload everything to AI.

Guess what this is probably fine. AI helps people do their work on a daily basis and probably makes them memorize less things. That’s how the world is moving. You don’t need to be able to memorize something for an exam to show mastery of a topic. And we shouldn’t organize our society around memorizing things for exams.

When you get out into the real world, what will matter is how effectively you use AI. When you are done with school, you can take the "exam" using AI. The problem is that teaching is trying to teach skills that are no longer relevant and is always behind what the latest going on in the industry. Every time without fail you got interns, who after two years at stanford would learn more in the 3 months on a google internship than in those two years at stanford. They learn useless shit.

My own experience is that college is fun. You can spend the time wisely, but school stuff, is not what makes you money.

Make student loans dischargeable in bankruptcy and make colleges under write them. The problem will get solved quickly.