Sebastian Raschka's from-scratch video course on building reasoning models

I've got a question. I'm having a bit of trouble phrasing it, so please don't assume I'm against learning for learning's sake, that's not the point.

Whenever I've tried to learn about LLMs, machine learning, etc. I always come out with a bad feeling, compared to learning about software.

Theoretical courses make me feel like it's all baby steps, unless you assume a level of commitment equivalent to a math degree. Practical/applied stuff seems to require levels of compute, training data, etc. that make it unapproachable for a home hacker for everything beyond basic exercises like a handwritten digit classifier.

Now this feeling of "impracticality" is perfectly fine. It would be the expected outcome if I was taking a course on classic literature or modern art. But machine learning stuff feels like it could have room for learning-by-doing applications that aren't entirely pointless, so I wonder if there's some way to approach the topic I'm simply not finding.

Did anyone have the same experience? any tips if so?

I feel there's a misconception in a lot of the comment threads about what an LLM-from-scratch tutorial is for. Of course you're not going to come out of this kind of tutorial being able to train a useful LLM from scratch. (It requires too much compute, and datasets you don't have access to, and there are a lot of tricks and operational problems to solve for real training runs.) If that's your bar for "impracticality", then sure, this is impractical.

But that's not the point; the point is understanding these things better by seeing how they are made (or grown, if you prefer). Building something at a level further down the stack than you will ever go forces you to confront details that shape the layers above that you do deal with. This is one reason why Crafting Interpreters is such a beloved book here on HN.

LLMs are now at the core of tools we all use everyday... don't you want to know how they are made? Maybe not everyone has this itch to understand the platforms and tools they use, but I sure do, and it's helped me over and over again. Sure, LLMs aren't like operating systems or compilers. They're a quasi-organic mass of billions of parameters learned during training and honestly nobody (not even the mech interp folks) really deeply understands why they work so well. But that's no reason to completely give up and there is still a lot you can learn.

Take RLVR ("Reinforcement Learning from Verifiable Rewards"). This is the training technique that made LLMs get so good at programming and math in 2025 and 2026. A lot of people who confidently talk about LLM capabilities either haven't heard of it (and thus end up parroting out-of-date factoids about LLM training derived from the pre-2025 era) or have only a very dim idea of what it is. In this course you actually do RL training of an LLM to get it solving math problems better. You're right down in the details. It's interesting for its own sake but it will also give you a grounding for discussing questions such as: "why did LLM writing get so weird and bad in mid-2026?" and "why is it so hard to train LLMs to be good at things that aren't like math and programming?"

Anyway, Sebastian's two books are great and highly recommended; I worked through the first one a year ago and it was one of the best investments of my time in all of 2025. If you prefer video for learning this is great as well (although I'd still say - try the books first).

I got his book Build Your Own LLM From Scratch and it's super readable and does a great job at holding your hand through the entire process...need to finish reading it though!

Highly recommended! Sebastian does a great job explaining complex AI topics in a very intuitive way.

What would be the computing cost to go through the book (or the course)? Do you need Cloud GPUs, or can this be run locally, if so how powerful your computer has to be?

I've been wondering about getting a PC to poke around LLM stuff (not a practical use, but more for learning purposes) but I don't have any intuition about how much is enough and whether it is worth it to me.

Maybe someone can clarify this:

He says:

> A reasoning model is essentially an upgrade from a conventional LLM, that is, adding more post-training via reinforcement learning with verifiable rewards (and optionally inference scaling techniques) to make it more capable of solving more challenging problems.

But this isn't really true is it? When reasoning models came around with OpenAI's o1, the main feature it had was to be able to try several paths and select the best one. This is what a reasoning model is. This was before agents and reinforcement learning with verifiable rewards wasn't it? And then RLVR and agents were added on later. Or am I wrong?

I’ve read, worked through, and watched his LLM book as well as this one! Things finally clicked, can’t recommend both books enough—his pedagogy is great (at least for me).

All of these build your own model guides are like someone putting out a build your own CPU guide 10 years ago. Interesting, but not of any practical use.

It seems going forward AI is just going to build more AI and no human will understand it.

I just started reading his book, and he makes it easy to grasp difficult concepts. As a matter of fact, I have been working on platform where you can learn how AI models work in an interactive manner.

The way he explained the attention module helped me a lot. I have read other people explanation but his technique helped me 8n remembering it easily.

This is an essential resource and his book is a must read from this author. I have the upmost respect for Raschka for his research and detailed explanations in AI including him dissecting research papers.

I have read both and the are excellent

saved

[flagged]

[flagged]

someone's getting auction-poached at openai and anthropic

Do you not just say “do reasoning” and leave it an open “<reasoning>” tag?

I'm a bit confused as to who this is for - apart from learning how the tech works that is.

If you're truly going to build your own model (and commit to the cost of that), then video tutorials will not cut it.

And if you don't really, then I wonder if this is really the best style of conveying the information you'd be interested in?

I mean I might be wrong and maybe this helps emerging players or something like that, but to me it is more likely that this is more of a monetisation strategy of a personal brand. Living off selling to people that think they _should_ but then in the end don't.