Vibecoding an inference chip
ankitg.me - 59 poäng - 20 kommentarer - 10072 sekunder sedan
Kommentarer (8)
- cpldcpu - 4308 sekunder sedanI did something like this with Opus 4.5: https://github.com/cpldcpu/smollm.c
It seems implementing transformers is the new "hello world" of AI assisted coding.
- datadrivenangel - 6911 sekunder sedanDoes not say what type of model or size they were serving, but one of the traces mentions Maple, which is a 20b model. Still, getting ~70+ tok/s is quite good.
- codekansas - 5585 sekunder sedanThis is cool
Next step is tuning your post training pipeline to optimize your model for your chosen FPGA architecture...
- chalkycrimp - 4840 sekunder sedanGood chance I'm not the only one thinking this: share the code
- lxe - 1059 sekunder sedanRepetitive "make it faster" prompting is essentially what all these "RSI" claims are
- saejox - 4297 sekunder sedani wonder if we can emulate an FPGA on a rtx 3090? then test it on an AWS EC2 F2 few hours a day.
- spwa4 - 1879 sekunder sedanI must say I do see one application for this, that you can't really match reasonably otherwise: you can bake weights into an FPGA and then run them for 5W at a speed that will be competitive with a big Nvidia card.
(unfortunately for a decent model it will also be price competitive with a big nvidia card)
- Archit3ch - 2327 sekunder sedanNow do silicon. ;)
Nördnytt! 🤓