Nördnytt! 🤓The Waymo World Model
- mattlondon - 16852 sekunder sedanSuddenly all this focus on world models by Deep mind starts to make sense. I've never really thought of Waymo as a robot in the same way as e.g. a Boston Dynamics humanoid, but of course it is a robot of sorts.
Google/Alphabet are so vertically integrated for AI when you think about it. Compare what they're doing - their own power generation , their own silicon, their own data centers, search Gmail YouTube Gemini workspace wallet, billions and billions of Android and Chromebook users, their ads everywhere, their browser everywhere, waymo, probably buy back Boston dynamics soon enough (they're recently partnered together), fusion research, drugs discovery.... and then look at ChatGPT's chatbot or grok's porn. Pales in comparison.
- yummypaint - 2815 sekunder sedanBy leveraging Genie’s immense world knowledge, it can simulate exceedingly rare events—from a tornado to a casual encounter with an elephant—that are almost impossible to capture at scale in reality. The model’s architecture offers high controllability, allowing our engineers to modify simulations with simple language prompts, driving inputs, and scene layouts. Notably, the Waymo World Model generates high-fidelity, multi-sensor outputs that include both camera and lidar data.
How do you know the generated outputs are correct? Especially for unusual circumstances?
Say the scenario is a patch of road is densely covered with 5 mm ball bearings. I'm sure the model will happily spit out numbers, but are they reasonable? How do we know they are reasonable? Even if the prediction is ok, how do we fundamentally know that the prediction for 4 mm ball bearings won't be completely wrong?
There seems to be a lot of critical information missing.
- xnx - 22794 sekunder sedan> The Waymo World Model can convert those kinds of videos, or any taken with a regular camera, into a multimodal simulation—showing how the Waymo Driver would see that exact scene.
Subtle brag that Waymo could drive in camera-only mode if they chose to. They've stated as much previously, but that doesn't seem widely known.
- caycep - 6851 sekunder sedanAll this work is impressive, but I'd rather have better trains
- ra7 - 20729 sekunder sedanThe novel aspect here seems to be 3D LiDAR output from 2D video using post-training. As far as I'm aware, no other video world models can do this.
IMO, access to DeepMind and Google infra is a hugely understated advantage Waymo has that no other competitor can replicate.
- joshuamerrill - 11267 sekunder sedanIt’s impressive to see simulation training for floods, tornadoes, and wildfires. But it’s also kind of baffling that a city full of Waymos all seemed to fail simultaneously in San Francisco when the power went out on Dec 22.
A power outage feels like a baseline scenario—orders of magnitude more common than the disasters in this demo. If the system can’t degrade gracefully when traffic lights go dark, what exactly is all that simulation buying us?
- nightpool - 10212 sekunder sedanInteresting, but it feels like it's going to cope very poorly with actually safety-critical situations. Having a world model that's trained on successful driving data feels like it's going to "launder" a lot of implicit assumptions that would cause a car to get into a crash in real life (e.g. there's probably no examples in the training data where the car is behind a stopped car, and the driver pulls over to another lane and another car comes from behind and crashes into the driver because it didn't check its blindspot). These types of subtle biases are going to make AI-simulated world models a poor fit for training safety systems where failure cannot be represented in the training data, since they basically give models "free reign" to do anything that couldn't be represented in world model training.
- londons_explore - 476 sekunder sedanDo wayno models really use side cameras at only like 4 FPS?
- hazrmard - 15874 sekunder sedancue the bell curve meme for learning autonomy:
Seems like it, no?____.----.____ ______/ \______ _____/ \_____ ________________________________________ (simulations) (real world data) (simulations)We started with physics-based simulators for training policies. Then put them in the real world using modular perception/prediction/planning systems. Once enough data was collected, we went back to making simulators. This time, they're physics "informed" deep learning models.
- mellosouls - 18353 sekunder sedanDeepmind's Project Genie under the hood (pun intended). Deepmind & Waymo both Alphabet(Google) subsidiaries obv.
https://deepmind.google/blog/genie-3-a-new-frontier-for-worl...
Discussed here,eg.
Genie 3: A new frontier for world models (1510 points, 497 comments)
https://news.ycombinator.com/item?id=44798166
Project Genie: Experimenting with infinite, interactive worlds (673 points, 371 comments)
- ok_dad - 8721 sekunder sedanI'd like to see Waymo have a few of their Drivers do some sim racing training and then compete in some live events. It wouldn't matter much to me if they were fast at all, I'd like to see them go into the rookie classes in various games and see how they avoid crashes from inexperienced players. I believe that it would be the ultimate "shitty drivers vs. AI" test.
- 0xTJ - 1832 sekunder sedanInteresting, but I am very sceptical. I'd be interested in seeing actual verified results of how it handles a road with heavy snow, where the only lane references are the wheel tracks of other vehicles, and you can't tell where the road ends and the snow-filled ditch begins.
- joshfee - 2704 sekunder sedanIt is great being able to generate a much larger universe of possibilities than what they can gather from real world data collection, but I'd be curious to learn how they check that the generated data is a superset of the possibility-space seen in the real world (e.g. confirm that their models closely match what is seen in the real world too)
- AceJohnny2 - 11400 sekunder sedanIIUC, there's a confusion of meaning for "World Model", between Waymo/Deepmind's which is something that can create a consistent world (for use to train Waymo's Driver), vs Yann LeCun/Advanced Machine Intelligence (AMI) which is something that can understand a world.
- jrm4 - 13211 sekunder sedan1. Still hard not to think that this is a huge waste of time as opposed to something that's a little more like a public transport train-ish thing, i.e. integrate with established infrastructure.
2. No seriously, is the filipino driver thing confirmed? It really feels like they're trying to bury that.
- phailhaus - 12010 sekunder sedanFinally I understand the use case for Genie 3. All the talk about "you can make any videogame or movie" seems to have been pure distraction from real uses like this: limited, time-boxed simulated footage.
- ActorNightly - 13209 sekunder sedanThis is cool, but they are still not going about it the right way.
Its much easier to build everything into the compressed latent space of physical objects and how they move, and operate from there.
Everyone jumped on the end-2-end bandwagon, which then locks you into the input to your driving model being vision, which means that you have to have things like genie to generate vision data, which is wasteful.
- NullHypothesist - 20925 sekunder sedanI wonder if they can simulate the Beatles crossing the street at Abbey Road in the late '60s
- 999900000999 - 17156 sekunder sedanIt doesn't look like they're going to open sources or anything, but I could imagine this would be great for city planning.
Or the most realistic game of SimCity you could imagine.
- fabmilo - 11817 sekunder sedanVery impressive work from Waymo. The driving with a tornado in the horizon example kind of struck my imagination, many people actually panic in such scenarios. I wonder though the compute requirements to run these simulations and producing so many data points.
- Kapura - 18348 sekunder sedanInteresting that this should come out right as lawmakers are beginning to understand that Waymos have overseas operators making major decisions.
[*] https://futurism.com/advanced-transport/waymos-controlled-wo...
- pcurve - 7462 sekunder sedanDumb question - Why would Waymo disclose this much information to public and competitors?
- anigbrowl - 5958 sekunder sedanSeems relevant: Waymo exec admits remote operators in Philippines help guide US Robotaxis
- KeyBoardG - 5381 sekunder sedanand I literally just saw the other headline "Waymo says its robotaxis get help from remote workers in the Philippines"
- LightBug1 - 2032 sekunder sedanHave been seeing Waymo test vehicles regularly around central London recently, operating at speed.
For shits and giggles, I did stop randomly while crossing the road and acted like a jerk.
The Waymo did, in fact, stop.
Kudos, Waymo
- t1234s - 10291 sekunder sedanCould these world models be used to build some sort of endless GranTurismo type street racing game?
- mgaunard - 21069 sekunder sedanStill needs to be trained on the final boss: dense cities with narrow streets.
- threethirtytwo - 6609 sekunder sedanWhat if we put this mechanism of recording the world on people. We have mics listening to people talking to us and noises we hear.
Also we record body position actuation and self speech. As output then we put this on thousands of people to get as much data as Waymo gets.
I mean that’s what we need to imitate agi right? I guess the only thing missing is the memory mechanism. We train everything as if it’s an input and output function without accounting for memory.
- cratermoon - 4067 sekunder sedan
- LowLevelKernel - 13432 sekunder sedanInstructions to load it on WAYMAX simulator?
- PeterStuer - 20198 sekunder sedanImagine driving in a Waymo 'out of a raging fire'.
Talk about edge cases.
But, what would you do? Trust the Waymo, or get out (or never get in) at the first sign of trouble?
- ge96 - 13896 sekunder sedanWhat is the 5/3 tiles? Cameras?
- AndrewKemendo - 14913 sekunder sedanFor whatever it’s worth World models is going to be the dominant computing structure of the future
I started working heavily on realizing them in 2016 and it is unquestionably (finally) the future of AI
- 01100011 - 13883 sekunder sedanNvidia has had this for years. What am I missing?
- tgrowazay - 15591 sekunder sedanThis page crashes my browser.
Vivaldi 7.8.3931.63 on iOS 26.2.1 iPhone 16 pro
- mempko - 18069 sekunder sedanOne interesting thing from this paper is how big of a LiDaR shadow there is around the waymo car which suggests they rely on cameras for anything close (maybe they have radar too?). Seems LiDaR is only useful for distant objects.
- jimt1234 - 20047 sekunder sedanThis might be relevant to the timing here: https://eletric-vehicles.com/waymo/waymo-exec-admits-remote-...
- Vosporos - 16596 sekunder sedanThe new frontier is manifestly the Phillipines.
- m0llusk - 19785 sekunder sedanSeems interesting, but why is it broken. Waymo repeatedly directed multiple automated vehicles into the private alley off of 5th near Brannan in SF even after being told none of them have any business there ever, period. If they can sense the weather and stuff then maybe they could put out a virtual sign or fence that notes what appears to be a road is neither a through way nor open to the public? I'm really bullish on automated driving long term, but now that vehicles are present for real we need to start to think about potentially getting serious about finding some way to get them to comply with the same laws that limit what people can do.
- selenajennifer - 3821 sekunder sedan[dead]
- andrewmcwatters - 18564 sekunder sedan[dead]
- xvxvx - 11795 sekunder sedan[flagged]
- devmor - 20758 sekunder sedanWow, interesting timing for this PR blast considering the admission in the Senate Commerce Committee hearing. Not transparent at all!
- turtlesdown11 - 18874 sekunder sedanHow many Filipinos, who do not have US drivers licenses, does it take to drive this new model?
- add-sub-mul-div - 21546 sekunder sedan
- smcl - 13926 sekunder sedanThe Waymo driving model: hire some guys in Philippines: https://futurism.com/advanced-transport/waymos-controlled-wo...
- OGEnthusiast - 21661 sekunder sedanWhat's going to happen to all the millions of drivers who will lose their job overnight? In a country with 100 million guns, are we really sure we've thought this through?