Nördnytt! 🤓Uber’s COO says it’s getting harder to justify money spent on tokenmaxxing
- izanton - 4167 sekunder sedanWhat if... we stop for a moment, and then, after thinking for a moment, we stop hammering nails with a microscope, and stop using token usage as a metric of productivity?
I know it's sounds stupid, but what if
- mrkeen - 3480 sekunder sedanI always used to wonder this about software stacks even prior to LLMs, but it seems more relevant now somehow:
When will Uber (or your favourite company) be 'done'? They've been writing software for 16 years.
They match drivers to passengers. More software isn't going to increase the chance that I seek them out instead of taking a bus or train.
Will their software be finished in 20 years? 80?
- bilater - 3166 sekunder sedanThe black bill that is coming that nobody is prepared for is that the value of a token varies greatly depending on the human. Companies will quickly find out its much better to give your top 10% engineers a lot more tokens and lay off your average engineers. The 10x engineer will become the 1000x engineer.
Wrote about this and the impact of to jobs here: https://x.com/deepwhitman/status/2058324179506831372
- FartyMcFarter - 4071 sekunder sedanIf any company announces that they use token consumption as an employee performance signal, for me that's close to a red flag to stay away from that company.
No company with good engineering leadership should act like this is remotely a good idea.
- crorella - 3601 sekunder sedanTokenmaxxing makes no sense, it is akin to write extremely inefficient SQL / Spark Jobs, full of cartesian joins, ultra skewed datasets, etc, just for the sake of using as much compute / memory / IO as possible.
This always happens when the metric becomes the goal, companies should nurture and foster an environment where AI is used in the most efficient way possible, first asking "do we really need an agent for this" and if so, what kind of agent is needed, what model, reasoning level, etc.
They should also promote projects that aim at saving tokens, increasing cache hits, codifying the information in ways such they use as less context as possible (graphs of knowledge are pretty good for this!)
- simonw - 3591 sekunder sedanI'd be interested to know if this is about individual employee AI usage, or use of AI tokens in production features, or both - and assuming both, what the split is.
I can see how Uber could burn unbelievable amounts of tokens if they start running internal features that run a bunch of prompts against every completed ride, or every customer profile, for example.
Or maybe this is about employee usage, but they introduced some stupid "you get evaluated on how many tokens you used" thing a couple of months ago when that was trendy and are just beginning to notice how much that cost?
- jhack - 3960 sekunder sedanMaybe don't use the most expensive models on the planet? Maybe use AI like a tool and not this black box that grants wishes?
- InsideOutSanta - 3137 sekunder sedan"He said that, based on talks with Uber's senior engineering leaders, he realized higher token usage did not translate into a proportional increase in useful consumer features."
He's saying that like it's some grand epiphany and not the most self-evident, obvious thing I've heard this month. Some of the literal dumbest people on earth are in charge of these major companies.
- JackDanMeier - 3511 sekunder sedanAt what point is there a difference between a burn rate and tokenmaxxing? Isn't it the same as during the dotcom bubble?
- cryo32 - 4581 sekunder sedanWaiting for tokenedging next.
- rcvassallo83 - 3501 sekunder sedanOof leader of bubble are starting to take a step back?
- - 3790 sekunder sedan
- chihuahua - 4705 sekunder sedanIt's amazing that it took months to figure this out. "Well we thought that if engineers are told to maximize costs through AI use, to consume as much as possible of a resource that costs us money, then obviously good things will happen. Imagine my surprise when it didn't turn out that way."
Imagine if engineers were ranked based on their AWS spend. People allocate VMs and fill databases with terabytes of random bits, to get to the top of the AWS leaderboard. If you don't do this, you're ranked at the bottom, and good luck at the next review cycle. Who could have expected that this is not the road to success?
- yapyap - 3225 sekunder sedanwtv
- lorecore - 3542 sekunder sedanNot all tokens are created equal. It's easy to use a ton of tokens by having agents work together in parallel. That's basically the equivalent as people spending time in meetings, hardly a productivity win. As with everything in development, results matter, how you get there doesn't (unless you're a bad manager).
- irishcoffee - 3928 sekunder sedanI just realized my company is months behind this curve. About to blow my token allocation. Before I do, anyone have requests? Sincerely.
- paulpauper - 3506 sekunder sedanmany of these leading AI companies are operating at large losses and subsidizing users with VC money. Profitability will entail having to impose greater limits and raising prices, so this will reduce to some degree the value proposition of AI compared to humans.
- egypturnash - 4804 sekunder sedanUber COO says he just decided to short a bunch of AI company stock.
- pocksuppet - 3726 sekunder sedanwhat the fuck is this timeline I am stuck living in
- 7777777phil - 4824 sekunder sedanAs soon as tokens stop stop being subsidized, heavy agentic use will become as least as expensive than paying an (entry level) employee. When this happens many companies will trade off havy tolen usage for (maybe a bit slower, bit less accurate) employees again.
- illithid0 - 5046 sekunder sedan>"He said that, based on talks with Uber's senior engineering leaders, he realized higher token usage did not translate into a proportional increase in useful consumer features."
Goodhart's law strikes again at someone with enough power to be both ignorant of it and make others suffer their ignorance. You cannot simply measure productivity by tokens spent just like you can't measure it by hours spent in a chair at a desk.
- Rohunyyy - 4177 sekunder sedanNow we are going to get a new profession. Token Engineer! They will be experts on tokenmaxxing! The job growth that the billionaire CEOs promised us from AI is finally here!
- nekzn - 4845 sekunder sedanIt’s funny that “maxxing” entered the common vocabulary.
- gigatexal - 5536 sekunder sedanI find it useful that if they cut the use altogether I will pay for it out of pocket.