RuneScape Bench: MMOs for Agents [video]

Thanks HN Author here to answer questions if helpful!

If this inspired you, would encourage people to try it out by pointing their agents to this repo and giving a goal:

https://github.com/MaxBittker/rs-sdk

You can also view the realtime activity of the server here: https://maxbittker.github.io/rs-world/

What a great talk. My introduction to programming as a kid was also writing Runescape bots, using some Delphi tool called SCAR [1] (together with writing plugins for my Counter Strike 1.6 server using AMX Mod X). Start at Varrock bank, wizard hut, dialogue picker, teleport, find out where you are, mine, mine, mine, teleport back, back to bank, deposit, and again.

My great essence mining bot contraption horribly broke when Runescape started randomizing all colors slightly, after making a paltry amount in dollars selling the essences.

[1] https://scar-divi.com/

This is a great talk. I've been working on a very similar concept for World of Warcraft at https://github.com/agent-wow/agent-wow.

A lot of the techniques and pitfalls he talks about also aligns with my experience on WoW agents too. It's also interesting to see how agents coordinate in these kinds of game environments.

Was fun exploring this months ago. Back then even the smallest amount of game knowledge you could provide a model would drastically improve their performance. Most of it is in their training data but they hallucinate a ton of incorrect information.

This is a good talk. There is some similar things in the private server / self hosted server space for World of Warcraft. There are bots that will play the game, talk with you, etc. using classical game aI, but people are also hooking them up with LLMS for more planning, strategy, and conversation.

I've wondered if these types of casual open world, resource collection games-- which are basically a series of mini-games-- would make good model training envs. Each mini-game has a pretty simple mechanic that needs to be learned, as well as the navigation and interactions to begin it. Those could be pretty useful generic skills I think.

One of my side projects is a game, and I've made it so agents can spawn off their own world shard to play in. There's a little Chrome-driven agent harness that can grab low res screenshots while it speaks directly to the game over the wire to control things.

I've had Luna play it and it can drive around (you play as a bike) but it did struggle to play one of the mini-games (Disc Golf). My idea was that model trainers could run it an RL loop using the harness to get feedback data and screenshots to use, then reset the game world as needed during the process.

A smart model capable of multi-modal inference on screenshots/game context at 30-60fps could be pretty interesting, and I'm sure it's coming. No idea how much it would generalize but would be fun to see done.

Saw it this morning, this video is a goldmine of ideas on how to build agent harnesses and benchmark them.

The author did a great job capturing the essence of what made RS such a great game, and the soul that Jagex has totally lost over the years with updates to make the game a softer place.

This formative moment could never happen today, because the wilderness is behind 3 confirmation screens instead of just being something you get to discover in the game:

> You have to know that [items] are worth something in order to not get taken advantage of. And it goes to kind of extreme levels too because not only will people, try to give you a bad price, but the game is just really well known for the scamming and the lying and stealing your password. And it's really the wild west out there.

> And so, and really this is true. My very first memory of the very first time that I entered the world of RuneScape, I was asking somebody for help. I said like I was trying to make bread and I was like, "Where can I find a mill so I can make some bread?" And a guy was like, "Follow me. I'm going to show you." And I followed him like into these woods. And you know, you can imagine he turned around. He started to attack me. And I realized what was happening. I was probably 11 years old. I realized what was happening. And I made a break for it. I ran back south for cover and I got away. I didn't die. And I remember like, you know, closing the game.

> I think that my parents hadn't been home, so I had time to think about this. And I was like, that was the coolest thing that's ever happened to me. And I think that this is kind of the beauty and the magic of this game is that it's a little bit of like a rough and tumble world where you have to look out for yourself. And as a kid, it's like that's actually really great practice to be out in the world with real people and have to understand what's going on and learn things like, oh yeah, if somebody offers you a really great deal, maybe be careful.

Cheers to this kind of game design. Wish we'd see it again some day.

Games are a huge untapped training potential. Especially sandbox games that have distant goals with a complex dependency tree and structured/semi-open solution space like Factorio. There are so many dimensions that factor in to make a solution good.

Somebody is working on this for Factorio at: https://www.youtube.com/@FactoryArenaAI

RuneScape was what led me to become a software engineer (through the private server communities) and it's refreshing to see someone describe this game in layman terms, all the way up through to having AI agents drive the player interactions. I worry, though, about Jagex's stance on all of this. Perhaps this is what pushed them towards the recent RuneScape 4 announcement?

If this was for UO, you'd have a bloodbath. RuneScape's systems were pretty tame compared to UO:SA where anywhere outside of towns was PvP allowed and when you died you lost all your gear.

full circle. i remember downloading an autominer back in like 2004 to auto click through mining coal in falador.

It's really shocking that there aren't more games like (old-school) RuneScape. Like where is OSRS but you're a mafia boss in NYC, or OSRS but you're on a space station? Obviously games with these themes exist, but not games in this genre, which I'd say is defined by fairly shallow but very broad content building on a cartoonishly simple engine. Modern games seem to be going relatively long on engine capabilities but short on content instead. You'd think AI would have no problem designing 50 good quests for an NYC mob boss in a few days or less.

Maybe RS's origins from the text MUD era led it to a weird path-dependent place. The other day I watched a whole video essay about how weird it is that everything has an "examine" option when no other graphical game has this, but every MUD has it.

[dead]

[flagged]