What Is It in Minecraft?

Numen is a server-side player that AI can control, and the AI controls it through sets of "workflows".

Acknowledgements

The projects in this chapter all helped a great deal along the way, so first, a deep bow of thanks to each of them!

• Voyager: Numen started as the idea of "a setup-free Voyager". Voyager was the first thing that showed me a large language model really can explore and act on its own in Minecraft.
• Mindcraft: An excellent project, almost exactly what I wanted. Too bad its body is still borrowed from mineflayer.
• mineflayer: Numen didn't end up taking the network-packet route, but studying mineflayer is what made me figure out what Numen actually wants to be.
• Carpet: The way its fake players get work done inspired me too, and Numen's body is now growing flesh and bone at a frantic pace.
• Mekanism: Its machines stick to vanilla Minecraft's own container logic instead of reinventing it, so Numen, without a single line of compatibility code, can still walk up to its furnace, open it and shove a couple of ores in. That "it kind of works even without support" experience is one of the reasons I was confident going with a server-side player.

The Original Idea

Numen began as an idea for a setup-free Voyager: install a mod on your client and you get your own AI. That AI calls the API provided by mineflayer to control a character, and just like that, you have your own agent.

How does mineflayer "pretend" to be a player?

Minecraft is split into a client and a server. Every action you take on the client is, underneath, a matter of sending and receiving packets with the server to get data back and forth, and vanilla Minecraft normally handles all of that. mineflayer takes over this sending and receiving, so as far as the server can tell, it is a real player.

I suspect most online games can be handled this way: crack a game's network packets and you can "play" it without ever opening the client. Programs like this also go by a name everyone knows: cheats. Of course, that's just one kind of cheat.

Why We Dropped mineflayer

Being called a cheat certainly wasn't going to make me give up on this route: Minecraft already has plenty of client cheats, so one more called Numen hardly matters. The real problems were elsewhere, and there were three of them.

1. All of mineflayer would have to be translated into Java. mineflayer is a reverse-engineered network-packet implementation written in Node. My original goal was to give players a setup-free experience, which meant porting the whole Node stack to Java by following mineflayer. That sounds like a lot of work, but it isn't actually that hard. The real difficulty lies in the other two.

2. Servers won't tolerate it. I wanted players to use Numen freely on servers and enjoy AI together with their friends. But a purely client-side "cheat" mod would need all kinds of anti-ban workarounds. From what I found at the time, servers generally ban automated bots like this, and such cheat clients can do real damage to a server too: the server has no way to manage these bots, which may bring a huge performance cost and all sorts of conflicts between players.

3. Supporting other mods is hard. mineflayer works by reverse-engineering network packets, so supporting another mod would, as far as I could tell, mean reverse-engineering that mod's packets as well. For open-source mods that isn't hard, but it still drives up development cost. Worse, a mod without explicit support simply doesn't work at all. What I wanted was this: even without support, it should at least be able to do a little bit, and broaden its horizons.

A Server-Side Player

That led to the second architecture of Numen: a server-side player, the same kind as Carpet's. From this point on, my thinking parted ways with mineflayer completely.

A server-side player gives Numen a nearly native way of interacting with the game. Even without any compatibility work, Numen can try opening containers from other mods, such as Mekanism's furnace. That's the benefit of being "native". (Admittedly, all it can do is put things in, but at least it isn't completely useless. That's a bit better, right?)

There were two reasons for letting AI control a server-side player:

• The server can manage it: in the future, server owners can allocate resources and set permissions sensibly;
• Nearly native interaction: compatibility with vanilla is much higher.

Conclusion

So the first question has its answer: Numen is a server-side player that AI can control.