On Making A Useful Minecraft AI

Minecraft is, without a doubt, my favorite game I have ever played. Granted, I don't play a lot of video games, because I don't really find them to have much of a point. Minecraft is so pointless (there being no real "goal" of the game) that it ends up being a wonderful sandbox for all sorts of creative projects. It's like legos for your computer, except it's way cooler!

One of the things that I don't like about Minecraft, though, is mining for materials and gathering said materials up in an organized fashion. Strip mining for enough redstone to make a finite state machine, or enough stone to make a train station, really isn't my idea of fun. I'd rather just get straight to building whatever it is that I want to make! Some would say that "Creative" mode offers players the opportunity to do just that: with an unlimited amount of resources, no constraints on how long it takes to break blocks, and the ability to fly, people can make impressive structures far quicker than the average "Survival" mode player. Still, it feels like cheating. I want to find a way to keep the pleasure of creating from the materials offered by the environment, and not just materializing whatever I need whenever I need it.

Enter a more ridiculous way of cheating: the Minecraft bot. When I was a Freshman at CMU, I made such a bot for my term project. It couldn't do much beyond basic pathfinding, parsing chunk data, and saying pre-programmed things in the in-game chat, but it was definitely a start. I wanted something that I could leave on overnight, so that when I woke up I would have chests full of materials that were strip-mined by my hardy, diligent worker bot. Unfortunately, this is obviously way easier to say than to do. Strip mining requires that the bot know which materials are valuable, that it knows to make a new pickaxe when its old one breaks, when to stop mining due to caves, lava flows, water, gravel, etc. And this is just one aspect of the game! (Just to be clear, I'm talking about a bot that pretends it's a client on a Minecraft server, over an internet connection. Recognizing patterns in pixels on the screen in a programmatical way would be absolutely crazy.)

Lately I've been thinking of going even further. Don't just teach the bot how to strip mine, teach the bot how to avoid danger, to fight creepers, to farm wheat, craft tools, build structures, and make me coffee in the morning. The best way to go about this would be to create an AI that would plan out the best possible sequence of separate things to do (which is, admittedly, totally arbitrary), all the while tracking the resources it would gain, the probability that certain events might happen (such as not finding the gold it was looking for, or that a mob in the area might end up overpowering it), its current status (like health and hunger), and, most importantly, the likelihood of it dying at any given moment (which, I assume, it would try to avoid at all costs.)

Not being terribly versed in game theory, the only thing that I can remember doing that resembled this was an AI that I wrote in a functional programming class that played a deterministic variant of Risk. Using alpha-beta pruning over a minimax tree, in addition to a simple scoring algorithm, the computer generally beat me any time that I tried to play. Admittedly, however, the "rules" of Minecraft are much more complex.

Using some variant of a minimax tree, I imagine that a Minecraft AI would actually perform reasonably well. If the bot was written using a procedural programming style, then a "score" value could be assigned to obtaining a certain item, destroying a certain mob, or eating a certain food. Some sort of scoring algorithm based on the "Four F's" (fighting, fleeing, feeding, and fucking, though maybe not really that last one) could associate some sort of value with each procedure, and a node's children in the decision tree would only constitute states that could evolve from the node itself. (i.e., you can't "make bread" if you don't "have 3 wheat".) Some amount of randomness would also have to be introduced, since it's not always certain if diamond ore will drop 1, 2, or 3 diamonds upon being mined, and you can't always be sure if the creeper you're fighting is going to succeed in blowing you up. The real work that's going to have to go into this, though, is scoring every possible procedure and action that the bot can take. Which is a lot of stuff.

Enter machine learning, which would likely make this scoring process easier (at the cost of taking forever and a day). Some procedures could be pre-scored, and a neural network or decision forest could be introduced into the scoring to make it more of a learning experience for the machine. Essentially, the way that a scoring function works in a minimax tree is it takes in the player's state (surroundings, possessions, hunger, health, etc) and "scores" the situation, turning it into a single number. The idea here is that we approximate the scoring function with an ML algorithm, assuming the machine has the necessary resources to carry this out. Granted I know little about which ML model to choose, but this would be sweet with something like a neural net. It might even be possible to train the machine learning component by capturing the packets between a Minecraft client and a server, and interpreting them as sequences of events: walk to (x, y, z), switch to pickaxe, break block at (x, y, z), creeper appeared, switch to sword, et cetera. Paths down the game tree that are chosen get positive feedback, paths down the game tree that are not chosen get negative feedback. This way the ML algorithm can approximate how to score any given situation in the game, and it'd be possible to capture the packets of multiple Minecraft players, making training the bot take even less time.

I have some starter code written in Python that can send packets to the server to control the bot, as well as receive chunk data from the server to be able to tell its environment, but I think a project like this would be better implemented in a functional language, since this involves far too many computer science concepts to implement imperatively (and comfortably). Between the neural networks, pathfinding, decision trees, and oodles of key-value stores this would involve, I'd probably die of old age if I tried writing this in something like C.

All this said, though, as much as it's a cool project, it would probably take me ages to complete (at least, to a stage where I'd be proud of it). Definitely not something to undertake while I'm busy with schoolwork, but this will definitely be simmering on my backburner for a while.

Comments !