Alt-F4 #33 - Vanilla: Shaken, not Stirred 30-04-2021
In this delightful issue #33 of Alt-F4, Villfuk02 presents their most recent mod creation: The Recipe Randomizer! A spotlight is put on all the design problems that needed to be overcome to make a recipe randomizer that is actually good. After that, redlabel announces the most recent COMFY event, which will likely be pretty explodey.
Vanilla: Shaken, not Stirred Villfuk02
Vill’s Recipe Randomizer does exactly what the name suggests - it randomises recipes. On first sight, this might seem like a very stupid, but very simple idea. However, once you try to make it less stupid, it gets very complicated pretty quickly.
How it started
I first started working on this project in September 2020. At that time I was trying to make a mod that’s both simple and unique. It wasn’t supposed to be playable, or even good, just something fun to make. After running into some issues, knowing full well that I’m making a mod nobody would play anyway, I gave up and never touched programming again.
After two other failed projects I came back to Factorio, this time, as a player. I played vanilla, then Industrial Revolution 2, and then I tried Space Exploration. I love playing overhaul mods because it lets me learn new recipes, discover new factory layouts and solve new logistic problems. But most overhaul mods are very grindy and I get bored quickly. That’s when I realised that a randomiser mod has in fact a purpose: It’s like an unlimited supply of overhaul mods while keeping the vanilla feel! The biggest bonus for me was that I don’t have to make any new machines or graphics as it’s just messing with numbers and letters.
Not as simple as it seems
The mod could simply take each recipe and make it take random items as ingredients. This would be pretty useless because, for example, red science might take nuclear fuel as an ingredient, but you won’t have it researched yet. So I decided to make it crawl through the tech tree, and for each recipe that’s being unlocked, take only the items unlocked before it as potential ingredients. There are of course exceptions in some mods, where you can unlock a recipe before you can even craft it, so I have to be a bit more careful.
You probably wouldn’t want to make an underground belt from steam engines and labs. That would be way too expensive. I needed a way to make the recipes more balanced. So I decided to implement an algorithm that calculates the value of each item based on the resources you need to make it. We can easily calculate that one electronic circuit takes 1 iron ore and 1.5 copper ore:
Let’s say that both copper ore and iron ore have a value of 1, this means electronic circuits would have a value of 2.5. Now the randomiser knows it can use for example one transport belt (value 1.5) and one copper ore (value of 1) as a new recipe for electronic circuits. There is some variance in the final value. It makes the recipes a bit more unique, but most importantly, it’s easier to find a valid combination, speeding up the randomisation process.
At this point, I published the mod and from feedback and some testing, it became very apparent there were many bugs (of course) and two “big” issues. The word big is in quotes because the mod was still more playable than you’d expect even from a randomiser. I however wanted the players to enjoy it, not just experience it.
The first few issues
The first issue was wood. The default settings made sure wood was not randomised, but many recipes used wooden power poles as an ingredient! As you can imagine, the fix should have been was pretty simple and boring. I added a flag for non-randomizable resources, like wood, and any item that strictly needs an item with this flag as it’s ingredient inherits it.
The second one was that even though recipes needed the right amount of resources, they often were too complex and felt unfair. The mod did not account for the number of crafting steps or crafting time. A great example is the automation (red) science pack. It is very cheap, having a value of only 3, but it takes 5 seconds to craft. It was used in many recipes, like a basic component, similarly to iron gears. Its long crafting time means you could need a lot of assemblers making red science just for something simple like underground belts. Another problematic example is that simple items like iron gears could require many steps like making them from stone bricks, which could be made from iron sticks, which could be made from pipes, etc. So I made the mod calculate a second value, called complexity, for each item. Complexity takes into account the amount of crafting steps required, the amount of various ingredients (and results) and also the crafting times.
Balancing raw resources
It seemed that I’ve arrived at the perfect randomiser, but again, some issues (and bugs) revealed themselves. Often, many recipes used primarily one resource (often iron or stone) which was kind of bland and you had to scout for far away ore patches because you needed just so much of the chosen resource. Also, oil was often not needed for launching the rocket. For a given recipe, all valid items were equally likely to be picked, so the items without oil massively outnumbered the few oil products. I tried some strategies to push the algorithm in the right direction but none of them really worked. I could just tell it to use more plastic or solid fuel and such, but what if you want to play with mods and there are even more items to be prioritised? I wanted a universal solution.
I already knew one approach that could solve it, but I was afraid to implement it because it would make everything way more complicated. This means more development time, more bugs and potentially way slower loading times. After unsuccessfully trying to come up with some simpler way to achieve a similar result, I finally decided to go with this complicated solution.
The main idea was to use the amounts of raw resources instead of just the total value. This means the mod can make sure that each randomised recipe needs the right amount of raw resources, ensuring that all raw resources are used. Similarly to the total value, the raw resource amounts for each recipe may vary a bit, or they can be completely random based on the mod settings. Pretty simple, right? Well, now instead of doing maths with one number I have to do maths with several and the logic is a lot more complicated. Fortunately, with some optimisations and rigorous testing, the mod works fine and is still pretty fast.
And that’s the whole history of Vill’s Recipe Randomizer!
We need to go deeper
Except I did not talk about some BIG issues (notice the missing quotes and the capital letters) which were present from the start. One of them is recipe loops. The algorithm could get stuck when trying to evaluate the value of one of the recipes or it could calculate it improperly. Loops contained within one recipe are simple to deal with. For evaluating the recipe, the algorithm cuts out the items that appear both as an ingredient and as a result, removing the loop. It can then add back the cut items because their values are now known, making the recipe ready for randomising. In vanilla, this applies to Kovarex enrichment process and coal liquefaction:
Loops with more than one recipe are somewhat problematic. There are no loops in vanilla with more than one recipe (except for barreling and unbarrelling, which is not randomised by default anyway), so they’re pretty easy to miss, but in mods, they are everywhere! The randomiser might also accidentally create a loop in certain conditions. To illustrate the problem we’ll need an example. Let’s take synthetic sapphires in Industrial Revolution 2. You need pure nickel mineral, silica and sapphire dust to make a sapphire, however, sapphire dust is made by crushing sapphires, forming a loop:
The way I resolved this issue is that when a loop is detected, it gets cut open in one point and then the recipes are merged into one. The new recipe is then used for the calculation instead of the one the straightened loop ended with. Here, the two recipes are merged by removing the three sapphire dust in the middle, then we can remove the two sapphires that are both an ingredient and a result:
We end up with a recipe to make sapphires while leaving out the problematic details! But what happens when there are branching loops or loops within loops?
Let’s just say that things get loopy. I think I somewhat solved it in the mod, but I won’t go into details here, this is not a graph theory course, and I’m not even sure the solution works reliably.
My nemesis
And now, time for the biggest issue of them all: recipes with multiple outputs. Let’s take a look at advanced oil processing as an example.
By default, crude oil has a value of 0.4 (we’ll ignore water for now). That means the recipe has a total value of 40. So we know that 25 heavy oil + 45 light oil + 55 petroleum gas should have a value of 40 in total, but how much of that value is contained within each product? There are many ways to approach this and tackling this issue took most of my time and caused many headaches. Let’s go through the two most different options.
The simplest: Just make each of the results worth an equal part of the total value. This is mostly reasonable, but it breaks when you introduce waste products. For example, the Kovarex enrichment process makes 41 U-235 and 2 U-238, the algorithm would assume they both have the same value, but you probably know that U-235 is much more valuable. So this approach is not really usable.
The “best”: Just make each part worth the full value, but keep track of the byproducts which you can subtract later. For example, heavy oil has light oil and petroleum gas as its byproducts in advanced oil processing. That means lubricant has them as byproducts too because it’s made from heavy oil only. If we want to make an express splitter, we need, among some other ingredients, lubricant and advanced circuits. Advanced circuits need petroleum, but we have some as a byproduct of the lubricant so we can subtract it. The only byproduct that’s left is light oil. What do we do with the light oil? We can of course crack it to save even more petroleum. But that takes some water. Is it cheaper to use more water or more petroleum? Well, let’s calculate both options just in case.
As you can tell, this approach is insanely complicated. There are many edge cases and questions to be answered. Like what do I do with excess byproducts when other recipe’s ingredients don’t want them? I tried to implement a version of this approach and I must say, I was pretty successful. It took me only about 100 hours and I had to fix about a million bugs. And what was the reward? Well, this solution is still not perfect, although it comes very close. I also could no longer make sense of my own code. Everything broke every time I tried to adjust something. And most importantly, loading the mod with vanilla Factorio took anywhere from 10 seconds to 10 minutes. Since I want the mod to be compatible with as many other mods (and it working with vanilla Factorio would be nice too), I decided this is not the way to go.
Finally, after much more thinking, I arrived at the algorithm I use now. It is basically a compromise between the two approaches described above. It works well enough and is pretty simple. Though not simple enough for me to explain because I’d need to explain the many other quirks of the evaluation algorithm. It does not evaluate all multiple-output recipes incredibly well in all cases, but it also does not break in edge cases (I hope).
Final thoughts
That concludes our peek into the inner workings of the Recipe Randomizer. If you have any questions or suggestions, you can message me on my Discord server. Speaking of which, I’d like to thank all the people who joined it and posted bugs, ideas, tested things or otherwise supported me during the development. The mod wouldn’t have come this far without you!
COMFY Minesweeper! redlabel
To celebrate a new map design by legendary map maker mewmew and a shiny new Ryzen 9 5950X powered by Gerkiz, COMFY Factorio is organizing a special event on May 1st @ 5PM UTC. Everyone and their families are invited, no need to RSVP.
If anyone is wondering what Minesweeper is about, well, it is a new map where base expansion happens by solving a minesweeper puzzle. To clear a mine, a furnace must be placed over it. Placing it over a clear spot will trigger an explosion, and stepping on it without clearing first will yield the same result. Mistakes cost dearly, as they release evil biters and cause nuclear explosions. Players can also earn (or lose) points depending on how good they are, and can compare themselves to others through a scoreboard that keeps a track of valuable internet points.
COMFY is one of the leading multiplayer Factorio communities and some of our creations are Mountain Fortress, Biter Battles, Fish Defense, Chronotrain and Cave Miner. And very, very soon it will include a map about pirates and sailing ships! Stay tuned for that one.
You can join the COMFY community with nearly 3500 members on our Discord.
Het aftellen is voorbij
Contributing
As always, we’re looking for people that want to contribute to Alt-F4, be it by submitting an article or by helping with translation. If you have something interesting in mind that you want to share with the community in a polished way, this is the place to do it. If you’re not too sure about it we’ll gladly help by discussing content ideas and structure questions. If that sounds like something that’s up your alley, join the Discord to get started!