Creating the perfect modding language
grug is the name of my modding language, and its name is based on the legendary article The Grug Brained Developer:
The article Video game modding on Wikipedia describes modding pretty well:
Video game modding (short for “modification”) is the process of alteration by players or fans of one or more aspects of a video game, such as how it looks or behaves, and is a sub-discipline of general modding. Mods may range from small changes and tweaks to complete overhauls, and can extend the replay value and interest of the game.
Mods and plugins are the same thing, though the word “mod” is normally used by games. Keep in mind that when I say “mod” or “game”, my modding language’s goal is to work for any application written in any programming language, so not just games.
complexity very, very bad
grug is a modding framework that makes the integration of mods to an existing project as easy as possible.
grug is also a compiled programming language, making the experience of writing and maintaining mods as pleasant as possible.
It was designed alongside the writing of this article, and is based on two modding observations:
- Most mods just want to add basic content, like more guns and creatures
- Most mods just want to run some basic code whenever a common event happens, like having a human spawn three explosions when they die
Very few data types
These are grug’s data types:
stringbooli32(int32_t)f32(float)id(uint64_t)
There are also resource and entity, which are just strings that grug will check for existence. So if a mod passes "sprites/m60.png" to a function that expects a resource, grug will check that the PNG exists.
The same goes for "ww2:m1_garand" when it is passed to a function that expects an entity, where grug will check that there is a ww2 mod that contains an m1_garand entity.
You might now think “But what if a mod needs a more complex data type, like a pointer, struct, or dynamic array”? The simple answer is that it is the game developer’s responsibility to add functions for this.
So a modder might call vector_string_create(), which returns an i32 ID, and is then used when calling vector_string_push(id, "foo") and vector_string_get(id, index). Note how it is up to the game developer here to decide whether index is 0-based or 1-based.
The game developer could add a vector_string_free(id) function, but this is discouraged, as modders shouldn’t be burdened with and counted on calling this function. grug might smell like C, but its goal is to be friendlier to newcomers.
Instead, the game developer should take the responsibility of freeing the vector, when there are no more references to it.
But since reference counting isn’t always trivial to do, and since most mods don’t actually need more complex data types, game developers are recommended to hold off on exposing memory allocating functions to modders.
grug is stupidly easy to set up
The game developer only needs to drop grug.c into their existing project, which is a roughly 7500 line long file, and grug.h, which is under 100 lines long.
grug.c contains an entire compiler and linker, currently capable of outputting 64-bit ELF shared objects (which only runs on Linux), containing x86-64 instructions (which won’t run on ARM CPUs).
Its GitHub repository is found here.
I am currently in the process of writing games and non-games that show off grug.
Since most languages can either call functions from grug.c directly, or are able to load it as a library, grug can be used by almost every programming language under the sun.
In a nutshell, the game developer:
- Periodically calls a function from
grug.c, which will recompile any modified mods, and will store the modified mods in an array. - Loops over this array, and copies the data and functions from these modified mods into their own game.
So the game might have a Gun class, and the modified mod might up the firerate of the gun, and have a different on_fire function.
The “How a game developer might use grug” section of this blog post shows an example of how grug can be used by a game written in C.
Runtime error handling
Every possible runtime crash in a grug file is caught.
In this video, look at the console at the bottom of the game for the only possible grug runtime errors:
- Division by 0
- Functions taking too long, often caused by an accidental infinite loop (with Lua the game would hang!)
- A stack overflow, often caused by recursing too deep
If you’re curious how grug catches runtime errors, I wrote a post about the implementation.
It’s important to note that the game developer is expected to give the player a setting, for whether they want their on_ functions to be in “safe” or “fast” mode. The mode can be changed on the fly by calling grug_switch_on_fns_to_safe_mode() and grug_switch_on_fns_to_fast_mode() respectively.
The “fast” mode has zero overhead, sometimes making it up to 1000x faster than “safe” mode. It does not protect against mod runtime errors, however. The default mode is “safe”. See my grug benchmark repository for more details and nice pictures.
grug example
Here’s a zombie.grug file that a mod might have:
define() human {
return {
.name = "Zombie",
.price = 49.95,
.sprite_path = "sprites/zombie.png",
}
}
on_kill(killed: id) {
print_string(get_human_name(me))
print_string(" killed ")
print_string(get_human_name(killed))
print_newline()
}
The define function adds a new human variant to the game.
The on_kill function is called by the game whenever the zombie kills someone.
That same mod can then add a marine.grug file, having its own on_kill function:
define() human {
return {
.name = "Marine",
.price = 420.0,
.sprite_path = "sprites/marine.png",
}
}
kills: i32 = 0
on_kill() {
kills = kills + 1
if kills == 3 {
helper_spawn_sparkles()
kills = 0
}
}
helper_spawn_sparkles() {
i: i32 = 0
while i < 10 {
x: i32 = get_human_x(me) + random(-30, 30)
y: i32 = get_human_y(me) + random(-30, 30)
spawn_particle("sprites/sparkle.png", x, y)
i = i + 1
}
}
The helper_spawn_sparkles function is a helper function, which the game can’t call, but the on_ functions in this file can.
The game can allow grug entities to edit each other’s data
The game could be responsible for giving every entity a map (think hash maps/Lua tables/JavaScript objects/Python dictionaries), where mods can then read from and write to each other’s maps:
In this video:
- The gun’s
on_spawn()function spawns a “counter” entity. - The gun’s
on_fire()function increments the counter’s “shots” map value by 1. - The counter’s
on_tick()function prints its “shots” map value.
Instead, or additionally, entities could send each other messages.
Here is what the code from the video could look like, when using messages:
There is a big difference between the options of giving every entity a map, and letting entities send messages to each other:
- With a map, entity A can put something in the map of entity B, even when entity B doesn’t ever look at that thing.
- With messages, entity B can choose to ignore a message.
The map approach can be more suitable when there is a blade entity that needs to apply a lasting “poison” effect on a human entity, assuming the poison effect is something the mod came up with. If a human doesn’t want to be poisoned, it could put unpoisonable in its own map, which the blade could check for. If the message approach were to instead be taken, then every human would need to be modified, to handle a potential poisoned message.
The message approach on the other hand is stateless, in the sense that it just processes/ignores a message and moves on, which can be nice. If the game developer allows mods to allocate their own data structures, then the message approach might make the most sense. A blade could for example just maintain a dynamic array of human IDs that it has poisoned, though it would mean that other blade entities can’t tell whether the human has been poisoned by someone else.
Documentation, security, and type checking in one
The game developer is responsible for maintaining a mod_api.json file, which declares which entities and game functions modders are allowed to call. This ensures that malicious modders have no way of calling functions that might compromise the security of the user. It also allows grug.c to catch any potential issues in mods, like passing an i32 to a game function that expects a string.
The game developer can safely share mod_api.json with players, as it also functions as the game’s mod API documentation. The optional work of writing and hosting a pretty website around this file, like a wiki, could then be left to the players.
The mod_api.json file can just be shipped sitting next to the game’s executable, because even if the user uses it to declare the game function exit() exists, mods still can’t call that function. This is because any mod calling exit() in grug will actually be calling game_fn_exit() under the hood, which the runtime loader will fail to find, which grug will report with a nice error message.
This screenshot shows all there is to the mod_api.json file:
Resources and entities are checked at game startup, and during runtime
The game developer can specify which types of resources and entities they expect to receive from mods:
So if a gun entity gets passed a sprite_path field with the value "foo.jpg", grug will throw an error, because the resource_extension specifies that only .png files are accepted.
The same goes for game functions, where play_sound("foo.mp3") might only accept some sound formats, like .flac files.
For entity arguments, spawn_rabbit("ferrari") of course doesn’t make any sense, assuming ferrari is a car entity. That’s why that argument should have an entity_type with the value "rabbit".
The game developer can use "resource_extension": "" or "entity_type": "" where they want to do the type checking themselves. This is necessary when there’s a game function that needs to accept both .wav and .flac files, or that needs to accept both rabbit and jumpy entities.
grug files are easy to convert to JSON, and JSON is easy to convert to grug
In this video there is a small Python script on the right, which uses grug.c its grug_dump_file_ast() and grug_apply_file_ast() functions to double the gun’s rounds per minute:
This makes it easy to automatically update mods, but it could also be useful for VS Code extensions, or for generating the grug files using an in-game Scratch/Blender-like node system.
Stability through hundreds of tests and fuzzing
237 handwritten tests (at the time of writing) that run automatically on every commit using GitHub Actions, ensure that there are no bugs. The actions run the tests on Linux with AddressSanitizer and valgrind, which check for memory access bugs.
There are three test categories:
- Error tests:
grug.cshould find an issue in a.grugfile, like an unexpected character, and return a descriptive error message. - Runtime error tests: During the execution of an
on_function there is a runtime error, like a division by 0, and a descriptive error message should be returned. - OK tests: All
.grugfiles should be compiled and linked without any errors, and every single grug feature (statements, operators, etc.) is extensively tested for correctness.
libFuzzer is a tool that is used to ensure that even the strangest and corrupt looking .grug files won’t ever crash the game.
A fuzzer is basically a neural network that generates a random string, throws it into the fuzzed program, and gets a reward if it walked a new path through the fuzzed program’s code. If it got a reward, it uses the knowledge that there is a good chance it’ll get more rewards if it tries similar strings. In this manner it quickly finds most possible paths through the fuzzed program’s code, also finding a few inputs that crashed grug.c, which I of course patched.
How a game developer might use grug
The below snippets are based on the grug terminal game repository, so if anything confuses you, feel free to check out the full program.
Games typically have an update loop, so by calling grug_regenerate_modified_mods() in there you can tell grug to recompile any modified mods, where the function returns true if there was an error:
int main() {
while (true) {
if (grug_regenerate_modified_mods()) {
if (grug_error.has_changed) {
printf(
"%s:%d: %s (detected in grug.c:%d)\n",
grug_error.path,
grug_error.line_number,
grug_error.msg,
grug_error.grug_c_line_number
);
}
continue;
}
if (grug_mod_had_runtime_error()) {
fprintf(stderr, "Runtime error: %s\n", grug_get_runtime_error_reason());
fprintf(
stderr,
"Error occurred when the game called %s(), from %s\n",
grug_on_fn_name,
grug_on_fn_path
);
continue;
}
static bool initialized = false;
if (!initialized) {
initialized = true;
init();
}
reload_modified_entities();
// Since this is a terminal game, there are no PNGs/MP3s/etc.
// reload_modified_resources();
update();
}
}
The call to reload_modified_entities() in the main function loops over all regenerated mods, reinitializing the tool’s globals and using the new on_ fns:
void reload_modified_entities() {
// For every reloaded grug file
for (size_t reload_idx = 0; reload_idx < grug_reloads_size; reload_idx++) {
struct grug_modified reload = grug_reloads[reload_idx];
// For the player and opponent tools
for (size_t i = 0; i < 2; i++) {
// If the reloaded grug file has the same tool type
if (reload.old_dll == data.tool_dlls[i]) {
data.tool_dlls[i] = reload.new_dll;
// Reinitialize the tool's globals
free(data.tool_globals[i]);
data.tool_globals[i] = malloc(reload.globals_size);
reload.init_globals_fn(data.tool_globals[i]);
// Use the new on fns
data.tools[i].on_fns = reload.on_fns;
}
}
}
}
The call to init() in the main function gives the player tool 0, and the opponent tool 1 from the mods/ directory:
void init() {
pick_tool(0, PLAYER_INDEX);
pick_tool(1, OPPONENT_INDEX);
}
void pick_tool(size_t chosen_tool_index, size_t human_index) {
struct grug_file *tool_files = get_type_files("tool");
struct grug_file file = tool_files[chosen_tool_index];
// This calls the grug file's define fn, which calls our game_fn_define_tool()
file.define_fn();
// The previous file.define_fn() line caused tool_definition to be filled
tool tool = tool_definition;
tool.on_fns = file.on_fns;
data.tools[human_index] = tool;
data.tool_dlls[human_index] = file.dll;
// Initialize the tool's globals
free(data.tool_globals[human_index]);
data.tool_globals[human_index] = malloc(file.globals_size);
file.init_globals_fn(data.tool_globals[human_index]);
}
struct tool tool_definition;
// This gets called by the define() function in grug mods
void game_fn_define_tool(string name, i32 damage) {
tool_definition = (struct tool){
.name = name,
.damage = damage,
};
}
Finally, the call to update() in the main function is the gameplay logic. The most important line is use(player_tool_globals, PLAYER_INDEX);, which calls the tool’s on_use() grug function:
void update() {
void *player_tool_globals = data.tool_globals[PLAYER_INDEX];
void *opponent_tool_globals = data.tool_globals[OPPONENT_INDEX];
tool *player_tool = &data.tools[PLAYER_INDEX];
tool *opponent_tool = &data.tools[OPPONENT_INDEX];
printf("You have %d health\n", player->health);
printf("The opponent has %d health\n\n", opponent->health);
printf("You use your %s against your opponent\n", player_tool->name);
typeof(on_tool_use) *use = player_tool->on_fns->use;
use(player_tool_globals, PLAYER_INDEX);
if (opponent->health <= 0) {
printf("The opponent died!\n");
exit(0);
}
printf("The opponent uses their %s against the player\n", opponent_tool->name);
use = opponent_tool->on_fns->use;
use(opponent_tool_globals, OPPONENT_INDEX);
if (player->health <= 0) {
printf("You died!\n");
exit(0);
}
}
Why grug
Like any good programming language, grug was born from frustration. Specifically, over 4 years of frustration keeping the configuration and Lua files of nearly 200 old Cortex Command mods up-to-date with the game.
The configuration language is a bespoke, cursed format that was only readable by the game’s buggy parser. It required me to write a pretty complex tokenizer and parser, which I had to write many tests for, compared to if it had been say JSON.
And while I love Lua, it was the bane of the community’s existence, as it resulted in an endless flood of mod bug reports in our Discord server. These were incredibly hard to find and fix (though we did our best), due to Lua’s interpreted nature. If for example gun.property_that_was_removed_from_the_game was used in a mod, then it’d just evaluate to nil. Ideally this mod would refuse to run at all until the bug was fixed, like with any compiled language.
Maintaining these mods was a never-ending amount of work.
Lua was also way too complex for most people (since most gamers are not programmers), which meant more cool mods would have been made, had an even simpler scripting language been used.
grug is a stupidly simple configuration and scripting language that only allows mods to use simple functions that either act directly on the game’s state, or act on “global” variables that are only visible to the functions in the same grug file (where Zombie 1 isn’t able to access Zombie 2’s global variables).
Base game content can also be turned into mods in this fashion, which even players who don’t want to install mods will appreciate, as it will allow them to disable content that would have otherwise been hardcoded into the game.
Future work
It is important to note that grug will still very much be a work in progress for the coming months.
I have many plans, but the biggest undertakings will be:
- Supporting Windows
- Supporting ARM
- Outputting debug symbols again, so that grug files can be stepped through with a debugger
For the time being you can try out the list of small example programs. :-)