One-Click Save Game System -- Part 1

Hello fellow developers! It’s Leo here.

I’ve been working on a Save Game system for The Machinery for quite a while now. A first version of the system, which we call the “Gamestate”, will be available for you to play around with in 2021.6. Considering how interesting and challenging the experience was for me, I thought it would be nice to share what went wrong, what went right, but most importantly how the design process went from “paper” to a first working prototype to the final system implementation and integration.

Note: I’ll be referring to “The user” quite a lot in this post. What I mean is not the end-user of the game engine, but rather the “user” of the Gamestate API, which will be the game code in most cases.

Requirements, part 1

A Save Game System is a pretty generic term. So the first thing we did was to clarify what the system needed to do. The most important functionalities we identified were:

1. Telling the system what the State of the application looks like (what kind of objects exist, what members/fields they have, etc). We want to be flexible here, and not assume any particular layout of the State, since it can vary a lot from game to game.

2. Notifying the Gamestate about changes to the application’s state. For example, when an Entity moves, the game will notify the Gamestate of its new position. You can think of this as mirroring the actual state of the application (position of an entity) into the more abstract Gamestate. (But it does not necessarily have to be a mirror operation, some games may choose to keep all their state data in the Gamestate.)

3. Being able to Save and later Load the state. This involves serializing the Gamestate to a binary representation and later recreating first the Gamestate from the serialized file and then the application’s concrete state from the abstract Gamestate.

4. The system will eventually play a crucial role in supporting multiplayer games, so we want to always keep that in the back of our heads. (At the end of the day, you can see sending data to another machine as dumping data to a file and sending that file to the other machine, the other machine will just read back that file and apply those changes to its own state.)

We noticed that the first three requirements (let’s consider the fourth one a bonus requirement) map very well to three distinct “phases” in the lifecycle of the Gamestate*,* so we tried to design our API around that:

1. Configuration. (Things that will only happen at the startup of the application)

2. Runtime. (Things that potentially happen many times per frame, as the simulation is run)

3. One-time actions. (“Special” things that will only happen from time to time under specific circumstances)

Based on this, we sketched out a basic pseudo-code API:

// Configuration
void add_object_type(game_state* gs, char* name);
void add_struct_type(game_state* gs, char* name, member* members, uint32_t n_members);

// Runtime
object_id create_object(game_state* gs, char* type_name);
struct_id create_struct(game_state* gs, object_id, char* struct_name);
void change_member(game_state* gs, object_id o, struct_id s, char* member_name, void *value);

struct buffer
{
  uint8_t* data;
  uint32_t size;
};

// One-time actions
buffer dump_to_buffer(game_state* gs);
void load_from_buffer(game_state* gs, buffer b);

Basic prototype

an exploratory proof-of-concept if you will. Once we were satisfied with this rough on-paper design, I proceeded to implement a basic prototype,

I decided to create a special purpose tab in The Machinery with a simple mini-game inside it. This “game” consisted of a couple of buttons and some shapes changing color and moving around on the screen:

Once I’d implemented this and was able to store and restore the state of this minigame, I took another look at the original requirements with much clearer ideas in my head. Back at the whiteboard, I knew that there were other things that needed to happen for the system to serve all of the use cases. I was also able to make the previous requirements much less vague:

1. There’s an explicit Configuration phase of the system where the user defines what the Persistent State looks like. (This is still quite generic, but we’ll come back to that later.)

// Sample pseudocode:
struct tm_object_type;
struct tm_struct_type;
struct tm_member_id;
struct square;
struct circle;

struct application_state
{
  tm_object_type square;
  tm_object_type circle;

  tm_struct_type position;
  tm_struct_type color;
  tm_struct_type parent;
  tm_struct_type dimension;

  uint32_t square_count;
  square* squares;

  uint32_t circle_count;
  circle* circles;

  tm_gamestate* persistent_state;
};

static global_application_state global;

static void configure_saved_game()
{
  tm_gamestate* gamestate = global.persistent_state;
  global.square = tm_gamestate_api->configure_object_type(gamestate, "square", square_restored_from_saved_game_callback);
  global.circle = tm_gamestate_api->configure_object_type(gamestate, "circle", circle_restored_from_saved_game_callback);

  global.position = tm_gamestate_api->configure_struct_type(gamestate, "position");
  tm_gamestate_api->add_vec3_member(gamestate, global.position, "position_member");

  global.color = tm_gamestate_api->configure_struct_type(gamestate, "color");
  tm_gamestate_api->add_vec4_member(gamestate, global.color, "color_member");
  //...
}

2. As the application runs, the State changes. We push those changes to the Gamestate so it can keep track of them. Note that we don’t necessarily need to push the state changes every frame… it is sufficient if we make sure that all changes have been pushed when the Save happens. We leave it up to the user to decide how often (or not) they want to push changes.

struct square
{
  tm_vec3_t P;
  tm_vec4_t color;
  tm_vec2_t dim;
  uint32_t parent_index;
};

static void update_square_position(uint32_t square_index, tm_vec3_t P)
{
  square* square = global.squares + square_index;
  square->P = P;

  // This will much probably be a lookup in a hash table (it will be filled when
  // objects are first created).
  object_id persistent_id = get_persistent_id_for(square_index);
  tm_gamestate_api->member_changed(global.gamestate, persistent_id, global.position, "position_member", &P);
}

3. Saving the State of the simulation means serializing data to a buffer while Restoring it will involve deserializing data from a buffer and into the actual game structures.

// Very high level pseudocode.
static buffer serialize_to_disk(tm_gamestate* gamestate)
{
  buffer result = {0};
  for(uint32_t i = 0; i < gamestate->change_count; ++i)
  {
    change* change = gamestate->changes + i;
    serialize_to_buffer(change, buffer);
  }

  return result;
}

static void deserialze_to_disk(tm_gamestate* gamestate, buffer b)
{
  while(b.size)
  {
    change c;
    deserialize_from_buffer(&c, &buffer);
    // This will callback into the gamecode so that it can do the proper thing.
    // (Create object, change data, etc)
    apply_change_to_application_state(gamestate, &c);
  }
}

4. Every persistent object needs a unique identifier. So It’s either the gamestate that automatically assigns an object_id to each persistent object or it’s the user’s responsibility to generate unique object_ids and pass them to the API.

At this point of the design process, we didn’t know what solution could work best in the end, so we decided to go with the one that seemed more flexible at the time: the user is responsible for generating unique IDs.

struct object_id
{
  uint64_t u64;
};

// You can generate persistent id's however you want, the important thing is that
// each object has a unique persistent_id.
static object_id generate_persistent_id()
{
  object_id result = {0};
  result.u64 = (uint64_t) random_ensure_unique();
  return result;
}

static void create_persistent_square()
{
  object_id id = generate_persistent_id();
  tm_gamestate_api->create_object(gamestate, application_state->square, id);
}

static void create_persistent_circle()
{
  object_id id = generate_cpersistent_id();
  tm_gamestate_api->create_object(gamestate, application_state->circle, id);
}

// This is the callback that the Saved Game will call when an object needs to be
// created.
static void square_restored_from_saved_game_callback(object_id to_create)
{
  create_square();
}

static void circle_restored_from_saved_game_callback(object_id to_create)
{
  create_circle();
}

5. The Gamestate should be able to work at a Member granularity. I.e., the user should be able to push a change to a single Member of a Struct in an Object. This will help us immensely with multiplayer: only the actual member that changed will be sent across the network (instead of the full struct), and in addition to this users will have the possibility to encode/decode the members however they want.

6. We want to support references. The state of an application might involve objects referencing other objects, so the Gamestate needs to support it.

struct circle
{
  tm_vec3_t P;
  tm_vec4_t color;
  float r;
  uint32_t parent_index;
};

static void update_circle_parent(uint32_t circle_index, uint32_t parent_index)
{
  circle* circle = global.circles + circle_index;
  square->parent_index = parent_index;

  // Note that we have to "query" for both the object itself and the object we want to
  // reference.
  object_id persistent_id = get_persistent_id_for(circle_index);
  object_id reference_persistent_id = get_persistent_id_for(parent_index);
  tm_gamestate_api->member_changed(global.gamestate, persistent_id, global.parent, "parent_member", reference_persistent_id);
}

And this Concludes the first big “iteration” that was done on the system: the one that dealt mostly with the initial on-paper requirements and prototype.

Next time around we’ll take a look at how we approached the problem of integrating the system with our Entity-Component System.

Take care, Leonardo