I recently wrote a post which explained how Brackeys' Save & Load system could be fixed so that it isn't victim to a major security flaw regarding BinaryFormatter, by instead using BinaryWriter and BinaryReader. This guide will explain how to write a better, portable, more scalable save & load system from the ground up.

Buckle your seatbelts, we have a lot to cover.

Introduction

As I hinted at in the end of my post where I fix Brackeys' Save & Load system, we'll be using Protocol Buffers to serialize and deserialize game save data. Protocol Buffers are a language-agnostic specification on binary data serialization, which means there is a way to use them in almost every programming language (C# included!)

Aside from the security flaw, one of the problems with BinaryFormatter is that it was specifically built to serialize and deserialize CLR objects. If you saved your game data with BinaryFormatter in C#, and then attempted to load that data in Java, you would have an extremely difficult task ahead of you trying to figure out how exactly the data is encoded.

Protocol Buffers solve this because the specification for how data is encoded is very well documented, and extremely easy to implement; not that we'll have to worry about it.

Why not…

... PlayerPrefs?

I see a lot of people suggest PlayerPrefs to save and load game state such as high scores and player positions. I wholeheartedly disapprove of this advice for a few reasons:

• On Windows it stores the data in the registry, which can lead to very inefficient and non-optimal programming if not done properly.

• It's only capable of handling 3 types - float, int, and string. This is even worse than using BinaryReader and BinaryWriter! At least they support double, long, bool, and all the other built-in types. Unity did not even bother to add overloads for Color, or Vector3, a type you'd think would be very important to saving the player position or that of objects.

... JSON¹ (or similar)?

Of course, another alternative to serializing game state is by using a human-readable format such as JSON¹. This would entirely solve the issue of serializing complex types such as vectors, or other custom classes.

The problem, however, is that formats such as these were designed to be easily read by humans - not computers. Computers don't understand JSON¹, and so precious CPU time must be spent either constructing a JSON¹ string when writing, and attempting to parse it back to an object when reading.

This makes JSON ideal for storing configuration (e.g. the player's chosen settings), things which are okay to be modified by the player - but less than ideal for things which you want to keep relatively untouched.

It also drastically increases the size it occupies on disk. Remember that characters use - at the very least - 8 bits each. Even a simple JSON structure like so:

{
  "health": 100.0,
  "score": 90
}

If you viewed the hex dump of this string, you would see the following:

EF BB BF 7B 0D 0A 20 20 22 68 65 61 6C 74 68 22 3A 20 31 30 30 2E 30 2C 0D 0A 20 20 22 73 63 6F 72 65 22 3A 20 39 30 0D 0A 7D

A whopping 42 bytes, for a float value and an int value. That's insane. Using binary serialization would mean writing only 8 bytes total; 4 bytes for a float, 4 bytes for an int. That's a massive ~81% reduction. If you're curious, this is the equivalent data represented in pure binary:

00 00 C8 42 5A 00 00 00

Bringing Protocol Buffers to C#

Method 1: Using the NuGet outside of Unity

If you aren't using Unity, all you need to do to follow along with the rest of this guide is install the protobuf-net NuGet into your project and skip to the next section.

Method 2: Using the NuGet with Unity

If you are using Unity, you can use NuGetForUnity which I personally find to be a fantastic way of adding NuGet dependency support to Unity. I won't go into detail about how to use that, the README explains it. Add the protobuf-net NuGet to your project, and skip to the next section.

Method 3: I did Method 4 for you

If neither method 1 nor 2 work for you, I have compiled the assembly references you need so that protobuf-net can be imported into Unity with ease.

Simply download this .unitypackage _{(version 3.0.101; last updated June 21^st, 2021)}, and import the files into your project. However, be warned: I will not be keeping this file up to date on a strict and regular basis. If you are reading this post far in the future, and the references are dangerously out of date, follow the steps in Method 4.

Method 4: Manually downloading references

Open the NuGet page for protobuf-net and choose to download the package directly:

The .nupkg file it gives you is actually just a ZIP archive with a misleading file extension. You can either rename it from protobuf-net.x.x.xx.**nupkg** to protobuf-net.x.x.xx.**zip**, and open it as normal, or you can right click it and choose to Open With an app such as WinRAR or 7-zip.

Once inside the file, navigate to the lib directory, and then netstandard2.0. This directory contains two files: protobuf-net.dll and protobuf-net.xml. Extract them both to somewhere in your Assets.

Repeat these steps for the following NuGets:

• protobuf-net.Core

• System.Collections.Immutable

• System.Memory

• System.Buffers

Testing the library

For the remainder of this guide, I will be giving my examples in the context of Unity. Do keep in mind, however, that protocol buffers are completely platform independent and so this guide should be useful to you whether or not you are using Unity (by making changes like replacing Debug.Log with Console.WriteLine, etc.)

Create a new behaviour and copy the following code into the Start method.

using (var stream = new MemoryStream())
{
    Serializer.SerializeWithLengthPrefix(stream, new { X = int.MaxValue }, PrefixStyle.Base128);
    var buffer = stream.ToArray();
    Debug.Log(BitConverter.ToString(buffer));
}

Ensure the behaviour is attached to a live game object and play the scene. If you see 06-08-FF-FF-FF-FF-07 in your console, this means everything is working as expected:

As an aside, the four bytes FF FF FF FF is the X value on the third line! This is the representation of int.MaxValue in hexadecimal, a value whose 32 binary digits are all 1. Try changing the value of X and watch how the bytes change.

Creating a save data model

Let's start off by assuming you have a behaviour which is attached to the player, creatively named PlayerBehaviour, within which are a couple of properties that represent some data about the player we wish to save, such as their Score and how much Health they have.

using UnityEngine;

public class PlayerBehaviour : MonoBehaviour
{
    <mark>[field: SerializeField]</mark>
    <mark>public int Score { get; set; } = 0;</mark>
    
    <mark>[field: SerializeField]</mark>
    <mark>public float Health { get; set; } = 100;</mark>

    private void Awake()
    {
        // ...
    }

    private void Update()
    {
        // ...
    }

    private void OnApplicationQuit()
    {
        // ...
    }
}

Our first step would be to create a POCO model containing the data - and only the data - we wish to save. Let's create a simple SaveData class to encapsulate it.

public class SaveData
{
    public int Score { get; set; } = 0;

    public float Health { get; set; } = 100;
}

There are two attributes we need to use so that this class is easily understood by protobuf-net.

The first is ProtoContractAttribute. This tells protobuf that the type is one we wish to serialize. It is similar to having to mark a class Serializable when using BinaryFormatter.

<mark>using ProtoBuf;</mark>

<mark>[ProtoContract]</mark>
public class SaveData
{
    public int Score { get; set; } = 0;

    public float Health { get; set; } = 100;
}

The second attribute is ProtoMemberAttribute. This is similar to using JsonProperty or JsonPropertyName when using JSON serialisation libraries - in that it indicates the “name” of the property we want to serialize.

protobuf-net unfortunately does not support named properties, but it does support integer “tagged” properties. Decorate each property with ProtoMemberAttribute like so:

using ProtoBuf;

[ProtoContract]
public class SaveData
{
    <mark>[ProtoMember(1)]</mark>
    public int Score { get; set; } = 0;

    <mark>[ProtoMember(2)]</mark>
    public float Health { get; set; } = 100;
}

You can use any tags you like - they do not have to be 1 and 2, nor do they have to be in any particular order. The only rule with protocol buffers is that each member's tag must be unique. This is because if you decide to change your model later on by removing or adding new properties, it won't interfere with the existing data.

The save model is now complete!

Loading the data

To get started, let's create a field in PlayerBehaviour which will store the save path of the save file, as well as two new empty methods named LoadGame and SaveGame.

<mark>using System.IO;</mark>
<mark>using ProtoBuf;</mark>
using UnityEngine;

public class PlayerBehaviour : MonoBehaviour
{
    <mark>private string _saveFile;</mark>

    [field: SerializeField]
    public int Score { get; set; } = 0;

    [field: SerializeField]
    public float Health { get; set; } = 100;

    private void Awake()
    {
        <mark>_saveFile = Path.Combine(Application.persistentDataPath, "save001.dat");</mark>
        
        // ...
    }

    private void Update()
    {
        // ...
    }

    private void OnApplicationQuit()
    {
        // ...
    }

    <mark>private void LoadGame()</mark>
    <mark>{</mark>
    <mark>}</mark>

    <mark>private void SaveGame()</mark>
    <mark>{</mark>
    <mark>}</mark>
}

We'll focus on LoadGame first. There are only four things we need to do here.

• Check if a save file exists

• If it does, open a read-only stream to the file

• Deserialize the stream to a SaveData instance

• Set Score and Health in our PlayerBehaviour

protobuf-net offers a Serializer class which, unlike classes such as BinaryFormatter, is accessed statically. Call the generic Deserialize method, passing in the input stream.

private void LoadGame()
{
    if (!File.Exists(_saveFile))
        return;

    using var file = File.OpenRead(_saveFile);
    <mark>var data = Serializer.Deserialize<SaveData>(file);</mark>

    Score = data.Score;
    Health = data.Health;
}

All you need to do is call LoadGame when the game starts (Awake) or when the player loads their save.

Saving the data

The Serializer class also contains a Serialize method, which works in the opposite direction. Construct a SaveData instance, and then serialize it to a writable stream:

private void SaveGame()
{
    using var file = File.<mark>OpenWrite</mark>(_saveFile);

    <mark>var data = new SaveData</mark>
    <mark>{</mark>
    <mark>    Score = this.Score,</mark>
    <mark>    Health = this.Health,</mark>
    <mark>};</mark>

    <mark>Serializer.Serialize(file, data);</mark>
}

All you need to do now is call SaveGame when the game quits (OnApplicationQuit) or the player manually saves their game. Whatever works for your needs.

Serializing Unity types

We can now serialize and deserialize custom types! We can even have nested types (i.e. custom classes inside SaveData) too as long as those are also decorated with ProtoContract. However, we can't serialize Unity types such as Vector3 or Color.

To show what I mean, let's store the player position in SaveData as a Vector3, and adjust our SaveGame and LoadGame methods to reflect it.

using ProtoBuf;
<mark>using UnityEngine;</mark>

[ProtoContract]
public class SaveData
{
    [ProtoMember(1)]
    public int Score { get; set; }

    [ProtoMember(2)]
    public float Health { get; set; }
    
    <mark>[ProtoMember(3)]</mark>
    <mark>public Vector3 Position { get; set; }</mark>
}

private void LoadGame()
{
    if (!File.Exists(_saveFile))
        return;

    using var file = File.OpenRead(_saveFile);
    var data = Serializer.Deserialize<SaveData>(file);

    Score = data.Score;
    Health = data.Health;
    <mark>transform.position = data.Position;</mark>
}

private void SaveGame()
{
    using var file = File.Create(_saveFile);
    var data = new SaveData
    {
        Score = this.Score,
        Health = this.Health,
        <mark>Position = transform.position</mark>
    };

    Serializer.Serialize(file, data);
}

This will compile - after all there's nothing syntactically wrong here - but protobuf-net won't handle it. Running this code alone will throw us InvalidOperationException.

Vector3 (and all other Unity types) are not decorated with the ProtoContract attribute, therefore protobuf-net has no idea about how to serialize it. We also have no way to access the source code to add it. What can we do in this situation?

Surrogates to the rescue

We can define a surrogate type, which essentially acts as a “middle man” between protobuf-net and the types we don't have access to modify. In a nutshell, surrogates tell protobuf-net “Hey, whenever you see a Vector3, convert it to a different type and serialize that instead.”

To do this, let's create a very basic struct which will serve as the surrogate for a Vector3, called Vector3Surrogate. All we need to care about are the X, Y and Z components - and so that is all we need to declare. For example, we don't need the magnitude because that is calculated from these 3 components.

using ProtoBuf;

[ProtoContract]
public struct Vector3Surrogate
{
    [ProtoMember(1)]
    public float X { get; set; }
    
    [ProtoMember(2)]
    public float Y { get; set; }
    
    [ProtoMember(3)]
    public float Z { get; set; }
}

Next, we need to define a couple of conversion operators so that we can convert between Unity's Vector3 and our surrogate.

using ProtoBuf;
<mark>using UnityEngine;</mark>

[ProtoContract]
public struct Vector3Surrogate
{
    [ProtoMember(1)]
    public float X { get; set; }
    
    [ProtoMember(2)]
    public float Y { get; set; }
    
    [ProtoMember(3)]
    public float Z { get; set; }

    <mark>public static implicit operator Vector3(Vector3Surrogate vector) =></mark>
        <mark>new Vector3(vector.X, vector.Y, vector.Z);</mark>

    <mark>public static implicit operator Vector3Surrogate(Vector3 vector) =></mark>
        <mark>new Vector3Surrogate { X = vector.x, Y = vector.y, Z = vector.z };</mark>
}

If you don't understand operator overloading yet, you can read the documentation about them. But do feel free to copy/paste this simple surrogate.

Now we need to tell protobuf-net about our struct. For beginners, this is arguably the most complicated part of the entire guide (which is why this guide is classified as “Intermediate”), so bear with me.

The way protobuf-net knows what types to serialize - and how to serialize them - is through something called a “type model”. We can get the model that's in use by accessing Protobuf.Meta.RuntimeTypeModel.Default, and storing it in a variable.

using ProtoBuf.Meta;

// in a method
var model = RuntimeTypeModel.Default;

Now we can add our struct as a known type to serialize. To do this, call the Add method and pass the surrogate type we made as a generic argument:

var model = RuntimeTypeModel.Default;
<mark>model.Add<Vector3Surrogate>();</mark>

The last step is to tell protobuf-net to actually use this as a surrogate in place of UnityEngine.Vector3. Call Add again, but this time pass Vector3 as a generic argument, and false as the first argument (this tells protobuf-net not to serialize Vector3).

Then, call SetSurrogate on the result, and pass the type of our struct as the argument.

var model = RuntimeTypeModel.Default;
model.Add<Vector3Surrogate>();
<mark>model.Add<Vector3>(false).SetSurrogate(typeof(Vector3Surrogate));</mark>

At this point you're probably asking "but where do I write this?", since earlier my only comment was merely “in a method”. The most important thing to note here is that these 3 lines should only be executed once. And when I say once, I don't mean “once per behaviour” or “once per scene load”, I mean once throughout the entire lifetime of the game. Once protobuf-net knows about the surrogate, it doesn't need to be told again (you will actually get errors if you do this more than once.)

This is down to you. Depending on how your game is structured, you might have a GameManager which has some one-time initialization logic as soon as the game launches. If you do, put it there. Or, instead favouring the use of the RuntimeInitializeOnLoadMethod attribute.

But purely for the purposes of demonstration, I'm going to shove this logic into the static constructor for PlayerBehaviour. This is a dirty trick that will at least let us test our code is working.

using System.IO;
using ProtoBuf;
<mark>using ProtoBuf.Meta;</mark>
using UnityEngine;

public class PlayerBehaviour : MonoBehaviour
{
    // ...

    <mark>static</mark> PlayerBehaviour()
    {
        <mark>var model = RuntimeTypeModel.Default;</mark>
        <mark>model.Add<Vector3Surrogate>();</mark>
        <mark>model.Add<Vector3>(false).SetSurrogate(typeof(Vector3Surrogate));</mark>
    }

    // ...
}

If at any point you lost track of the code, you can see how my full PlayerBehaviour looks here.

You should be able to launch the game, play around with Score and Health, and even move the player around. Stop the game, relaunch it, and it should have loaded your the previous state! Player position included.

Making a structural improvement

The more properties you add, the more cumbersome this can become to exchange values between SaveData and PlayerBehaviour; just like with BinaryReader and BinaryWriter.

Here you could instead add the Serializable attribute to SaveData (so that it shows in the inspector properly) and store an instance of it within the PlayerBehaviour itself like so:

public class PlayerBehaviour : MonoBehaviour
{
    private string _saveFile;

    [field: SerializeField]
    <mark>public SaveData Data { get; private set; }</mark>

    private void Awake()
    {
        _saveFile = Path.Combine(Application.persistentDataPath, "save001.dat");
        
        LoadGame();
    }

    private void LoadGame()
    {
        if (!File.Exists(_saveFile))
            return;

        using var file = File.OpenRead(_saveFile);
        <mark>Data = Serializer.Deserialize<SaveData>(file);</mark>

        transform.position = Data.Position;
    }

    private void SaveGame()
    {
        Data.Position = transform.position

        using var file = File.OpenWrite(_saveFile);
        <mark>Serializer.Serialize(file, Data);</mark>
    }

    // ...
}

We do still have to set the transform position, since that Vector3 and the one defined in SaveData are unrelated values. You can work around that by passing a reference to the player for SaveData to access. The Position property could automatically set/get the player's position.

However, this alone greatly reduces the amount of code in both the Save and Load methods - only having to serialize a single type similar to BinaryFormatter. 👌

Adding compression

For now this is good. We only have 3 properties (Score, Health, and Position) and they serialize to just a few bytes. But as a game grows, it will need to start serializing more and more data. Skills, inventories, enemy data, game progression. A savefile can get quite large pretty quickly. This is where compression comes in, and thanks to .NET it is remarkably easy to implement.

Keep in mind this will introduce a breaking change, so be sure to delete your current save file so that deserialization errors don't happen! On Windows, Application.persistentDataPath points to %LOCALAPPDATA%low\<COMPANY>\<PRODUCT>, the save001.dat file will be in there.

protobuf-net works with streams. However, it doesn't care what stream. We opened a FileStream by calling File.OpenRead and File.OpenWrite, but we can also encapsulate that stream in another stream - a GzipStream - and tell protobuf-net to use that instead. GZip compression functionality is built right into .NET, so we don't need any other third party libraries!

Let's modify SaveGame to create a new GZipStream under “Compress” mode - wrapping the FileStream. Then we simply pass that GZipStream to Serialize.

<mark>using System.IO.Compression;</mark>

// ...

private void SaveGame()
{
    Data.Position = transform.position;

    using var file = File.Create(_saveFile);
    <mark>using var gzip = new GZipStream(file, CompressionMode.Compress);</mark>
    Serializer.Serialize(<mark>gzip</mark>, Data);
}

LoadGame will also open a GZipStream but instead under “Decompress” mode.

private void LoadGame()
{
    if (!File.Exists(_saveFile))
        return;

    using var file = File.OpenRead(_saveFile);
    <mark>using var gzip = new GZipStream(file, CompressionMode.Decompress);</mark>
    Data = Serializer.Deserialize<SaveData>(<mark>gzip</mark>);

    transform.position = Data.Position;
}

The last thing we need to do to ensure maximum compatibility is to add a length prefix to our data, so that protobuf-net knows how much data to deserialize.

Thankfully, this functionality is included so we don't need to do it ourselves. Replace Serialize and Deserialize with SerializeWithLengthPrefix and DeserializeWithLengthPrefix respectively, passing in a prefix encoding. It doesn't necessarily matter which you choose, but they do need to match. I typically choose PrefixStyle.Fixed32BigEndian.

The final versions of SaveGame and LoadGame should now look like this:

private void LoadGame()
{
    if (!File.Exists(_saveFile))
        return;

    using var file = File.OpenRead(_saveFile);
    using var gzip = new GZipStream(file, CompressionMode.Decompress);
    Data = Serializer.DeserializeWithLengthPrefix<SaveData>(gzip<mark>, PrefixStyle.Fixed32BigEndian</mark>);
    
    transform.position = Data.Position;
}

private void SaveGame()
{
    Data.Position = transform.position;
    
    using var file = File.Create(_saveFile);
    using var gzip = new GZipStream(file, CompressionMode.Compress);
    Serializer.SerializeWithLengthPrefix(gzip, Data<mark>, PrefixStyle.Fixed32BigEndian</mark>);
}

And there we have it. We now have the foundations of save/load system! One which can be easily scaled by simply adding more things to SaveData - including child classes.

I do want to stress that the code I've shown in this guide is only for demonstration purposes. I'm aware that I've committed some code sins here, but I am confident that as a developer, you will find ways to improve it. You may also feel like this is a rather convoluted way to save data, but the fact is that data serialization is not exactly trivial - especially if you want it done right. Game development takes a lot of effort as it stands. Good game development will take some time.

Besides, we didn't need to use BinaryFormatter.