Modding:Deserializing protobufs from the command line
Protocol Buffers are one of the serialization formats used by Vintage Story. If one can obtain a file containing one of these serialized protobufs, then it is possible to deserialize it and view the contents from the command line.
Obtaining the serialized data
The main vcdbs file is a sqlite file, which contain tables, which contain protobuf serialized blobs. It is possible to use the sqlite3 command line tool to extract the serialized blob from a row into a file.
For example, the following reads the row with savegameid=1 from the gamedata table of the modtestworld world.
sqlite3 modtestworld.vcdbs "SELECT writefile('gamedata_1.binpb', data) FROM gamedata WHERE savegameid=1;"
Alternatively, all of the rows can be selected and written to a filename that contains the primary key of the table. This dumps all rows of the gamedata table (there's actually only one anyway).
sqlite3 modtestworld.vcdbs "SELECT writefile('gamedata_' || savegameid || '.binpb', data) FROM gamedata;"
Obtaining the schema
Without the schema, it is only possible to perform a decode_raw of the protobuf, which produces difficult to read output. It is better to obtain Vintage Story's protobuf schema so that the output contains properly labelled fields.
The Vintage Story source code does not contain a normal .proto schema file. Instead, the schema is implicitly defined by applying protobuf-net annotations on types. However, a .proto schema file can be extracted from those types using the vs-proto tool. Alternatively, one can use the already extracted schema available on the vs-proto GitHub page. Protobufs are resilient to errors. So even if the already extracted schema is slightly out of date, it will still mostly work.
Deserializing the file
protoc is a command line tool that can deserialize the binary file. The tool can be downloaded as an asset of the Protocol Buffers releases on GitHub.
The --decode option is used to tell protoc to deserialize from stdin. The option is undocumented on the official Protocol Buffers page where the tool is downloaded from. However, this stack overflow answer documents it. Specifically, the tool must be given the type name of the serialized data and the schema file that defines the type. It reads the serialized data from stdin, so the binary file must be redirected into the program. Annoyingly, if the schema file is outside the current directory (the path contains a slash), then that folder name must be given both in the schema file path, and with the --proto_path option.
The following example deserializes gamedata_1.binpb using the schema-1.19.8.proto schema file in the home directory.
$ protoc --decode SaveGame --proto_path=$HOME $HOME/schema-1.19.8.proto <gamedata_1.binpb MapSizeX: 1024000 MapSizeY: 256 MapSizeZ: 1024000 Seed: 1727735926 LastEntityId: 82 ModData { key: "BlockIDs" value: "\n\007\010\000\022\003air\n\ ...
Wondering where some links have gone?
The modding navbox is going through some changes! Check out Navigation Box Updates for more info and help finding specific pages.
Modding | |
---|---|
Modding Introduction | Getting Started • Theme Pack |
Content Modding | Content Mods • Developing a Content Mod • Basic Tutorials • Intermediate Tutorials • Advanced Tutorials • Content Mod Concepts |
Code Modding | Code Mods • Setting up your Development Environment |
Property Overview | Item • Entity • Block • Block Behaviors • Block Classes • Block Entities • Block Entity Behaviors • World properties |
Workflows & Infrastructure | Modding Efficiency Tips • Mod-engine compatibility • Mod Extensibility • VS Engine |
Additional Resources | Community Resources • Modding API Updates • Programming Languages • List of server commands • List of client commands • Client startup parameters • Server startup parameters Example Mods • API Docs • GitHub Repository |