Quantcast
Proposal: A Version Control System For Game Content

Forums - General Discussion - Proposal: A Version Control System For Game Content

I'm working on a solo indie game (nothing to announce yet) and I've been thinking about the problems version controls systems run into when dealing with large, non-merge-able binary files, specifically game content. A commit-based system (like Git) is the model I'm going to follow. I feel like source code should be handled by a separate VCS, as they have good merge tools already in place, and deal with a different problem (many small text files over time). I'd like some feedback from any developers on the forum. The plan is to release it as an open-source project on GitHub.

Here's what I'm thinking: split files into chunks, compress them and store those in the repository. To map these chunks back into files, store an object whose contents are just an ordered list of those chunks. This will give you de-duplication on two levels: per file and per file chunk. That solves the problem of having many similar version of the same file.

The next thing I was thinking of was having partial checkouts - an incomplete "view" that only has what you're working on as writable. Every other file you don't need to edit is a UNIX symbolic link (or Windows junction) pointing to the most recent version of the corresponding file on the server. This means checkouts are fast, as a single, complete commit may be dozens or hundreds of gigabytes, and that can never be fast. Artists should be free to work and be creative, not deal with version control, especially doing version control by hand (abusing Save-As and including version numbers in file names). They should be able to configure the tool to create a temporary "checkpoint" commit on every save that doesn't get included in the master version history.

The other issue to total repository size. I think there should be a mechanism to move part of the repository off the server and on to permanent storage (either Blu-Rays or external hard drives). There should be a simple command to back-up version history.



Currently (Re-)Playing: Starcraft 2: Legacy of the Void Multiplayer, The Legend of Zelda: A Link to the Past

Currently Watching: The Shield, Stein's;Gate, Narcos