Tag Archive for 'software'

Mercurial, file browsers, and the pipeline

browser

So, we’ve more or less finished the first version of our file browser – a custom Qt/PyQt interface written for the project primarily by Mikkel Jans.  Above is the version view component of the browser.  The browser is the start to managing our pipeline data processes – publishing, versioning (I’ll get into that below), launching files, and connecting (at the moment through socket interfaces) with our pipeline applications.

Versioning:

logo-droplets-200We’ve chosen to use Mercurial, a distributed version control system, for our file versioning.  A user might typically save, say, a Maya file dozens of times during a day, but she’ll commit a new revision only a handful of those dozen times.  Each commit requires a comment which describes pertinent information about that commit, and the user is able to revert to any commit in the history, or restore a commit as a separate file to a local folder.  Branching is also possible, which means that it’s relatively simple to ‘try out’ ideas in your work files.  In a typical programming environment, you might then merge that branch with your main branch, but it’s a bit more difficult with Maya or Shake files to merge through a text editor (so we’re not addressing that aspect yet).

We’ve also written a standard view into our browser which displays the revision history on a selected file, with a visual graph that displays the DAG (directed acyclic graph) of the file history.

What this means is that we don’t add user names or version increments in our naming convention – each file’s history is saved in a mercurial repository, along with comments, usernames, and other tags that we include in the changesets.  Since mercurial saves only file differences, and compresses that file history into a binary format, the entire mercurial repository for a file, including dozens of commits, is usually considerably less than the size of the actual file (at least for most of our work files which are saved in ascii formats).

Mercurial forms one of the components of our pipeline.  It allows us to easily roll-back assets, keep track of asset relationships (dev -> publish), monitor user activity on work files, and have an overview of the different iterations a file goes through.  It’s relatively lightweight, cross-platform, and integrates well with our primarily Python based setup.

Earth

In preparation for the shoot, I’ve installed Earth from Rising Sun Pictures, which keeps track of disk usage.  It’s browser based, and runs off a Postgre database and RubyOnRails.  I think I’ll be expanding the server soon – since the pipeline is based around 16bit float (RAW/EXR), and the motion control shots will likely have more than one pass, our data for the shoot will take at least a terabyte of space.  The production, I’m anticipating, will easily eclipse that number.

Along with showing usage statistics, Earth also has a neat radial display:

earth_radial

Concept Mapping

I’ve recently discovered this open source tool for planning the production:

http://vue.tufts.edu/

The below picture is my first try at mapping out some of the general processes in this film.

vue_concept_mapping




Warning: require_once(/home/sunit_parekh/blog.machinefilm.com/wp-content/themes/k2/footer.php): failed to open stream: Permission denied in /home/sunit_parekh/blog.machinefilm.com/wp-includes/template.php on line 572

Fatal error: require_once(): Failed opening required '/home/sunit_parekh/blog.machinefilm.com/wp-content/themes/k2/footer.php' (include_path='.:/usr/local/lib/php:/usr/local/php5/lib/pear') in /home/sunit_parekh/blog.machinefilm.com/wp-includes/template.php on line 572