Alembic the VFX Pipeline Overhaul...

Alembic the VFX Pipeline Overhaul...

What exactly is Alembic? I know that some of you will have a fair idea, but I can guess that an awful lot of you simply won't have the first idea about what it does and why it should matter to you.

There's actually quite a lot of information out there about it and the Alembic website is a good place to start. You will find a lot of information on there, much of it is full of very technical jargon... So, to put it extremely simply, Alembic is an industry-level caching technique that's highly optimised for VFX industry pipelines. 

"What?" I hear you say, another caching technique? Who wants another one of those? Well, bare with me on that one. Alembic is actually very different to what's already available and has the backing of most of the influential players in the VFX industry.

Let's face it most of us have tried to transfer data between 3D packages at some point or another - the worst example I can remember was from about 20 years ago with DXF... Least said the better! Then there was the advent of OBJ's, but that wasn't the panacea it was meant to be and each package had its foibles. You could indeed import these different formats into different software applications, but the best you could hope to achieve was to just get the model.

And then along comes FBX, formerly Kaydara and now owned by Autodesk. FBX is doing an increasingly better job of converting files between packages - especially between the Autodesk applications - but it still 'bakes' animation in some instances. Those of you who have used it will know that weird conversions of data types occur. This is due to how packages handle certain types of data, normaly rotational data, at the lowest level.

We have particle, fluid, hair and cloth caches all taking massive amounts of disk space, and all in their own formats. The larger facilities have for years been writing their own proprietary "special sauce" plug-ins and data formats that they use internally to move a lot data from one package to another or from one department to another. But for most smaller productions facility, the same file, with all the data in it, is passed from one person to the next. This is pretty inefficient and means that huge files, with exactly the same data, are duplicated unnecessarily. Do lighting artists really need the animation rig in their scene? Do they want it to calculate the rig every time they move a time slider? The same goes for the FX TDs. The only person who actually needs the rig is the animator.

Vertex caching was a very early concept aimed at resolving these issues, and is now known as geometry caching. Once animators are finished animating the version they are on, they hit a script or button and it will generate a complete list of all the vertices and their positions on a per frame basis, allowing the lighting department or FX artists to simply hook up to the cache. It is very fast and the cache can be updated at anytime, regardless of the animator. This has been implemented in numerous packages and in numerous different ways. Point Oven was a commercial attempt to do this, and it was relatively successful. The problem is that each VFX facility and each software company still have their own pipeline, their own data formats, and their own 'special sauce'. This is a big challenge in an industry where the norm is now for multiple vendors to all work on the same film projects and share assets. And for every time assets are transferred from one vendor to another, data has to be converted or recomputed locally.

Now that I have exposed the challenges that we face, I can return to Alembic, a product which promises to fix most of these issues. It can basically send a model with animation and curves (used for hair and fur), but without rig and without the additional proprietary stuff added from one company to another, or one software package to another, or even from one department to department.

The Foundry - makers of NUKE and MARI - have KATANA in the works. This is a lighting and look development tool which leverages Alembic files. It allows you to pull data in from any package that supports Alembic, and allows you to use Katana as a central hub for lighting and look dev for entire scenes and shots.

Alembic has been designed by VFX facilities for use in VFX workflows and can cope with transfers from department to department and from business to business. Imagine a world where animation is done by one company and all the FX work is done at a completely separate facility seamlessly... Alembic is going to change the way pipelines function and how films are made.

3 Comments Lee Danskin

Posted by
Lee Danskin
Thu 6 Oct 2011: 10:49am

Tags: , , , ,

Comments

  • gustavo schrank:

    Nice ... tnx

  • sandra:

    Awesome explanation.

  • D1g1talr10t:

    Last but not least, it's really fast to load data from the scene along the timeline and the diskspace is extremely smaller compared to the other formats.
    Pure alchemy!

comments powered by Disqus