All Blogs

The Machinery Asset Pipeline

In my last post, I covered the idea of building data pipelines for buffer and image processing using a concept called Creation Graphs. Today I’ll give a more holistic view of how importing assets created in other DCC-packages (Blender, Maya, etc) will work in The Machinery. I say “will work” as while I have a pretty good feeling of how everything will fit together, I’ve yet to implement some of the stuff in this post, so it might drift a bit from what’s described here.

Also worth noting is that while this post only covers import of assets, our goal is to make it just as simple to export data out of The Machinery. This will make it possible to use The Machinery as a standalone pipeline tool, to complement other DCC tools and game engines if desired.

Up until recently, The Machinery has only been able to import glTF files using our own importer, and while we are big believers in glTF there’s no way to ignore the fact that a lot of game assets are still stored in other formats (with Autodesk’s FBX being one of the most popular). Since one of our primary goals with Our Machinery is to empower and simplify the life of content creators we’ve known that we eventually would need to address this. As a first step, we’ve just added a new plugin wrapping the popular Open Asset Import Library (or Assimp in short) making it possible to easily import a lot of other scene formats.

Importing into dcc_asset

Both our own glTF importer plugin and the Assimp plugin are exposed to The Machinery through an interface called tm_asset_io_i which is registered to an API called tm_asset_io_api. The role of tm_asset_io_api is simply to keep track of all active tm_asset_io_i interfaces. The API exposes methods for enumerating those as well as retrieving the first interface capable of either importing or exporting a certain file format (looked up using its file extension).

Importing data runs as an asynchronous task, and when finished the imported scene ends up in our own intermediate representation in The Truth (called tm_dcc_asset_scene). This gets exposed to the user as an asset in the asset browser with the file extension .dcc_asset. The Truth definition of the dcc_asset is kept in its own plugin, together with a helper component called “DCC Asset Component” which makes it possible to render and reason about the imported data.

While we could use the DCC Asset Component directly to build out entities consisting of one or multiple imported scenes, that’s typically not what we want. We tend to need more fine-grained control over the individual objects in the imported scene, both so that we can tweak them further after import, but also so we can reason about them from the simulation/gameplay code.

So instead of doing a bunch of data massaging when loading data into the tm_dcc_asset_scene we try to do as little as possible, just enough so that we can render the scene with decent quality, trying to stay as close to the original file representation as possible.

If we breakdown the contents of tm_dcc_asset_scene we find the regular things typically found in a 3D scene:

  • A scene tree representing the hierarchical relationship and transforms of all nodes in the scene

  • Various buffers storing vertex data, index data, image bits, animation data, etc

  • Objects, such as meshes, cameras, lights, materials, textures, skins, etc

When creating a generic target scene representation like this, it’s inevitable that you end up with some form of least common denominator, which can result in not being able to represent valuable information available in some DCC files. The way we are currently trying to combat that is by allowing importer plugins to extend the tm_dcc_asset_scene with arbitrary new truth objects in an “extensions” subobject set. This means we can serialize plugin specific data together with the tm_dcc_asset_scene and as long as the importer plugin extending The Truth is loaded we can also reason about this data.

So we got our DCC asset imported into The Truth and we can visualize the entire scene as a single monolithic object using the DCC Asset Component. Now let’s take a look at how we break it apart into individual objects, making it possible to tweak the data inside The Machinery.

Scene Tree Component

In The Machinery, an entity has two components for positioning it in the world, the Transform Component and the Link Component. The Transform Component holds the final world transform of the entity and the Link Component holds the local transform in relation to its parent entity. If the Link Component is present it will take control over writing the world transform to the Transform Component.

When importing large DCC scenes, or scenes with skinned meshes, it’s not uncommon that the number of nodes in the scene tree can be rather significant (>1000). And while we, in theory, could break that apart into an entity hierarchy with the nodes represented as child entities with a Transform and Link Component, it’s rare that we need that kind of fine-grained control over the imported scene node hierarchy.

So instead, our current idea is that we will store the imported node hierarchy in a single Scene Tree Component and have the Link Component be able to link to a named node within a Scene Tree Component owned by a parent entity. That way, we can keep a more or less flat entity tree, with each object type represented as its own entity, something like this:

E <dcc-scene-name>
  - Transform Component
  - Link Component
  - Scene Tree Component
  E <mesh-name.material-name>
  - Transform Component
  - Link Component -- References a node within the Scene Tree Component
  - Render Component -- References an imported creation graph representing a mesh
  E <mesh-name.material-name>
  - Transform Component
  - Link Component -- References a node within the Scene Tree Component
  - Render Component -- References an imported creation graph representing a mesh
  E <light-name>
  - Transform Component
  - Link Component -- References a node within the Scene Tree Component
  - Light Component
...

While we probably will allow overriding the transforms of the nodes within the Scene Tree Component, we do not intend to allow changing the link hierarchy. The reason for not making the link hierarchy mutable is that it would drastically complicate re-importing (iterating over) the same DCC-scene. If we allow changing the hierarchy, it becomes unclear who’s authoritative over it — The Machinery or the DCC tool used to create the scene in the first place. This is also true for transform overrides, but in that case, we can restrict those to only happen on instanced entities rather than the entity prototype itself, so in a sense, we can consider the transforms to be immutable as well.

Entity rigging using Creation Graphs

When it comes to the actual objects inside the tm_dcc_asset_scene, most of those will be rigged by executing Creation Graphs. The idea is fairly simple, but yet very powerful. For each object type (mesh, camera, light, texture, material, etc), we will map a Creation Graph prototype that describes what to do with the object data.

For some object types, like cameras and lights, the mapped Creation Graph prototype will simply describe what kind of component will be used to represent its data and how to map the dcc-asset object’s properties to that component. For other objects, like meshes, textures, and materials, those will output new Creation Graph resources located under a <dcc-file>-resources/ sub-folder. Depending on the object type, the graph will do different things:

  • For textures, the Creation Graph would be responsible for generating mipmaps, compress them, etc.

  • For materials, the Creation Graph would contain a “shader graph” that’s able to represent the material definition found in the dcc_asset.

  • For meshes, the Creation Graph would handle vertex cache optimizations and potentially quantization of the vertex attributes. It would then pipe the result to the input of a Material node which in turn would reference the Creation Graph representing the assigned material, and finally, output a draw call for rendering the mesh.

With this in place, we can split up the .dcc_asset into multiple resources and rig the final entity prototype by simply iterating over all objects for each object type found in the .dcc_asset and execute the corresponding Creation Graphs.

Conclusions

While this might sound like an overly complicated setup it’s important to remember that all of this will happen automatically when viewed from an artist’s perspective. All of the needed Creation Graph prototypes would already have been rigged by a Technical Artist.

The real benefits with this type of asset pipeline don’t show until you want to do something a bit more exotic with the imported data. By running all rigging through Creation Graphs, it becomes natural where to plug in things like scene voxelization or other types of pre-processing or bake steps.

A question I recently got was how I would go about visualizing an imported mesh as a set of sphere’s drawn in the center of each triangle — my take is that this type of asset pipeline is a perfect fit for doing those kinds of things. Also, as mentioned in my last post, it’s worth keeping in mind that this way of transforming data is not restricted to run only as an “offline” or import step, we can use the exact same framework for doing runtime procedural content generation, or various forms of particle effects, VFXs, etc.

by Tobias Persson