All Blogs

The Machinery Beta — September 2020 (version 2020.9)

Just as mushrooms are popping up in the forest, it’s time for another The Machinery beta release. We are getting back in the swing of things after vacations and have improvements for you.

As always, registered beta users can download the build from our forum. If you want to join the beta, sign up! If you find bugs in this release or have requests for specific features, post them to our issue tracker. For more general discussions, hit us up on the forum or Discord.

One thing you will notice right away is that our Beta EULA is now signed within The Machinery itself. When you start The Machinery you will be asked to agree to the EULA before continuing. Don’t worry, you only have to do this once.

Other key highlights in this release are:

As always, we also have a number of smaller updates.

Asset Database

This release introduces a new file format for The Machinery projects which we call the Asset Database.

Since before, The Machinery has had two different file formats. A Directory format, where each asset in the project is saved to a separate file and a Single File format, where the entire project is saved to a single file. The Directory format works well with version control tools, such as Perforce or Git, while the Single File format is convenient for example if you want to mail a project to someone.

The Asset Database format replaces the previous single file format. In the previous format, the entire project was saved as a single stream, so any change would require a complete re-save of the whole project. This can be pretty time-consuming when you start to have projects with GBs of assets.

In contrast, the new format lets individual assets be saved and loaded without touching any other parts of the file. This means that if you change an asset, only that asset needs to be saved, not the whole project.

The new format also lets you save and revert individual assets using the Modified Assets tab. (Previously, this was only supported for Directory projects.)

With this release, the old single file format is considered deprecated. You can still open projects in the old format, make changes to them and save, but new projects can only be saved in the Directory or the Asset Database format.

Download Tab

The Machinery has a new Download tab that lets you download new versions of the executable as well as sample projects. Note that we no longer distribute the sample projects through the forum, since they can now be downloaded via the Download tab.

Download The Machinery executables and projects.

Download The Machinery executables and projects.

Creation Graph Instancing

This is a rather technical update so if you’re not interested in the low-level details of how Creation Graphs works on the engine side, feel free to skip this section.

In earlier versions of The Machinery, there hasn’t been any concept of creation graph instances, meaning that all data generated by a graph had to be considered immutable since it could be shared by multiple creation graph owners (typically a .creation asset or a Render Component) referencing the same graph. This caused a lot of complexity in the owner code since it had to understand what parts of the data outputted from the graph that needed to be instantiated and somehow know how to implement that functionality.

A typical example is Constant Buffers (CB) and Resource Binders (RB) coming from graph authored shaders instantiated by the graph. In this case, the render component had to view the output of these CBs and RBs as “templates” and implement logic for creating unique instances of them for each entity instance and entity context (viewport) referencing the graph. While this was pretty manageable for simple graphs, where we only dealt with known node outputs, things became way more complicated and ugly when we added the GPU Simulation system in the April release.

If you’ve tried creating a GPU Simulation in earlier versions, you’ve probably noticed it behaves as if the systems are shared between all viewports instead of correctly simulated per viewport, that’s because they were, and this was caused by the lack of proper Creation Graphs Instancing.

In this release, we’ve done a big refactoring of how the Creation Graph interpreter works. We’ve made a clear distinction between the compile step of the graph and the execution of the graph. During the compile step, the graph is first validated and a creation graph interpreter is created. Any node in the graph implementing the tm_creation_graph_node_type_i::compile() will also get called during this step.

To be able to trigger events or requesting the output of a graph a tm_creation_graph_instance_t first needs to be created by calling tm_creation_graph_api->create_instance() and then passed as an argument to the event triggering and output functions. The returned instance holds its own copy of the wire and variable data, making it possible to operate on any number of creation graph instances in parallel without locking.

When an instance is created a new event called TM_CREATION_GRAPH_EVENT_CREATE_INSTANCE is automatically triggered, and likewise when an instance is destroyed an event called TM_CREATION_GRAPH_EVENT_DESTROY_INSTANCE gets triggered. Nodes in need to create per instance-specific data can listen to these events to create/destroy the data directly in the node implementations rather than relying on external logic in the owner to somehow deal with it.

New API Documentation Format

We have improved the documentation format generated by the docgen tool to look better and be more easily navigated:

New API documentation format.

New API documentation format.

Have a look at doc/apidoc.md.html in the beta package to see the changes. Note that it’s a pretty hefty web page, so it takes a while to load and generate the Markdeep documentation. We are considering breaking it up into smaller parts for a future release.

In addition to the format changes, docgen also has a new flag. If you run docgen -e FILE, we will generate documentation for that specific file and open it immediately in your web browser. This is a nice way of quickly checking the documentation of a single file.

UI

  • Fixed a crash when deleting multiple entities in the Entity Tree. [#283]

  • Triple clicking the mouse in a text box now selects all the content.

  • When the core folder is updated, the user is notified with a message box and a log entry details which entries were updated.

  • Graph nodes are now created with a width that is guaranteed to fit their title.

  • Fixed a bug where the graph view jumps the first time the user pans.

  • Nodes in the graph no longer receive hover highlight when another window blocks the cursor.

  • Full project path is shown in title bar. [#288]

  • In the Asset Browser, you can hover over a file or folder to see its full path (in directory projects).

  • You can also show the file in explorer using the context menu.

  • The Console now has a context menu

  • It is now possible to navigate Animation State Machine Trees using the arrow keys [#287].

Runner and simulate tab

  • The Simulate tab now has its own tm_ui_o and tm_ui_renderer_o objects for rendering in-game UI, instead of piggybacking on the editor’s instances of those objects.

  • The Runner now has tm_ui_o and tm_ui_renderer_o objects for rendering in-game UI.

Samples

  • The First Person and Third Person samples now come with example UI code, mouse capturing and the possibility to quit by pressing escape

  • Fixed a bug that allowed camera movement in the Animation sample even if the simulation was paused.

HTTPS plugin

  • Added tm_https_api.get_to_file().

  • The https library now follows redirects (status code 301 and 302) automatically.

  • tm_https_api.get() now provides progress reports and offers an opportunity to cancel the download.

Build system

  • Fix for new empty project containing dirty assets. [#278]

  • Additional tmbuild options for more detailed control: --no-unit-test, --no-premake, --quiet, --install-build-tools, --gen-hash, --gen-doc, --gen-nodes, --clean, --``just-``clean, -``-project etc. Run tmbuild -h for more information.

  • The output from trim-includes.exe is now less noisy.

Creation Graphs

  • Fix for issue with migration to new core creation graphs leaving some graph instances with broken inputs. [#284]

  • The underlying system for caching the result of nodes in the Creation Graph has been generalized and improved. Every graph interface now has a subobject set called tm_graph_cached_node_result.

  • A Compress Image (Crunch) node has been added to the Creation Graph. It compresses textures using the Crunch library.

  • Fixed a bug in migration that broke some Creation Graph Inputs.

  • The Creation Graph system has gone through an in-depth rewrite to support instancing, the graph interpreter and compile step are now separated from the data it operates on (i.e the wire and variable data).

  • Two new events have been added to Creation Graphs:

    • TM_CREATION_GRAPH_EVENT_CREATE_INSTANCE — called when an instance is created.
    • TM_CREATION_GRAPH_EVENT_DESTROY_INSTANCE — called when an instance is destroyed.

  • The Compress Image (Crunch), GPU Image->Image, and all shader output nodes now have a refresh button for forcing a refresh of their cached node results.

  • Buffer data for GPU Simulation is now unique per entity and entity context.

Shaders

  • lit and unlit output nodes now support tessellation with displacement. The displacement vector is specified in world space.

  • lit and unlit output nodes now correctly work with emissive even when Surface Type is Opaque.

  • display_condition on a shader node setting has been renamed to condition. Any action defined referenced by the setting won’t take effect unless the condition is met.

  • User imported system semantics (import_system_semantics in .shader files) are now declared outside of the stage input struct.

Visual Scripting Language

  • It’s now possible to specify the node category directly in the C source code: it will get parsed by the generate-graph-nodes executable and automatically applied.

  • The function that registers the nodes is now automatically generated with the name generated__register_``<``filename``>``_graph_nodes by generate-graph-nodes.

Task system

  • When saving a project, The Machinery now waits for any running background tasks to finish before saving.

  • Functions for flagging tasks as canceled have been added. This flag is now checked in a number of tasks. When the user clicks Don``'``t Save in the save dialog, all background tasks are canceled.

2020.9.a

  • Fix for crash when adding components using the properties tab. [#298]

  • Fix for crash when deleting scene tree component. [#299]

  • Fix for UTF-8 error when using tmbuild --project.

  • Fix for context menu not working in Console tab. [#311]

  • Fix for not being able to expand multiline comments in Console tab. [#311]

by The Machinery Team