The Machinery Beta — March 2021 (version 2021.3)

Happy anniversary, The Machinery!

It’s hard to believe it’s been a full year since our first beta release. With the global pandemic, it sometimes feels like time is standing still, and sometimes like it’s racing by. We’re happy to see that vaccines are rolling out and can’t wait to see you all in person once we’ve beat this thing. Stay strong, this is the home stretch! In the meantime, hope you like what we’ve been cooking up in our home offices.

If you are already running The Machinery, this new release should pop up in your Downloads tab. Otherwise, get it from our beta download page and have a look at this quick introduction video. If you find bugs in this release or have requests for specific features, post them to our issue tracker. For more general discussions, hit us up on the forum or Discord.

Key highlights in this release are:

• Math Expressions — Expressions are now supported in graph nodes and property text boxes.
• Tab Overlays — Floating overlays on top of the tabs.
• Animation Compression — Compressed animations are now supported.
• Atmospheric Sky — An atmospheric sky rendering component.
• Box Select in the Scene Tab — You can now drag to select multiple entities.

As always, we also have a number of smaller updates.

Math Expressions

One of the things that can be frustrating when using visual scripting languages is that simple math expressions can turn into big blobs of nodes and wires.

In the 2021.3 release, we support Math expression nodes in the graph. These allow you to simply type out the expression, instead of creating a network of nodes and wires. In the example below, the four math nodes can be replaced with a single expression node making the graph more compact and readable:

We’ve also added support for mathematical expression to our property editor. So you can now type both numerical values and expressions.

You can use x in the expression to mean whatever value the property had before, so if you type x + 1 you will increase the current value by 1.

Let us know if you can think of other cool uses for expressions!

Tab Overlays

As an extension to the previously added dockable toolbars; we now support overlays that hover on top of the tabs. Also, any toolbar can be pulled off and be made into a hovering overlay.

An overlay is just a toolbar that does not belong to any of the four toolbar containers that run along the edge of the tab. Toolbars have three rendering modes — horizontal, vertical, and widget. The widget mode is new, it is the richer, window-like mode seen in the picture above.

In the scene and simulate tabs, we’ve added:

• A rendering visualization overlay. Found in Render → Lighting Module → Show as overlay in the top right toolbar.
• A Statistics button (also top right toolbar) that makes it possible to popup statistics overlays, previously found within the Statistics tab.

As previously, the tab should return all toolbars it wishes to draw each frame, see tm_tab_vt->toolbars(). If you wish to support widget mode drawing, then make sure to set the bitmask tm_toolbar_i->draw_mode_mask to a value that contains TM_TOOLBAR_DRAW_MODE_WIDGET.

Additionally, the toolbars have been generalized. They no longer have a coupling to the docking system and the tabs, so you could use them within other contexts if you wish. See the documentation under How to use toolbars outside of The Machinery tabs in toolbar.h.

Animation Compression

We now support compressed animations. Compressed animations have the extension .animation. Note that with this, we have three kinds of animation resources:

.dcc_asset

Animation imported from a Digital Content Creation (DCC) software, such as Max, Maya, Blender, etc. Note that .dcc_asset is used for all imported content, so it could be animations, textures, models, etc.

.animation

A compressed animation. The compressed animation is generated from the .dcc_asset by explicitly compressing it.

.animation_clip

Specifies how an animation should be played: playback speed, whether it plays forward or backward, if it drives the root bone or not, etc. An .animation_clip references either an uncompressed .dcc_asset or a compressed .animation to access the actual animation data.

To create a compressed animation, right-click a .dcc_asset file that contains an animation and choose Create xxx.animation in the context menu:

When you first do this, the animation shows a white skeleton in T-pose and a moving blue skeleton. The blue skeleton is the reference .dcc_animation and the white skeleton is the compressed animation. By comparing the skeletons you can see how big the error in the animation is.

At first, the white skeleton is in T-pose because we haven’t actually generated the compressed data yet. To do that, press the Compress button:

This will update the Format and Buffer fields and we can see that we have 9.2 KB of compressed data for this animation and that the compression ratio is x 6.66. I.e, the compressed data is 6.66 times smaller than the uncompressed one. The white and the blue skeletons overlap. The compression error is too small to be noticed in this view, we have to really zoom in to see it:

When you compress an animation like this, The Machinery tries to come up with some good default compression settings. The default settings work in a lot of cases, but they’re not perfect, because The Machinery can’t know how the animation is intended to be viewed in your game.

Are you making a miniature fighting game, and all the models will be viewed from a distant overhead camera? In that case, you can get away with a lot of compression. Or are you animating a gun sight that will be held up really close to the player’s eye? In that case, a small error will be very visible.

To help the engine, you can create an .animation_compression asset. (New Animation Compression in the asset browser.) The Animation Compression asset control the settings for all the animations in the same folder or in its subfolders (unless the subfolders override with a local Animation Compression asset):

The Animation Compression settings object has two properties:

Max Error specifies the maximum allowed error in the compressed animation. The default value is 0.001 or 1 mm. This means that when we do the compression we allow bones to be off by 1 mm, but not more. The lower you set this value, the less compression you will get.

Skin Size specifies the size we assume for the character’s skin. It defaults to 0.1, or 10 cm. We need the skin size to estimate the effects of rotational errors. For example, if the rotation of a bone is off by 1°, the effect of that in mm depends on how far away from the bone the mesh is.

10 cm is a reasonable approximation for a human character, but notice that there are situations where the skin size can be significantly larger. For example, suppose that a 3 m long staff is attached to the player’s hand bone. In this case, rotational errors in the hand are amplified by the full length of the staff and can lead to really big errors in the position of the staff end. If this gives you trouble, you might want to up the skin size to 3 for animations with the staff.

We don’t support setting a per-bone skin size, because it’s unclear if the effort of specifying per-bone skin sizes is really worth it in terms of the memory savings it can give. (Also, even a per-bone skin size might not be enough to estimate errors perfectly. For example, an animator could have set up a miter joint where the extent of the skin depends on the relative angle of two bones and goes to infinity as the angle approaches zero.)

Note that sometimes animations are exported in other units than meters. In this case, the Skin Size and the Max Error should be specified in the same units that are used in the animation file.

Atmospheric Sky

We’ve added a new sample component implementing Sébastien Hillaire’s excellent “A Scalable and Production Ready Sky and Atmosphere” adapted for The Machinery.

To simplify the workflow to get up and running we now ship a sky.entity as part of the core project. In addition to the Atmospheric Sky component, it also contains a Light and a Cubemap capture component. If any settings are tweaked affecting the sky the Cubemap capture component will automatically recapture the scene and new diffuse and specular IBLs will be computed and assigned to the light.

The sun’s position is controlled by the “Sun Entity” property on the Atmospheric Sky component. Typically this should point to another entity in the scene containing a directional light representing the sunlight. The default world.entity that is automatically created in the root of any new project has everything pre-rigged.

The implementation is based on the reference shader code found at https://github.com/sebh/UnrealEngineSkyAtmosphere and released as an open-source sample together with The Machinery SDK under the same MIT license agreement.

Box Select in the Scene Tab

The Machinery now supports a long awaited feature — box selection.

To select multiple items in the scene, simply drag out a selection rectangle with the mouse:

The touched entities will become selected in the scene:

UI

• Better display of file names in tm_properties_api->ui_open_path().
• Ctrl+Left/Right now works to skip words in text edit boxes. [#499]
• Fixed layout of project loading dialog box.[#503]
• Better sequential numbering of duplicated assets. [#500]
• Cloned and duplicated entities now get sequentially numbered [#501]
• Fix for Microsoft IME keyboards not working. [#406]
• Fix for not being able to switch keyboard layouts when The Machinery is in focus. [#515]
• Improved the readability of the Profiler tab.
• Text edit boxes now consume Home and End keypresses so that they don’t scroll the view.
• The workflow for overriding inherited properties has been streamlined. Instead of sometimes using a context menu and sometimes providing a button that pops up a menu, we now always provide a context menu with the possibility of double-clicking the property in order to override it.
• Picker textboxes start filtering after typing [#514] [#519]

Plugin Assets

• Fixed an issue where dropping a DLL into a new project wouldn’t load it. [#497] [#492]
• Support for helper DLL plugin assets.
• Removed naming requirement for Asset Plugins and misleading log messages. [#483]
• The Machinery can now remember the choice of whether to Allow or Deny plugin assets to run for a specific project.
• Plugin assets are now disabled by default during collaboration. If so desired, you can enable them with the Allow Project Plugins checkbox.

Asset Browser

• Fix for occasional crash on creating a new Folder. [#517]
• Filter buttons do not disappear anymore if the asset browser is too small.
• Double-click on a folder does not select the first element in the folder [#512]

Rendering

• Added RenderDoc viewport capturing support.
• Added --gpu-debugging to enable GPU debugging tools support.
• Fixed profiling scopes not working correctly for render graph extension points at the start of a render graph.
• Render Component: Always trigger Init Event of creation graph when needing to refresh graph outputs.
• Fixed bug where contents of tm_shader_constant_buffer_instance_t wasn’t always correctly mirrored to GPU.
• Fixed bug causing staging memory to get recycled prematurely when using tm_renderer_resource_command_buffer_api->append_buffers().
• Render Component: Fixed rare crash caused by not noticing that the creation graph output needed to be refreshed.
• Added a 512x512 blue noise LUT image from http://momentsingraphics.de/BlueNoise.html exposed as a global resource accessed through the frame_system.
• Render Component: API for retrieving all draw calls generated by the component.
• Cubemap Capture Component: Support for setting the resolution of captured cubemap.
• Support for specular IBLs with an arbitrary number of mip-levels.

Runner

• Fixed an issue with the Runner not working correctly with DPI scaling on Windows. [#493]
• The runner now defaults to not hot-reloading plugins. You can still enable hot reload in the runner with --hot-reload.

Scene

• When dragging out entities from the Asset Browser into the Scene, they are now placed on top of any hovered entity already in the scene.

Creation Graph

• The Input Image Archive node now has the ability to import raw files, i.e. files with no header. The user can toggle a special Raw mode where they can provide the image description manually.
• Fixed a bug where using multiple subgraphs of the same prototype within a graph could cause node IDs to become non-unique, resulting in undefined behavior when interpreting the graph.
• Added GPU Node /Shader/Math/Floor
• GPUSim Write node now supports conditional appends using InterlockedAdd() when writing to a buffer.
• Fixed bug causing Sampler States node to sometimes fail to compile.
• Significantly optimized dependency tracking mechanism.
• Shader/Output/Lit node now supports reading transforms from an instance_transforms buffer indexed by instance_id.
• Image/Filter Image node now supports in-place generation of mipmaps by also accepting an Image input.

Graph

• Provided default values for nodes are now shown correctly in the graph editor.
• Graphs automatically show the current value of a wire if hovered over during simulation.
• Dragging an entity_graph or creation asset into a graph now spawns a subgraph node with the dragged asset as its prototype.
• Fixed a bug where subgraph nodes showed the input property editor even though the input had a connection.

Documentation

• Better docgen output when run manually on .c files.
• Code blocks are now line broken in the documentation.
• A review pass has been made over parts of the API and documentation, fixing references and improving documentation.
• Types in code samples and function headers can now be clicked to jump directly to that part of the documentation.

Foundation

• open_output() has been split into open_output() and open_append().

Utilities

• Added missing unit-test.exe from Beta distribution. [#498]
• localize.exe now patches files with missing localization tags instead of printing a report.
• tmbuild makes use of vswhere to find the location of Visual Studios[#420]

Animation

• Fixed a crash when editing an ASM Blend State without setting the Preview unit. [#495]
• When creating animation clips, we now make sure that the name is unique.
• Clicking in the animation scrubber sets its position.
• Animation bones are now sorted by depth — roots always come first in the bone list.
• Fix for animation previews being framed badly in the Preview window from invalid bounds values.

Profiler

• Fixed Profiler freeze when opening the Save dialog. [#487]
• You can now right-click on any profiler item to open up a Statistics view of that item.

Entity

• Introduced a wrapped tm_component_type_t to replace the confusing component index.

Physics

• Fixed a crash when applying forces to a kinematic rigid body. [#494]

Collaboration

• Fixed a Collaboration freeze when disconnecting from a non-existing server. [#488]

Linux

• Added gamepad support.

2021.3.a Hotfix Release

• Fix for crash when opening projects from download tab. [#524]
• Fix for bug where selection occurred after gizmo was moved, thereby selecting a different object. [#539] [#533]
• Fix for crash when destroying cubemaps. [#536] [#535] [#530] [#527]
• Fix for crash when deleting sun entity.
• Fix for crash when importing OBJ files.
• Fix for icon clipping. [#534]
• Fix for ringing artifact in rendering. [#531]

2021.3.b Hotfix Release

• Fix for bug when connecting Position node to Base Color node. [#541]
• Fix for Atmospheric Sky overwriting sun light color / intensity.
• Fix for log messages missing from log file. [#538]
• Fix for “New Plugin” code not compiling. [#545]
• Fix for hang in tmbuild trying to execute vswhere. [#525]
• Proper cleanup of cubemap images.