A Better Maya to Unity Workflow

In the past, I’ve implemented tools and fixes to Unity’s import pipeline of native Maya files by manually modifying the FBXMayaExport.mel script. Unfortunately, these types of modifications are difficult to maintain. For one, they need to be redone every time the Unity installation is upgraded. Second, the files are buried in obscure places that require administrative privileges to modify them. It would be ideal if changes could be made in one place, one time, in a way that supported more modular design. So, I finally decided to take the time to sit down and tackle this problem.

Unity’s Export Process

Unity uses the FBX SDK to import complex models. As such, one popular way to get models into projects is to simply export FBX files from some DCC application (e.g., Maya, 3D Studio Max). Although this approach has some advantages, the manual exportation step means that two versions of the file are kept in separate places. Files can become out of sync, and (depending on your tools) this workflow can introduce the possibility of human error in the configuration of export settings or location of files. Another option is to drop files in the DCC application’s native format (e.g., .ma, .mb, .max) into the project. Depending on the DCC application, the Unity editor spawns a headless child process that executes some script to automatically convert the native file into a temporary FBX that is imported into the project.*

In the case of Maya, there are some template scripts in Unity’s install location (e.g., FBXMayaMain.mel, FBXMayaExport.mel). When your Unity project imports a Maya file, it duplicates these scripts into your project’s temp folder, modifies some file paths inside of them, and then launches Maya as a child process, passing a command line argument that sources the duplicated FBXMayaMain.mel when Maya launches. This script ensures the FBX plug-ins are loaded, and then sources the duplicated FBXMayaExport.mel script, which handles the FBX export.

In the past, this latter approach was less attractive to larger teams. For one, each file had to be reimported by every user, unless the team were using the Unity Asset Server (which cached the imported data in Unity’s native asset format). If you were using something like Perforce, however, each user needed to have Maya installed in order to import the models, and the process could be slow for very large projects with hundreds or more Maya files. The introduction of Unity’s Cache Server, however, makes the direct workflow potentially more appealing. The Cache Server is compatible with any VCS, and can cache the imported data when they are committed, just like the Asset Server workflow.

A Solution

Although in past work, I have often incorporated a combination of direct modifications to the FBXMayaExport.mel template, it can be a hassle to maintain, for the reasons I stated at the start of this post. As such, I had a few design goals for this exercise:

  1. I did not want to have to make any modifications to the template files in the Unity install
  2. I wanted an automated process that could be handled from a studio-wide userSetup.py script
  3. I wanted any users to be able to easily register their own modifications in a modular way

Taking these considerations in mind, I basically wanted a script that, when imported in userSetup.py, would know if it were being launched in a child process of the Unity editor, and would register callbacks using the messaging system built into Maya’s API if so.

Determining whether the Maya instance is a child of the Unity editor is as simple as reading the command-line arguments. Specifically, we look for the -script flag and see what the argument’s value is:

import os.path
import sys
try:
    startup_script = sys.argv[sys.argv.index('-script') + 1]
except Exception:
    startup_script = None
if startup_script is not None:
    directory, script = os.path.split(startup_script)
    is_maya_child_of_unity = (
        os.path.basename(directory) == 'Temp' and
        script = 'FBXMayaMain.mel'
    )
else:
    is_maya_child_of_unity = False

The trick at this point is registering callbacks. The FBXMayaExport.mel script uses the FBXExport command. Why wouldn’t it? Autodesk explicitly says you should, as opposed to using the file command. Unfortunately, the FBXExport command does not presently broadcast messages that you can hook into with the MSceneMessage class.

Knowing that Unity makes copies of its mel scripts in a writable location, however, opens up a really simple possibility. Namely, if we detect that the Maya instance is owned by Unity using the command-line arguments, we can also find the duplicated FBXMayaExport.mel script in the same location as FBXMayaMain.mel. Moreover, because it is writable, we can make a simple modification to use the file command.

# continued from above
import re
if is_maya_child_of_unity:
    path_to_fbx_export_script = os.path.join(
        directory, 'FBXMayaExport.mel'
    )
    with open(path_to_fbx_export_script) as f:
        contents = f.read()
    contents = re.sub(
        'FBXExport -f ',
        'file -force -type "FBX export" -exportAll ',
        contents
    )
    with open(path_to_fbx_export_script, 'w+') as f:
        f.write(contents)

Now, after this code has executed, if we know that the Maya instance is a child of Unity, we can use MSceneMessage.addCallback() using the MSceneMessage.kBeforeExport message type.

If you’re interested in easily dropping a solution like this into your pipeline, I have put up a simple Python package on github. In order to use it, all you have to do is import the unityexport package in your studio-wide userSetup.py script. You can then read its unity_project_version attribute to determine if you are running as a child of the Unity editor (and what version the project is) in order to register callbacks with the MSceneMessage class. As an example, the package also registers a callback to automatically adjust the FBX export settings for blend shapes, as per my previous post.

*I don’t work on Windows, but in the couple of tests I have done, it seems like Unity 4.3.x currently hangs on exit if it has an active Maya child process. There have been some assorted vague reports of similar behavior for Max.

One thought on “A Better Maya to Unity Workflow”

Leave a Reply

Your email address will not be published. Required fields are marked *