glTF to USDZ research

Research and proof of concept of converting glTF to USDZ for AR Quick Look (iOS 12+).

Reasoning

Even though I think the intentions of Apple / Pixar are great with the open source USD pipeline I think we as an industry should be relying more on truly open formats that are not controlled by a single entity. Installing USD is cumbersome, requires a lot of disk space and is completely overkill for most situations (if your goal is to convert some 3D models to USDZ and show them using AR Quick Look). I'm hoping that as we get closer to the public release of iOS 12 more tools will pop up that directly convert to USDZ so that we can get cross-platform support.

In order to move away from using the USD pipeline solution offered by Pixar I think it would be wise to try and manipulate the intermediary readeable USDA format. Unfortunately there are very little examples available of USDA files.

The general idea is to dynamically generate / manipulate the intermediary a general USDA file-structure and pass that to the usdz-converter to handle the further conversion to USDZ.

After some work, debugging and friendly help from @domenicopanacea and @virakri I got it to work. Most of the findings on OBJ support come from @trayser who posted details regarding OBJ to USDZ conversion on developers.apple.com. The OBJ files that are generated by gltf-to-usdz only use the supported tags as mentioned by @trayser.

Please note that this is just an experimental setup and should be seen as a proof of concept. As it stands I've only tested it with the DamagedHelmet glTF asset included in /assets/. You will very likely run into issues with the current code.

My goal of this research is to inspire others to make proper converters that handle all the complexities.

Live demo

Live demo

Limitations

• .glb files are not yet accepted. Please use glTF files with seperate textures.

• Only the first mesh in the glTF file will be parsed and exported. Make sure you merge all meshes.

• Only glTF files with external textures are currently handled (as opposed to embedded base64 encoded textures).

• Animations from the glTF are not yet transferred, it seems possible to transfer skeleton animation and regular transformations. Unfortunately due to the shadow being baked on the first frame of the animation you can't really do large moving animations. Besides that the filesize increases quite dramatically. It is not recommended to have animations that move a character away from the origin as it results in tracking problems. Your best bet are micro-animations that enhance a character or model instead of large animations.

To do

• Convert the example USDZ examples to USDA structures by converting USDC to USDA. Unfortunately I think this requires the installation of the USD pipeline and the use of usdcat.

  Thanks to @wave-electron the USDC's of the examples of the AR Quick Look gallery have been converted to USDA. Due to copyright concerns I unfortunately won't be able to upload them to this repo. I've had a look at them and for the most part look very similar compared to the USDA file generated by gltf-to-usdz. I've since also been able to convert the examples myself using usdcat.

Installation

• Make sure you have Node.js installed

• Upgrade your operating system to macOS High Sierra 10.13.4 or newer

• Download Xcode 10 beta and put it in /Applications/

https://developer.apple.com/download/

• Link to the beta version instead of the normal version

sudo xcode-select --switch /Applications/Xcode-beta.app

• Construct a new .usda using gltf-to-usdz and run it through the usdz_converter

node ./bin/gltf-to-usdz.js -i ./assets/DamagedHelmet/DamagedHelmet.gltf -o ./assets/DamagedHelmet.usda -s 10.0 && xcrun usdz_converter ./assets/DamagedHelmet.usda ./assets/DamagedHelmet-`date +"%H-%M-%S"`.usdz

• On succes the following should be outputted to the console

2018-06-20 17:21:23.364 usdz_converter[82749:13335731]


Converting asset file 'DamagedHelmet.usda' ...

In order to see the contents of the outputted USDZ change the extension to .zip and unzip it. You will see a .usdc and several textures (if the original glTF file had textures).

Development

In order to install USD on MacOS please follow the following instructions:

• Upgrade your operating system to macOS High Sierra 10.13.4 or newer

• Download Xcode 10 beta and put it in /Applications/

https://developer.apple.com/download/

• Link to the beta version instead of the normal version

sudo xcode-select --switch /Applications/Xcode-beta.app

• Install Cmake and QT. If you get any issues during compiling or installing you might have to install QT5.

brew install cmake
brew install qt@4

• Install PyOpenGL

pip install PyOpenGL

• Install PySide2

pip install --index-url=http://download.qt.io/snapshots/ci/pyside/5.9/latest/ pyside2 --trusted-host download.qt.io

• Update OpenImageIO release version from Release-1.7.14.zip to Release-1.8.12.zip and add the following section in build_scripts/build_usd.py as mentioned as a solution by @robpieke to an earlier Python crash I was experiencing.

diff --git a/build_scripts/build_usd.py b/build_scripts/build_usd.py
index 37fb2a5e..3c940b3b 100644
--- a/build_scripts/build_usd.py
+++ b/build_scripts/build_usd.py
@@ -692,7 +692,7 @@ PTEX = Dependency("Ptex", InstallPtex, "include/PtexVersion.h")
 ############################################################
 # OpenImageIO

-OIIO_URL = "https://github.com/OpenImageIO/oiio/archive/Release-1.7.14.zip"
+OIIO_URL = "https://github.com/OpenImageIO/oiio/archive/Release-1.8.12.zip"

 def InstallOpenImageIO(context, force):
     with CurrentWorkingDirectory(DownloadURL(OIIO_URL, context, force)):
@@ -855,6 +855,12 @@ def InstallUSD(context):

         if context.buildPython:
             extraArgs.append('-DPXR_ENABLE_PYTHON_SUPPORT=ON')
+            if MacOS():
+                import distutils.sysconfig
+                pyLibPath = distutils.sysconfig.get_config_var('LIBDIR')
+                pyIncPath = distutils.sysconfig.get_config_var('INCLUDEPY')
+                extraArgs.append('-DPYTHON_LIBRARY=' + pyLibPath + '/libpython2.7.dylib')
+                extraArgs.append('-DPYTHON_INCLUDE_DIR=' + pyIncPath)
         else:
             extraArgs.append('-DPXR_ENABLE_PYTHON_SUPPORT=OFF')

• Run python USD/build_scripts/build_usd.py BUILD, it will take roughly 1 hour the first time you build because of all the dependencies that need to be compiled, resulting in the following output if succesfully installed:

➜  pixar python USD/build_scripts/build_usd.py BUILD

Building with settings:
  USD source directory          /Users/timvanscherpenzeel/Projects/pixar/USD
  USD install directory         /Users/timvanscherpenzeel/Projects/pixar/BUILD
  3rd-party source directory    /Users/timvanscherpenzeel/Projects/pixar/BUILD/src
  3rd-party install directory   /Users/timvanscherpenzeel/Projects/pixar/BUILD
  Build directory               /Users/timvanscherpenzeel/Projects/pixar/BUILD/build
  CMake generator               Default
  Downloader                    curl

  Building                      Shared libraries
    Imaging                     On
      Ptex support:             Off
    UsdImaging                  On
    Python support              On
    Documentation               Off
    Tests                       Off
    Alembic Plugin              Off
      HDF5 support:             Off
    Maya Plugin                 Off
    Katana Plugin               Off
    Houdini Plugin              Off

    Dependencies                zlib, boost, TBB, JPEG, TIFF, PNG, OpenEXR, GLEW, OpenImageIO, OpenSubdiv

STATUS: Installing zlib...
STATUS: Installing boost...
STATUS: Installing TBB...
STATUS: Installing JPEG...
STATUS: Installing TIFF...
STATUS: Installing PNG...
STATUS: Installing OpenEXR...
STATUS: Installing GLEW...
STATUS: Installing OpenImageIO...
STATUS: Installing OpenSubdiv...
STATUS: Installing USD...

Success! To use USD, please ensure that you have:

    The following in your PYTHONPATH environment variable:
    /Users/timvanscherpenzeel/Projects/pixar/BUILD/lib/python

    The following in your PATH environment variable:
    /Users/timvanscherpenzeel/Projects/pixar/BUILD/bin

Taking apart a USDZ file

To take apart a USDZ file one should follow the following steps:

1. Follow the steps mentioned in Development to install USD
2. Change the extension of the usdz file from .usdz to .zip
3. Unzip the file using Archive Utility (comes with MacOS) or likely any other unzipping tool

The folder will contain the included textures seperated out and a .udsc file.

4. Now use usdcat to convert the USDC file to readeable USDA: usdcat input.usdc -o output.usda

The resulting USDA file will contain the mesh data.

Resources

• https://graphics.pixar.com/usd/docs/Converting-Between-Layer-Formats.html

• https://forums.developer.apple.com/thread/104042

• https://graphics.pixar.com/usd/docs/UsdPreviewSurface-Proposal.html

• https://developer.apple.com/videos/play/wwdc2018/603/

• https://devstreaming-cdn.apple.com/videos/wwdc/2018/603augiuv41xoowslk8/603/603_integrating_apps_and_content_with_ar_quick_look.pdf?dl=1

Contributing

As it is my goal to create a community project I welcome you to submit any pull requests or open a ticket if you spot an issue. Please be aware that as of now the project is in a very experimental state and only serves as a proof of concept so there are probably lots of things wrong and the code is far from clean.

Licence

gltf-to-usdz is released under the MIT licence.