Skip to content

Commit

Permalink
Refactor docs to match new resolvers
Browse files Browse the repository at this point in the history
  • Loading branch information
LucaScheller committed Oct 28, 2023
1 parent 92574cd commit 4f749b7
Show file tree
Hide file tree
Showing 15 changed files with 89 additions and 71 deletions.
17 changes: 10 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,25 +10,28 @@ To build the various resolvers, follow the instructions in the [install guide](h
> This guide currently covers compiling against Houdini on Linux and Windows. Alternatively you can also download a pre-compiled builds on our [release page](https://github.com/LucaScheller/VFX-UsdAssetResolver/releases). To load the resolver, you must specify a few environment variables, see our [environment variables](https://lucascheller.github.io/VFX-UsdAssetResolver/resolvers/overview.html#environment-variables) section for more details.
> [!IMPORTANT]
> We also offer a quick install method for Houdini via our "Update Manager" that does the download of the compiled resolvers and environment variable setup for you. This is ideal if you want to get your hands dirty right away and you don't have any C++ knowledge or extensive USD developer knowledge. If you are a small studio, you can jump right in and play around with our "PythonResolver"/"CachedResolver" resolvers and prototype around to make it fit your production needs.
> We also offer a quick install method for Houdini that does the download of the compiled resolvers and environment variable setup for you. This is ideal if you want to get your hands dirty right away and you don't have any C++ knowledge or extensive USD developer knowledge. If you are a small studio, you can jump right in and play around with our resolvers and prototype them further to make it fit your production needs.
To run the "Update Manager" simply run this snippet in the "Houdini Python Source Editor" panel:

> [!IMPORTANT]
import urllib,ssl
update_manager_url = 'https://raw.githubusercontent.com/LucaScheller/VFX-UsdAssetResolver/main/tools/update_manager.py'
exec(urllib.request.urlopen(update_manager_url,context=ssl._create_unverified_context()).read(), globals(), locals())
run_houdini()

See our [Automatic Installation](https://lucascheller.github.io/VFX-UsdAssetResolver/installation/automatic_install.html) section for more information.


## Feature Overview
Asset resolvers that can be compiled via this repository:
- **File Resolver** - A file system based resolver similar to the default resolver with support for custom mapping pairs as well as at runtime modification and refreshing.
- **Python Resolver** - Python based implementation of the file resolver. The goal of this resolver is to enable easier RnD by running all resolver and resolver context related methods in Python. It can be used to quickly inspect resolve calls and to setup prototypes of resolvers that can then later be re-written in C++ as it is easier to code database interactions in Python for initial research.
- **Cached Resolver** - Still work in progress, more info soon.
- **Http Resolver** - A proof of concept http resolver. This is kindly provided and maintained by @charlesfleche in the [arHttp: Offloads USD asset resolution to an HTTP server
](https://github.com/charlesfleche/arHttp) repository. For documentation, feature suggestions and bug reports, please file a ticket there. This repo handles the auto-compilation against DCCs and exposing to the automatic installation update manager UI.
- **Production Resolvers**
- **File Resolver** - A file system based resolver similar to the default resolver with support for custom mapping pairs as well as at runtime modification and refreshing.
- **Cached Resolver** - Still work in progress, more info coming soon.
- **RnD Resolvers**
- **Python Resolver** - Python based implementation of the file resolver. The goal of this resolver is to enable easier RnD by running all resolver and resolver context related methods in Python. It can be used to quickly inspect resolve calls and to setup prototypes of resolvers that can then later be re-written in C++ as it is easier to code database interactions in Python for initial research.
- **Proof Of Concept Resolvers**
- **Http Resolver** - A proof of concept http resolver. This is kindly provided and maintained by @charlesfleche in the [arHttp: Offloads USD asset resolution to an HTTP server
](https://github.com/charlesfleche/arHttp) repository. For documentation, feature suggestions and bug reports, please file a ticket there. This repo handles the auto-compilation against DCCs and exposing to the automatic installation update manager UI.

For more information check out the [building guide](https://lucascheller.github.io/VFX-UsdAssetResolver/installation/building.html) as well as the [individual resolvers](https://lucascheller.github.io/VFX-UsdAssetResolver/resolvers/overview.html) to see their full functionality.

Expand Down
6 changes: 3 additions & 3 deletions docs/src/SUMMARY.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,12 +11,12 @@
- [Requirements](./installation/requirements.md)
- [Building/Deployment](./installation/building.md)
- [Resolvers](./resolvers/overview.md)
- [Production Resolvers]()
- [Production Resolvers](./resolvers/production.md)
- [File Resolver](./resolvers/FileResolver/overview.md)
- [Python API](./resolvers/FileResolver/PythonAPI.md)
- [RnD Resolvers]()
- [RnD Resolvers](./resolvers/rnd.md)
- [Python Resolver](./resolvers/PythonResolver/overview.md)
- [Python API](./resolvers/PythonResolver/PythonAPI.md)
- [Proof Of Concept Resolvers]()
- [Proof Of Concept Resolvers](./resolvers/proofofconcept.md)
- [Http Resolver ('arHttp' by @charlesfleche)](./resolvers/HttpResolver/overview.md)
- [Example Setup](./resolvers/ExampleSetup/overview.md)
4 changes: 2 additions & 2 deletions docs/src/installation/automatic_install.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Automatic/Quick Installation

```admonish tip
We also offer a quick install method that does the download of the compiled resolvers and environment variable setup for you. This is ideal if you want to get your hands dirty right away and you don't have any C++ knowledge or extensive USD developer knowledge. If you are a small studio, you can jump right in and play around with our "PythonResolver"/"CachedResolver" resolvers and prototype around to make it fit your production needs.
We offer a quick install method that does the download of the compiled resolvers and environment variable setup for you. This is ideal if you want to get your hands dirty right away and you don't have any C++ knowledge or extensive USD developer knowledge. If you are a small studio, you can jump right in and play around with our resolvers and prototype them further to make it fit your production needs.
```

Currently we only support doing this in Houdini.
Expand All @@ -19,7 +19,7 @@ As mentioned in our [Resolvers Plugin Configuration](../resolvers/overview.md#us
In your install directory you will find a "launch.sh/.bat" file, which does this for you based on what host app you ran the installer in.
All you then have to do is run the "launch.sh/.bat" file by double clicking it and then your app should open as usual with the resolver running. In the launch file we have enabled the "TF_DEBUG=AR_RESOLVER_INIT" environment variable, so there will be debug logs where you can see if everything worked correctly.

![Install folder and launcher](./media/AutomaticInstallFolder.jpg)
![Install folder and launcher](./media/AutomaticInstallFolder.png)

## Houdini
In Houdini we simply need to open the "Python Source Editor" from the "Windows" menu and run the following code to get access to the update manager. You should preferably do this in a clean Houdini session as a safety measure.
Expand Down
15 changes: 4 additions & 11 deletions docs/src/installation/building.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ In the [setup.sh](https://github.com/LucaScheller/VFX-UsdAssetResolver/blob/main

It will then automatically set the `PATH`, `PYTHONPATH`, `PXR_PLUGINPATH_NAME` and `LD_LIBRARY_PATH` environment variables to the correct paths so that after your run the compile, the resolver will be loaded correctly (e.g. if you launch Houdini via `houdinifx`, it will load everything correctly). The build process also logs this information again.

By default it also sets the `TF_DEBUG` env var to `<ResolverName>_RESOLVER` so that you'll get debug logs of what the resolver is doing. Check out the debug codes section of the resolvers for more information.
By default it also sets the `TF_DEBUG` env var to `AR_RESOLVER_INIT` so that you'll get logs of what resolver is loaded by USD's plugin system, which you can use to verify that everything is working correctly.

### Manually setting up the environment
If you don't want to use our convenience script, you can also setup the environment manually.
Expand Down Expand Up @@ -48,8 +48,9 @@ build.bat
The `build.sh/.bat` files also contain (commented out) the environment definition part above, so alternatively just comment out the lines and you are good to go.

## Testing the build
Currently we don't include tests as part of the build process yet, so to test if everything worked, you'll have to run Houdini.
If you didn't use our convenience script as noted above, you'll have to specify a few environment variables, so that our plugin is correctly detected by USD.
Unit tests are automatically run post-build on Linux using the Houdini version you are using. You can find each resolvers tests in its respective src/<ResolverName>/testenv folder.

Alternatively you can run Houdini and check if the resolver executes correctly. If you didn't use our convenience script as noted above, you'll have to specify a few environment variables, so that our plugin is correctly detected by USD.

Head over to our [Resolvers > Environment Variables](../resolvers/overview.md#environment-variables) section on how to do this.

Expand All @@ -59,14 +60,6 @@ After that everything should run smoothly, you can try loading the examples in t
If you want to further configure the build, you can head into the [CMakeLists.txt](https://github.com/LucaScheller/VFX-UsdAssetResolver/blob/main/CMakeLists.txt) in the root of this repo. In the first section of the file, you can configure various things, like the environment variables that the resolvers use, Python module namespaces and what resolvers to compile.
This is a standard `CMakeLists.txt` file that you can also configure via [CMake-GUI](https://cmake.org/cmake/help/latest/manual/cmake-gui.1.html). If you don't want to use the `build.sh` bash script, you can also configure and compile this project like any other C++ project via this file.

If you want to change the resolver names, you'll have to additionally edit the `CMakeLists.txt` files in the corresponding resolver folder by un-commenting the following line in the beginning of the file:
~~~admonish info title=""
```cmake
add_compile_definitions(<ResolverName>=${AR_<RESOLVERNAME>_USD_CXX_CLASS_NAME})
```
~~~
This is considered experimental and is not guaranteed to work. It is easier to just rename all Python/C++ class via a quick search & replace.

# Documentation
If you want to locally build this documentation, you'll have to download [mdBook](https://github.com/rust-lang/mdBook) and [mdBook-admonish](https://github.com/tommilligan/mdbook-admonish) and add their parent directories to the `PATH`env variable so that the executables are found.

Expand Down
1 change: 1 addition & 0 deletions docs/src/installation/overview.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
# Installation
- [Automatic Installation](./automatic_install.md)
- [System Requirements](./requirements.md)
- [Building/Deployment](./building.md)
9 changes: 1 addition & 8 deletions docs/src/overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,14 +16,7 @@ We also offer a quick install method for Houdini that does the download of the c

## Feature Overview
Asset resolvers that can be compiled via this repository:
- **Production Resolvers**
- **File Resolver** - A file system based resolver similar to the default resolver with support for custom mapping pairs as well as at runtime modification and refreshing.
- **Cached Resolver** - Still work in progress, more info coming soon.
- **RnD Resolvers**
- **Python Resolver** - Python based implementation of the file resolver. The goal of this resolver is to enable easier RnD by running all resolver and resolver context related methods in Python. It can be used to quickly inspect resolve calls and to setup prototypes of resolvers that can then later be re-written in C++ as it is easier to code database interactions in Python for initial research.
- **Proof Of Concept Resolvers**
- **Http Resolver** - A proof of concept http resolver. This is kindly provided and maintained by @charlesfleche in the [arHttp: Offloads USD asset resolution to an HTTP server
](https://github.com/charlesfleche/arHttp) repository. For documentation, feature suggestions and bug reports, please file a ticket there. This repo handles the auto-compilation against DCCs and exposing to the automatic installation update manager UI.
{{#include ./resolvers/shared_features.md:resolverOverview}}

For more information check out the [building guide](./installation/building.md) as well as the [individual resolvers](./resolvers/overview.md) to see their full functionality.

Expand Down
5 changes: 5 additions & 0 deletions docs/src/resolvers/ExampleSetup/overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,11 @@
## Simple Example

A very simple setup can be found in In the <REPO_ROOT>/files folder.

```admonish info
These examples currently work with the [File Resolver](../FileResolver/overview.md) and [Python Resolver](../PythonResolver/overview.md)
```

Before launching a Usd related software, you'll have to set these env vars:
```bash
export AR_SEARCH_PATHS=${REPO_ROOT}/files
Expand Down
6 changes: 4 additions & 2 deletions docs/src/resolvers/FileResolver/overview.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,11 @@
# File Resolver
## Overview
A file system based resolver similar to the default resolver with support for custom mapping pairs.
- It implements the features listed in the [resolvers overview section](../overview.md).
This resolver is a file system based resolver similar to the default resolver with support for custom mapping pairs.
{{#include ../shared_features.md:resolverSharedFeatures}}
- You can adjust the resolver context content during runtime via exposed Python methods (More info [here](./PythonAPI.md)). Refreshing the stage is also supported, although it might be required to trigger additional reloads in certain DCCs.

{{#include ../shared_features.md:resolverEnvConfiguration}}

## Debug Codes
Adding following tokens to the `TF_DEBUG` env variable will log resolver information about resolution/the context respectively.
* `FILERESOLVER_RESOLVER`
Expand Down
6 changes: 3 additions & 3 deletions docs/src/resolvers/PythonResolver/overview.md
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@

# Python Resolver
## Overview

Python based implemention of the file resolver:
- It implements the features listed in the [resolvers overview section](../overview.md).
Python based implemention of the [file resolver](../FileResolver/overview.md):
{{#include ../shared_features.md:resolverSharedFeatures}}
- This resolver has feature parity to the file resolver, but the implementation is slightly different. The goal of this resolver is to enable easier RnD by running all resolver and resolver context related methods in Python. It can be used to quickly inspect resolve calls and to setup prototypes of resolvers that can then later be re-written in C++ as it is easier to code database related pipelines in Python.
- Running in Python does not allow proper multithreading due to Python's [Global Interpreter Lock](https://wiki.python.org/moin/GlobalInterpreterLock), so this resolver should not be used in (large scale) productions.

{{#include ../shared_features.md:resolverEnvConfiguration}}

## Debug Codes
Adding following tokens to the `TF_DEBUG` env variable will log resolver information about resolution/the context respectively.
Expand Down
38 changes: 5 additions & 33 deletions docs/src/resolvers/overview.md
Original file line number Diff line number Diff line change
@@ -1,22 +1,11 @@
# Overview
All resolvers share these common features:
- A simple mapping pair look up in a provided mapping pair Usd file. The mapping data has to stored in the Usd layer metadata in an key called ```mappingPairs``` as an array with the syntax ```["sourcePathA.usd", "targetPathA.usd", "sourcePathB.usd", "targetPathB.usd"]```. (This is quite similar to Rodeo's asset resolver that can be found [here](https://github.com/rodeofx/rdo_replace_resolver) using the AR 1.0 specification.)
- The search path environment variable by default is ```AR_SEARCH_PATHS```. It can be customized in the [CMakeLists.txt](https://github.com/LucaScheller/VFX-UsdAssetResolver/blob/main/CMakeLists.txt) file.
- You can use the ```AR_ENV_SEARCH_REGEX_EXPRESSION```/```AR_ENV_SEARCH_REGEX_FORMAT``` environment variables to preformat any asset paths before they looked up in the ```mappingPairs```. The regex match found by the ```AR_ENV_SEARCH_REGEX_EXPRESSION``` environment variable will be replaced by the content of the ```AR_ENV_SEARCH_REGEX_FORMAT``` environment variable. The environment variable names can be customized in the [CMakeLists.txt](https://github.com/LucaScheller/VFX-UsdAssetResolver/blob/main/CMakeLists.txt) file.
- The resolver contexts are cached globally, so that DCCs, that try to spawn a new context based on the same pinning file using the [```Resolver.CreateDefaultContextForAsset```](https://openusd.org/dev/api/class_ar_resolver.html), will re-use the same cached resolver context. The resolver context cache key is currently the pinning file path. This may be subject to change, as a hash might be a good alternative, as it could also cover non file based edits via the exposed Python resolver API.
- ```Resolver.CreateContextFromString```/```Resolver.CreateContextFromStrings``` is not implemented due to many DCCs not making use of it yet. As we expose the ability to edit the context at runtime, this is also often not necessary. If needed please create a request by submitting an issue here: [Create New Issue](https://github.com/LucaScheller/VFX-UsdAssetResolver/issues/new)

# Resolvers
- [File Resolver](./FileResolver/overview.md)
- [Python Resolver](./PythonResolver/overview.md)

# Environment Variables
Asset resolvers that can be compiled via this repository:
{{#include ./shared_features.md:resolverOverview}}

## USD Plugin Configuration
In order for our plugin to be found by USD, we have to specify a few environment variables.
Run this in your terminal before running your USD capable app. If your are using a pre-compiled release build, redirect the paths accordingly.


~~~admonish tip
If you are using our quick install method, this will already have been done for you via the "launch.sh/.bat" file in the directory where you downloaded the compiled release to. See our [Automatic Installation](../installation/automatic_install.md) section for more information.
~~~
Expand All @@ -43,25 +32,8 @@ ArGetResolver(): Found primary asset resolver types: [FileResolver, ArDefaultRes
```
~~~

## Resolver Configuration

- `AR_SEARCH_PATHS`: The search path for non absolute asset paths.
- `AR_SEARCH_REGEX_EXPRESSION`: The regex to preformat asset paths before mapping them via the mapping pairs.
- `AR_SEARCH_REGEX_FORMAT`: The string to replace with what was found by the regex expression.

The resolver uses these env vars to resolve non absolute asset paths relative to the directories specified by `AR_SEARCH_PATHS`. For example the following substitutes any occurrence of `v<3digits>` with `v000` and then looks up that asset path in the mapping pairs.

~~~admonish info title=""
```bash
export AR_SEARCH_PATHS="/workspace/shots:/workspace/assets"
export AR_SEARCH_REGEX_EXPRESSION="(v\d\d\d)"
export AR_SEARCH_REGEX_FORMAT="v000"
```
~~~

# Debugging

## Using the `TF_DEBUG` environment variable
## Debugging
### By using the `TF_DEBUG` environment variable
To check what resolver has been loaded, you can set the `TF_DEBUG` env variable to `AR_RESOLVER_INIT`:
~~~admonish info title=""
```bash
Expand All @@ -85,7 +57,7 @@ ArGetResolver(): Using package resolver Usd_UsdzResolver for usdz from plugin us
```
~~~

## Loading the Python Module
### By loading the Python Module
When importing the Python module, be sure to first import the Ar module, otherwise you might run into errors, as the resolver is not properly initialized:
~~~admonish info title=""
```bash
Expand Down
4 changes: 4 additions & 0 deletions docs/src/resolvers/production.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
# Production Resolvers
Here you can find production ready asset resolvers, checkout our [Resolvers Overview](../resolvers/overview.md) section for an outline of their features:
- [File Resolver](./FileResolver/overview.md)
- [Cached Resolver]()
Loading

0 comments on commit 4f749b7

Please sign in to comment.