[for reference] Add Support for Image Planes (as a concrete schema type) #1

nxkb · 2023-09-25T22:44:21Z

Description of Change(s)

This is a full feature reference Pull Request (used in production at Luma) to help guide development of the ultimate Image Plane support in the USD core.
usd-proposal-pr
link to wg draft

Adds ImagePlane prim type, as a textured plane parented to the camera that will display in usdview to support VFX plate workflows.

implementation author: @sirpalee

I have verified that all unit tests pass with the proposed changes

I have submitted a signed Contributor License Agreement

…ured surface.

meshula · 2023-10-28T18:16:07Z

pxr/imaging/lib/hdSt/shaders/simpleTexturedSurface.glslfx

@@ -0,0 +1,25 @@
+-- glslfx version 0.1


is the introduction of imaging/lib/hdSt/shaders/... intentional?

meshula · 2023-10-28T18:18:26Z

pxr/usd/usdGeom/CMakeLists.txt

@@ -12,11 +12,13 @@ pxr_library(usdGeom
        usd
        work
        ${Boost_PYTHON_LIBRARY}
-        ${TBB_tbb_LIBRARY}
+        ${OIIO_LIBRARIES}


OpenUSD 23.11 has OpenEXR support built in, so it shouldn't be necessary to directly link OIIO or OpenEXR

exr does not support 8-16 etc formats

Sure, to be more clear about my comment, we won't be able to add a dependency on OpenImageIO to usdGeom. Loading textures is a function of the Hio library, and Hio already has a plugin to support OpenImageIO reading and writing.

meshula · 2023-10-28T18:35:53Z

pxr/usd/usdGeom/schema.usda

+        doc = "Rotation value for the image plane. Specified in degrees."
+    )
+    int2 coverage = (0, 0) (
+        doc = "Coverage of the image in pixels. 0 is equal to the image's size."


we typically use coverage to mean geometric coverage of a texel. OpenEXR uses the word window ~ perhaps windowSize and windowOrigin would work here?

I think he means protection (0.05) or padding +0.05 (same as overscan 1.05)

meshula · 2023-10-28T18:36:39Z

pxr/usd/usdGeom/schema.usda

+        doc = "Alpha Gain"
+    )
+    float depth = 100.0 (
+        doc = "Depth"


Does depth mean distance from the local origin, in scene units? I wonder if the word distance might be the right choice?

meshula · 2023-10-28T18:37:45Z

pxr/usd/usdGeom/schema.usda

+        doc = "Fitting mode for the image plane."
+    )
+    float2 offset = (0.0, 0.0) (
+        doc = "Image plane offset in inches."


How do imageCenter, and offset relate? How do inches relate to scene units?

meshula · 2023-10-28T18:40:02Z

pxr/usd/usdGeom/schema.usda

+        doc = "3D offset for the image plane."
+    )
+    float2 size = (-1.0, -1.0) (
+        doc = "Image plane size in inches. Less than zero means equal to the camera aperture size."


if we know the camera parameters, and the distance of the image plane from the camera, wouldn't we want to specify this in NDC? I'm not sure why inches is the choice here.

units should be inherited - imperial or metric. Would call this Physical Dimension,
In practical cameras, sensor are defined in mm, with 4 digits of precision being the tiniest photosite known (0.7 nm on some samsung phone sensor).
Not clear what negative value reference is? Not a fan of semantic associated to sign...

meshula · 2023-10-28T18:41:15Z

pxr/usd/usdGeom/schema.usda

+        doc = "Using frame extension"
+    )
+    int frameOffset = 0 (
+        doc = "Frame offset"


I realize this is necessary since USD hasn't got support for multiframe textures. I have a feeling this should be addressed upstream. In terms of this proposal, this is probably the right choice, but we should bookmark the problem for further discussion.

meshula · 2023-10-28T18:41:30Z

pxr/usd/usdGeom/schema.usda

+        doc = "Coverage origin in pixels. Clamped to -image's size and image's size."
+    )
+    bool useFrameExtension = false (
+        doc = "Using frame extension"


What is a frame extension? Perhaps a little more doc.

meshula · 2023-10-28T18:42:55Z

pxr/usd/usdGeom/schema.usda

+        doc = "Depth"
+    )
+    float squeezeCorrection = 1.0 (
+        doc = "Squeeze Correction"


Might we name this pixelAspectRatio? squeeze is fine, but I'm biased to OpenEXR conventions :)

-pedantic:

squeeze correction?

theory/convention is you have "Anamorphic Squeeze" = Optical Compression (lens thing, how image is projected on sensor). A camera lens is described as 1.33X, 2X anamorphic... not 0.5, Yes?
As in, to account at lens distortion level.

and PAR- Pixel Aspect Ratio (a sampling thing, what is a pixel?...) which in contemporary cameras is always 1.0 (only legacy video formats, NTSC | PAL had non-square pixels). Yes?
For example a squeezed SBS stereo is not anamorphic, is squeezed when recording video (storage aspect ratio versus display aspect ratio).
And something to deal with I guess for a background image that is not square pixels if you overlay render over.
[e.g: In "scope" the desqueeze is performed by the projector lens- reversing the camera lens anamorphic]

In practice at fileIn level: Anamorphic Stretch = Anamorphic Desqueeze = 1/Anamorphic_Squeeze = Pixel Aspect Ratio = is all the same transform.
There might be a restoration project somewhere in the universe having to deal with scope telecined in an NTSC or PAL context, but not a fan of supporting legacy complexity.

the oval shape bokeh "effect" as if it was shot anamorphic (as in some shots in Toy Story 4)
vs

putting a background image on the "film back" (sensor crop dimension) of Maya does have lens squeeze ratio as parameter, I forget if people still have to compensate with overscan so the back plate matches?

I see the expression Anamorphic Desqueeze as most used to describe stretching image to fit the desired image aspect ratio.

I'd be fine with a naming that includes the word "anamorphic" and clarity in the rest of the name around whether one is to multiply or divide in order to undistort an image.

meshula · 2023-10-28T18:44:11Z

pxr/usd/usdGeom/schema.usda

+        doc = "Height"
+    )
+    float alphaGain = 1.0 (
+        doc = "Alpha Gain"


alpha gain could use an explanation.

+1 - same question, not a thing

meshula · 2023-10-28T18:47:37Z

pxr/usd/usdGeom/schema.usda

+        doc = "The frame offset for animated textures."
+    )
+    token fit = "best" (
+        allowedTokens = ["fill", "best", "horizontal", "vertical", "to size"]


best and to size should be described. I can't guess what they are without reading the code....

In ASC-FDL template, best was ignored as an option as it's not the same meaning in different apps... Also what does fill mean?

My understanding from previous discussion is that fill means "stretch the image arbitrarily such that it fills the rectangle".

meshula · 2023-10-28T18:47:49Z

pxr/usd/usdGeom/schema.usda

+        doc = "Asset path to the file containg the image data."
+    )
+    double frame = 0.0 (
+        doc = "The frame offset for animated textures."


frameOffset might be more descriptive

meshula · 2023-10-28T18:48:50Z

pxr/usd/usdGeom/schema.usda

+        doc = "Precached frame count"
+    )
+    float width = 0.0 (
+        doc = "Width"


width of the image plane in NDC, same question for height?

meshula · 2023-10-28T18:49:27Z

pxr/usd/usdGeom/wrapImagePlane.cpp

@@ -0,0 +1,368 @@
+//
+// Copyright 2016 Pixar


year of introduction would be 2023 (wherever a new file is introduced)

meshula · 2023-10-28T18:57:12Z

This is looking pretty good. It's pretty obvious, I imagine, from my questions/comments that I'm trying to get a mental picture of the model. I'm wondering if a diagram might be created that shows a camera frustum, the image plane, and how all the values modify the display of the image plane? That would really help me understand all the moving parts :)

chadrik · 2023-11-21T22:18:54Z

I love that this PR is getting some traction right now, but the authors of this PR are no longer at Luma, and nor am I, so I can't speak to whether there are resources at Luma to work on this. I would encourage others to carry this forward and make a proposal to the USD community.

nxkb · 2023-11-22T03:10:56Z

You can carry this forward by contributing to the latest proposal or adapting this PR to match that proposal.
Here is a link to the proposal as far as we got: PixarAnimationStudios/OpenUSD-proposals#22

sirpalee added 2 commits September 19, 2023 18:13

[usdView] Loading default shaders for usdview and adding a plain text…

2076be5

…ured surface.

[USD] Support for Image Planes in Maya / Hydra.

779f47f

meshula reviewed Oct 28, 2023

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[for reference] Add Support for Image Planes (as a concrete schema type) #1

[for reference] Add Support for Image Planes (as a concrete schema type) #1

nxkb commented Sep 25, 2023 •

edited

Loading

meshula Oct 28, 2023

meshula Oct 28, 2023

revisionfx Nov 21, 2023

meshula Nov 21, 2023

meshula Oct 28, 2023

revisionfx Nov 21, 2023 •

edited

Loading

meshula Oct 28, 2023

meshula Oct 28, 2023

meshula Oct 28, 2023

revisionfx Nov 21, 2023 •

edited

Loading

meshula Oct 28, 2023

meshula Oct 28, 2023

meshula Oct 28, 2023

revisionfx Nov 21, 2023

meshula Nov 21, 2023

meshula Oct 28, 2023 •

edited

Loading

revisionfx Nov 21, 2023

meshula Oct 28, 2023

revisionfx Nov 21, 2023

meshula Nov 21, 2023

meshula Oct 28, 2023 •

edited

Loading

meshula Oct 28, 2023

meshula Oct 28, 2023 •

edited

Loading

meshula commented Oct 28, 2023

chadrik commented Nov 21, 2023

nxkb commented Nov 22, 2023

[for reference] Add Support for Image Planes (as a concrete schema type) #1

Are you sure you want to change the base?

[for reference] Add Support for Image Planes (as a concrete schema type) #1

Conversation

nxkb commented Sep 25, 2023 • edited Loading

Description of Change(s)

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

revisionfx Nov 21, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

revisionfx Nov 21, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

meshula Oct 28, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

meshula Oct 28, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

meshula Oct 28, 2023 • edited Loading

Choose a reason for hiding this comment

meshula commented Oct 28, 2023

chadrik commented Nov 21, 2023

nxkb commented Nov 22, 2023

nxkb commented Sep 25, 2023 •

edited

Loading

revisionfx Nov 21, 2023 •

edited

Loading

revisionfx Nov 21, 2023 •

edited

Loading

meshula Oct 28, 2023 •

edited

Loading

meshula Oct 28, 2023 •

edited

Loading

meshula Oct 28, 2023 •

edited

Loading