Support overlay mode?

[dantman]

Is Valve willing to support using the SteamVR Unity Plugin for creating SteamVR Overlays?

After experimenting a bit and digging into the code. It appears as if nearly everything needed to make overlays is available in the plugin. There's enough here it would likely be more productive to tweak the plugin to support overlays, than to implement overlays from scratch using openvr_api.

From what I can tell, the primary thing stopping overlays built in Unity with this plugin from being viable is that when Unity's XR handling – which initializes itself as a scene renderer – is disabled so you can initialize your app as overlay-only. The SteamVR class decides it cannot function and never creates an instance. And as a result the majority of the plugin just simply fails on SteamVR.instance. Even when the code is just doing something as simple as checking the textureType. Even when the code doesn't need UnityXR's rendering. And even though the plugin already appears to do most of the pose tracking, etc... itself without UnityXR.

[alexbu92]

Any updates on this? We really just need an option under the native openvr support settings to set what kind of EVRApplicationType type we want OpenVR to be initialized as. Right now it seems to be hardcoded as VRApplication_Scene, we just need a dropdown to be able to change that to whatever application type we need.

[zite]

I'll add that to the steamvr settings object.

[zite]

Actually it looks like this is done at the unity level now, we may have to roll this into a unity 2019 xr sdk thing. I'll look into it further.

[dantman]

The initialization is done by Unity, if you have OpenVR turned on in Unity. However the Unity XR stuff does not do overlays and IMHO just about everything that is done in Unity XR is not needed for building overlays.

IMHO building overlays should be done by using the SteamVR Plugin, turning Unity's VR handling off, calling the OpenVR Initialization with an overlay type yourself, and setting a setting that tells the SteamVR Plugin NOT to disable itself when it sees that the Unity XR code is not running.

[dantman]

I'm already building an overlay using the SteamVR Plugin:
https://github.com/dantman/elite-vr-cockpit

I have the Unity XR stuff disabled. I init the overlay stuff myself. And I copy a few parts of the SteamVR plugin like the EventLoop for which the built-in one doesn't run because the StreamVR Plugin disabled itself. The one caveat is there are some places like SteamVR Input that I can't copy things to get working because the references to the SteamVR class assuming that it's running are too hardcoded.

The latter two things should not be required. The plugin already has a built in event loop and APIs like the Input API also seem to work in an overlay. The only problem is that when the SteamVR class that everything depends on starts up and sees that Unity XR is not running (because custom overlay code is running instead) and then shuts down completely and turns very simple calls to non-UnityXR things like getting the SteamVR Plugin's own settings values, into fatal errors. And does not allow things like the event loop to start up (which does not actually rely on Unity XR).

[fragmentvr-dev]

Yeah, same here. The SteamVR Unity Plugin is unusable without heavy modification, and even then only parts can be utilized, not to mention that it becomes impossible to update it when updates come around.

[bddckr]

It looks like some parts of the plugin can be called independently and will then initialize OpenVR if needed (most often as the Utility app type) and calls Shutdown once done.

I would like to see a single place handling the automatic initialization and there being a way to disable it. If the same method it calls to initialize is public API that would be even better - this way users don't have to reinvent the wheel in case they want to delay turning on VR or support non-VR gameplay and want to allow switching at runtime.

[dantman]

I can confirm there is very little in the plugin that actually references the Unity XR runtime. UnityXR is of course relied on to handle rendering scenes, but that functionality is not needed/desired when running an overlay.

• SteamVR.cs just checks XRSettings to see if OpenVR support is setup.
• SteamVR_Behaviour.cs uses XRSettings/XRDevice loading and XRSettings.enabled = true; only when you initialize with forceUnityVRMode = true.
• SteamVR_Camera.cs uses XRSettings to get the eyeTextureResolutionScale/renderScale. However you only use SteamVR_Camera when you are using UnityXR to run a game, not using it for an overlay.

I got SteamVR to startup by adding an extra standalone mode that calls OpenVR.Initialize and disables the XRSettings checks. And of course turning off autoEnableVR, turning off Unity XR, including the OpenVR dll myself. I'll verify how much I have working later.

[dantman]

Most things seem to work well:

• SteamVR_Actions bindings work well and I'm able to listen to bindings for my overlay.
• The built-in event loop in SteamVR_Renderer works, i.e. I can remove my own custom EventLoop implementation and things still work.
• The SteamVR_TrackedObject works. Though I may stick with my own TrackedHMD fork for a little while longer since my customizations handle the changes in origin that affect the overlay when games change the tracking origin.

The issues I've had so far are minor and mostly not actually issues related to running the plugin in an overlay:

• OpenVR recognizes my overlay's Input API bindings and they work with the Unity plugin. However my overlay does not show up in the Steam VR dashboard's Controller Bindings list to edit; i.e. You can only edit them by manually visiting the browser url. However this is likely a general SteamVR issue and not an issue with the Unity plugin or overlay usage of the Unity plugin.
• My old VREvent_Button{Press,Unpress,Touch,Untouch} code no longer works, so I can't test the full functionality of my overlay yet. However I assume this is intentional and I just have to replace all my legacy input code with actions based code before I can test anything. It will be a little bit troubling having to work on both InputAPI-ifying my overlay and refactoring to use built-in plugin functionality simultaneously. But it's doable.
• Using the old style device index based tracking for the HMD position works. But I can't seem to figure out what the "Input API/Action-based" method of HMD tracking is and how to make it work. Using SteamVR_Behaviour_Pose works for LeftHand/RightHand but doesn't seem to work for Head or Camera.

[dantman]

Here's a branch with the changes I made #336. I implemented it adding a SteamVR.InitializeStandalone(applicationMode); API since I was already calling OpenVR.Init to initialize an overlay; and in overlay mode you inevitably need to write code since that's all custom.

However if it is preferable to implement an explicit "overlay mode" (perhaps also with a "background mode") and not write code to initialize it. Then I could redo it so I add a setting to SteamVR_Settings to allow switching to overlay/background mode.

[zite]

I believe this feature has been added and should work now. If not please create a new issue with repro steps and ideally an example project. Thanks!

[dantman]

@zite I strongly doubt that this feature has been added. As the SteamVR code checking for XRSettings.enabled , etc... appears to be exactly the same without any changes aside from disabling it when you are using OpenXR. And I doubt the plugin works correctly trying to run a SteamVR/OpenVR overlay when it is set to use OpenXR.

This is a request for adding support for running in overlay mode. There is no way to write an example project for something that doesn't exist.

I did however go further than that. I wrote a PR implementing standalone/overlay support #336.

[TitansWhale]

@zite我强烈怀疑是否添加了此功能。由于SteamVR检查XRSettings.enabled等的代码似乎完全相同，除了在使用 OpenXR 时禁用它之外没有任何更改。我怀疑该插件在设置为使用 OpenXR 时能否正常运行 SteamVR/OpenVR 覆盖。

这是添加对在覆盖模式下运行的支持的请求。无法为不存在的东西编写示例项目。

然而我确实走得更远。我写了一个 PR 实现独立/覆盖支持#336。

Do you have an upgraded the overlay in OpenXR?