Emulated oculus and vive controls #2423

machenmusik · 2017-02-20T18:37:53Z

(Per request, this PR no longer extends #2413 / #2415 to oculus-touch-controls as thumbstickmoved)

Should fix #2379.

machenmusik · 2017-02-21T13:57:37Z

Verified on both Vive (via SteamVR/OpenVR support) and Touch (direct Oculus support) with latest experimental Chromium.

cmvanb · 2017-02-21T15:47:16Z

Looks like a nice upgrade. After #2415 was merged, I noticed that the trigger on the HTC Vive also fires onAxisMove (z axis) events. Does this PR handle the trigger axis as 'triggermoved' or as 'trackpadmoved'?

machenmusik · 2017-02-21T22:07:47Z

trigger is not an axis, it is a button with a value associated -- or at least that is the case with Chromium. per discussion with @dmarcos in Slack, we should definitely avoid browser-specific ids and button/axis mappings if possible, bleah.

machenmusik · 2017-02-22T08:45:09Z

axismove events should have been fired before #2415 as well, meaning that you could easily handle axismove yourself if need be. but for buttons with values, as trigger should be (and grip for Touch as well), it looks like no specific event was fired when the value changed, only the more generic buttonchanged that is also fired on up/down/touchstart/touchend.

cmvanb · 2017-02-22T13:30:57Z

Ok, so 'buttonchanged' still needs to be fired correctly for this specific case (htc vive trigger), with the correct value? Does the oculus touch grip work like this already?

machenmusik · 2017-02-22T15:48:23Z

buttonchanged should be firing now, see https://github.com/aframevr/aframe/blob/master/src/components/tracked-controls.js#L151

oculus-touch-controls and vive-controls provide mappings to friendly names for up/down/touchstart/touchend, but looks like they don't handle changed. that can be added, but would like to settle the button/axis disparity between Chromium and Nightly first to avoid redoing work. @dmarcos any more word from Gecko folks?

machenmusik · 2017-02-22T17:48:14Z

@cmvanb if you have time in next few days, can you confirm which environments (browser / HMD+controllers) receive events such as triggerchanged with the latest PR?

cmvanb · 2017-02-22T18:04:30Z

I can test FF nightly + Vive for you. I will also try Chromium + Vive, although Chromium didn't work last time I tried. Will do that tonight or tomorrow. I don't have an Oculus to test with.

machenmusik · 2017-02-22T18:25:12Z

(FYI, there is a brand new experimental chromium that resolves previous issues with latest steamvr)

cmvanb · 2017-02-23T18:43:59Z

I tested the branch with vive + FF nightly 54.0a1 (2017-02-23) (64-bit).

When clicking the trackpad, 'trackpadchanged' event is fired (multiple times/frames) as expected.
When sliding a finger over the trackpad, 'trackpadmoved' event (multiple times/frames) is fired as expected. The x/y values are correct.
When pulling the trigger down (fast), several events are fired in this order:
'trackpadmoved'
'trackpadmoved'
'triggerdown'
'triggerchanged'
'trackpadmoved'
'triggerchanged'

On releasing the trigger:
'trackpadmoved'
'triggerup'
'triggerchanged'
'trackpadmoved'
'triggerchanged'

It seems like the 'trackpadmoved' event is still being fired for the trigger. Otherwise, the values seem correct to me.

PS: Chromium still isn't working for me. I'm using the version made available here: https://webvr.info/get-chrome/

machenmusik · 2017-02-23T19:03:11Z

thanks @cmvanb !

I believe there is currently a fundamental difference in how the latest experimental Chromium and Nightly model Vive and Touch expose controllers with respect to buttons and axes.

Chromium represents trigger (Vive and Touch) as a button that has an analog value, and not as an axis... same thing for grip on Touch. The only axes are trackpad (Vive) and thumbstick (touch).

Nightly appears to be representing buttons that have analog values (like trigger) as axes, for example the new Oculus Touch support was exposing controllers with 3 buttons and 4 axes in Nightly, rather than the 6 buttons and 2 axes in Chromium, This causes problems since axismove events are currently fired by tracked-controls when trigger changes, with no indication of which axes changed.

@cmvanb can you confirm whether the triggerchanged button state provides an analog value using Nightly? If not, then even the button events are inconsistent, so suppressing spurious trackpadmoved events would not suffice.

IMO it is highly undesirable to have browser-specific gamepad API mappings for the same controllers on the same machine. @cvan @kearwood @toji - would appreciate your thoughts on how best to resolve discrepancies.

ngokevin · 2017-03-01T00:10:39Z

src/components/oculus-touch-controls.js

+
+  update: function (oldData) {
+    if ((oldData.emulated !== undefined) && (this.data.emulated !== oldData.emulated)) {
+      this.checkIfControllersPresent();


This LOC isn't getting covered by the tests. Maybe something wrong with the condition?

I think just if (this.data.emulated !== oldData.emulated) alone should work

the original reason for that was to avoid flagging the first update call (right after init) as needing the controllers check, since play() will do it.

I see. Maybe emulated in oldData + a comment is a minor improvement.

in order for that line to be covered, the entity would have to have emulated be true right at initial definition, and the tests use a setup function where no attributes are provided initially.

ngokevin · 2017-03-01T00:11:57Z

How difficult to split the emulated changes from the axismove additions?

machenmusik · 2017-03-01T00:30:28Z

Easiest way to "split" the axismove handling is to simply comment out the onAxisMoved handler, in which case the code won't fire. but remember there is already a different version of axismove committed into master for vive only, not for oculus, so we'd be disabling that vive-only functionality by doing so

machenmusik · 2017-03-01T01:14:17Z

so rather than split out the axismove handling, I added axis change propagation, which allows culling of spurious moves.

machenmusik · 2017-03-01T14:02:56Z

@cmvanb if you get the chance, can you check to see whether the latest version of this PR prevents the spurious trackpadmoved events you were seeing before?

cvan · 2017-03-01T23:57:53Z

docs/components/oculus-touch-controls.md

@@ -34,34 +35,43 @@ mappings, events, and a Touch controller model.
 | triggerup            | Trigger released.              |
 | triggertouchstart    | Trigger touched.               |
 | triggertouchend      | Trigger no longer touched.     |
+| triggerchanged       | Trigger changed.               |


FYI: Firefox ships with an implementation of experimental Gamepad events: gamepadbuttondown, gamepadbuttonup, and gamepadaxismove (fired on window).

If you're curious, might want to check these links out:

Should fire events instead of using passive model w3c/gamepad#4

Mapping of HID Events to the Gamepad API w3c/gamepad#56

spec gamepadbuttondown and gamepadbuttonup events (issue #4) w3c/gamepad#15

good to know, thanks. this code needs to support other browsers that currently lack those, let's revisit when more widely available?

cvan · 2017-03-02T00:00:56Z

src/components/oculus-touch-controls.js

-
-    // since each hand is named differently, avoid enumeration
+    // Inject with specific gamepad id, if provided. This works around a temporary issue
+    // where Chromium uses `Oculus Touch (Right)` but Firefox uses `Oculus Touch (right)`.
    el.setAttribute('tracked-controls', {


might want to check out these Gamepad librarys I wrote; maybe there are some parts you can reuse.

https://github.com/MozVR/oculus/blob/5aa680f/assets/js/gamepad.js

https://github.com/MozVR/oculus/blob/e73ea2c/assets/js/common.js#L107

https://github.com/MozVR/gamepad-plus

https://github.com/MozVR/gamepad-plus/blob/b3cc035/demo.js

yeah, I have a similar not-just-tracked-controls set for aframe-arcade, but that's beyond this PR's scope, and last we discussed, core team preference was to keep non-tracked controllers (gearvr, daydream, oculus remote, xbox controller, etc) out of core. maybe file new issue to support controllers included with VR equipment including 0DOF (emulated pose and orientation) and 3DOF (emulated pose, true orientation)?

cvan · 2017-03-02T00:40:47Z

src/components/vive-controls.js

@@ -202,8 +202,8 @@ module.exports.Component = registerComponent('vive-controls', {
    Object.keys(axesMapping).map(function (key) {
      var value = axesMapping[key];
      var detail = {};


where is detail used? perhaps do var detail = evt.detail || {};?

no, detail is used to build the translated/remapped event detail.

ah, I didn't see it in the diff - my bad

cvan · 2017-03-02T00:41:17Z

src/components/oculus-touch-controls.js

-      var changed = false;
-      value.map(function (axisNumber) { changed |= evt.detail.changed[axisNumber]; });
+      var changed = !evt.detail.changed;
+      if (!changed) { value.map(function (axisNumber) { changed |= evt.detail.changed[axisNumber]; }); }


you can use forEach instead of map

machenmusik · 2017-03-02T01:05:37Z

@dmarcos relayed discussion with Mozillans where different Oculus Touch button/axis mapping is not likely to change soon (unlike id string). Can I urge Mozillans to reconsider? At least make analog buttons like trigger and grip generate actual button events, rather than treating them like sliders... otherwise even simple trigger button down/up will become unreliable

dmarcos · 2017-03-02T19:00:48Z

I don't understand at first sight what the emulated property does. If I don't get it quickly that I wrote the original controls many other people will get confused too.

machenmusik · 2017-03-02T19:39:14Z

See #2446 (comment)

ngokevin · 2017-03-02T21:59:47Z

emulated won't be used often, almost purely for motion capture replaying. Where you want to force controller event listeners to be added despite no controllers being present. Currently, we add controller event listeners regardless which causes events to be double-emitted.

machenmusik · 2017-03-02T22:13:39Z

@ngokevin you might actually be better off also capturing emitted events from oculus-touch-controls and vive-controls (perhaps daydream and gearvr, not sure about leap hands) as well as those from tracked-controls, at which point you don't even need emulated true, you just need the app to be paying attention.

ngokevin · 2017-03-09T19:39:47Z

@dmarcos Any thoughts? Issue is when we made controls always add event listeners, controller events would double emit for hand-controls since both Oculus + Vive event handlers are attached. The proposal is an emulated property which controls when we force event listeners to be added. Let us know if you have other proposals (e.g., naming forceEventListeners, forcePresent), but the use case is for emulating + motion capture.

machenmusik · 2017-03-09T22:14:26Z

should we deprecate this in favor of #2446?

ngokevin · 2017-03-21T18:04:56Z

It's more easy to review if things are in isolation, so we'll try not to clump non-cohesive changes together in one PR. @dmarcos still needs to take a look and understand, but the tl;dr is that we forced event listeners to be added regardless of Gamepad present in order for motion capture, but that causes doubly-emitted events (vive + oculus controls). The proposed emulated property will only add all event listeners if specifically enabled.

machenmusik · 2017-03-21T18:21:17Z

Right. if you merge this PR, #2446 will simplify itself :-)

ngokevin · 2017-03-21T19:19:49Z

Would you mind breaking out the axismove changes to another PR?

machenmusik · 2017-03-21T19:48:29Z

so which things between this and #2446 want to be separate PRs? there is some overlap so may be nontrivial (e.g. #2423 (comment)), and i am tied up for a little while

i think we definitely want the fix for #2448 that @caseyyee confirmed via cherry-pick, that one is easy (now #2510), but maybe not entirely done with Nightly Rift+Touch without #2446

i think we need to fix the double-event issue introduced unless we move back to auto-detect-controllers paradigm, and IIRC the axismove changes here are needed now to prevent spurious events from prior vive-specific PR / Nightly having control elements be both buttons and axes, so a bit hesitant to separate temporarily only to have them merged back together shortly

machenmusik · 2017-03-21T21:22:42Z

w.r.t. separating PRs: I think these delineations would work

cherry-picked also apply userHeight to controller when no stage parameters (fixes #2448) #2510 - already done
axismove and nightly touch support - need new mapping structure for both
emulated

may get a chance to look at that late this evening.

ngokevin · 2017-03-22T17:30:15Z

After discussing in Slack, it seems we'll revert #2314 for another solution, so we don't need emulated. Thanks!

machenmusik mentioned this pull request Feb 24, 2017

Expose axismove events as trackpadmoved on vive-controls #2413

Open

ngokevin reviewed Mar 1, 2017

View reviewed changes

cvan reviewed Mar 2, 2017

View reviewed changes

machenmusik mentioned this pull request Mar 2, 2017

Nightly oculus touch #2446

Closed

machenmusik mentioned this pull request Mar 20, 2017

Emit event when oculus or vive controllers are connected #2505

Merged

add emulated property (separate from axismove changes)

1751b6c

machenmusik force-pushed the emulated-oculus-and-vive-controls branch from 3b8199b to 1751b6c Compare March 21, 2017 23:23

machenmusik changed the title ~~Emulated oculus and vive controls with axismove handler~~ Emulated oculus and vive controls Mar 21, 2017

add explanation for why emulated property exists

efb4c8b

ngokevin closed this Mar 22, 2017

machenmusik mentioned this pull request Mar 22, 2017

Axismove for touch and vive #2513

Merged

Emulated oculus and vive controls #2423

Emulated oculus and vive controls #2423

Conversation

machenmusik commented Feb 20, 2017 • edited Loading

machenmusik commented Feb 21, 2017

cmvanb commented Feb 21, 2017

machenmusik commented Feb 21, 2017

machenmusik commented Feb 22, 2017 • edited Loading

cmvanb commented Feb 22, 2017 • edited Loading

machenmusik commented Feb 22, 2017

machenmusik commented Feb 22, 2017

cmvanb commented Feb 22, 2017

machenmusik commented Feb 22, 2017

cmvanb commented Feb 23, 2017

machenmusik commented Feb 23, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ngokevin commented Mar 1, 2017

machenmusik commented Mar 1, 2017

machenmusik commented Mar 1, 2017

machenmusik commented Mar 1, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

machenmusik commented Mar 2, 2017 • edited Loading

dmarcos commented Mar 2, 2017

machenmusik commented Mar 2, 2017

ngokevin commented Mar 2, 2017

machenmusik commented Mar 2, 2017

ngokevin commented Mar 9, 2017

machenmusik commented Mar 9, 2017

ngokevin commented Mar 21, 2017

machenmusik commented Mar 21, 2017

ngokevin commented Mar 21, 2017

machenmusik commented Mar 21, 2017 • edited Loading

machenmusik commented Mar 21, 2017

ngokevin commented Mar 22, 2017

machenmusik commented Feb 20, 2017 •

edited

Loading

machenmusik commented Feb 22, 2017 •

edited

Loading

cmvanb commented Feb 22, 2017 •

edited

Loading

machenmusik commented Mar 1, 2017 •

edited

Loading

machenmusik commented Mar 2, 2017 •

edited

Loading

machenmusik commented Mar 21, 2017 •

edited

Loading