Add trackpad button support

[marcoscaceres]

The current W3C Gamepad specification does not include mappings for the trackpad button on Sony DualShock and DualSense controllers. This omission leads to inconsistencies across browsers, as observed in WebKit (Safari) and Chromium (Chrome).

Suggested Update:

1. Standardize Trackpad Button Mapping: Introduce a standard mapping for the trackpad button, possibly as button 17, to align with Chromium's existing implementation.
2. Uniform Button Array Size: Ensure a 17-element button array for all gamepad controllers to maintain consistency and aid in fingerprinting mitigation.

Additional Information:

This inconsistency was noted in WebKit bug report 266293, with a test conducted on gamepad-tester.com showing different behaviors in Safari Technology Preview and Chrome.

Impact:

Addressing this will enhance cross-browser compatibility and user experience for gamepad users, especially those using Sony's controllers.

References:

• WebKit Bug Report: 266293 – Gamepad: Map DualShock/DualSense Trackpad Button

[nondebug]

I think we should not add assign a Standard Gamepad button index specifically for the trackpad button. The Standard Gamepad buttons are defined by their layout on the gamepad rather than their intended functionality. Instead of a trackpad button, call it "fourth button in center cluster" or something like that so we can map it on gamepads with no trackpad.

Specifying trackpad button at index 17 would match Chromium's mappings for DualShock4 and DualSense but would invalidate several other mappings that map other buttons to button index 17. For example, Capture button on Switch controllers and Share button on Xbox controllers are both mapped to button 17. If we don't require button 17 to be a trackpad button then we can keep the existing mappings.

Uniform Button Array Size

Limiting the button array size would be confusing and frustrating to users. Gamepads with more than 18 buttons will have "dead" buttons that can't be used in games. For example, Stadia Controller has 19 buttons total. Even if we add button 17 we'll still have to pick a button to leave unmapped.

Using a uniform button array size to limit fingerprinting makes little sense unless the ID string is also uniform because the length of the buttons array doesn't expose any more fingerprinting surface than the ID string itself. Two gamepads with the same ID string will typically have the same number of buttons.

I think it's reasonable to expose a limited view of the gamepad as long as there's a simple way for the user to unlock the full view. In the past we've discussed the possibility of adding a Gamepad permission. Perhaps we can expose a limited view of the gamepad when a page lacks the Gamepad permission and a full view once the permission is granted.

[xingri]

@marcoscaceres Thanks for bring this issue up.

@nondebug Thanks for sharing the opinion. Based on our internal discussion, we are good to go with the "fourth button in center cluster" approach. And we are agreeing with you regarding the limiting the Button Array Size. We also do not want to make more restrictions on the current specification.

[nondebug]

We discussed this topic at our recent meeting and decided against assigning any new standard button indices. Instead, we want to add a new GamepadButton attribute that would expose a standard name for the button (ex: 'trackpad', 'capture', 'share'). This would allow apps to more easily discover "extra" buttons and would not leave gaps in the array when uncommon buttons are missing.

[xingri]

We discussed this topic at our recent meeting and decided against assigning any new standard button indices. Instead, we want to add a new GamepadButton attribute that would expose a standard name for the button (ex: 'trackpad', 'capture', 'share'). This would allow apps to more easily discover "extra" buttons and would not leave gaps in the array when uncommon buttons are missing.

@nondebug Thanks for the update and good news. Is there a task to track the activity of adding new Gamepad Button attribute?

[nondebug]

I would prefer to adopt an existing button naming scheme instead of inventing something new. I'm looking at OpenXR /user paths since it aims to solve the same problem. OpenXR organizes gamepad inputs into a subtree rooted at /user/gamepad. The specification gives standard identifiers for common gamepad components (buttons, triggers, thumbsticks, trackpads, etc).

For example, the Standard Gamepad buttons are:

/user/gamepad/input/diamond_down/click
/user/gamepad/input/diamond_right/click
/user/gamepad/input/diamond_left/click
/user/gamepad/input/diamond_up/click
/user/gamepad/input/shoulder_left/click
/user/gamepad/input/shoulder_right/click
/user/gamepad/input/trigger_left/click
/user/gamepad/input/trigger_right/click
/user/gamepad/input/select/click
/user/gamepad/input/start/click
/user/gamepad/input/thumbstick_left/click
/user/gamepad/input/thumbstick_right/click
/user/gamepad/input/dpad_up/click
/user/gamepad/input/dpad_down/click
/user/gamepad/input/dpad_left/click
/user/gamepad/input/dpad_right/click
/user/gamepad/input/system/click

The PlayStation trackpad button is:

/user/gamepad/input/trackpad/click

It's not clear how to name buttons like Xbox Elite paddles or DualSense Edge back buttons. I think they are considered "squeeze" buttons similar to HTC Vive's Grip button.

Xbox Elite squeeze buttons:

/user/gamepad/input/squeeze_left_upper/click
/user/gamepad/input/squeeze_left_lower/click
/user/gamepad/input/squeeze_right_upper/click
/user/gamepad/input/squeeze_right_lower/click

DualSense Edge squeeze buttons:

/user/gamepad/input/squeeze_left/click
/user/gamepad/input/squeeze_right/click

For many gamepads there isn't a standard OpenXR identifier for every button. OpenXR allows extensions to add new identifiers with the "_ext" suffix.

Several gamepads have something like a "Share" button. Maybe these can use the same identifier?

/user/gamepad/input/share_ext/click ("Share" button, Xbox Wireless Controller Model 1914)
/user/gamepad/input/share_ext/click ("Capture" button, Nintendo Switch controllers)
/user/gamepad/input/share_ext/click ("Capture" button, Stadia controller)

DualShock4 has a button labeled "Share" but it's already included in the Standard Gamepad as "left button in center cluster". DualSense labels this button "Create". We should prefer the Standard Gamepad button name.

/user/gamepad/input/select/click ("Share" button, DualShock4)
/user/gamepad/input/select/click ("Create" button, DualSense)

Google and Amazon gamepads have a dedicated button for calling the assistant:

/user/gamepad/input/assistant_ext/click ("Google Assistant" button, Stadia Controller)
/user/gamepad/input/assistant_ext/click ("Alexa Push-to-talk" button, Amazon Luna controller)

@marcoscaceres Can you think of any reason why we shouldn't use OpenXR's /user paths as identifiers in Gamepad API?

[xingri]

@marcoscaceres @nondebug I have added the PR#196 for this.
Please review this and let me know your opinion.
Also, could we discuss it on upcoming working group meeting schedule on 3/14?

[DanielHerr]

What about devices which have 2 touchpads? Steam Controller, Steam Deck, and Ayaneo Kun have left and right touchpads.

[xingri]

What about devices which have 2 touchpads? Steam Controller, Steam Deck, and Ayaneo Kun have left and right touchpads.

@DanielHerr Thanks for the feedback. As I already mentioned on the PR#196, The table defines the extended buttons used by some gamepad models. It's important to note that this list is not exhaustive, and user agents may utilize different or additional buttons for these or other gamepad models.

I guess we can propose something like "trackpad_left" and trackpad_right" button types for the multiple touchpads case.