Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add trackpad button support #191

Open
marcoscaceres opened this issue Dec 19, 2023 · 8 comments · May be fixed by #196
Open

Add trackpad button support #191

marcoscaceres opened this issue Dec 19, 2023 · 8 comments · May be fixed by #196
Assignees
Labels
TPAC2024 Topic for discussion at TPAC 2024

Comments

@marcoscaceres
Copy link
Member

The current W3C Gamepad specification does not include mappings for the trackpad button on Sony DualShock and DualSense controllers. This omission leads to inconsistencies across browsers, as observed in WebKit (Safari) and Chromium (Chrome).

Suggested Update:

  1. Standardize Trackpad Button Mapping: Introduce a standard mapping for the trackpad button, possibly as button 17, to align with Chromium's existing implementation.
  2. Uniform Button Array Size: Ensure a 17-element button array for all gamepad controllers to maintain consistency and aid in fingerprinting mitigation.

Additional Information:

This inconsistency was noted in WebKit bug report 266293, with a test conducted on gamepad-tester.com showing different behaviors in Safari Technology Preview and Chrome.

Impact:

Addressing this will enhance cross-browser compatibility and user experience for gamepad users, especially those using Sony's controllers.

References:

@nondebug
Copy link
Collaborator

nondebug commented Jan 3, 2024

I think we should not add assign a Standard Gamepad button index specifically for the trackpad button. The Standard Gamepad buttons are defined by their layout on the gamepad rather than their intended functionality. Instead of a trackpad button, call it "fourth button in center cluster" or something like that so we can map it on gamepads with no trackpad.

Specifying trackpad button at index 17 would match Chromium's mappings for DualShock4 and DualSense but would invalidate several other mappings that map other buttons to button index 17. For example, Capture button on Switch controllers and Share button on Xbox controllers are both mapped to button 17. If we don't require button 17 to be a trackpad button then we can keep the existing mappings.

Uniform Button Array Size

Limiting the button array size would be confusing and frustrating to users. Gamepads with more than 18 buttons will have "dead" buttons that can't be used in games. For example, Stadia Controller has 19 buttons total. Even if we add button 17 we'll still have to pick a button to leave unmapped.

Using a uniform button array size to limit fingerprinting makes little sense unless the ID string is also uniform because the length of the buttons array doesn't expose any more fingerprinting surface than the ID string itself. Two gamepads with the same ID string will typically have the same number of buttons.

I think it's reasonable to expose a limited view of the gamepad as long as there's a simple way for the user to unlock the full view. In the past we've discussed the possibility of adding a Gamepad permission. Perhaps we can expose a limited view of the gamepad when a page lacks the Gamepad permission and a full view once the permission is granted.

@xingri
Copy link

xingri commented Jan 26, 2024

@marcoscaceres Thanks for bring this issue up.

@nondebug Thanks for sharing the opinion. Based on our internal discussion, we are good to go with the "fourth button in center cluster" approach. And we are agreeing with you regarding the limiting the Button Array Size. We also do not want to make more restrictions on the current specification.

@nondebug
Copy link
Collaborator

We discussed this topic at our recent meeting and decided against assigning any new standard button indices. Instead, we want to add a new GamepadButton attribute that would expose a standard name for the button (ex: 'trackpad', 'capture', 'share'). This would allow apps to more easily discover "extra" buttons and would not leave gaps in the array when uncommon buttons are missing.

@xingri
Copy link

xingri commented Jan 26, 2024

We discussed this topic at our recent meeting and decided against assigning any new standard button indices. Instead, we want to add a new GamepadButton attribute that would expose a standard name for the button (ex: 'trackpad', 'capture', 'share'). This would allow apps to more easily discover "extra" buttons and would not leave gaps in the array when uncommon buttons are missing.

@nondebug Thanks for the update and good news. Is there a task to track the activity of adding new Gamepad Button attribute?

@nondebug nondebug self-assigned this Feb 21, 2024
@nondebug
Copy link
Collaborator

I would prefer to adopt an existing button naming scheme instead of inventing something new. I'm looking at OpenXR /user paths since it aims to solve the same problem. OpenXR organizes gamepad inputs into a subtree rooted at /user/gamepad. The specification gives standard identifiers for common gamepad components (buttons, triggers, thumbsticks, trackpads, etc).

For example, the Standard Gamepad buttons are:

/user/gamepad/input/diamond_down/click
/user/gamepad/input/diamond_right/click
/user/gamepad/input/diamond_left/click
/user/gamepad/input/diamond_up/click
/user/gamepad/input/shoulder_left/click
/user/gamepad/input/shoulder_right/click
/user/gamepad/input/trigger_left/click
/user/gamepad/input/trigger_right/click
/user/gamepad/input/select/click
/user/gamepad/input/start/click
/user/gamepad/input/thumbstick_left/click
/user/gamepad/input/thumbstick_right/click
/user/gamepad/input/dpad_up/click
/user/gamepad/input/dpad_down/click
/user/gamepad/input/dpad_left/click
/user/gamepad/input/dpad_right/click
/user/gamepad/input/system/click

The PlayStation trackpad button is:

/user/gamepad/input/trackpad/click

It's not clear how to name buttons like Xbox Elite paddles or DualSense Edge back buttons. I think they are considered "squeeze" buttons similar to HTC Vive's Grip button.

Xbox Elite squeeze buttons:

/user/gamepad/input/squeeze_left_upper/click
/user/gamepad/input/squeeze_left_lower/click
/user/gamepad/input/squeeze_right_upper/click
/user/gamepad/input/squeeze_right_lower/click

DualSense Edge squeeze buttons:

/user/gamepad/input/squeeze_left/click
/user/gamepad/input/squeeze_right/click

For many gamepads there isn't a standard OpenXR identifier for every button. OpenXR allows extensions to add new identifiers with the "_ext" suffix.

Several gamepads have something like a "Share" button. Maybe these can use the same identifier?

/user/gamepad/input/share_ext/click ("Share" button, Xbox Wireless Controller Model 1914)
/user/gamepad/input/share_ext/click ("Capture" button, Nintendo Switch controllers)
/user/gamepad/input/share_ext/click ("Capture" button, Stadia controller)

DualShock4 has a button labeled "Share" but it's already included in the Standard Gamepad as "left button in center cluster". DualSense labels this button "Create". We should prefer the Standard Gamepad button name.

/user/gamepad/input/select/click ("Share" button, DualShock4)
/user/gamepad/input/select/click ("Create" button, DualSense)

Google and Amazon gamepads have a dedicated button for calling the assistant:

/user/gamepad/input/assistant_ext/click ("Google Assistant" button, Stadia Controller)
/user/gamepad/input/assistant_ext/click ("Alexa Push-to-talk" button, Amazon Luna controller)

@marcoscaceres Can you think of any reason why we shouldn't use OpenXR's /user paths as identifiers in Gamepad API?

xingri added a commit to xingri/gamepad that referenced this issue Mar 5, 2024
@xingri xingri linked a pull request Mar 5, 2024 that will close this issue
4 tasks
@xingri
Copy link

xingri commented Mar 5, 2024

@marcoscaceres @nondebug I have added the PR#196 for this.
Please review this and let me know your opinion.
Also, could we discuss it on upcoming working group meeting schedule on 3/14?

xingri added a commit to xingri/gamepad that referenced this issue Mar 13, 2024
xingri added a commit to xingri/gamepad that referenced this issue Mar 13, 2024
xingri added a commit to xingri/gamepad that referenced this issue Mar 13, 2024
xingri added a commit to xingri/gamepad that referenced this issue Mar 13, 2024
xingri added a commit to xingri/gamepad that referenced this issue Mar 13, 2024
xingri added a commit to xingri/gamepad that referenced this issue Mar 13, 2024
xingri added a commit to xingri/gamepad that referenced this issue Mar 14, 2024
xingri added a commit to xingri/gamepad that referenced this issue Mar 27, 2024
xingri added a commit to xingri/gamepad that referenced this issue Mar 28, 2024
xingri added a commit to xingri/gamepad that referenced this issue Mar 28, 2024
xingri added a commit to xingri/gamepad that referenced this issue Mar 28, 2024
xingri added a commit to xingri/gamepad that referenced this issue Mar 29, 2024
@DanielHerr
Copy link

What about devices which have 2 touchpads? Steam Controller, Steam Deck, and Ayaneo Kun have left and right touchpads.

@xingri
Copy link

xingri commented Apr 8, 2024

What about devices which have 2 touchpads? Steam Controller, Steam Deck, and Ayaneo Kun have left and right touchpads.

@DanielHerr Thanks for the feedback. As I already mentioned on the PR#196, The table defines the extended buttons used by some gamepad models. It's important to note that this list is not exhaustive, and user agents may utilize different or additional buttons for these or other gamepad models.

I guess we can propose something like "trackpad_left" and trackpad_right" button types for the multiple touchpads case.

xingri added a commit to xingri/gamepad that referenced this issue Apr 8, 2024
xingri added a commit to xingri/gamepad that referenced this issue Apr 22, 2024
xingri added a commit to xingri/gamepad that referenced this issue May 9, 2024
xingri added a commit to xingri/gamepad that referenced this issue May 9, 2024
xingri added a commit to xingri/gamepad that referenced this issue May 9, 2024
xingri added a commit to xingri/gamepad that referenced this issue May 10, 2024
xingri added a commit to xingri/gamepad that referenced this issue May 10, 2024
xingri added a commit to xingri/gamepad that referenced this issue May 10, 2024
xingri added a commit to xingri/gamepad that referenced this issue May 10, 2024
xingri added a commit to xingri/gamepad that referenced this issue May 10, 2024
xingri added a commit to xingri/gamepad that referenced this issue May 10, 2024
xingri added a commit to xingri/gamepad that referenced this issue May 10, 2024
xingri added a commit to xingri/gamepad that referenced this issue May 10, 2024
xingri added a commit to xingri/gamepad that referenced this issue May 11, 2024
xingri added a commit to xingri/gamepad that referenced this issue Jul 8, 2024
xingri added a commit to xingri/gamepad that referenced this issue Jul 11, 2024
xingri added a commit to xingri/gamepad that referenced this issue Jul 11, 2024
@nondebug nondebug added the TPAC2024 Topic for discussion at TPAC 2024 label Sep 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
TPAC2024 Topic for discussion at TPAC 2024
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants