Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Wayland and PipeWire support #46

Open
ehfd opened this issue Aug 25, 2022 · 13 comments
Open

Wayland and PipeWire support #46

ehfd opened this issue Aug 25, 2022 · 13 comments
Labels
enhancement New feature or request funding Requires funding to implement help wanted External contribution is required interface OS input, display, or audio interfaces

Comments

@ehfd
Copy link
Member

ehfd commented Aug 25, 2022

Spun out from #34.

This is a placeholder issue for all discussions associated with supporting Wayland capture.
NVIDIA has introduced GBM support to its 510.xx drivers on Linux, and GStreamer's Pipewire plugin to capture Wayland would be mature enough by now. This also means that pulseaudio and Xorg no longer has to be matched with a timer.

Implementation of capturing Wayland using GStreamer's Pipewire plugin is accessible with https://github.com/H-M-H/Weylus or https://github.com/pavlobu/deskreen and some more Wayland capturing references are available in https://github.com/SunshineStream/Sunshine.
How to deploy a headless Sway DE on Wayland: https://github.com/bbusse/swayvnc-firefox.

@ehfd
Copy link
Member Author

ehfd commented Aug 25, 2022

Relevant code: https://github.com/selkies-project/selkies-gstreamer/blob/master/src/selkies_gstreamer/gstwebrtc_app.py
Anyone with GStreamer experience, or willing to learn can solve this.
The PipeWire plugin of GStreamer should be able to do it.

https://github.com/selkies-project/selkies-gstreamer/blob/master/src/selkies_gstreamer/webrtc_input.py
This also needs to include input for Wayland interfaces. https://github.com/any1/wayvnc is an interesting reference.

@ehfd
Copy link
Member Author

ehfd commented Sep 26, 2022

Since there must be another option to use keyboard and mouse operations using Wayland without uinput (in order to keep supporting Docker and Kubernetes), the virtual keyboard and virtual pointer specs may be an alternative. This is only supported for Sway and wlroots compositors.

xdg-desktop-portal can work out better for GNOME or KDE, as long as supporting Sway and wlroots as well.

The virtual keyboard and virtual pointer specs have nothing to do with uinput, confirmed.

@ehfd
Copy link
Member Author

ehfd commented Sep 26, 2022

Luckily, it looks like WLR_BACKENDS=headless and WLR_LIBINPUT_NO_DEVICES=1 exist in wlroots.

VirtualGL for Wayland with VirtualGL/virtualgl#43 combined with WLR_BACKENDS=headless will be sufficient to replace VirtualGL + Xvfb.

@ehfd
Copy link
Member Author

ehfd commented Sep 27, 2022

People are welcome to suggest or contribute attempts from using xdg-desktop-portal as it should be possible to do even now. It can be combined with the Pipewire plugin of GStreamer.

@ghost
Copy link

ghost commented Oct 2, 2022

It seems that team-viewer is working on a wayland support

No idea how they get the input working.

@ehfd
Copy link
Member Author

ehfd commented Oct 2, 2022

It seems that team-viewer is working on a wayland support

No idea how they get the input working.

I believe this would be through xdg-desktop-portal.

@ehfd
Copy link
Member Author

ehfd commented Oct 5, 2022

@ehfd ehfd added enhancement New feature or request funding Requires funding to implement help wanted External contribution is required upstream Requires upstream development from dependencies labels Oct 12, 2022
@ehfd ehfd changed the title Wayland support [WAY-UPSTREAM] Wayland support Oct 12, 2022
@ehfd ehfd changed the title [WAY-UPSTREAM] Wayland support [WL-UPSTREAM] Wayland support Oct 13, 2022
@ehfd ehfd added the interface OS input, display, or audio interfaces label Oct 16, 2022
@ehfd
Copy link
Member Author

ehfd commented Oct 18, 2022

xdg-desktop-portal with GStreamer:

https://gitlab.gnome.org/-/snippets/39

@ehfd
Copy link
Member Author

ehfd commented Feb 12, 2023

Strong reference: Xpra-org/xpra#3750 (comment)

@ehfd
Copy link
Member Author

ehfd commented Sep 26, 2023

Because of #95, we might be able to use uinput in Wayland without privileges.

@ehfd ehfd removed the upstream Requires upstream development from dependencies label Nov 19, 2023
@ehfd
Copy link
Member Author

ehfd commented Nov 19, 2023

Weylus (https://github.com/H-M-H/Weylus) has all of the initial answers and since we are already on GStreamer, GStreamer-PipeWire can be promptly used with xdg-desktop-portal. However Wolf (https://github.com/games-on-whales/wolf) came up with a way to not rely on KDE's or GNOME's session manager and simply wrap a compositor inside a custom virtual screen.

Overriding uinput (and the keyboard/mouse as well?) is also very useful for gamepads in Wayland.

@ehfd
Copy link
Member Author

ehfd commented Mar 12, 2024

gst-pipewire is essential for this capability. This GStreamer plugin is included in pipewire, not GStreamer.

Since PipeWire is not meant to be bundled with GStreamer unless Conda is used, the gst-env script for GStreamer build has been edited to find it with the environment variable GST_PLUGIN_PATH.

Check #34 (comment) for properties of pipewiresrc and pipewiresink.

https://gstreamer.freedesktop.org/documentation/gstreamer/running.html

https://gstreamer.freedesktop.org/documentation/installing/building-from-source-using-meson.html#how-does-it-work

For Conda, need conda-forge/staged-recipes#25800.

@ehfd ehfd changed the title [WL-UPSTREAM] Wayland support Wayland and PipeWire support Jun 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request funding Requires funding to implement help wanted External contribution is required interface OS input, display, or audio interfaces
Projects
None yet
Development

No branches or pull requests

1 participant