Hololens 2 support

[rahulswa08]

Does the repo support build for hololens2?

[hoffmann-stefan]

Hi @rahulswa08,

I'm not sure about the current state of ros2_dotnet for Hololens 2 and therefore UWP.

As UWP is an sandboxed environment that the software needs to support, for example not using normal WIN32 API that are not available in UWP.
As far as I know there was support for at least some of the core ROS 2 packages for UWP, but with the time those packages did loose support for UWP again, as UWP isn't actively testet in CI on those packages.
In that sense this repo should support UWP as I wouldn't know about something that broke it, but the depended ROS 2 packages would need to support it again first so we can test this.

ooeygui did try to fix this and upstream the changes to the needed ROS 2 packages and dependencies, see #92, but there isn't much recent information on the issue.

@ooeygui: Do you have any updates on this? Is this all done and we can try enabling it again or is there work left to do? Is there some more recent TODO list somewhere?
In some recent news article I saw that Hololens 2 gets Windows 11 support. Maybe this means we don't have to use UWP for this but rather can depend on normal WIN32 API and modern .NET versions?

@rahulswa08: If you mind to share, what would be your use-case for ROS 2 on the Hololens? Is this some student project, a hobby or some professional use? Maybe you could assist in getting this to work again?

[ooeygui]

Hi @hoffmann-stefan,
In order to run ROS2 on current generation of Hololens, all of the native code needs to be compiled to be UWP container friendly. (This was not the case for Hololens 1). I had made significant changes to the ROS2 runtime itself, FastDDS, and numerous dependencies. The current codebase, which includes the repos file - lives at http://aka.ms/ros/mrtk_native.

I'm not actively working on it.

[rahulswa08]

Hi @rahulswa08,

I'm not sure about the current state of ros2_dotnet for Hololens 2 and therefore UWP.

As UWP is an sandboxed environment that the software needs to support, for example not using normal WIN32 API that are not available in UWP. As far as I know there was support for at least some of the core ROS 2 packages for UWP, but with the time those packages did loose support for UWP again, as UWP isn't actively testet in CI on those packages. In that sense this repo should support UWP as I wouldn't know about something that broke it, but the depended ROS 2 packages would need to support it again first so we can test this.

ooeygui did try to fix this and upstream the changes to the needed ROS 2 packages and dependencies, see #92, but there isn't much recent information on the issue.

@ooeygui: Do you have any updates on this? Is this all done and we can try enabling it again or is there work left to do? Is there some more recent TODO list somewhere? In some recent news article I saw that Hololens 2 gets Windows 11 support. Maybe this means we don't have to use UWP for this but rather can depend on normal WIN32 API and modern .NET versions?

@rahulswa08: If you mind to share, what would be your use-case for ROS 2 on the Hololens? Is this some student project, a hobby or some professional use? Maybe you could assist in getting this to work again?

Hi, @hoffmann-stefan

Thanks for your reply. we are building a human robot interaction platform for a research study using hololens 2. we need to stream a video into hololens 2 and the other way communication too.For this purpose I am looking for some resources . I don't need ros2 in particular and not sure if needed ros at all. Any suggestions would be appriciated.

Thank you

[ooeygui]

Take a peek at https://github.com/microsoft/MixedReality-WebRTC. (although, It is deprecated). You can use WebRTC to display remote cameras. I've been contributing to https://github.com/RobotWebTools/webrtc_ros, including supporting ROS2 - which allows you to subscribe to a ROS camera, and publish it as RTC. (I've done a video on how to use this ROS node to output to a steamdeck, but webrtc to webrtc should work as well - https://www.youtube.com/watch?v=osiSNGhKwQo&list=PL2dJBq8ig-vh42rfm8QEZFoCpYaBG7x2u&index=3.)

[rahulswa08]

Thanks @ooeygui for your input. I have actually tried Mixedreality-WebRTC. But the repo is only supported on windows where as we are running a linux system on robot end. Can i use webrtc_ros repo to stream video to MR-webrtc app?

[ooeygui]

Sorry if I wasn't clear - the WebRTC_Ros node only works on linux - you would run that end on your ROS robot. The MRWebRTC component runs on the hololens. With some custom signaling, you should be able to connect them.

[rahulswa08]

Ohh great!!. I will definitely try that.

[hoffmann-stefan]

I had made significant changes to the ROS2 runtime itself, FastDDS, and numerous dependencies. The current codebase, which includes the repos file - lives at http://aka.ms/ros/mrtk_native.

@ooeygui Thanks for the information, I crated an PR that points to http://aka.ms/ros/mrtk for the current Hololens support and removes the duplicate install instructions from this repository's README.md

• Update Readme to point to http://aka.ms/ros/mrtk for Hololens support #113

[hoffmann-stefan]

closing this ass duplicate of #82

@rahulswa08 feel free to ask more questions or report bugs in new issues :)