WinUI 3 support: Open issues

[lukasf]

This is to keep track of the things that do not work 100% on WinUI.

• Slight flickering on subtitles, because we cannot access WinUI dispatcher
  • Possible solutions:
    • Create separate WinUI build and use WinUI DispatcherQueue
    • Use passthrough approach to move subtitle rendering down to MF framework
• BasicVideoEffect not working
  • Probably a bug in prerelease MediaPlayerElement
  • Let's re-test once the official 1.2 release is out
• Only works with packaged apps
  • Not sure if we can do anything about that
  • Self contained WinUI 3 app deployment is still being worked on

[brabebhin]

The video effect is probably not a bug in mpe. I tried with a mediaplayer object in frame server and still doesn't work.

It's probably some shenanigans related to COM activation.

[Minmoose]

Unpackaged apps can use the Microsoft.VCRTForwarders.140 Nuget package to make the project work.

See this blog post under Handling Dependencies for more information
https://blogs.windows.com/windowsdeveloper/2019/04/30/enhancing-non-packaged-desktop-apps-using-windows-runtime-components/

[lukasf]

Interesting read @Minmoose. There's still quite some unpleasant manual work required as it seems. I hope that MS is working on improving the copy deployment scenario. The compiler should be able to generate the required info automatically from the WinMD file and the dll.

[Minmoose]

My apologies, I wasn't very clear, nothing should have to be changed, it is all automatic, all you have to do is add the Microsoft.VCRTForwarders.140 NuGet package to the app consuming FFmpegInteropX
as I've done here with this demo project: https://github.com/Minmoose/FFmpegInteropX-WinUI-UnpackagedDemo

I would suggest adding Microsoft.VCRTForwarders.140 as a dependency to the WinUI NuGet package for FFmpegInteropX
but I don't think there's anyway for NuGet to detect whether it was installed into a package or unpackaged app, so the packaged app would also contain the Microsoft.VCRTForwarders.140 package which is useless for package apps causing a sort of dll bloat.

So perhaps a separate NuGet package for unpackaged apps that has the Microsoft.VCRTForwarders.140 dependency?
But that also seems kind of messy.

[lukasf]

I don't think it would be a good idea to include the forwarders as a NuGet dependency in a WinUI build. It is only required for unpackaged apps. For packaged apps, this would mean unneccessary dlls being deployed (and it could even cause trouble). Someone who wants to create an unpackaged app can add the dependency themselves. If it is really just adding the Nuget, then it is much easier than expected. I could add a note in the readme about that.

[lukasf]

Indeed I could get our WinUI sample app working as unpackaged app. Besides adding the forwarders NuGet, I had to edit the .csproj file and add <WindowsPackageType>None</WindowsPackageType>. After I installed the AppSdk runtime 1.2 preview, the app launched successfully. I had to comment out FutureAccessList, because that is not available to unpackaged apps, but besides that, everything worked correctly.

It is good to know that getting unpackaged apps working is rather painless. Still I guess that most or many apps will use packaged deployment, so we should not add stuff that is not required by that deployment type.

[insomniachi]

Unpackaged apps can use the Microsoft.VCRTForwarders.140 Nuget package to make the project work.

See this blog post under Handling Dependencies for more information https://blogs.windows.com/windowsdeveloper/2019/04/30/enhancing-non-packaged-desktop-apps-using-windows-runtime-components/

adding this fix the issue, but it broke my CI builds, i'm unable to run dotnet publish, this giving this error.

C:\Users\runneradmin\.nuget\packages\microsoft.vcrtforwarders.140\1.0.7\build\netcoreapp2.0\Microsoft.VCRTForwarders.140.targets(61,5): error MSB4801: The task factory "CodeTaskFactory" is not supported on the .NET Core version of MSBuild. [D:\a\Totoro\Totoro\Totoro.WinUI\Totoro.WinUI.csproj] 

is there a i away could bundle the dlls in the installer and remove the nuget reference?

[lukasf]

I am not sure if that is possible. Is it not possible use the normal MSBuild instead of the .NET Core MSBuild on your CI?

[brabebhin]

Just a thought, for winUI we technically can use c/c++ APIs to open the files by path, and not rely on the winRT IRandomAccessStream.

[lukasf]

Indeed, we could (and should) do that.

[insomniachi]

I am not sure if that is possible. Is it not possible use the normal MSBuild instead of the .NET Core MSBuild on your CI?

How would i do that?
microsoft/vcrt-forwarders#38
same issue is open for over 2 years.

[brabebhin]

I'm not so sure the basic video effect bug is related to the MediaPlayerElement.
It seems to complain about class not registered now. It appears to be something in our projection.

[lukasf]

There is an open issue on IBasicVideoEffect in the CsWinRT repo: microsoft/CsWinRT#1088

It looks like there are multiple unresolved issues, which prevent proper usage of the IBasicVideoEffect interface with CsWinRT. The ClassNotRegistered can be worked around with manual registration in the manifest, but then additional problems occur. If we make a WinRT version, it has to come without IBasicVideoEffect support (at least initially).