($150, Typescript) video embedding in UI app does not render properly on windows

[louis030195]

eg

[linear]

MED-97 video embedding in UI app does not render properly

[louis030195]

/bounty 150

[algora-pbc]

💎 $150 bounty • Screenpi.pe

Steps to solve:

1. Start working: Comment /attempt #319 with your implementation plan
2. Submit work: Create a pull request including /claim #319 in the PR body to claim the bounty
3. Receive payment: 100% of the bounty is received 2-5 days post-reward. Make sure you are eligible for payouts

Thank you for contributing to mediar-ai/screenpipe!

Add a bounty • Share on socials

Attempt	Started (GMT+0)	Solution
🟢 @tribhuwan-kumar	Sep 22, 2024, 11:54:52 AM	#372

[louis030195]

fixed

[louis030195]

not fixed

idk whats the issue

[louis030195]

issue is here https://github.com/mediar-ai/screenpipe/blob/main/screenpipe-app-tauri/components/video.tsx

[tribhuwan-kumar]

/attempt #319

Options

• Cancel my attempt

[tribhuwan-kumar]

@louis030195 is this issue similar to this ?

[louis030195]

@tribhuwan-kumar no, are you on windows? it works on macos

[tribhuwan-kumar]

@louis030195 I'm trying to build it windows 11, the pre_build.js script isn't working as I expected.

[louis030195]

@tribhuwan-kumar what is the error?

[louis030195]

increased bounty to 150

[tribhuwan-kumar]

@louis030195 I'm trying to build the screenpipe in windows 11 but the rusty_v8 library url keeps pointing to rusty_v8_release_x86_64-pc-windows-gnu.lib.gz which doesn't exist. it should be pointing on rusty_v8_release_x86_64-pc-windows-msvc.lib.gz.

  Looking for download in '"C:\\Users\\eirae\\.cargo\\.rusty_v8\\https___github_com_denoland_rusty_v8_releases_download_v0_105_1_rusty_v8_release_x86_64_pc_windows_gnu_lib_gz"'
  Downloading (using Python) https://github.com/denoland/rusty_v8/releases/download/v0.105.1/rusty_v8_release_x86_64-pc-windows-gnu.lib.gz
  Downloading https://github.com/denoland/rusty_v8/releases/download/v0.105.1/rusty_v8_release_x86_64-pc-windows-gnu.lib.gz...
  HTTP Error 404: Not Found

I found a Github issue about it, also there is Stackoverflow question about this same issue. but none of them are working.
I was able to build it on Arch Linux, but building it on windows seems rough, Sorry for the late reply!

Update:

build it!!

[tribhuwan-kumar]

not fixed

idk whats the issue

the problem is, video generated by backend encoded with MPEG-4 Part 2 which is an older codec. I tried to manually play those video in microsoft edge & firefox, i got same result like the screenpipe application. we can fix it by generating those videos in backend with any supported codec or converting those video in frontend using ffmpeg. I'd like to know which approach you want to apply.

Video encoding:

PS C:\Users\eirae\.screenpipe\data> ffprobe -v quiet -print_format json -show_streams '.\monitor_65537_2024-09-27_02-53-30.mp4'
{
    "streams": [
        {
            "index": 0,
            "codec_name": "mpeg4",
            "codec_long_name": "MPEG-4 part 2",
            "profile": "Simple Profile",

MPEG-4 part 2 vs H.264 encoding in microsoft edge:

[tribhuwan-kumar]

@louis030195 I've a working PR tested in windows 11 & the video is rendering properly!!
if you say I'll open a pull request

[louis030195]

nice send a PR ❤️ @tribhuwan-kumar

[louis030195]

@louis030195 I'm trying to build the screenpipe in windows 11 but the rusty_v8 library url keeps pointing to rusty_v8_release_x86_64-pc-windows-gnu.lib.gz which doesn't exist. it should be pointing on rusty_v8_release_x86_64-pc-windows-msvc.lib.gz.

  Looking for download in '"C:\\Users\\eirae\\.cargo\\.rusty_v8\\https___github_com_denoland_rusty_v8_releases_download_v0_105_1_rusty_v8_release_x86_64_pc_windows_gnu_lib_gz"'
  Downloading (using Python) https://github.com/denoland/rusty_v8/releases/download/v0.105.1/rusty_v8_release_x86_64-pc-windows-gnu.lib.gz
  Downloading https://github.com/denoland/rusty_v8/releases/download/v0.105.1/rusty_v8_release_x86_64-pc-windows-gnu.lib.gz...
  HTTP Error 404: Not Found

I found a Github issue about it, also there is Stackoverflow question about this same issue. but none of them are working. I was able to build it on Arch Linux, but building it on windows seems rough, Sorry for the late reply!

Update: build it!!

would also appreciate anything to make windows building more smooth :)

[louis030195]

not fixed
idk whats the issue

the problem is, video generated by backend encoded with MPEG-4 Part 2 which is an older codec. I tried to manually play those video in microsoft edge & firefox, i got same result like the screenpipe application. we can fix it by generating those videos in backend with any supported codec or converting those video in frontend using ffmpeg. I'd like to know which approach you want to apply.

Video encoding:

PS C:\Users\eirae\.screenpipe\data> ffprobe -v quiet -print_format json -show_streams '.\monitor_65537_2024-09-27_02-53-30.mp4'
{
    "streams": [
        {
            "index": 0,
            "codec_name": "mpeg4",
            "codec_long_name": "MPEG-4 part 2",
            "profile": "Simple Profile",

MPEG-4 part 2 vs H.264 encoding in microsoft edge:

cool, send a PR lets see

[louis030195]

@tribhuwan-kumar i will have to revert this PR - this broke windows completely, does crash and not fix the issue

[tribhuwan-kumar]

@tribhuwan-kumar i will have to revert this PR - this broke windows completely, does crash and not fix the issue

okay, let me find out what's causing this crash.

[louis030195]

trying to find the exact error but it seem ffmpeg is crashing for me on my windows

[tribhuwan-kumar]

trying to find the exact error but it seem ffmpeg is crashing for me on my windows

I just build the screenpipe and tested in my windows machine, Its working for me

the version of ffmpeg, I'm using is from pre_build.js:

PS C:\Users\eirae\Downloads\app\screenpipe\screenpipe-app-tauri> C:\Users\eirae\Downloads\app\screenpipe\screenpipe-app-tauri\src-tauri\ffmpeg\bin\x86\ffmpeg.exe -encoders | findstr "h264_mf"
ffmpeg version 7e69129-avbuild Copyright (c) 2000-2024 the FFmpeg developers
  built with Microsoft (R) C/C++ Optimizing Compiler Version 19.40.33813 for x86
  configuration: --extra-version=avbuild --disable-doc --disable-debug --enable-shared --enable-runtime-cpudetect --enable-libvpl --disable-ptx-compression --enable-mediafoundation --disable-postproc --toolchain=msvc --windres=llvm-windres --extra-windresflags='--target=i386-pc-windows-msvc -I/c/Program\ Files/Microsoft\ Visual\ Studio/2022/Enterprise/VC/Tools/MSVC/14.40.33807/include -I/c/Program\ Files/Microsoft\ Visual\ Studio/2022/Enterprise/VC/Tools/MSVC/14.40.33807/ATLMFC/include -I/c/Program\ Files/Microsoft\ Visual\ Studio/2022/Enterprise/VC/Auxiliary/VS/include -I/c/Program\ Files\ \(x86\)/Windows\ Kits/10/include/10.0.22621.0/ucrt -I/c/Program\ Files\ \(x86\)/Windows\ Kits/10/include/10.0.22621.0/um -I/c/Program\ Files\ \(x86\)/Windows\ Kits/10/include/10.0.22621.0/shared -I/c/Program\ Files\ \(x86\)/Windows\ Kits/10/include/10.0.22621.0/winrt -I/c/Program\ Files\ \(x86\)/Windows\ Kits/10/include/10.0.22621.0/cppwinrt -I/c/Program\ Files\ \(x86\)/Windows\ Kits/NETFXSDK/4.8/include/um' --enable-pic --extra-cflags='-I/d/a/avbuild/avbuild/tools/Vulkan-Headers/include -MD -Zi -FS -guard:cf -I/d/a/avbuild/avbuild/tools/dep/include -D_WIN32_WINNT=0x0600' --extra-ldflags='-NODEFAULTLIB:libcmt -DEBUG -guard:cf -OPT:REF -SUBSYSTEM:CONSOLE' --disable-filter=scale_cuda
  libavutil      59.  8.100 / 59.  8.100
  libavcodec     61.  3.100 / 61.  3.100
  libavformat    61.  1.100 / 61.  1.100
  libavdevice    61.  1.100 / 61.  1.100
  libavfilter    10.  1.100 / 10.  1.100
  libswscale      8.  1.100 /  8.  1.100
  libswresample   5.  1.100 /  5.  1.100
 V....D h264_mf              H264 via MediaFoundation (codec h264)

[louis030195]

@tribhuwan-kumar still does not render when using my windows machine which uses ffmpeg from the script (as written in the app logs)

you should be:

1. building the app bun tauri build
2. installing with windows installer
3. use the screenpipe exe from the installation folder and ffmpeg from installation folder
4. make sure the used ffmpeg is the one from the app installation folder and not another one which will not be installed on users runtime

you can also try to run the release-app.yml ci using https://github.com/nektos/act but not sure it works on windows

[tribhuwan-kumar]

@louis030195 checks I made:

• make sure the ffmpeg version
• change the video generating encoder for windows in screenpipe-server\src\video.rs to h264_mf
• build screenpipe backend with your latest commit
• build screenpipe-app-tauri with bun tauri build, but i got a missing private key error, however it generates the screenpipe_0.2.79_x64-setup.exe file. i ran that executable and it installed the screenpipe desktop application. then checked the application & the videos are playing fine!

Here is the executable of screenpipe desktop application for windows and a demo video about working

Missing TAURI_SIGNING_PRIVATE_KEY key error:

        C:\Users\eirae\Downloads\app\screenpipe\screenpipe-app-tauri\src-tauri\target\release\bundle\nsis\screenpipe_0.2.79_x64-setup.exe

    Error A public key has been found, but no private key. Make sure to set `TAURI_SIGNING_PRIVATE_KEY` environment variable.
error: script "tauri" exited with code 1