go2rtc is laggy/slow 2.0

[clsferguson]

Checklist:

• Please try updating to the latest available version.
• Please try clearing your browser cache.
  done and done

[REQUIRED] Card diagnostic information:

card_version: 6.0.0-beta.8
browser: >-
  Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like
  Gecko) Chrome/131.0.0.0 Safari/537.36 Edg/131.0.0.0
date: 2024-11-30T07:15:52.741Z
frigate_versions:
  95d02a0efa42332dfb375b526e010897: 5.5.1/0.15.0-e76f4e9
lang: en
timezone: America/Denver
git:
  hash: a869109
  build_date: Mon, 14 Oct 2024 00:01:23 GMT
  commit_date: Sun, 13 Oct 2024 16:50:47 -0700
ha_version: 2024.11.3
config:
  type: custom:frigate-card
  panel: true
  cameras:
    - camera_entity: camera.front_1
      live_provider: go2rtc
      id: front_1_hd
      go2rtc:
        modes:
          - webrtc
          - mse
      capabilities:
        disable_except:
          - substream
    - camera_entity: camera.doorbell
      live_provider: go2rtc
      id: doorbell_hd
      go2rtc:
        modes:
          - webrtc
          - mse
      capabilities:
        disable_except:
          - substream
    - camera_entity: camera.driveway_cam
      live_provider: go2rtc
      id: driveway_cam_hd
      go2rtc:
        modes:
          - webrtc
          - mse
      capabilities:
        disable_except:
          - substream
    - camera_entity: camera.garage
      live_provider: go2rtc
      id: garage_hd
      go2rtc:
        modes:
          - webrtc
          - mse
      capabilities:
        disable_except:
          - substream
    - camera_entity: camera.rear_garage_1
      live_provider: go2rtc
      id: rear_garage_1_hd
      go2rtc:
        modes:
          - webrtc
          - mse
      capabilities:
        disable_except:
          - substream
    - camera_entity: camera.shop
      live_provider: go2rtc
      id: shop_hd
      go2rtc:
        modes:
          - webrtc
          - mse
      capabilities:
        disable_except:
          - substream
    - camera_entity: camera.shop_shed
      live_provider: go2rtc
      id: shop_shed_hd
      go2rtc:
        modes:
          - webrtc
          - mse
      capabilities:
        disable_except:
          - substream
    - camera_entity: camera.front_1
      live_provider: go2rtc
      id: front_1_sub
      go2rtc:
        modes:
          - webrtc
          - mse
        stream: front_1_sub
      dependencies:
        cameras:
          - front_1_hd
    - camera_entity: camera.doorbell
      live_provider: go2rtc
      id: doorbell_sub
      go2rtc:
        modes:
          - webrtc
          - mse
        stream: doorbell_sub
      dependencies:
        cameras:
          - doorbell_hd
    - camera_entity: camera.driveway_cam
      live_provider: go2rtc
      id: driveway_cam_sub
      go2rtc:
        modes:
          - webrtc
          - mse
        stream: driveway_cam_sub
      dependencies:
        cameras:
          - driveway_cam_hd
    - camera_entity: camera.garage
      live_provider: go2rtc
      id: garage_sub
      go2rtc:
        modes:
          - webrtc
          - mse
        stream: garage_sub
      dependencies:
        cameras:
          - garage_hd
    - camera_entity: camera.rear_garage_1
      live_provider: go2rtc
      id: rear_garage_1_sub
      go2rtc:
        modes:
          - webrtc
          - mse
        stream: rear_garage_1_sub
      dependencies:
        cameras:
          - rear_garage_1_hd
    - camera_entity: camera.shop
      live_provider: go2rtc
      id: shop_sub
      go2rtc:
        modes:
          - webrtc
          - mse
        stream: shop_sub
      dependencies:
        cameras:
          - shop_hd
    - camera_entity: camera.shop_shed
      live_provider: go2rtc
      id: shop_shed_sub
      go2rtc:
        modes:
          - webrtc
          - mse
        stream: shop_shed_sub
      dependencies:
        cameras:
          - shop_shed_hd
  menu:
    style: outside
    buttons:
      substreams:
        icon: mdi:high-definition
      microphone:
        enabled: true
        alignment: opposing
        toggle: true
  dimensions: {}

[REQUIRED] Description of problem:
webrtc doesn't work well. I have no way to test webrtc in frigate ui I don't think.

I am using opnsense as my firewall. Hass-os for home assistant, frigate integration, frigate-card. Frigate is installed in a docker container on a different machine, using frigate 0.15.0 beta 2 now.

frigate config contains

  webrtc:
    candidates:
      - 10.0.1.14:8555
      - stun:8555

locally, and through cloudflare tunnel, the behavior is the same. if I leave the go2rtc mode as "- mse" the frame rate is full frames. But if I only use "- webrtc" I only get about 1fps.

streaming through nabu casa is very limited using mse, but won't play at all using webrtc

similar to this #1671

I tried looking for help in https://www.reddit.com/r/frigate_nvr/comments/1gwqoob/nabu_casa_and_frigate_in_a_docker_container_laggy/ too.

[OPTIONAL] Last working release (if known):

[OPTIONAL] Javascript errors shown in the web inspector:

[OPTIONAL] Additional information:

[dermotduffy]

I only get about 1fps

I am guessing the WebRTC stream just isn't loading at all (for whatever reason), instead the card is trying to show you something and it's just showing you a HomeAssistant snapshot every second. You could verify this with:

live:
  show_image_during_load: false

... and the Frigate card will make no attempt to "cover up" if the stream doesn't load. You'd just get empty video players. Then you need to figure out why that's not loading. I'd start here and make sure you have followed all of these steps. Your Javascript console (F12 in your browser) may also show something useful. Good luck.

[clsferguson]

I only get about 1fps

I am guessing the WebRTC stream just isn't loading at all (for whatever reason), instead the card is trying to show you something and it's just showing you a HomeAssistant snapshot every second. You could verify this with:

live:
  show_image_during_load: false

... and the Frigate card will make no attempt to "cover up" if the stream doesn't load. You'd just get empty video players. Then you need to figure out why that's not loading. I'd start here and make sure you have followed all of these steps. Your Javascript console (F12 in your browser) may also show something useful. Good luck.

show image: false helped. Doing this I realized my h264 streams were working, but my h265 streams were not.

I did have the stun stuff set up, but what I discovered was it is a codec issue. webrtc needs h264 streams. Some of my 4k cams were h265, and I had my hikvision as h265.

I almost had it licked, but then something happened and I can't figure that out now, lol.

But one issue with h264/webrtc was with my hikvision ultra wide cams. I tried setting them to h264 native and webwtc wouldn't work. So I tried h265 and transcoded to h264, but ffmpeg said it couldn't do wider then 4096px.

I tried updating gpu drivers now the whole thing is shot and I can't get it working again. Some other issue I guess. Details

thanks though, your guidance did help.