Warning no provider set, not transcribing

[philipgrueter]

Hello all,

we tried to implement this in our bbb setup but we are having some issues.
When starting the nodeJS server and starting a conference with voice stream, we get the message "warn: [transcript-manager] No provider set, not transcribing"

Also we disabled gladia, because we don't want our services talking to an external service. Is this supported?

This is our config:

log:
  filename: /var/log/bigbluebutton/bbb-transcription-controller.log
  level: info
  stdout: true
  gladiaProxy: /var/log/bigbluebutton/gladia-proxy.log

redis:
  host: 0.0.0.0
  port: "6379"
  # password: foobared
  publishChannel: 'from-akka-apps-redis-channel'

includePartialResults: true

# If set to false connection to the transcription provider will be kept open from
# the call start to the call hangup. Otherwise it will be opened/closed whenever a
# user start/stops talking. Setting to true might yield lower quality results
disconnectOnSilence: false
# If disconnectOnSilence is true we'll wait this time in seconds
# before closing the connection. If this is too low we might have too much
# connections/disconnections and final results might be cut too short
closeConnectionAfterSeconds: 10

# in mhz
# Valid values are '16' and '8', but 8 seems to be
# too low to get accurate transcriptions
sampleRate: '16'

# Each transcription server has its own config section
# startMessage and endMessage are the messages sent whenever mod_audio_fork's
# connection starts/ends

# servers/server registers transcription server websocket URLs
# I've configured my servers using an nginx reverse proxy for each
# language, see the example below

vosk:
  startMessage: '{"config": { "sample_rate": "" } }'
  endMessage: '{"eof" : 1 }'
  servers:
    en-US: wss://127.0.0.1/voskEN
#    es-ES: wss://HOST/voskES
#    pt-BR: wss://HOST/voskPT
#    fr-FR: wss://HOST/voskFR

# TODO: whispering is still not supported, need to get
# the start/end messages right

gladia:
#  startMessage: '{"x_gladia_key": "", "sample_rate": 0, "bit_depth": 16, "model_type": "fast", "endpointing": 300 }'
#  endMessage: '{}'
#  server: ws://localhost:8777
  proxy:
    enabled: false
#    address: "wss://api.gladia.io/audio/text/audio-transcription"
#    minPartialDuration: 3
#    languageManual: true

#whispering:
#  startMessage: '{"config": { "sample_rate": ""} }'
#  endMessage: '{"eof" : 1 }'
#  server:  wss://HOST/whispering

# Setup freeswitch credentials to make ESL work

freeswitch:
  ip: 0.0.0.0
  port: 8021
  password: REDACTED

And this is our log:

2024-03-20T10:31:34.452Z - info: [transcript-manager] Connected to FreeSWITCH ESL
2024-03-20T10:31:52.475Z - info: [transcript-manager] FS: Associating channel fc99362d-325b-43c1-b324-36dbc87afbc9 2pbp9fvaajpj2dhbh7hc userId: w_hstui65d0oje
2024-03-20T10:31:52.477Z - info: [transcript-manager] Start mod_audio_fork connection fc99362d-325b-43c1-b324-36dbc87afbc9 w_hstui65d0oje
2024-03-20T10:31:52.483Z - warn: [transcript-manager] No provider set, not transcribing
2024-03-20T10:31:52.533Z - info: [transcript-manager] FS: floor changed 96954 10
2024-03-20T10:31:56.546Z - info: [transcript-manager] FS: Start talking fc99362d-325b-43c1-b324-36dbc87afbc9 userId: w_hstui65d0oje
2024-03-20T10:31:59.034Z - info: [transcript-manager] FS: Stop Talking fc99362d-325b-43c1-b324-36dbc87afbc9 userId: w_hstui65d0oje
2024-03-20T10:32:00.399Z - info: [transcript-manager] FS: Start talking fc99362d-325b-43c1-b324-36dbc87afbc9 userId: w_hstui65d0oje
2024-03-20T10:32:04.033Z - info: [transcript-manager] FS: Stop Talking fc99362d-325b-43c1-b324-36dbc87afbc9 userId: w_hstui65d0oje

[bwildenhain]

Did you tell BBB to use vosk?

yq w -i /etc/bigbluebutton/bbb-html5.yml public.app.audioCaptions.enabled true yq w -i /etc/bigbluebutton/bbb-html5.yml public.app.audioCaptions.provider vosk

[philipgrueter]

(Edit: I checked again, I forgot to start the docker when testing it again today.)
I don't get the warning that the provider isn't set, but still nothing is being transcribed in BBB.
I can active the "Write closed captions" Feature when using chrome, but it's neither writing in the text field or generating CC on my screen anywhere.

[arbdevml]

Feel free to try these step by step:

1. Make the directory /home/docker/kaldi/
   sudo mkdir -p /home/docker/kaldi/

2. Add kaldi docker-compose.yml

sudo cat > /home/docker/kaldi/docker-compose.yml << "EOF"
version: "2.4"
networks:
  kaldi-intranet:
    driver: bridge
    internal: true
    ipam:
      config:
        - subnet: 172.27.100.0/25
services:
    kaldien:
        image: alphacep/kaldi-en:latest
        container_name: kaldien
        networks:
            kaldi-intranet:
                ipv4_address: 172.27.100.10
        restart: unless-stopped
EOF

3. Start kaldi
   sudo docker-compose -f /home/docker/kaldi/docker-compose.yml up -d

4. Add /etc/bigbluebutton/bbb-html5.yml

sudo cat > /etc/bigbluebutton/bbb-html5.yml << "EOF"
public:
  app:
    audioCaptions:
      enabled: true
      mobile: true
      provider: vosk
      language:
        available:
          - en-US
  captions:
    enabled: true
    dictation: true
    showButton: true
EOF

5. Install bbb-transcription-controller
   sudo apt install bbb-transcription-controller

6. Add /usr/local/bigbluebutton/bbb-transcription-controller/config/default.yml

sudo cat > /usr/local/bigbluebutton/bbb-transcription-controller/config/default.yml << "EOF"
log:
  filename: /var/log/bigbluebutton/bbb-transcription-controller.log
  level: debug
  stdout: true
  gladiaProxy: /var/log/bigbluebutton/gladia-proxy.log

redis:
  host: 0.0.0.0
  port: "6379"
  # password: foobared
  publishChannel: 'from-akka-apps-redis-channel'

includePartialResults: true
disconnectOnSilence: false
closeConnectionAfterSeconds: 10
sampleRate: '16'

vosk:
  startMessage: '{"config": { "sample_rate": "" } }'
  endMessage: '{"eof" : 1 }'
  servers:
    en-US: ws://172.27.100.10:2700

gladia:
  proxy:
    enabled: false

freeswitch:
  ip: 0.0.0.0
  port: 8021
  password: bigbluebutton_freeswitch_ESL_password
EOF

7. Create a variable FREESWITCH_ESL_PASSWORD

FREESWITCH_ESL_PASSWORD=$(yq r /usr/local/bigbluebutton/bbb-webrtc-sfu/config/default.yml freeswitch.esl_password)

8. Insert the variable FREESWITCH_ESL_PASSWORD into /usr/local/bigbluebutton/bbb-transcription-controller/config/default.yml

sudo yq w -i /usr/local/bigbluebutton/bbb-transcription-controller/config/default.yml freeswitch.password $FREESWITCH_ESL_PASSWORD

9. Restart bbb-conf

sudo bbb-conf --restart

10. Visit your BigBlueButton and turn on the button CC, enjoy.