Skip to content

Latest commit

 

History

History
339 lines (315 loc) · 8.75 KB

README.md

File metadata and controls

339 lines (315 loc) · 8.75 KB

Voice Verification API

Introduction

This project is built with 2 Django applications: SpeechEngine and VoiceVerifier.

  • SpeechEngine
    SpeechEngine takes care of handling streams of audio data, feeding the data to different models and fetching results. Currently, the application only uses one voice verification model (ECAPA TDNN) implemented by Speechbrain.
    More information on the open-source code can be found in this link: https://github.com/speechbrain/speechbrain

  • VoiceVerifier
    VoiceVerifier is the API with routes exposed to end-users. It uses SpeechEngine API to enroll users using their user IDs, storing and handling user voiceprints, and verifying new audio streams against enrolled users.

Python Environment

Both SpeechEngine and VoiceVerifier uses Python 3.8 and pip3. Python environment is already set up in Docker images given as .tar files.

Docker Environment

The project uses docker-compose to run SpeechEngine and VoiceVerifier in parallel.

Images

Before building the docker-compose project, you must load the docker images included as voiceverifier.tar and speechbrain.tar files in the project folder.

IP Address

SpeechEngine and VoiceVerifier are given fixed ip addresses of 172.19.0.2 and 172.19.0.3. Only VoiceVerifier (172.19.0.3) can be accessed from outside the local server through port 3000.

Configuration

All docker-compose configurations can be found in docker-compose.yml.

version: '3.5'
services:
  speechbrain: 
    image: speechbrain
    container_name: speechbrain_compose
    tty: true
    volumes: 
      - "{local folder directory for ./SpeechEngine}:/home/SpeechEngine"
    command: 
      bash /home/SpeechEngine/run.sh
    networks: 
      voice_verifier_net:
        ipv4_address: 172.19.0.2

  voice_verifier:
    image: voice_verifier
    container_name: voice_verifier_compose
    tty: true
    ports: 
      - 3000:3000
    volumes: 
      - "{local folder directory for ./VoiceVerifier}:/home/VoiceVerifier"
    command:
      bash /home/VoiceVerifier/run.sh
    networks: 
      voice_verifier_net:
        ipv4_address: 172.19.0.3

networks: 
  voice_verifier_net:
    ipam:
      driver: default
      config:
        - subnet: 172.19.0.0/16

Configure

All settings for SpeechEngine and VoiceVerifier can be found in "/SpeechEngine/SpeechEngine/settings.py" and "/VoiceVerifier/VoiceVerifier/settings.py".

Running

  • Install Docker and Docker Compose
    Install Docker: https://docs.docker.com/engine/install/
    Install Docker Compose: https://docs.docker.com/compose/install/

  • Configure Volumes
    Edit "volumes: " sections in docker-compose.yml to create volumes to local directories.

  • Load images
    Using image tar files

    # edit image names in docker-compose.yml from jihee4375/speechbrain, jihee4375/voice_verifier to speechbrain, voice_verifier
docker load < speechbrain.tar  
docker load < voice_verifier.tar

    Load from Docker hub

    # https://hub.docker.com/repository/docker/jihee4375/speech_engine
docker pull jihee4375/speech_engine:latest  
# https://hub.docker.com/repository/docker/jihee4375/voice_verifier
docker pull jihee4375/voice_verifier:latest

  • Run application

    docker-compose up

    Configuration in docker-compose.yml is set to automatically execute run.sh in SpeechEngine and VoiceVerifier at start of the application.

Documentation

SpeechEngine and VoiceVerifier have separate Swagger documentation that can be accessed once the server is running.

SpeechEngine: http://172.19.0.2:3000/swagger/
VoiceVerifier: http://{ip address of local server}:3000/swagger/ or http://171.19.0.3:3000/swagger/ inside local server

Logging

Logs for SpeechEngine:
All logs are stored "./SpeechEngine/logs/"

  • request_log
    Logs all incoming request and any error messages incurred if applicable 
    2021-08-20 04:46:52,537 "POST /http_stream HTTP/1.1" 200 48
2021-08-20 04:46:52,553 "POST /http_stream HTTP/1.1" 200 48
2021-08-20 04:46:52,567 "POST /http_stream HTTP/1.1" 200 48
2021-08-20 04:46:52,584 "POST /http_stream/242e10f5-1a00-46a8-85ab-5fc41d36e6ee HTTP/1.1" 200 0
2021-08-20 04:46:52,599 "POST /http_stream/242e10f5-1a00-46a8-85ab-5fc41d36e6ee HTTP/1.1" 200 0
  • system_log
    Any other logs incurred by SpeechEngine application

Logs for VoiceVerifier:

  • request_log
    Same as SpeechEngine
  • system_log
    Same as SpeechEngine
  • activity_log
    Logs all enrollment, deletion of enrollment and verification or any failures. 
    2021-08-24 06:35:59,278 Enrolled test1
2021-08-24 06:35:59,379 Enrolled test2
2021-08-24 06:35:59,452 Verified test1 (result: True, score: 0.6241418719291687)
2021-08-24 06:35:59,517 Verified test2 (result: False, score: -0.09651359170675278)
2021-08-24 06:35:59,560 All enrollments (2) deleted

API Routes

SpeechEngine

Stream Routes

Method Route Body Response
GET/http_streamNA 200: OK 
{
  "streams": [
    "4d038bd4-8d92-4d16-8697-36770463c4d8",
    "8af9b39d-7830-4215-9244-fc2027730639"
  ]
}
500: Internal Server Error
POST /http_stream NA 200: OK 
{
    "uuid": "4d038bd4-8d92-4d16-8697-36770463c4d8"
}
500: Internal Server Error
POST /http_stream/{uuid} 
{ "data": "..."}
200: OK
400: Malformed request
404: No such UUID
500: Internal Server Error
512: Something went wrong connecting to Redis database
DELETE /http_stream/{uuid} NA 200: OK
400: Malformed request
404: No such UUID
500: Internal Server Error
512: Something went wrong connecting to Redis database

Voiceprint Routes

Method Route Body Response
GET /http_stream/voiceprint/{uuid} NA 200: OK 
{ "voiceprint": [29.842374801635742, -15.604244232177734, 1.7878642082214355, ...] }
404: No stream found with this UUID
500: Internal Server Error
POST /speakeridt/compare_vp 
{  "voiceprint1": [29.842374801635742, 1.7878642082214355, ...], "voiceprint2": [-7.919283866882324, 2.2755887508392334, ...]}
200: OK 
{ "score": 0.678 }
500: Internal Server Error

VoiceVerifier

Stream Routes

Method Route Body Response
GET/get_all_streamsNA 200: OK 
{
  "streams": [
    "4d038bd4-8d92-4d16-8697-36770463c4d8",
    "8af9b39d-7830-4215-9244-fc2027730639"
  ]
}
500: Internal Server Error
POST /start_stream NA 200: OK 
{
    "uuid": "4d038bd4-8d92-4d16-8697-36770463c4d8"
}
500: Internal Server Error
POST /upload_stream_data/{uuid} 
{ "data": "..."}
200: OK
400: Malformed request
404: No such UUID
500: Internal Server Error

Verifier Routes

Method Route Body Response
DELETE/vv/delete_all_enrollmentNA 200: OK 
{ "deleted_count": 21 }
500: Internal Server Error
DELETE/vv/delete_enrollment/{uuid}NA 200: Deleted
400: A user with this external ID was not found
500: Internal Server Error
POST /vv/enroll 
{
  "uuid": "4d038bd4-8d92-4d16-8697-36770463c4d8",
  "external_id": "test"
}
200: OK
400: uuid or external ID was not passed over
404: Stream with this uuid doesn't exist
409: user_id already exists
500: Internal Server Error
GET/vv/get_voiceprint/{id}NA 200: OK 
{ "voiceprint": [29.842374801635742, -15.604244232177734, 1.7878642082214355, ...] }
460: No user enrolled with this external ID
500: Internal Server Error
GET/vv/list_enrollments/{id}NA 200: OK 
{
  "ids": [
    "test1",
    "test2"
  ]
}
500: Internal Server Error
POST /vv/verify/{id} 
{ "uuid": "4d038bd4-8d92-4d16-8697-36770463c4d8",
"external_id": "test1"}
200: OK 
{ "score": 0.678 }
460: A user with this external ID doesn't exist
461: A stream with this uuid doesn't exist
500: Internal Server Error