Vad plugin

[aaronchantrill]

Description

VAD Plugin

naomi/application.py

Attached input device parameters (input_samplerate, input_samplewidth, input_channels, input_chunksize) to the input_device object so they are all available to the mic and vad objects as the input_device object is passed around.

For consistency, also moved the output_chunksize and output_padding parameters to the output_device object.

Added initialization of Voice Activity Detection object and passed it to the initialization of the mic object.

naomi/mic.py

Removed the main logic around listen and active listen so I could move it into the VADPlugin class. This should make it easier to implement the "Passive Listen for Commands" project, because once
the passive listener identifies a keyword in the audio returned, we can just pass the same block of audio to the active listener for transcription. Simplified a lot of stuff. The original authors
were running two threads constantly scanning the audio input for keywords, and it appears that the only reason was to speed up keyword detection. The new VAD method works much differently, but
I'm interested in hearing whether anyone notices a difference.

naomi/plugin.py

Added skeleton for VADPlugin class

naomi/pluginstore.py

Added the VADPlugin as a new plugin class

naomi/testutils.py

Added a test audio_device class for my VAD tests

plugins/vad

Added two new plugins, snr_vad and webrtc_vad.

plugins/vad/snr_vad

This is based on the way voice activity currently works with naomi, which is basically just waiting for audio levels to go above a certain threshold and below a certain threshold.

I have always had trouble with this method, as different sound cards and different microphones register sound quite differently, so choosing a proper threshold level is often problematic.

I am now saying that anything over the mean plus one and a half times the standard deviation should be considered audio to pay attention to. I also reset every 100 samples by cutting all the
counts in half, thus ensuring that we aren't counting to ridiculous numbers over time and allowing noise levels to adjust fairly quickly to changes in the environment.

plugins/webrtc_vad

Uses the webrtcvad module, which can be installed via pip. This module requires that all chunks be 10, 20, or 30 ms. The default chunk size for Naomi is 64 ms, so you have to adjust the value of

audio:
  input_chunksize:

to either 160 (10ms), 320 (20ms), or 480 (30ms) in profile.yml, assuming a rate of 16000 samples/sec.

Related Issue

VAD plugin #144
[Feature-Request] - Passive Listening for commands #48
Automate STT training #103

Motivation and Context

This allows us to quickly and easily write and test Voice Activity Detection plugins without having to modify the main structure of Naomi. It also simplifies some handling of audio which should make some other projects simpler. And all of this should allow us to improve overall speech capture for building catalogs of data samples for training the STT engines.

How Has This Been Tested?

I have tested both plugins on both my x86 Raspbian Stretch VirtualBox machine and my Raspberry Pi 3B+.

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

• My code follows the code style of this project. (In fact, I fixed a bunch of flake8 complaints)
• My change requires a change to the documentation. (Need to explain the new plugin type)
• I have updated the documentation accordingly. (Added VAD Plugin documentation #4)
• I have added tests to cover my changes.
• All new and existing tests passed. (or at least didn't get worse)