Skip to content

How To: Work With WAVs

Jump to bottom
Tyler Gamvrelis edited this page Feb 29, 2020 · 10 revisions

Introduction

This page walks through the process of transforming motion data to and from audio waveforms (.wav), which can be easily manipulated in audio editing software, such as Adobe Audacity. We begin by diving into how to use the python program lamp.py to accomplish this translation, and end with an overview the implementation of these features.

Required tools

  • Python 3
  • Although not an exhaustive list, these are some Python libraries you will need to have installed (at the minimum):
    • numpy
    • scipy
    • pyserial
    • prettytable
    • matplotlib
    • opencv

Using the Python script lamp.py

For the following section, open up a terminal and navigate to oil_lamp/pc on your computer.

How to translate .dat to .wav

The form of the command to accomplish this translation is as follows:

python3 lamp.py --analyze=filename --make_wav=True

Where filename is the name of the .dat file, relative to oil_lamp/pc/data.

Example:

python3 lamp.py --analyze=oct_13_2019\lamp_data_0.dat --make_wav=True

This will produce 2 files in oil_lamp/pc/data/oct_13_2019 called lamp_data_0_outer.wav and lamp_data_0_inner.wav

How to translate .csv to .wav

The --csv2wav command converts simulated data (.csv) into audio files (.wav) for playback. The form is as follows:

python3 lamp.py --csv2wav=filename

Where filename is the name of the .csv file, relative to oil_lamp/pc/data.

Example:

python3 lamp.py --csv2wav=simulation1/simlog1.csv

How to use a .wav for playback

The form of the playback command is identical when working with .dat and .wav files; lamp.py detects the file extension and changes its behaviour appropriately:

python3 lamp.py --playback=filename

However there is one counter-intuitive thing about playing back .wav files: since there are separate files for the outer and inner angle waveforms, which filename do you use? The answer is neither. Since the filename of these is identical up to the suffix "_outer" or "_inner", the chosen convention is to use the identical part as the filename. To make this clear, consider the following example.

Example:

WAV Files:

  • oil_lamp/pc/data/oct_13_2019/lamp_data_0_outer.wav
  • oil_lamp/pc/data/oct_13_2019/lamp_data_0_inner.wav

Notice how the root "oct_13_2019/lamp_data_0" is identical. This means we should use the file name "oct_13_2019/lamp_data_0.wav".

Command used:

python3 lamp.py --playback=oct_13_2019/lamp_data_0.wav

Editing .wav files

The .wav files produced by the above procedure can be edited in your favourite respectable audio editing software. :) However, one thing to be wary of is that you must save the .wav files using the format "WAV 16-bit signed PCM", otherwise you will not be able to play them back using lamp.py.

How to calibrate a recording

There are two steps to calibration:

  1. Computing calibration constants given a baseline recording
  2. Extracting a subset of a larger recording to use as a baseline

1. Computing calibration constants

If we run python3 lamp.py --help we will notice the following description for the set_baseline command:

  --set_baseline SET_BASELINE
                        Specifies a file to use for generating calibration
                        offsets. This can be used to account for the IMUs
                        being mounted at angles relative to the lamp and base

At the moment, we can only set the baseline based on a .dat file (as opposed to .wav).

Example:

Suppose we have a file at oil_lamp/pc/data/baseline.dat. This file should capture the steady-state conditions of the recording environment (at rest; no motion). The command to update the calibration constants as per this baseline recording would be:

python3 lamp.py --set_baseline=baseline.dat

Note that the file name is relative to oil_lamp/pc/data. Running this command will update oil_lamp/pc/settings.ini with the new calibration constants, as well as the time at which the new baseline was computed and the name of the file it was calculated from.

2. Extracting a subset of a larger recording

Suppose that for whatever reason we did not record a good baseline prior to doing our main recordings, but we do have some segments of our main recordings that are fairly steady and could therefore act as baselines. For this purpose, we have the --file_slice command:

python3 lamp.py --file_slice=filename,starttime,stoptime

This command has 3 arguments:

  1. The name of the file to split, relative to oil_lamp/pc/data
  2. The time at which to start the slice, specified in seconds relative to the beginning of the recording. Must be >= 0
  3. The time at which to end the slice, specified in seconds relative to the beginning of the recording. Must not exceed the duration of the recording; if so, an error message will be displayed that tells you the max length of the recording

Example:

python3 lamp.py --file_slice=oct_13_2019\lamp_data_0.dat,360,370

This will produce the following messages:

D:GitHub\oil_lamp\pc>python3 lamp.py --file_slice=oct_13_2019\lamp_data_0.dat,360,370
18.05.34.623210 Starting PC-side application
18.05.34.627199 Attempting to open data D:\GitHub\oil_lamp\pc\data\oct_13_2019\lamp_data_0.dat
18.05.35.855912 Saved split data to D:\GitHub\oil_lamp\pc\data\oct_13_2019\lamp_data_0_slice360.00to370.00.dat

See the previous section for how to use this new file as a baseline.

Under the Hood (Optional)

The python program lamp.py is a jack of all trades for this project: it handles data storage for recording, angle estimation and streaming for playback, analysis of recorded waveforms (including animations!) and so on. Here we will go over its ability to work with WAV representations of the angle data.

When out in the field, our equipment records 12 channels of raw data to a ".dat" file. These 12 channels are the acceleration and angular velocity along the x, y and z axes for both the base and lamp IMU.

How does playback work for.dat files?

Assuming a .dat file is used for playback, the first step that lamp.py performs is loading this .dat file from the hard disk into RAM—this might take a while for long recordings! Next, the data is calibrated using the constants in settings.ini (which are set via a baseline recording). Next, the raw data is used to estimate the pitch and roll angles for the lamp. Finally, these angles are sent to the microcontroller, which communicates with the servos to realize the motion.

Here's what playback might look like for a .dat file:

D:\GitHub\oil_lamp\pc>python3 lamp.py --playback=oct_13_2019\lamp_data_0.dat
16.43.45.051550 Starting PC-side application
16.43.45.052548 Starting playback
16.43.45.053545 Loading data file D:\GitHub\oil_lamp\pc\data\oct_13_2019\lamp_data_0.dat
16.43.45.056524 Attempting to open data D:\GitHub\oil_lamp\pc\data\oct_13_2019\lamp_data_0.dat
16.43.52.688259 Loaded existing calibration data
16.43.54.695849 Interpolated 32130 points in time series
...
16.44.05.320492 Attempting connection to embedded
16.44.05.325536         Port: COM15
16.44.05.326477         Baud rate: 115200
...

How do we make a .wav from a .dat?

We use the analyze module of lamp.py to transform the representation of our data from .dat to .wav. The process begins the same way as before—we load the .dat from the disk and estimate pitch and roll angles. However, we soon encounter the main challenge with going from .dat to .wav: our sample rate. Our IMU data is recorded at a rate of 100 samples per second, i.e. 100 Hz, but audio is typically sampled between 6 kHz and 192 kHz. Thus, unless we can pull new data samples out of a hat, we will not be able to work with our waveforms in audio editing software.

Luckily, we can pull new data samples out of a hat! The technique is called spline interpolation, and it is pretty reasonable to employ for our purposes. Basically, we use our recorded data samples to define a time-dependent mathematical function that can be thought of as a "waveform of best fit" for our data. This mathematical function satisfies 2 conditions: (1) it perfectly matches our data when it is evaluated at the times that we have real data for, and (2) between any 3 real data samples, it is smooth as opposed to jagged. Since our data is now represented by a mathematical function which depends on time, we can generate data at any time that we want. It's as simple as plugging in a time and seeing what the function spits out. As we can see, it is now no problem to go from the 100 Hz sample rate of our IMU data to an arbitrary sample rate, such as 6 kHz.

Finally, all that we need to do is take our new (interpolated) sequence of samples and write this into a .wav file. But there is one last thing we need to sort out, and this is the amplitude of our waveform. The angle data varies between roughly +/- 40 degrees, but the .wav format only accepts integer amplitude values between +/- 32767. Thus, an angle of, say, 10.5 degrees cannot be directly written to a .wav file. The approach taken here is to map amplitudes according to the following formula:

amp = angle * 32767.0 / 40.0

This way, an angle of +40 degrees corresponds to an amplitude value of 32767 in the .wav file (max amplitude), while -40 degrees corresponds to -32767. Note that we do clip amplitudes, i.e. if a recording contains an angle of +50 degrees, or -40.0001 degrees, they will be capped to +32767 and -32767 when written to the .wav file, respectively. Clipping is necessary, because if later we want to load a .wav for playback, we need to know what amplitude values correspond to what angles. It is clear that a maximum allowed angle is necessary for this translation to occur.

This new sequence of amplitudes is then written to the .wav file, and it's good to go! It can now be opened in Audacity, Audition, etc.

Note that separate .wav files are produced for the pitch and roll angle waveforms (also called outer and inner).

How does playback work for .wav files?

First we load two .wav files from the disk: one for the outer angles and one for the inner angles. We then downsample the waveform back down to 100 Hz. Playback then proceeds the same as for .dat files (send angles to microcontroller forever). The messages displayed will be slightly different than in the .dat case, since for the .wav case 2 files need to be loaded instead of 1, and there is no calibration or interpolation.

D:\GitHub\oil_lamp\pc>python3 lamp.py --playback=oct_13_2019\lamp_data_0.wav
17.40.22.306985 Starting PC-side application
17.40.22.308981 Starting playback
17.40.22.308981 Loading data files:
17.40.22.311975         D:\GitHub\oil_lamp\pc\data\oct_13_2019\lamp_data_0_outer.wav
17.40.22.312971         D:\GitHub\oil_lamp\pc\data\oct_13_2019\lamp_data_0_inner.wav
...
17.40.22.342203 Attempting connection to embedded
17.40.22.347401         Port: COM15
17.40.22.348769         Baud rate: 115200
...
Clone this wiki locally