-
Notifications
You must be signed in to change notification settings - Fork 0
How To: Work With WAVs
This page walks through the process of transforming motion data to and from audio waveforms (.wav), which can be easily manipulated. We begin with an overview the implementation, then dive into how to use the python program lamp.py to accomplish our needs.
- Python 3 with the following libraries: TODO
For the following section, open up a terminal and navigate to oil_lamp/pc on your computer.
The form of the command to accomplish this translation is as follows:
python3 lamp.py --analyze=filename --make_wav=TrueWhere filename is the name of the .dat file, relative to oil_lamp/pc/data.
Example:
python3 lamp.py --analyze=oct_13_2019\lamp_data_0.dat --make_wav=TrueThis will produce 2 files in oil_lamp/pc/data/oct_13_2019 called lamp_data_0_outer.wav and lamp_data_0_inner.wav
The form of the playback command is identical when working with .dat and .wav files; lamp.py detects the file extension and changes its behaviour appropriately:
python3 lamp.py --playback=filenameHowever there is one counter-intuitive thing about playing back .wav files: since there are separate files for the outer and inner angle waveforms, which filename do you use? The answer is neither. Since the filename of these is identical up to the suffix "_outer" or "_inner", we simply use the identical part as the filename. To make this clear, consider the following example.
Example:
WAV Files:
- oil_lamp/pc/data/oct_13_2019/lamp_data_0_outer.wav
- oil_lamp/pc/data/oct_13_2019/lamp_data_0_inner.wav
Notice how the root "oct_13_2019/lamp_data_0" is identical. This means we should use the file name "oct_13_2019/lamp-data_0.wav".
Command used:
python3 lamp.py --playback=oct_13_2019/lamp_data_0.wavTODO (go over split and set baseline commands)
The python program lamp.py is a jack of all trades for this project: it handles data storage for recording, angle estimation and streaming for playback, analysis of recorded waveforms (including animations!) and so on. Here we will go over its ability to work with WAV representations of the angle data.
When out in the field, our equipment records 12 channels of raw data to a ".dat" file. These 12 channels are the acceleration and angular velocity along the x, y and z axes for both the base and lamp IMU.
Assuming a .dat file is used for playback, the first step that lamp.py performs is loading this .dat file from the hard disk into RAM—this might take a while for long recordings! Next, the data is calibrated using the constants in settings.ini (which are set via a baseline recording). Next, the raw data is used to estimate the pitch and roll angles for the lamp. Finally, these angles are sent to the microcontroller, which communicates with the servos to realize the motion.
Here's what playback might look like for a .dat file:
D:\GitHub\oil_lamp\pc>python3 lamp.py --playback=oct_13_2019\lamp_data_0.dat
16.43.45.051550 Starting PC-side application
16.43.45.052548 Starting playback
16.43.45.053545 Loading data file D:\GitHub\oil_lamp\pc\data\oct_13_2019\lamp_data_0.dat
16.43.45.056524 Attempting to open data D:\GitHub\oil_lamp\pc\data\oct_13_2019\lamp_data_0.dat
16.43.52.688259 Loaded existing calibration data
16.43.54.695849 Interpolated 32130 points in time series
...
16.44.05.320492 Attempting connection to embedded
16.44.05.325536 Port: COM15
16.44.05.326477 Baud rate: 115200
...
We use the analyze module of lamp.py to transform the representation of our data from .dat to .wav. The process begins the same way as before—we load the .dat from the disk and estimate pitch and roll angles. However, we soon encounter the main challenge with going from .dat to .wav: our sample rate. Our IMU data is recorded at a rate of 100 samples per second, i.e. 100 Hz, but audio is typically sampled between 6 kHz and 192 kHz. Thus, unless we can pull new data samples out of a hat, we will not be able to work with our waveforms in audio editing software.
Luckily, we can pull new data samples out of a hat! The technique is called spline interpolation, and it is pretty reasonable to employ for our purposes. Basically, we use our recorded data samples to define a time-dependent mathematical function that can be thought of as a "waveform of best fit" for our data. This mathematical function satisfies 2 conditions: (1) it perfectly matches our data when it is evaluated at the times that we have real data for, and (2) between any 3 real data samples, it is smooth as opposed to jagged. Since our data is now represented by a mathematical function which depends on time, we can generate data at any time that we want. It's as simple as plugging in a time and seeing what the function spits out. As we can see, it is now no problem to go from the 100 Hz sample rate of our IMU data to an arbitrary sample rate, such as 6 kHz.
Finally, all that we need to do is take our new (interpolated) sequence of samples and write this into a .wav file. But there is one last thing we need to sort out, and this is the amplitude of our waveform. The angle data varies between roughly +/- 40 degrees, but the .wav format only accepts integer amplitude values between +/- 32767. Thus, an angle of, say, 10.5 degrees cannot be directly written to a .wav file. The approach taken here is to map amplitudes according to the following formula:
amp = angle * 32767.0 / 40.0
This way, an angle of +40 degrees corresponds to an amplitude value of 32767 in the .wav file (max amplitude), while -40 degrees corresponds to -32767. Note that we do clip amplitudes, i.e. if a recording contains an angle of +50 degrees, or -40.0001 degrees, they will be capped to +32767 and -32767 when written to the .wav file, respectively. Clipping is necessary, because if later we want to load a .wav for playback, we need to know what amplitude values correspond to what angles. It is clear that a maximum allowed angle is necessary for this translation to occur.
This new sequence of amplitudes is then written to the .wav file, and it's good to go! It can now be opened in Audacity, Audition, etc.
Note that separate .wav files are produced for the pitch and roll angle waveforms (also called outer and inner).
First we load two .wav files from the disk: one for the outer angles and one for the inner angles. We then downsample the waveform back down to 100 Hz. Playback then proceeds the same as for .dat files (send angles to microcontroller forever). The messages displayed will be slightly different than in the .dat case, since for the .wav case 2 files need to be loaded instead of 1, and there is no calibration or interpolation.
D:\GitHub\oil_lamp\pc>python3 lamp.py --playback=oct_13_2019\lamp_data_0.wav
17.40.22.306985 Starting PC-side application
17.40.22.308981 Starting playback
17.40.22.308981 Loading data files:
17.40.22.311975 D:\GitHub\oil_lamp\pc\data\oct_13_2019\lamp_data_0_outer.wav
17.40.22.312971 D:\GitHub\oil_lamp\pc\data\oct_13_2019\lamp_data_0_inner.wav
...
17.40.22.342203 Attempting connection to embedded
17.40.22.347401 Port: COM15
17.40.22.348769 Baud rate: 115200
...