-
Notifications
You must be signed in to change notification settings - Fork 11
Audio Player
The audio player used in Freesound presents some issues with respect to UI/UX. The main issues are:
- interaction buttons (play/pause, loop sound, switch view) are too small, and thus not functional (#14);
- the looping functionality isn’t really needed, while being able to restart the clip from the beginning would be more useful and convenient;
- playing one sound won’t stop other sounds already playing (#96) Moreover, a custom audio player would allow us to implement easily other functionalities, such as an annotator for annotation tasks. To address these issues and give us more flexibility, we rely on wavesurfer.js. Wavesurfer.js is a library built on top of Web Audio API that provides a simple interface for visualization and playback of audio clips. However, since we needed to build our logic on top of it, some overrides to the source code were necessary. These overrides are described in the Modifications section.
Building a new player involves two steps. First, you need to include in your template file the source files needed to build the player. To do so, you have to add the following lines on top of your template file:
{% block extra_head %}
{% load_sound_player_files %}
{% endblock %}
The load_sound_player_files basically includes the content of
/datasets/templates/datasets/include_player_resources.html.
Then, from your template, you can call the following template tag wherever you need your player to be
displayed:
{% sound_player dataset sound size %}
Here, dataset is the dataset you are working with, sound is the Freesound ID of the audio clip you want to be loaded in the player, and size is the size of the player (currently supported sizes: “mini”, “small” and “medium”). Note that different player sizes imply slightly different interfaces. The sound_player template tag includes the content of /datasets/templates/datasets/player.html. You can head over there to check the basic structure of the player (though most of the components are defined in the Player Javascript prototype, see next section).
The prototype responsible for handling a player object is Player. You can find its implementation in /static/audio-annotator/js/player.js. The prototype takes care of putting together the various components of the player: the wavesurfer.js engine, visualizations and control buttons (the respective prototypes are located in the same file). It also takes care of destroying players, if needed.
As already mentioned, wavesurfer.js is taking care of most of the functionalities of our audio player, namely loading, seeking and playing back audio clips. Visualizations are handled by the View prototype. It takes care of loading and switching between background images (spectrogram and waveform), and of the progress bar located on top of them. It also handles mouse interaction on top of the player, that is, updating the current playback position when the user clicks on top of the visualization area. Controls are handled by the PlayBar prototype. It takes care of defining the components of the play bar (as of now: play/pause button, restart button, switch view button, and the time indicator), and their interaction with both the wavesurfer.js backend and the View component.
In order to use custom Freesound visualizations in the player, we need to prevent wavesurfer.js from
drawing its own waveform. To do so, we override both the drawBuffer and the createDrawer methods
directly inside the Player prototype. Check out the Player.setupWaveSurferInstance() method for details.
Hopefully, this will not prevent us to use other libraries (CrowdCurio annotator, for example).
The wavesurfer.js version currently used is the 2.0.5 version.
Wavesurfer.js comes with a BSD 3-Clause license. The content is reported below. BSD 3-Clause License Copyright (c) 2012-2018, katspaugh and contributors All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:
- Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
- Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
- Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE
OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
lorenzo-romanelli