Simone is a web interface for musical improvisation with distributed systems of devices using synthesis by audio mosaicing and audio inputs.
This is the second version of Simone. The first version can be found here.
Simone is developped using the soundworks framework.
Install Node.js (LTS version)
Clone the repo or download it to the location you want on your computer then in the terminal:
cd /path/to/simone
npm install
npm run build
You will first need to create a config file for your application.
To do that, execute the following command npx soundworks and choose the create a new environment config file.
Once this is done, execute
npm run start
This should start the server and prints several IP adresses like this:
+ https server listening on
https://127.0.0.1:8000
https://XXX.XXX.XXX.XXX:8000
You will need to put soundfiles in the simone/public/soundbank/ folder to be used by the application. Do not put them in any subfolders.
Simone only support wav and mp3 formats.
The Web client provides you with a interface that controls a sound synthesis process based on audio inputs from your microphone. The recorded sound is then transformed through a process of 'audio mosaicing' and played on the Node clients connected to the server.
Audio mosaicing refers to the process of recomposing the temporal evolution of a given target audio file from segments cut out of source audio materials. (cf. http://recherche.ircam.fr/equipes/temps-reel/audio-mosaicking/).
To go into more details, your input (called the target sound) is cut into small segments and for each of these segments we look for the 'most similar' segment in another sound (the source sound). The resulting sound then sound like the target sound but reconstructed from patches from the source sound.
In Simone the target sound is recorded by the microphone and the data from this target sound is sent simultaneously to all connected Node clients which can use a different source sound.
Two modes of control are available in Simone :
- A real-time mode in which audio input from the microphone is processed in real-time.
- An offline mode (not yet available) in which a pre-recorded sound is used to control synthesis.
The interface in divided in 3 parts
The top panel differs depending on the selected mode
In real-time mode, the top panel allows you to open the input stream from your microphone, to monitor the input level from your microphone and to calibrate the input sound
Calibration is an essential part as it will determine normalization value to use when analyzing the input sound. Poor calibration will probably lead to incoherent results in the mosaicing process.
To calibrate your sound, press the record button in the calibration section. This will start recording. Press again to stop recording and press the compute button to compute the calibration values. These values will be saved in a file (rename it if it is useful) in the public/calibration folder. You can load previous calibration files using the dropdown menu and the load button.
It is recommended that you record a calibration sound that encompasses the whole breadth (in volume, pitch, timbre, etc...) of the sound you intend to use when playing later.
The left panel allows you to monitor connected clients, to create groups of clients and to assign clients to groups.
The first part allows you to manage groups of clients. Groups will be used to control synthesis parameters. Clients assigned to the same group will share the same set of parameters and will play the same sound.
Use the + button to create a group. You can rename groups, assign a color to them and delete them. Upon creation, a control box for this group will be created in the right pansel.
The second part monitors all clients connected to the server. You can assign a group to a client using a dropdown menu. A client will not play sound until assigned to a group.
This is the main playing part of Simone. The right panel displays a box for each group you create. This box contains elements that control the synthesis for each client assigned to this group. The elements are the following :
- A play/stop button to start/stop synthesis
- A dropdown menu to select the source sound used by this group
- A slider to control volume
- A slider to control the pitch of the sound (from -1 octave to +1 octave)
- A slider to control the period between two grains
- A slider to control the duration of each grain
- A slider to randomize the result of the search for the most similar grain. A value of n means that the source grain played will be randomly selected among the n most similar grains to the target grain.
- recorder client
- loop record mode : automatic record every X seconds
- Play/stop in master controls
- Timer for changing source file
- loading seems very long for long soundfile
BSD-3-Clause