Feel the data

Try me: http://feel-the-data-rick1080p.onrender.com/

A machine learning-based data sonification and visualization

Map data to music and image, generating a data experience. The project is meant for anyone that is able to interact with a web app, from very young people (as it is almost usable just by looking at the images) to older ones

Goals

• to educate the highest number of people possible: to inform the young generations and to get detached people interested in today’s issues
• make an extendible platform
• make data understandable without having to know specific concepts

Challenges

• Not being able to work physically together and divide the work in such a way that it could be easily merged
• Automatic music composition
• Merging together our ideas and also using the latest technologies

Technologies

• map: p5.js, mappa.js, and the Mapbox API
• music: magenta.js, specifically the magenta RNN and magenta music VAE. We applied the valence-arousal plane concept to map the data to music
• data: OpenWeather API, specifically the Current Weather Data API and Air Pollution API
• visuals: JavaScript, the OpenAI Dall-e API, and the concepts of Particle systems and Perlin noise The whole project is hosted in a Node.js application

Contributors

• Eutizi Claudio: particle system and music mapping
• Perego Gabriele: particle system and front-end development
• Plandolit Ricard: automatic music composition and visuals
• Zezza Federica: map and menu front-end development