Generating watercolors from music

[mashumashupan]

Project name

Generating watercolors from music

Description

Overview

This AI application is developed in .NET 8. It uses AI to output a polka dot pattern from the pitch, volume, and time information of an input music file. By inputting the polka dot pattern for each note into the image-to-image AI, the pattern is converted into a real-time watercolor painting and output, finally producing a single picture.

Purpose

We were looking for a way to make music enjoyable for people who have difficulty enjoying music (e.g., the hearing impaired and the elderly).
To this end, we developed an application that allows people to visually enjoy sound images by outputting the input music in the form of watercolor paintings.
Painting is an art form that always exists, whereas music is an art form that disappears.
This application makes it possible to preserve the fading art as a painting by leaving the music as a watercolor.
By using AI to turn polka dots into watercolor paintings, we believe that the lingering effect of music and artistic perfection will be improved. In the future, we would also like to express the overlapping of music with watercolor paintings.
It is expected that watercolor paintings of the same piece of music will change depending on the performer, or watercolor paintings will change depending on the growth of the same performer, which will be enjoyed not only by the hearing-impaired and the elderly, but also by many music lovers.

Prerequisites

Before running the application, ensure that you have the following prerequisites installed:
.NET 8 SDK: Download .NET 8
Application Results

Application Results

https://docs.google.com/presentation/d/16v9NDRWUQ8lo2LzzX0b2dvCAizPgBR8tJiub6C3PnQ4/edit?usp=sharing

Project Repository URL

https://github.com/mashumashupan/dotnet_hackathon

Project video

https://youtu.be/niN-huWpk4Y

Team members

https://github.com/mashumashupan , https://github.com/SatomiHasegawa20 , https://github.com/har06git13