Skip to content

Unofficial implementation JEN-1: Text-Guided Universal Music Generation with Omnidirectional Diffusion Models(https://arxiv.org/abs/2308.04729)

Notifications You must be signed in to change notification settings

0417keito/JEN-1-pytorch

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

JEN-1-pytorch

Unofficial implementation JEN-1: Text-Guided Universal Music Generation with Omnidirectional Diffusion Models(https://arxiv.org/abs/2308.04729)

JEN-1

README

📖 Quick Index

💻 Installation

git clone https://github.com/0417keito/JEN-1-pytorch.git
cd JEN-1-pytorch
pip install -r requirements.txt

🐍Usage

Sampling

import torch
from generation import Jen1

ckpt_path =  'your ckpt path'
jen1 = Jen1(ckpt_path)

prompt = 'a beautiful song'
samples = jen1.generate(prompt)

Training

torchrun train.py

Dataset format

Json format. the name of the Json file must be the same as the target music file.

{"prompt": "a beautiful song"}
How should the data_dir be created?

'''
dataset_dir
├── audios
|    ├── music1.wav
|    ├── music2.wav
|    .......
|    ├── music{n}.wav
|
├── metadata
|   ├── music1.json
|   ├── music2.json
|   ......
|   ├── music{n}.json
|
'''

About config

please see config.py and conditioner_config.py

🧠TODO

  • Extension to JEN-1-Composer
  • Extension to music generation with singing voice
  • Adaptation of Consistency Model
  • In the paper, Diffusion Autoencoder was used, but I did not have much computing resources, so I used Encodec instead. So, if I can afford it, I will implement Diffusion Autoencoder.

🚀Demo

coming soon !

🙏Appreciation

Dr Adam Fils - Thank you for providing the GPU. I really appreciate Adam giving me this opportunity.

⭐️Show Your Support

If you find this repo interesting and useful, give us a ⭐️ on GitHub! It encourages us to keep improving the model and adding exciting features. Please inform us of any deficiencies by issue.

🙆Welcome Contributions

Contributions are always welcome.

About

Unofficial implementation JEN-1: Text-Guided Universal Music Generation with Omnidirectional Diffusion Models(https://arxiv.org/abs/2308.04729)

Resources

Stars

Watchers

Forks

Packages

No packages published

Languages