Master Deep Generative Models in Julia

The goal of this repo is to provide a list of deep generative models implemented in Julia.

Note that each implementation targets only a very specific case¹. Generally we'll provide a simple version for the education purpose and a performant version which demonstrates how to reach the state-of-the-art performance in Julia².

Model List

• MLP
• VAE
• VQ-VAE
• GAN
• GPT2
• LLAMA
• DDPM
• MoE
• VQGAN
• CLIP
• MaskGIT

Benchmark

Model	Environment	Performance	Description
MLP

Notable Implementations in Julia

• Transformers.jl
• AutoEncoderToolkit.jl
• DenoisingDiffusion.jl
• Jjama3.jl

Feel free to add your work here.

Q&A

1. Lux.jl vs Flux.jl?

   • There have been some discussions on this topic (see Deep learning in Julia). Since each algorithm is implemented independently under separate folder, just choose the one you are most comfortable with.

References

Following are some important blogs, papers, and codes that helped me a lot to understand the deep generative models implemented here.

• 6.S978 Deep Generative Models MIT EECS, Fall 2024
• Auto-Encoding Variational Bayes
• Tutorial on Variational Autoencoders
• A Beginner's Guide to Variational Methods: Mean-Field Approximation
• Understanding Variational Autoencoders (VAEs)
• A must-have training trick for VAE(variational autoencoder)
• http://www.openias.org/variational-coin-toss
• Neural Discrete Representation Learning
• Understanding Vector Quantization in VQ-VAE
• What is Residual Vector Quantization?
• Vector Quantization Pytorch
• jax-vqvae-vqgan
• pytorch-vqgan
• The Illustarated VQGAN
  • 
• CLIP
• llama2.c
• Diffusion Models — DDPMs, DDIMs, and Classifier Free Guidance
• The Annotated Diffusion Model

Footnotes

1. This is to follow the Tips & tricks ↩

2. See the discussions here. Ideally we'd like to keep updating this repo and demonstrate how to reach the state-of-the-art performance in Julia. ↩