This repository is an implementation that recreates the SketchGuidance feature of "ToonCrafter".
2024-07-13.2.52.44.mov
Model | Resolution | GPU Mem. & Inference Time (A100, ddim 50steps) | Checkpoint |
---|---|---|---|
ToonCrafter_512 | 320x512 | TBD (perframe_ae=True ) |
Hugging Face |
SketchEncoder | TBD | TBD | Hugging Face |
Currently, ToonCrafter can support generating videos of up to 16 frames with a resolution of 512x320. The inference time can be reduced by using fewer DDIM steps.
conda create -n tooncrafter python=3.8.5
conda activate tooncrafter
pip install -r requirements.txt
- Download pretrained ToonCrafter_512 and put the model.ckpt in checkpoints/tooncrafter_512_interp_v1/model.ckpt.
- Download pretrained SketchEncoder and put the model.ckpt in control_models/sketch_encoder.ckpt.
python gradio_app.py