diff --git a/README.md b/README.md index d0d82f4..71b13d4 100644 --- a/README.md +++ b/README.md @@ -76,7 +76,7 @@ The training GPU implements environment for this README is as follows: models ar 1. **Import the Dataset** - First, upload the dataset to the target folder `datasets`. After uploading, the folder structure (for example, under the `cifar10` folder, there are folders for each class; `class0` folder contains all images for class 0) should look like the following: + First, upload the dataset to the target folder `datasets` [**[issue]**](https://github.com/chairc/Integrated-Design-Diffusion-Model/issues/9#issuecomment-1882902085). After uploading, the folder structure (for example, under the `cifar10` folder, there are folders for each class; `class0` folder contains all images for class 0) should look like the following: ```yaml datasets @@ -145,7 +145,7 @@ The training GPU implements environment for this README is as follows: models ar python train.py --sample ddpm --conditional False --run_name df --epochs 300 --batch_size 16 --image_size 64 --dataset_path /your/dataset/path --result_path /your/save/path ``` 4. Wait for the training to complete. -5. If the training is interrupted due to any reason, you can resume it by setting `--resume` to `True` in the `train.py` file, specifying the epoch number where the interruption occurred, providing the folder name of the interrupted training (`run_name`), and running the file again. Alternatively, you can use the following command to resume the training: +5. If the training is interrupted due to any reason **[[issue]](https://github.com/chairc/Integrated-Design-Diffusion-Model/issues/9#issuecomment-1882912391)**, you can resume it by setting `--resume` to `True` in the `train.py` file, specifying the epoch number where the interruption occurred, providing the folder name of the interrupted training (`run_name`), and running the file again. Alternatively, you can use the following command to resume the training: **Conditional Resume Training Command** ```bash @@ -168,7 +168,14 @@ The training GPU implements environment for this README is as follows: models ar # This is not using --start_epoch, default use last checkpoint python train.py --resume True --sample ddpm --conditional False --run_name df --epochs 300 --batch_size 16 --image_size 64 --dataset_path /your/dataset/path --result_path /your/save/path ``` - +6. The pretrained models are released with every major [Release](https://github.com/chairc/Integrated-Design-Diffusion-Model/releases), so please stay updated. To use a pretrained model [**[issue]**](https://github.com/chairc/Integrated-Design-Diffusion-Model/issues/9#issuecomment-1886403967), download the model corresponding to parameters such as `network`, `image_size`, `act`, etc., and save it to any local folder. Adjust the `--pretrain` and `--pretrain_path` in the `train.py` file accordingly. You can also use the following command for training with a pretrained model: + + **Command for training with a pretrained model** + + ```bash + python train.py --pretrain True --pretrain_path /your/pretrain/path/model.pt --sample ddpm --conditional False --run_name df --epochs 300 --batch_size 16 --image_size 64 --dataset_path /your/dataset/path --result_path /your/save/path + ``` + #### Distributed Training 1. The basic configuration is similar to regular training, but note that enabling distributed training requires setting `--distributed` to `True`. To prevent arbitrary use of distributed training, we have several conditions for enabling distributed training, such as `args.distributed`, `torch.cuda.device_count() > 1`, and `torch.cuda.is_available()`. @@ -212,7 +219,7 @@ The training GPU implements environment for this README is as follows: models ar | --amp | | Automatic mixed precision training | bool | Enable automatic mixed precision training. It effectively reduces GPU memory usage but may affect training accuracy and results | | --optim | | Optimizer | str | Optimizer selection. Currently supports Adam and AdamW | | --act | | Activation function | str | Activation function selection. Currently supports gelu, silu, relu, relu6 and lrelu | -| --lr | | Learning rate | float | Initial learning rate. Currently only supports linear learning rate | +| --lr | | Learning rate | float | Initial learning rate. | | --lr_func | | Learning rate schedule | str | Setting learning rate schedule, currently supporting linear, cosine, and warmup_cosine. | | --result_path | | Save path | str | Path to save the training results | | --save_model_interval | | Save model after each training | bool | Whether to save the model after each training iteration for model selection based on visualization | @@ -237,16 +244,28 @@ The training GPU implements environment for this README is as follows: models ar 2. Set the necessary parameters such as `--conditional`, `--generate_name`, `--num_images`, `--num_classes`, `--class_name`, `--image_size`, `--result_path`, etc. If no parameters are set, the default settings will be used. There are two ways to set the parameters: one is to directly modify the `parser` in the `if __name__ == "__main__":` section of the `generate.py` file, and the other is to use the following commands in the console while in the `/your/path/Defect-Diffusion-Model/tools` directory: - **Conditional Generation Command** + **Conditional Generation Command (version > 1.1.1)** + + ```bash + python generate.py --generate_name df --num_images 8 --class_name 0 --image_size 64 --weight_path /your/path/weight/model.pt + ``` + + **Unconditional Generation Command (version > 1.1.1)** + + ```bash + python generate.py --generate_name df --num_images 8 --image_size 64 --weight_path /your/path/weight/model.pt + ``` + + **Conditional Generation Command (version <= 1.1.1)** ```bash - python generate.py --conditional True --generate_name df --num_images 8 --num_classes 10 --class_name 0 --image_size 64 --weight_path /your/path/weight/model.pt + python generate.py --conditional True --generate_name df --num_images 8 --num_classes 10 --class_name 0 --image_size 64 --weight_path /your/path/weight/model.pt --sample ddpm --network unet --act gelu ``` - **Unconditional Generation Command** + **Unconditional Generation Command (version <= 1.1.1)** ```bash - python generate.py --conditional False --generate_name df --num_images 8 --image_size 64 --weight_path /your/path/weight/model.pt + python generate.py --conditional False --generate_name df --num_images 8 --image_size 64 --weight_path /your/path/weight/model.pt --sample ddpm --network unet --act gelu ``` 3. Wait for the generation process to complete. @@ -263,10 +282,10 @@ The training GPU implements environment for this README is as follows: models ar | --num_images | | Number of generated images | int | Number of images to generate | | --weight_path | | Path to model weights | str | Path to the model weights file, required for network generation | | --result_path | | Save path | str | Path to save the generated images | -| --sample | | Sampling method | str | Set the sampling method type, currently supporting DDPM and DDIM. | -| --network | | Training network | str | Set the training network, currently supporting UNet, CSPDarkUNet. | -| --act | | Activation function | str | Activation function selection. Currently supports gelu, silu, relu, relu6 and lrelu. If you do not set the same activation function as the model, mosaic phenomenon will occur. | -| --num_classes | ✓ | Number of classes | int | Number of classes for classification | +| --sample | | Sampling method | str | Set the sampling method type, currently supporting DDPM and DDIM. **(No need to set for models after version 1.1.1)** | +| --network | | Training network | str | Set the training network, currently supporting UNet, CSPDarkUNet. **(No need to set for models after version 1.1.1)** | +| --act | | Activation function | str | Activation function selection. Currently supports gelu, silu, relu, relu6 and lrelu. If you do not set the same activation function as the model, mosaic phenomenon will occur. **(No need to set for models after version 1.1.1)** | +| --num_classes | ✓ | Number of classes | int | Number of classes for classification **(No need to set for models after version 1.1.1)** | | --class_name | ✓ | Class name | int | Index of the class to generate images. if class name is `-1`, the model would output one image per class. | | --cfg_scale | ✓ | Classifier-free guidance weight | int | Weight for classifier-free guidance interpolation, for better generation model performance | @@ -338,7 +357,7 @@ We conducted training on the following four datasets using the `DDPM` sampler wi #### Base on the 64×64 model to generate 160×160 (every size) images -Of course, based on the 64×64 U-Net model, we generate 160×160 `NEU-DET` images in the `generate.py` file (single output, each image occupies 21GB of GPU memory). Detailed images are as follows: +Of course, based on the 64×64 U-Net model, we generate 160×160 `NEU-DET` images in the `generate.py` file (single output, each image occupies 21GB of GPU memory). **Attention this [[issues]](https://github.com/chairc/Integrated-Design-Diffusion-Model/issues/9#issuecomment-1886422210)**! If it's an image with defect textures where the features are not clear, generating a large size directly might not have these issues, such as in NRSD or NEU datasets. However, if the image contains a background with specific distinctive features, you may need to use super-resolution or resizing to increase the size, for example, in Cifar10, CelebA-HQ, etc. **If you really need large-sized images, you can directly train with large pixel images if there is enough GPU memory.** Detailed images are as follows: ![model_499_ema](assets/neu160_0.jpg)![model_499_ema](assets/neu160_1.jpg)![model_499_ema](assets/neu160_2.jpg)![model_499_ema](assets/neu160_3.jpg)![model_499_ema](assets/neu160_4.jpg)![model_499_ema](assets/neu160_5.jpg) diff --git a/README_zh.md b/README_zh.md index 76c1a4b..71395e1 100644 --- a/README_zh.md +++ b/README_zh.md @@ -76,7 +76,7 @@ Integrated Design Diffusion Model 1. **导入数据集** - 首先,将数据集上传至目标文件夹`datasets`中。上传后文件夹格式(例如:cifar10文件夹下存放着所有类别;class0文件夹下存储着class0这个类别的所有图片)如下方列表所示: + 首先,将数据集上传至目标文件夹`datasets`中[**[issue]**](https://github.com/chairc/Integrated-Design-Diffusion-Model/issues/9#issuecomment-1882902085)。上传后文件夹格式(例如:cifar10文件夹下存放着所有类别;class0文件夹下存储着class0这个类别的所有图片)如下方列表所示: ```yaml datasets └── cifar10 @@ -144,7 +144,7 @@ Integrated Design Diffusion Model python train.py --sample ddpm --conditional False --run_name df --epochs 300 --batch_size 16 --image_size 64 --dataset_path /your/dataset/path --result_path /your/save/path ``` 4. 等待训练即可 -5. 若因异常原因中断训练,我们可以在`train.py`文件,首先将`--resume`设置为`True`,其次设置异常中断的迭代编号,再写入该次训练的所在文件夹(run_name),最后运行文件即可。也可以使用如下命令进行恢复: +5. 若因异常原因中断训练**[[issue]](https://github.com/chairc/Integrated-Design-Diffusion-Model/issues/9#issuecomment-1882912391)**,我们可以在`train.py`文件,首先将`--resume`设置为`True`,其次设置异常中断的迭代编号,再写入该次训练的所在文件夹(run_name),最后运行文件即可。也可以使用如下命令进行恢复: **有条件恢复训练命令** ```bash @@ -166,6 +166,12 @@ Integrated Design Diffusion Model # 此处为不输入--start_epoch参数,默认使用last权重 python train.py --resume True --sample ddpm --conditional False --run_name df --epochs 300 --batch_size 16 --image_size 64 --dataset_path /your/dataset/path --result_path /your/save/path ``` +6. 预训练模型在每次大版本[Release](https://github.com/chairc/Integrated-Design-Diffusion-Model/releases)中发布,请留意。预训练模型使用方法如下[**[issue]**](https://github.com/chairc/Integrated-Design-Diffusion-Model/issues/9#issuecomment-1886403967),首先将对应`network`、`image_size`、`act`等相同参数的模型下到本地任意文件夹下。直接调整`train.py`中`--pretrain`和`--pretrain_path`即可。也可以使用如下命令进行预训练: + **使用预训练模型训练命令** + + ```bash + python train.py --pretrain True --pretrain_path /your/pretrain/path/model.pt --sample ddpm --conditional False --run_name df --epochs 300 --batch_size 16 --image_size 64 --dataset_path /your/dataset/path --result_path /your/save/path + ``` #### 分布式训练 @@ -213,7 +219,7 @@ Integrated Design Diffusion Model | --amp | | 混合精度训练 | bool | 开启混合精度训练,有效减少显存使用,但无法保证训练精度和训练结果 | | --optim | | 优化器 | str | 优化器选择,目前支持adam和adamw | | --act | | 激活函数 | str | 激活函数选择,目前支持gelu、silu、relu、relu6和lrelu | -| --lr | | 学习率 | float | 初始化学习率,目前仅支持线性学习率 | +| --lr | | 学习率 | float | 初始化学习率 | | --lr_func | | 学习率方法 | str | 设置学习率方法,当前支持linear、cosine和warmup_cosine | | --result_path | | 保存路径 | str | 保存路径 | | --save_model_interval | | 是否每次训练储存 | bool | 是否每次训练储存,根据可视化生成样本信息筛选模型 | @@ -238,18 +244,29 @@ Integrated Design Diffusion Model 1. 打开`generate.py`文件,找到`--weight_path`参数,将参数中的路径修改为模型权重路径,例如`/your/path/weight/model.pt` 2. 设置必要参数,例如`--conditional`,`--generate_name`,`--num_images`,`--num_classes`,`--class_name`,`--image_size`,`--result_path`等参数,若不设置参数则使用默认设置。我们有两种参数设置方法,其一是直接对`generate.py`文件`if __name__ == "__main__":`中的`parser`进行设置;其二是在控制台在`/your/path/Defect-Diffiusion-Model/tools`路径下输入以下命令: - **有条件生成命令** + **有条件生成命令(1.1.1版本以上)** ```bash - python generate.py --conditional True --generate_name df --num_images 8 --num_classes 10 --class_name 0 --image_size 64 --weight_path /your/path/weight/model.pt + python generate.py --generate_name df --num_images 8 --class_name 0 --image_size 64 --weight_path /your/path/weight/model.pt ``` - **无条件生成命令** + **无条件生成命令(1.1.1版本以上)** ```bash - python generate.py --conditional False --generate_name df --num_images 8 --image_size 64 --weight_path /your/path/weight/model.pt + python generate.py --generate_name df --num_images 8 --image_size 64 --weight_path /your/path/weight/model.pt ``` - + **有条件生成命令(1.1.1版本及以下)** + + ```bash + python generate.py --conditional True --generate_name df --num_images 8 --num_classes 10 --class_name 0 --image_size 64 --weight_path /your/path/weight/model.pt --sample ddpm --network unet --act gelu + ``` + + **无条件生成命令(1.1.1版本及以下)** + + ```bash + python generate.py --conditional False --generate_name df --num_images 8 --image_size 64 --weight_path /your/path/weight/model.pt --sample ddpm --network unet --act gelu + ``` + 3. 等待生成即可 #### 生成参数 @@ -264,10 +281,10 @@ Integrated Design Diffusion Model | --num_images | | 生成图片个数 | int | 单次生成图片个数 | | --weight_path | | 权重路径 | str | 模型权重路径,网络生成需要加载文件 | | --result_path | | 保存路径 | str | 保存路径 | -| --sample | | 采样方式 | str | 设置采样器类别,当前支持ddpm,ddim | -| --network | | 训练网络 | str | 设置训练网络,当前支持UNet,CSPDarkUNet | -| --act | | 激活函数 | str | 激活函数选择,目前支持gelu、silu、relu、relu6和lrelu。如果不选择,会产生马赛克现象 | -| --num_classes | 是 | 类别个数 | int | 类别个数,用于区分类别 | +| --sample | | 采样方式 | str | 设置采样器类别,当前支持ddpm,ddim**(1.1.1版本后的模型可不用设置)** | +| --network | | 训练网络 | str | 设置训练网络,当前支持UNet,CSPDarkUNet**(1.1.1版本后的模型可不用设置)** | +| --act | | 激活函数 | str | 激活函数选择,目前支持gelu、silu、relu、relu6和lrelu。如果不选择,会产生马赛克现象**(1.1.1版本后的模型可不用设置)** | +| --num_classes | 是 | 类别个数 | int | 类别个数,用于区分类别**(1.1.1版本后的模型可不用设置)** | | --class_name | 是 | 类别名称 | int | 类别序号,用于对指定类别生成。如果输入为-1,则模型为每类输出一张图片 | | --cfg_scale | 是 | classifier-free guidance插值权重 | int | classifier-free guidance插值权重,用户更好生成模型效果 | @@ -341,7 +358,7 @@ Integrated Design Diffusion Model #### 基于64×64模型生成160×160(任意大尺寸)图像 -当然,我们根据64×64的基础模型,在`generate.py`文件中生成160×160的`NEU-DET`图片(单张输出,每张图片占用显存21GB)。详细图片如下: +当然,我们根据64×64的基础模型,在`generate.py`文件中生成160×160的`NEU-DET`图片(单张输出,每张图片占用显存21GB)。**请注意这个**[[**issue**]](https://github.com/chairc/Integrated-Design-Diffusion-Model/issues/9#issuecomment-1886422210):如果是缺陷纹理那种图片,特征物不明显的直接生成大尺寸就不会有这些问题,例如NRSD、NEU数据集。如果是含有背景有特定明显特征的则需要超分或者resize提升尺寸,例如Cifar10、CelebA-HQ等。**如果实在需要大尺寸图像,在显存足够的情况下直接训练大像素图片。**详细图片如下: ![model_499_ema](assets/neu160_0.jpg)![model_499_ema](assets/neu160_1.jpg)![model_499_ema](assets/neu160_2.jpg)![model_499_ema](assets/neu160_3.jpg)![model_499_ema](assets/neu160_4.jpg)![model_499_ema](assets/neu160_5.jpg) diff --git a/tools/generate.py b/tools/generate.py index 4c70e21..164ae61 100644 --- a/tools/generate.py +++ b/tools/generate.py @@ -96,7 +96,8 @@ def generate(args): # Generation name (required) parser.add_argument("--generate_name", type=str, default="df") # Input image size (required) - # [Warn] Compatible with older versions, version <= 1.1.1 need to be equal to model's image size + # [Warn] Compatible with older versions + # [Warn] Version <= 1.1.1 need to be equal to model's image size, version > 1.1.1 can set whatever you want parser.add_argument("--image_size", type=int, default=64) # Number of generation images (required) # if class name is `-1` and conditional `is` True, the model would output one image per class. @@ -112,18 +113,21 @@ def generate(args): # Set network # Option: unet/cspdarkunet # [Warn] Compatible with older versions, version <= 1.1.1 + # [Warn] Version <= 1.1.1 need to be equal to model's network, version > 1.1.1 can set whatever you want parser.add_argument("--network", type=str, default="unet") # Set activation function (needed) # [Note] The activation function settings are consistent with the loaded model training settings. # [Note] If you do not set the same activation function as the model, mosaic phenomenon will occur. # Option: gelu/silu/relu/relu6/lrelu # [Warn] Compatible with older versions, version <= 1.1.1 + # [Warn] Version <= 1.1.1 need to be equal to model's act, version > 1.1.1 can set whatever you want parser.add_argument("--act", type=str, default="gelu") # =====================Enable the conditional generation (if '--conditional' is set to 'True')===================== # Number of classes (required) # [Note] The classes settings are consistent with the loaded model training settings. # [Warn] Compatible with older versions, version <= 1.1.1 + # [Warn] Version <= 1.1.1 need to be equal to model's num classes, version > 1.1.1 can set whatever you want parser.add_argument("--num_classes", type=int, default=10) # Class name (required) # if class name is `-1`, the model would output one image per class.