feat(ml): dataloader for video generation

[wr0124]

Fix the data loader to improve the efficiency of loading videos

[wr0124]

dataloader works for video generation. It processes sequencdes of frames in both of the following formats:

1. /data/img(bbox)/mario_framenumber.jpg(.txt)
2. /data/img(bbox)/video_series/video_series_framenumber.jpg(.txt)

python3 -W ignore::UserWarning train.py
--dataroot /data1/juliew/ori_dataset/continental_20240827/code/conti
--checkpoints_dir /data1/juliew/checkpoints
--name test_conti
--gpu_ids 2
--train_save_epoch_freq 10000
--data_relative_paths
--data_temporal_number_frames 8
--data_online_creation_crop_size_A 32
--data_online_creation_crop_size_B 32
--data_crop_size 32
--data_load_size 32
--model_type palette
--output_print_freq 1
--output_display_freq 1
--data_dataset_mode self_supervised_temporal_labeled_mask_online
--train_batch_size 2
--train_iter_size 8
--data_num_threads 1
--model_input_nc 3
--model_output_nc 3
--data_relative_paths
--train_G_ema
--train_optim adamw
--G_netG unet_vid
--data_online_creation_rand_mask_A
--train_G_lr 0.0001
--dataaug_no_rotate
--G_diff_n_timestep_train 5
--G_diff_n_timestep_test 3
--data_temporal_frame_step 1
--alg_diffusion_cond_image_creation 'y_t' \

[beniz]

This needs to go into a different dataloader, dedicated to videos, since it requires a specific dataset structure, with one dir per video.

[wr0124]

This needs to go into a different dataloader, dedicated to videos, since it requires a specific dataset structure, with one dir per video.

This dataloader is specifically created for video generation, which did not exist before.

[wr0124]

it woks now with:

python3 -W ignore::UserWarning train.py
--dataroot /data1/juliew/dataset/online_mario2sonic_full_mario
--checkpoints_dir /data1/juliew/checkpoints
--name mario_vid
--gpu_ids 0
--model_type palette
--output_print_freq 1
--output_display_freq 1
--data_dataset_mode self_supervised_vid_mask_online
--train_batch_size 2
--train_iter_size 1
--model_input_nc 3
--model_output_nc 3
--data_relative_paths
--train_G_ema
--train_optim adamw
--G_netG unet_vid
--data_online_creation_crop_size_A 32
--data_online_creation_crop_size_B 32
--data_crop_size 32
--data_load_size 32
--data_online_creation_rand_mask_A
--train_G_lr 0.0001
--dataaug_no_rotate
--G_diff_n_timestep_train 6
--G_diff_n_timestep_test 3
--data_temporal_number_frames 8
--data_temporal_frame_step 1
--data_online_creation_mask_delta_A_ratio 0.12 0.12
--alg_diffusion_cond_image_creation computed_sketch
--alg_diffusion_cond_computed_sketch_list canny
--alg_diffusion_vid_canny_dropout 0.1 0.8
--alg_diffusion_cond_sketch_canny_range 500 1000
--G_unet_vid_max_sequence_length 15 \