feat: reference image conditioning [2306.08276]

[pnsuau]

This PR implements a dataloader for side / reference image in addition to the image input, used as additionnal conditioning.

• Dataloader
• Input concat as cond
• Bind embedding cond
• Clip embedding cond
• Test

Usage:

• --data_dataset_mode {unaligned_labeled_mask_ref,self_supervised_labeled_mask_ref}

  • conditions.txt format:

    /path/to/img.png /path/to/ref.png
    

• --data_dataset_mode {unaligned_labeled_mask_online_ref,self_supervised_labeled_mask_online_ref}

  • conditions.txt format:

    /path/to/img.png /path/to/cond.txt
    

  • /path/to/cond.txt format:

    /path/to/ref1.png # ref image for first bbox
    /path/to/ref2.png # ref image for second bbox
    # same number of lines and order as corresponding bbox.txt
    

• --alg_palette_conditioning ref conditioning with embedding

  • --alg_palette_ref_embed_net clip using CLIP
  • --alg_palette_ref_embed_net imagebind using ImageBind

• --alg_palette_cond_image_creation ref conditioning with concat to input