CUDA of memory during testing

[Grassyue]

It’s an awesome work, and thank you for sharing.😄

When I apply DA-CLIP to train on my own dataset for a dehazing task, I encounter an error during testing: "CUDA out of memory." It seems like the issue is due to the large size of my test images (1024x1024).
Could you please provide some suggestions to fix this? Thank you very much!

[Algolzw]

Hi, thanks!
The easiest way of reducing the CUDA memory is to 1) resize the input to a smaller size, 2) perform IR on this resized image, and 3) resize the result back to the same size as the input.

[Grassyue]

Thanks a lot!
I have one more question: would reducing the image size affect the model's performance?

[Algolzw]

Yes! So you need to carefully tune the scale factor such that the performance won't be affected too much :)

[jkhu29]

Maybe you could use the tile test.

if not opt["tile"]:
    noisy_state = sde.noise_state(LQ)

    model.feed_data(noisy_state, LQ, GT, text_context=degra_context, image_context=image_context)
    tic = time.time()
    model.test(sde, mode=sampling_mode, save_states=False)
    toc = time.time()
    test_times.append(toc - tic)

    visuals = model.get_current_visuals()
    SR_img = visuals["Output"]
else:
    # Tile test
    b, c, h, w = LQ.size()
    tile = 512
    tile_overlap = 32

    stride = tile - tile_overlap
    h_idx_list = list(range(0, h - tile, stride)) + [h - tile]
    w_idx_list = list(range(0, w - tile, stride)) + [w - tile]
    E = torch.zeros(b, c, h, w).type_as(LQ)
    W = torch.zeros_like(E)

    tic = time.time()
    for h_idx in h_idx_list:
        for w_idx in w_idx_list:
            in_patch = LQ[..., h_idx : h_idx + tile, w_idx : w_idx + tile]
            gt_patch = GT[..., h_idx : h_idx + tile, w_idx : w_idx + tile]
            noisy_state = sde.noise_state(in_patch)
            model.feed_data(noisy_state, in_patch, gt_patch, text_context=degra_context, image_context=image_context)
            model.test(sde, mode=sampling_mode, save_states=False)
            out_patch = model.get_current_visuals()["Output"].type_as(LQ)
            out_patch_mask = torch.ones_like(out_patch)
            x0 = h_idx
            y0 = w_idx
            x1 = (h_idx + tile)
            y1 = (w_idx + tile)

            E[..., x0:x1, y0:y1].add_(out_patch)
            W[..., x0:x1, y0:y1].add_(out_patch_mask)
    SR_img = E.div_(W)
    toc = time.time()
    test_times.append(toc - tic)

output = util.tensor2img(SR_img.squeeze())  # uint8

[TanMIMI]

Does stitching patches of image after they have been generated result in an image with distinct grid boundaries? @jkhu29

[jkhu29]

Does stitching patches of image after they have been generated result in an image with distinct grid boundaries? @jkhu29

In my test, such artifacts are rare.

Imagine we are cropping a 1024-resolution image to many 512-resolution images with a stride of 480, there will be many 32x32 overlapping areas. They belong to four cropped images and will be calculated four times during testing. Their results are cached (E in code) and averaged (.div_(W) in code) when stitching the images. This averaging can reduce (but not eliminate) boundary artifacts.

[TanMIMI]

Does stitching patches of image after they have been generated result in an image with distinct grid boundaries? @jkhu29

In my test, such artifacts are rare.

Imagine we are cropping a 1024-resolution image to many 512-resolution images with a stride of 480, there will be many 32x32 overlapping areas. They belong to four cropped images and will be calculated four times during testing. Their results are cached (E in code) and averaged (.div_(W) in code) when stitching the images. This averaging can reduce (but not eliminate) boundary artifacts.

thank you so much for your reply and good method!