dataloader num_workers=0

[Haijunlv]

hi, it seems that in DADA, dataloader num_workers have to be 0 in order to avioding mislocation between gradient and actual augment parameters.
But if num worker ==0, the speed advantages of DADA cannot be shown if comparsion with PBA. So do you have some trick to deal with this issue?

[latstars]

hi, it seems that in DADA, dataloader num_workers have to be 0 in order to avioding mislocation between gradient and actual augment parameters.
But if num worker ==0, the speed advantages of DADA cannot be shown if comparsion with PBA. So do you have some trick to deal with this issue?

As described in our paper, we utilize the straight-throught trick. Therefore, we only need to apply the image opeartion with maximum probability.

[Haijunlv]

hi, it seems that in DADA, dataloader num_workers have to be 0 in order to avioding mislocation between gradient and actual augment parameters.
But if num worker ==0, the speed advantages of DADA cannot be shown if comparsion with PBA. So do you have some trick to deal with this issue?

As described in our paper, we utilize the straight-throught trick. Therefore, we only need to apply the image opeartion with maximum probability.

so this means we cannot set num worker > 0 ? have to confirm single process reader?

[Euphoria16]

Hi, I tried num workers > 0, and got this error:

magnitude = self.magnitudes.clamp(0, 1)[self.weights_index.item()]
RuntimeError: CUDA error: initialization error
CUDA kernel errors might be asynchronously reported at some other API call,so the stacktrace below might be incorrect.

I think it is because the getitem function of AugmentDataset uses the parameters of Network in model_search.py including magnitudes, weights, probilities, which are cuda tensors.

[latstars]

This code do not support num_workers > 0, since the data loader need to sample the policy according to the parameters "magnitudes, probabilities", which are different in every iteration.

[siddagra]

Whats the point of DADA being faster if it can only operate with num_workers = 1. It would still be slower than Fast or Faster AutoAugment with num_workers = 12. Its not like most people use single CPU processor cores.

[siddagra]

Would it not be possible to set num_workers to a higher value and sample each batch based on magnitude/probabilities? within iteration, it would be the same