can't get the same augmentation policy(genotype) with the searching code

[skidsaint]

Thanks for the great work and code!
I want to reproduce the same augmentation policy(called genotype in your code) with the provided searching code. I folllow the description in ReadME.md and search augmentation policy in reduced ImageNet with Res50. However, I found the policy i get is different from the policy you gave in genotype.py, so i want to know whether i do something wrong in repoducing the result. Here are some reasons i guessed may affect the searching results:
1.In the searching code, default random seed is 2 in train_search_paper.py, is this the same random seed you used to get the final result?
2.In searching, i found the augmentations are insert after colorjitter, but in training code, augmentation policy is inserted after RandomHorizontalFlip and before Colorjitter(line 95 in fast-autoaugment/FastAutoAugment/data.py), this is not consistent in training and searching.
Are these two reasons affect the seraching? Or there are some other details i did not found in searching process process？I look forward to your reply，thanks.

[latstars]

The setting of random seed should use the code search_relax/train_search.py. Since we use random seed, different runs can lead to different searched results. If you want a deterministic results, you can try the code search_relax/train_search.py.

[bchao1]

Hi, does "due to our incorrent implementation for train_search_paper.py" mean the code is in all incorrect or just the random seed part? Is the differentiable augmentation part correct? Thanks!

[latstars]

This differentiable augmentation part is correct. The random setting part is useless since we apply https://github.com/VDIGPKU/DADA/blob/master/search_relax/train_search_paper.py#L61 before setting the random seed. You can move this line code https://github.com/VDIGPKU/DADA/blob/master/search_relax/train_search_paper.py#L61 to the main function train_search_paper.py#109.

[Ji4chenLi]

This differentiable augmentation part is correct. The random setting part is useless since we apply https://github.com/VDIGPKU/DADA/blob/master/search_relax/train_search_paper.py#L61 before setting the random seed. You can move this line code https://github.com/VDIGPKU/DADA/blob/master/search_relax/train_search_paper.py#L61 to the main function train_search_paper.py#109.

Hi,
Can you further elaborate this more? If I understand correctly, with the default value of num_policies=105, the set of sub_policies will remain unchanged, i.e. no randomness will be introduced.

In my case, I fail to reproduce the same augmentation policy by searching on reduced-cifar 10. FYI, I am using torch 1.6.0 instead of torch 1.2.0.

[latstars]

Can you further elaborate this more? If I understand correctly, with the default value of num_policies=105, the set of sub_policies will remain unchanged, i.e. no randomness will be introduced.

The sub-policies is the same but only the order is shuffle. Furthermore, we only validate the code on pytorch 1.2.0.