ValueError: num_samples should be a positive integeral value, but got num_samples=0

[hellopython123]

I run train_Mask-ShadowGAN.py and have errors as following:
Traceback (most recent call last):
File "train_Mask-ShadowGAN.py", line 140, in
batch_size=opt.batchSize, shuffle=True, num_workers=opt.n_cpu)
File "F:\python3.5\lib\site-packages\torch\utils\data\dataloader.py", line 802, in init
sampler = RandomSampler(dataset)
File "F:\python3.5\lib\site-packages\torch\utils\data\sampler.py", line 64, in init
"value, but got num_samples={}".format(self.num_samples))
ValueError: num_samples should be a positive integeral value, but got num_samples=0
I don't know where to modify the dataset path or why len(self.data_ info)= =0.Can you give me some advice?
Thanks.

[RewindL]

Change "from datasets_USR import ImageDataset" to "from datasets_ISTD import ImageDataset" in "train_Mask-ShadowGAN" and have a try again.

[Arican-Lee]

Hello,I encountered the same problem, and I also tried to use the above method(Change "from datasets_USR import ImageDataset" to "from datasets_ISTD import ImageDataset" in "train_Mask-ShadowGAN" and have a try again.). But it won't work.Is there any other way?

[RewindL]

Maybe you should check "dataroot" option in train_Mask-ShadowGAN is correct like "xxx/ISTD/", and also change path of "self.files_B" in ImageDataset from "train_C" to "train_C_fixed" or something else if you use adjusted ISTD dataset and shadow-free path is not end with "train_C". Good luck.

Hello,I encountered the same problem, and I also tried to use the above method(Change "from datasets_USR import ImageDataset" to "from datasets_ISTD import ImageDataset" in "train_Mask-ShadowGAN" and have a try again.). But it won't work.Is there any other way?

[Arican-Lee]

Thank you for your reply.Do you want to make changes like this?

However,the results did not change.

[RewindL]

Thank you for your reply.Do you want to make changes like this? However,the results did not change.

The dataroot option should be your own ISTD dataset's path, abviouly it should not be '/home/xwhu/dataset/train', make sure its correct. And also check "self.files_B" in ImageDataset, if you use adjusted ISTD dataset (ISTD+), shadow-free path might be "./train_C_fixed" or "./train_C_official_adjusted", make sure it's correct before you start training, or program can't find traning data with wrong training path.

[Mautong]

Hello,I'm the partner of Lee, I'm wondering why this situation still happens. I have changed my data path., which is a path without shadow image. Should I add more paths?Thank you

[RewindL]

Hello,I'm the partner of Lee, I'm wondering why this situation still happens. I have changed my data path., which is a path without shadow image. Should I add more paths?Thank you

Try setting opt.dataroot=r"D:\project\shadow-removal\Dataset\ISTD" (removing "train\train_C" in the end, and use prefix 'r' if you use \ in the path).

Again, if your ISTD dataset contains "train/train_A", "train/train_B" and "train/train_C" by default, DO NOT CHANGE "self.files_B" in ImageDataset as metioned above. Just change opt.dataroot is enough.

[Arican-Lee]

I have changed the path, but it showed this information, I don't understand what it means, maybe I can never run this code

Hello,I'm the partner of Lee, I'm wondering why this situation still happens. I have changed my data path., which is a path without shadow image. Should I add more paths?Thank you

Try setting opt.dataroot=r"D:\project\shadow-removal\Dataset\ISTD" (removing "train\train_C" in the end, and use prefix 'r' if you use \ in the path).

Again, if your ISTD dataset contains "train/train_A", "train/train_B" and "train/train_C" by default, DO NOT CHANGE "self.files_B" in ImageDataset as metioned above. Just change opt.dataroot is enough.

[RewindL]

I have changed the path, but it showed this information, I don't understand what it means, maybe I can never run this code

Hello,I'm the partner of Lee, I'm wondering why this situation still happens. I have changed my data path., which is a path without shadow image. Should I add more paths?Thank you

Try setting opt.dataroot=r"D:\project\shadow-removal\Dataset\ISTD" (removing "train\train_C" in the end, and use prefix 'r' if you use \ in the path).
Again, if your ISTD dataset contains "train/train_A", "train/train_B" and "train/train_C" by default, DO NOT CHANGE "self.files_B" in ImageDataset as metioned above. Just change opt.dataroot is enough.

I don't understand why you set opt.dataroot="A:/project/shadow-remove/Mask-ShadowGAN-master/home/xwhu/dataset"? You have put ISTD dataset in this path? Why it's not end with "ISTD"?

[Mautong]

me too，but mine is end with "ISTD"

I have changed the path, but it showed this information, I don't understand what it means, maybe I can never run this code

Hello,I'm the partner of Lee, I'm wondering why this situation still happens. I have changed my data path., which is a path without shadow image. Should I add more paths?Thank you

Try setting opt.dataroot=r"D:\project\shadow-removal\Dataset\ISTD" (removing "train\train_C" in the end, and use prefix 'r' if you use \ in the path).
Again, if your ISTD dataset contains "train/train_A", "train/train_B" and "train/train_C" by default, DO NOT CHANGE "self.files_B" in ImageDataset as metioned above. Just change opt.dataroot is enough.

I don't understand why you set opt.dataroot="A:/project/shadow-remove/Mask-ShadowGAN-master/home/xwhu/dataset"? You have put ISTD dataset in this path? Why it's not end with "ISTD"?

[RewindL]

me too，but mine is end with "ISTD"

I have changed the path, but it showed this information, I don't understand what it means, maybe I can never run this code

Hello,I'm the partner of Lee, I'm wondering why this situation still happens. I have changed my data path., which is a path without shadow image. Should I add more paths?Thank you

Try setting opt.dataroot=r"D:\project\shadow-removal\Dataset\ISTD" (removing "train\train_C" in the end, and use prefix 'r' if you use \ in the path).
Again, if your ISTD dataset contains "train/train_A", "train/train_B" and "train/train_C" by default, DO NOT CHANGE "self.files_B" in ImageDataset as metioned above. Just change opt.dataroot is enough.

I don't understand why you set opt.dataroot="A:/project/shadow-remove/Mask-ShadowGAN-master/home/xwhu/dataset"? You have put ISTD dataset in this path? Why it's not end with "ISTD"?

Change opt.log_path = os.path.join('output', str(datetime.datetime.now()) + '.txt') to opt.log_path = os.path.join('output/', str(datetime.datetime.now()) + '.txt')

[Mautong]

still

[Arican-Lee]

The authors only published the USR dataset. But previously obtained ISTD dataset from other place.

If I end with "ISTD",the “ValueError: num_samples should be a positive integer value, but got num_samples=0” happen once again, so I end with "Train"，the suitation is the same.

If I don't end with thing,when I ran the code ,it happened “OSError: [Errno 22] Invalid argument: 'output\2022-03-20 22:04:21.766201.txt'”

[RewindL]

The authors only published the USR dataset. But previously obtained ISTD dataset from other place.

If I end with "ISTD",the “ValueError: num_samples should be a positive integer value, but got num_samples=0” happen once again, so I end with "Train"，the suitation is the same.

If I don't end with thing,when I ran the code ,it happened “OSError: [Errno 22] Invalid argument: 'output\2022-03-20 22:04:21.766201.txt'”

First, the dataroot means your ISTD dataset's path, it should have two subfolders "train" and "test", and program will find training image according ImageDataset. If you met num_samples=0, that means there is no images in corresponding path, so you should check your setting of dataroot and ImageDataset.

Second, there might be some problem if you run the code directly in Windows environment, i have not tried run it in Windows, so I am not sure whether the path is valid (for example, opt.log_path = os.path.join('output', str(datetime.datetime.now()) + '.txt')), I strongly recommend running the project in Linux, so that you can just download dataset and change dataroot path and successfully run training program. For Windows, I'm not very skilled, but according to the error prompt, it's caused by wrong path format.

[Mautong]

The authors only published the USR dataset. But previously obtained ISTD dataset from other place.

If I end with "ISTD",the “ValueError: num_samples should be a positive integer value, but got num_samples=0” happen once again, so I end with "Train"，the suitation is the same.

If I don't end with thing,when I ran the code ,it happened “OSError: [Errno 22] Invalid argument: 'output\2022-03-20 22:04:21.766201.txt'”

First, the dataroot means your ISTD dataset's path, it should have two subfolders "train" and "test", and program will find training image according ImageDataset. If you met num_samples=0, that means there is no images in corresponding path, so you should check your setting of dataroot and ImageDataset.

Second, there might be some problem if you run the code directly in Windows environment, i have not tried run it in Windows, so I am not sure whether the path is valid (for example, opt.log_path = os.path.join('output', str(datetime.datetime.now()) + '.txt')), I strongly recommend running the project in Linux, so that you can just download dataset and change dataroot path and successfully run training program. For Windows, I'm not very skilled, but according to the error prompt, it's caused by wrong path format.

ohh,I see, I will try to use it in Linux, thank you for your precious time to help us

[Arican-Lee]

Oh,Oh.Thank for your guidance. I don't have a Linux system, so I don't know. I'll get a Linux system first and then run it.