Data generator: where use data augmentation and adaptive pixels weights

[pissw2016]

Hi, I am new to python and keras, so your code really help me a lot. When I am running segmentation.ipynb I got a error in data generation when trainning , so I did some debug.
The error happend during the middel of training some times at step76 some times step 99 like it is some picture error, so I stop shuffle. It was still random.
error:

Epoch 1/10
429/1238 [=========>....................] - ETA: 7:43 - loss: 0.2252 - Jaccard: 0.6086 - sparse_accuracy_ignoring_last_label: 0.9520multiprocessing.pool.RemoteTraceback:
"""
Traceback (most recent call last):
File "/home/DATA/liutian/envs/mask/lib/python3.6/multiprocessing/pool.py", line 119, in worker
result = (True, func(*args, **kwds))
File "/home/DATA/liutian/envs/mask/lib/python3.6/site-packages/keras/utils/data_utils.py", line 401, in get_index
return _SHARED_SEQUENCES[uid][i]
File "/home/DATA/liutian/tmp/deeplab/Keras-segmentation-deeplab-v3.1-master/utils.py", line 362, in getitem
class_weights = class_weight.compute_class_weight('balanced', u_classes, valid_pixels)
File "/home/DATA/liutian/envs/mask/lib/python3.6/site-packages/sklearn/utils/class_weight.py", line 55, in compute_class_weight
weight = recip_freq[le.transform(classes)]
IndexError: arrays used as indices must be of integer (or boolean) type
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/home/DATA/liutian/tmp/deeplab/Keras-segmentation-deeplab-v3.1-master/mainslic.py", line 83, in
history = SegClass.train_generator(model, train_generator, valid_generator, callbacks, mp=True)
File "/home/DATA/liutian/tmp/deeplab/Keras-segmentation-deeplab-v3.1-master/utils.py", line 226, in train_generator
workers=workers, use_multiprocessing=mp)
File "/home/DATA/liutian/envs/mask/lib/python3.6/site-packages/keras/legacy/interfaces.py", line 91, in wrapper
return func(*args, **kwargs)
File "/home/DATA/liutian/envs/mask/lib/python3.6/site-packages/keras/engine/training.py", line 1418, in fit_generator
initial_epoch=initial_epoch)
File "/home/DATA/liutian/envs/mask/lib/python3.6/site-packages/keras/engine/training_generator.py", line 181, in fit_generator
generator_output = next(output_generator)
File "/home/DATA/liutian/envs/mask/lib/python3.6/site-packages/keras/utils/data_utils.py", line 601, in get
six.reraise(*sys.exc_info())
File "/home/DATA/liutian/envs/mask/lib/python3.6/site-packages/six.py", line 693, in reraise
raise value
File "/home/DATA/liutian/envs/mask/lib/python3.6/site-packages/keras/utils/data_utils.py", line 595, in get
inputs = self.queue.get(block=True).get()
File "/home/DATA/liutian/envs/mask/lib/python3.6/multiprocessing/pool.py", line 644, in get
raise self._value
IndexError: arrays used as indices must be of integer (or boolean) type

Process finished with exit code 1

Then I comment out :

valid_pixels = self.F[n][self.Y[n] != self.n_classes]  # get all pixels (bg and foregroud) that aren't void
u_classes = np.unique(valid_pixels)
class_weights = class_weight.compute_class_weight('balanced', u_classes, valid_pixels)
class_weights = {class_id: w for class_id, w in zip(u_classes, class_weights)}
if len(class_weights) == 1:  # no bg\no fg
    if 1 in u_classes:
        class_weights[0] = 0.
    else:
        class_weights[1] = 0.
elif not len(class_weights):
    class_weights[0] = 0.
    class_weights[1] = 0.

sw_valid = np.ones(y.shape)
np.putmask(sw_valid, self.Y[n] == 0, class_weights[0])  # background weights
np.putmask(sw_valid, self.F[n], class_weights[1])  # foreground wegihts
np.putmask(sw_valid, self.Y[n] == self.n_classes, 0)
self.F_SW[n] = sw_valid

and I notice that the self.SWis just in [0,1] and not a weight consider the pixels amount relative also the model only got one input layer.
sample_dict = {'pred_mask': self.SW}
I change 'return self.X, self.Y, sample_dict' to
return self.X, self.Y

and it works. So it seems that these code is not used, right?

Data augmentation seems not working cause the step amount in an epoch equals to image amount.
How can I make it working? Or it is already working and I miss it?

[pissw2016]

After debug with the full class SegmentationGenerator(Sequence): I realize that with the official VOC2012 data set. You have to not only make your own self.image_path_list and self.label_path_list. You have to use label = np.array(Image.open(label_path)) and I believe that the class 21 refers to unlabeled class. And there is DA(data augmentation ) the data weight will work as long as your label load in the range of (0,21) . with opencv.imread you will get range (0,220).

[GuardSkill]

nice job!!! same issue