Multiple Output
Defined a secondary output with following probability distribution as ground truth:
| airplane | automobile | bird | cat | deer | dog | frog | horse | ship | truck | |
|---|---|---|---|---|---|---|---|---|---|---|
| airplane | 0.5 | 0.20 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.1 | 0.20 |
| automobile | 0.1 | 0.50 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.0 | 0.40 |
| bird | 0.0 | 0.00 | 0.75 | 0.05 | 0.05 | 0.05 | 0.05 | 0.05 | 0.0 | 0.00 |
| cat | 0.0 | 0.00 | 0.00 | 0.60 | 0.05 | 0.30 | 0.00 | 0.05 | 0.0 | 0.00 |
| deer | 0.0 | 0.00 | 0.00 | 0.05 | 0.60 | 0.05 | 0.00 | 0.30 | 0.0 | 0.00 |
| dog | 0.0 | 0.00 | 0.00 | 0.30 | 0.05 | 0.60 | 0.00 | 0.05 | 0.0 | 0.00 |
| frog | 0.0 | 0.00 | 0.05 | 0.05 | 0.05 | 0.05 | 0.75 | 0.05 | 0.0 | 0.00 |
| horse | 0.0 | 0.00 | 0.00 | 0.05 | 0.30 | 0.05 | 0.00 | 0.60 | 0.0 | 0.00 |
| ship | 0.2 | 0.05 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.7 | 0.05 |
| truck | 0.1 | 0.40 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.0 | 0.50 |
- Trained 9 networks.
- Adding the extra output of network at stack index i and block j
for stack in range(3):
for res_block in range(num_res_blocks):
y = resnet_layer(inputs=x, num_filters=num_filters)
y = resnet_layer(inputs=y, num_filters=num_filters, activation=None)
if stack > 0 and res_block == 0: # first layer but not first stack
# linear projection residual shortcut connection to match
# changed dims
x = resnet_layer(inputs=x, num_filters=num_filters, kernel_size=1)
x = keras.layers.add([x, y])
x = Activation('relu')(x)
if 'natural_vs_manmade' in output_types and wni == stack and wnj == res_block:
x_nvm = AveragePooling2D(pool_size=8)(x)
y_nvm = Flatten()(x_nvm)
natural_vs_manmade_outout = Dense(10, activation='softmax', name='natural_vs_manmade')(y_nvm)- Essentially network accuracy for the CIFAR-10 task is around 87% regardless of where the second output is place.
- The second output increases from 70 to 86% depending whether it's at the start or end of the architecture (this is expected).
- Similar findings for natural versus man-made objects:
- The primary accuracy reaches 90%
- The man-made versus natural ranges between 90 to 98%.