Possible reasons for similarity among generated images

[vasavig]

Hi,

I have been playing around with the DCGAN architecture and I have a question on the similarity among the generated images.

I trained the network on a 140 dimensional vectors sampled from normal(0,1). The results were good after a few epochs, and they look varied too. The generator's output looks like the following:

I modified the above network to take a table of inputs (one 100 dimensional vector and one 40 dimensional vector - both sampled from normal(0,1)) through a parallel table and joined them to make a 140 dimensional vector. The following are some results:

The above two networks are essentially the same because the parameter learning happens only in layers following the join table in 2nd network, and this part of network architecture is the same for both. But the results are more varied in the first network, and there are lots of similar pictures in the 2nd output.

I have observed this on other datasets too. According to my understanding, after training the DCGAN, the generator learns a mapping from Z-vector space to images. Is there a possibility of the generator learning only a "certain set" of images (which are not necessarily in training set, so there's no overfitting) for the whole Z-vector distribution, and output only those images for various Z vector inputs? It would be great if anyone can shed some light on why this might happen.

Thanks!

[soumith]

are your second set of generations on a pre-trained network?

[vasavig]

No, I had trained both the networks from scratch on Celeb dataset, with the only difference of how I created the input to generator, as I explained in my comment.

[vasavig]

Seems like this is a mode collapse problem where the generator is modeling the distribution to one Gaussian rather than a mixture of Gaussians. More pointers on the mode collapse problem can be found in Ian Goodfellow's 2016 NIPS tutorial, https://arxiv.org/abs/1701.00160.

[danieleghisi]

@vasavig Did you find a way to overcome this problem?

[danieleghisi]

In this paper the authors claim to have solved the mode collapse issue via "unrolling" the Adam objective.
I wouldn't be able to implement any of this in the current model, but it might be cool to consider it...?