An implementation of the arXiv preprint A Neural Algorithm of Artistic Style [1] & paper Image Style Transfer Using Convolutional Neural Networks [2].
Supports TensorFlow 2.4.1.
style-transfer.ipynb describes the style transfer process between a white noise image x, a content image p, and a style representation a. Performing gradient descent of the content loss and style loss with respect to x impressions the content of p into x, bearing local styles, and colors from a.
| Original Photograph Tubingen, Germany |  |
|
| Claude Monet Houses of Parliament |  |
 |
| Pablo Picasso Seated Nude |  |
 |
| Edvard Munch The Scream |  |
 |
| Vincent van Gogh The Starry Night |  |
 |
| William Turner The Shipwreck of The Minotaur |  |
 |
| Wassily Kandinsky Composition VII |  |
 |
content-reconstruction.ipynb describes the content reconstruction process from white noise. Performing gradient descent of the content loss on a white noise input x for a given content p yields a representation of the networks activation for a given layer l.
| Layer | Result |
|---|---|
block1_conv1 |
 |
block2_conv1 |
 |
block3_conv1 |
 |
block4_conv1 |
 |
block4_conv2 |
 |
block5_conv1 |
 |
style-reconstruction.ipynb describes the style reconstruction process on Wassily Kandinsky's Composition VII from white noise. Performing gradient descent of the style loss on a white noise input x for a given artwork a yields a representation of the networks activation for a given set of layers L.
| Layer | Result |
|---|---|
block1_conv1 |
 |
block1_conv1, block2_conv1 |
 |
block1_conv1, block2_conv1, block3_conv1 |
 |
block1_conv1, block2_conv1, block3_conv1, block4_conv1 |
 |
block1_conv1, block2_conv1, block3_conv1, block4_conv1, block5_conv1 |
 |
content-layer.ipynb visualizes how the style transfer is affected by using different layers for content loss.
| Layer | Result |
|---|---|
block1_conv1 |
 |
block2_conv1 |
 |
block3_conv1 |
 |
block4_conv1 |
 |
block5_conv1 |
 |
style-layers.ipynb visualizes how the style transfer is affected by using different sets of layers for style loss.
| Layers | Result |
|---|---|
block1_conv1 |
 |
block1_conv1, block2_conv1 |
 |
block1_conv1, block2_conv1, block3_conv1 |
 |
block1_conv1, block2_conv1, block3_conv1, block4_conv1 |
 |
block1_conv1, block2_conv1, block3_conv1, block4_conv1, block5_conv1 |
 |
optimizers.ipynb employs gradient descent, adam, and L-BFGS to understand the effect of different black-box optimizers. Gatys et. al use L-BFGS, but Adam appears to produce comparable results without as much overhead.
| Gradient Descent | Adam | L-BFGS |
|---|---|---|
 |
 |
 |
tv-loss.ipynb introduces total-variation loss to reduce impulse noise in the images.
| TV Loss Scale Factor | Result |
|---|---|
0 |
 |
1 |
 |
10 |
 |
100 |
 |
1000 |
 |
photo-realistic-style-transfer.ipynb describes the photo-realistic style transfer process. Opposed to transferring style from an artwork, this notebook explores transferring a nighttime style from a picture of Piedmont Park at night to a daytime picture of Piedmont Park.
| Content | Style | Result |
|---|---|---|
 |
 |
 |