Experiments with neural style transfer
For one of my weekend projects, I chanced upon this paper of L.A. Gatys (Gatys et al. 2015) describing how we can use a convolutional neural network to transfer artistic style from one image to another. Here, I used the neural style transfer technique to learn the artistic style of various Filipino paintings. Painters range from Amorsolo to Manansala, and other modern artists such as Zobel. Enjoy!
Figure 1: Vicente Manansala’s style in Communities was applied in a photo of the Manila Skyline.
Figure 2: Probably one of my favorites. Manansala’s brush strokes applied to the facade of his alma mater.
Figure 3: Fernando Zobel’s work as implemented into one of the buildings in Ateneo.
Figure 4: The work Sarimanok, aptly transfered to a pair of fighting cocks.
Here is also Ateneo’s Church of the Gesu, rendered in Van Gogh’s Starry Night. What beauty!
Figure 5: Van Gogh’s starry night in Church of the Gesu.
If you want to implement this technique, you can use my work written in Python 2.X and Tensorflow. I am running this above our lab’s GPU (NVIDIA Titan X), and it takes around 2-3 minutes for a single style transfer, depending on the resolution (I’m using 72dpi). I haven’t tested this with a normal CPU, but from previous experience, this may take more than 30 minutes to run.
References
- Gatys, Leaon, Echker, Alexander and Bethge, Matthias (2015). “A Neural Algorithm of Artistic Style”. In: arXiv:1508.06576[cs.CV]
Other neural style implementations
My neural style implementation is still naive and dirty, and doesn’t have the degree of freedom to blend various artistic styles perfectly. If you prefer using other implementations, take a look at the following below.
jcjohnson
’s original implementation, using Lua and Torch. (Probably one of the most comprehensive implementations around. It enables you to blend various styles, and control the degree of blending).fzliu
’s implementation, using Python and Caffe. (Here’s something that’s much closer to home. If you’re a Caffe user, this implementation might be more of a breeze. I don’t use Caffe so I opted for Tensorflow).anishathalye
s implementation, using Python and Tensorflow. (A straightforward implementation using Tensorflow. This also adopts various parameters set by jcjohnson, but works under the Tensorflow API)