It is a technique known as Neural Style Transfer, blending Style from one image into the base image without changing its content. The only change is the style configurations of the image to give it an artistic touch to the base image. The content image describes the layout or the sketch and Style of the painting. It is an application of Computer Vision related to image processing techniques and Deep Convolutional Neural Networks. Neural Style Transfer deals with two sets of images: Content image or base image and Style image. This technique helps to recreate the base image in the style of the reference image. It uses Neural Networks to apply the artistic style from one image to another. Neural style transfer opens up endless possibilities in design, content generation, and the development of creative tools.

Supplies

- Google Colab

- Google Drive

Step 1: Working Principle of Neural Style Transfer

The principle of neural style transfer is to define two distance functions, one that describes how different the content of two images are base, and one that describes the difference between the two images in terms of their style. Then, given three images, the desired style image, the desired base image, and the input image (initialized with the base image), we try to transform the input image to minimize the content distance with the base image and its style distance with the style image.

In short, we take the input image, a base image that we want to match, and the style image that we want to match. We’ll transform the input image by minimizing the base and style distances (losses) with backpropagation, creating an image that matches the content of the base image and the style of the style image.

Step 2: About Model

VGG19 pre-trained model is used and fed into our input tensor to the model. This will allow us to extract the feature maps (and subsequently the base and style representations) of the base, style, and generated images. In order to access the intermediate layers corresponding to our style and base feature maps, corresponding outputs by using the Keras Functional API to define our model with the desired output activations.

Content loss
It helps to establish similarities between the base image and the generated image. It is intuitive that higher layers of the model focus more on the features present in the image i.e. overall content of the image. Content loss is calculated by Euclidean distance between the respective intermediate higher-level feature representation of the input image and base image at layer l.

Style loss

Style loss is conceptually different from Content loss. It introduces a new term called Gram matrix. Gram matrix is a way to interpret style information in an image as it shows the overall distribution of features in a given layer. It is measured as the amount of correlation present between features maps in a given layer. Style loss is calculated by the distance between the gram matrices (or, in other terms, style representation) of the generated image and the style reference image.

Step 3: Generating Image Using Google Colab

Notebook Link: https://github.com/irongabriel7/AI_Painter

Initially download the python notebook and upload it to Google Drive. Also, change the runtime to GPU for faster generation of the image. Then open it using Google Colab. Before running the first cell of the python notebook, upload the required base and style images to Google Drive where the neural style transfer python notebook is stored. Next, either you can rename the corresponding base and content images as 'base.jpg' and 'style.jpg' otherwise change the filename and type in the fourth cell of the python notebook. Also, you can directly add the link of the images without saving it to google drive.

Run the cell one after the other without missing any cell, otherwise, an error will be generated. In the second cell, change the path to the location where the python notebook is stored. Finally, in the last cell, set the number of iterations required. The higher the iteration, the longer will it take, and the better will be the result.

Hence the generated image will be stored in your drive where the Neural Style Transfer python notebook is stored.