Image Super Resolution Using SRGAN in Keras and TensorFlow

 In this article, you will learn how to convert blur and low-resolution images into high super-resolution images using the generative adversarial network, in short called SRGAN.

What are Generative Adversarial Networks

Before knowing about Generative Adversarial Networks, we need to know about Generative Modeling. In machine learning, GM is an unsupervised learning approach, that involves automatically discovering and enhancing patterns and regularities in input data (e.g. in images samples) in such a way that the generated model can be used to regenerate new patterns, output, or images.

Enjoy Free Tools: Explore Free Online Image Resizer Tool

In simple words

Generative Adversarial Networks or GANs, are an approach to generative modeling using deep learning methods, such as convolutional neural networks. As a Generator and Discriminator, GAN generally works with image datasets and used Convolutional Networks (CNN).

How GAN Works

The architecture of Generative Adversarial Network (GAN Model) consists of two child models. A generator and a discriminator.

Model	Usage
Generator	Generate new samples or outputs from a given input or problem domain
Discriminator	Classify the regenerated (new samples or output) as fake or real

Types of Generative Adversarial Networks (GAN)

Below are the main types of generative adversarial networks.

Sr.No.	Name
1	Vanila GAN
2	Deep Convolutional GAN - DCGAN
3	Conditional GAN - CGAN
4	Super Resolution GAN - SRGAN
5	Info GAN

Usage and Application of Generative Adversarial Networks (GAN)

Below are the usage and application of generative adversarial networks.

Sr.No.	Application
1	Generate fake data for augmenting
2	Generate deep fake faces
3	Image to image translation
5	Text to image translation
6	Super resolution

Image Super-Resolution Using SRGAN in Keras and TensorFlow

To generate high and super-resolution images we are going to use SRGAN (Super-Resolution Generative Adversarial Networks), and to implement this we will be using Keras and TensorFlow python deep learning libraries.

Below are the main python libraries, which we are going to use in this tutorial.

Sr.No.	Package / Library	Version
1	Tensowflow	1.15.0
2	Keras	2.3.1
3	Numpy	1.19.5

Your Custom Data Sets (Images)

You can use your custom data set (images) to train the model. The minimum number of images will be 500, but I will suggest you to use the maximum number of images for better training. You can use any type of image like jpeg, jpg, png, etc. But remember you have to change the image extension in the code where you are going to fetch these images to feed your model.

Data Reprocessing

In SRGAN we are going to use the VGG19 pre-trained model that takes images as input shape (224,224,3). So to feed input images to train our model, we have to resize our data sets. Below python code will resize the data set in the required image shape.

Read More: Resize Image using Python for Deep Learning Model VGG19

Directory Structure of the Project

Below are the structure of directory and python files.

SRGAN

data

all_of_your_images_goes_here.jpeg

model

srgan.py

weights

weight files will be saved in this directory, files looks like (srgan_weights_epoch_160.h5)

init.py

utils.py

train.py

test.py

Below are the working code snippets, just copy and paste the below code in above created project (your have to create same directory structure) in PyCharm or Google Colab.

File Name	Description
srgan.py	Contains generator, discriminator and model (VGG19) training code
init.py	Contains the python command line argument structure to run the code
utils.py	Contains helping utility functions
train.py	Gets command line arguments and call training function, and after training save model weights in .h5 extension
test.py	Gets command line arguments and load saved model weights to generate super resolution images

srgan.py

init.py

utils.py

train.py

test.py

How to Run SRGAN Project using PyCharm

If you have successfully created the project as mentioned above (same directory structure), then set the configuration to run the project. Just click on edit configuration in PyCharm and follow the below steps.

To train the model set your pycharm configuration like below.

Configuration	Setting
Script	D:\Projects\SRGAN\init.py
Script Parameters	--mode train --dir-path ./data

To test the model set your pycharm configuration like below.

Configuration	Setting
Script	D:\Projects\SRGAN\init.py
Script Parameters	--mode test --dir-path ./data --load-weights-path ./weights/your_saved_model_name_goes_here.h5

How to Run SRGAN Project using Google Colab

To run Image Super-Resolution Using SRGAN in google colab, just login into google colab and create srgan.ipynb new project. To create a project structure like PyCharm, you can directly create folders and files in your google drive. To use image datasets you just need to upload all of your images into your google drive and mount your google drive with google colab to access these folders files and images (datasets).

In srgan.ipynb write the below code in a separate code cell.

#ount google drive in colab

from google.colab import drive

drive.mount('/content/gdrive')

#set current working directory in google colab

import os

os.chdir('/content/gdrive/My Drive/SRGAN/')

!pwd

#Import Dependencies

import  os,shutil

import cv2

import numpy as np

import matplotlib.pyplot as plt

import tensorflow as tf

from tqdm import tqdm_notebook as tqdm

from PIL import Image

from skimage.transform import resize

from models.SRGAN import SRGAN

from utils import *

# execute below code to train the model, this will save model weights in your given path

!python init.py --mode train --dir-path '/content/gdrive/My Drive/SRGAN/images' --load-weights-path '/content/gdrive/My Drive/SRGAN'