Integrating Deep Learning model With Django

In this project, we will be integrating a deep learning model with the Django framework. This deep learning model converts greyscale images to colorful images.

We will store the user-given greyscale image in our database and then resize the image, submit this resized image, to our DL model, and then provide the user with a token. This token can be used to download the colorful image.

Note :

1. This project implements the simplest frame/skeleton to integrate a deep learning model with the Django backend.
2. This project receives an image input, submits that image to the model for processing and then returns a download token to download the processed image.

Prerequisites:

1. There should be a deep learning project ready for implementation.
2. The project must include a file containing the trained model's weights. We are using a model trained with the Caffe deep learning framework, where the .prototxt and .caffemodel files are standard formats. The .prototxt file stores the architecture of the network, while the .caffemodel file contains the weights of the compiled and trained model.
3. You should be familiar with Django and database schemas, as well as handling static and media files in Django.

Flow chart for image post event

Flow chart for token post event

Project Setup:

Step 1: Create a directory named as myDL_project.

Step 2: Open that directory in Vscode.

Step 3: Create a virtual environment. We will be using pipenv to create a virtual environment.

pipenv shell

Step 4: Install Django, opencv and pillow required for this project by using command:

pipenv install django opencv pillow

Step 5: Start a project in django by using command:

django-admin startproject DLWebApp

Step 6: Create a new app home inside the project director (myDL_project/DLWebApp). It will handle the landing page and upload/download form for image.

cd DLWebApp
python manage.py startapp home

Step 7 : Register our new home app in the project settings.

In DLWebApp/DLWebApp/settings.py

Step 8: Create a template directory in the parent directory, at the same level as our project and app directories (DLWebApp/templates).

Setup the templates path in settings.py in the project directory.

• Inside template directory create a base.html.
• We will use bootstrap for css.
• We will use django-templatesto render webpages in this base.html.

In DLWebApp/templates/base.html

Create a upload_form.html in templates directory. This form will accept greyscale image from user.

Create a download_form.html in templates directory. This form will accept token for colored image from user.

Views And Model to Handle Image Upload

Step 1: Creating Views

Create an index view in home app to handle the loading of upload_form page.

In DLWebApp/home /views.py

We will add logic to handle image processing later.

Step 2: URL Setup:

• Create urls.py in home app directory.
• Create an URL path for Index view.

In DLWebApp/home /urls.py

Include home.urls. in projects urls.py.

In DLWebApp/DLWebApp/urls.py

Run the development server using command:

python manage.py runserver

Step 3: Creating Model

Create a model which can store greyscale image and its id. Create another model which can store colored image and its id. Create one more model which can store resized image.(DLWebApp/home/models.py)

Run migration commands to migrate these models into the data base.

python manage.py makemigrations
python manage.py migrate 

Step 4: Add View Logic to process image

Design a view logic which will handle the greyscale post event.

• The view will receive the greyscale image then call a resize function to resize that image.
• This greyscale and resized images were saved in the database.
• After successful resizing call the colorize_image, function to use DL to convert the resized grey image to colorful image.
• Save this colorful image to the database and return the token to download this image.

In DLWebApp/home/views.py

Step 5: Add Javascript code

Inside upload.html to display these signature and download tokens as alert messages.

Setup the media file path

Step 1: In settings.py file of the project update the media file root, url and startic_url.

In DLWebApp/DLWebApp /settings.py

Step 2 : Add url for media directory in the projects urls.py.

Step 3: Upload an image through the form and check if it is saved in the DLWebApp/media/grayscale_images directory.

Integrating Deep Learning Model:

Deep Learning Model :

1. The deep learning model is converting a grey image to colorized image.
2. We have used Caffe framework to develop my model so .caffemodel file contains the weight of my model. You can also use .h5 format to store your model weights.
3. Integration of a deep learning model and a machine learning model with Django follows different procedures.

Step 1 : Create a DL directory in same level as the project and home app.

Step 2 : Upload all model files to this directory. We are the Caffe framework for model training, so the model weights are saved in .caffemodel files, and the network architecture is defined in .prototxt. Additionally, we may have some data saved in .npy format.

Step 3 : Import all the dependencies we will need to handle image processing and your model. Additionally import os module.

In DLWebApp/home/views.py

Step 4: Load the model

In DLWebApp/home/views.py

Step 5 : Complete the colorize_image function which converts the resized grey image to colorful image.

In DLWebApp/home/views.py

Step 6 : Upload a greyscale image and check if our backend is producing a colourful image in media/colorized_images directory.

We should receive alerts containing the token for signature and download.

Views to handle download

In DLWebApp/home/views.py

Step 1 : Create a Download class in home/views.py which will render download_form.html for get request.

Step 2 : Create POST method in Download class to handle the form submission. This method will search and return a colorized image for the provided token.

URL setup: Add path to post the token to Download class.

In DLWebApp/home/urls.py

Step 3 : Add the URL of download class to the action of the download_form.html .

Step 4 : Create a download link in the upload form which will redirect user to the download form.

In DLWebApp/templates/upload_form.html

Output:

Step 5 : Upload a grey scale image to the upload form. It will return you two tokens signature and download. Copy download token and go to download form, paste the download token and you will receive a colored image.

Scope for optimization :

1. As this project aims to provide a basic skeleton to integrate a Deep Learning Model with django framework, it doesn't focus much on optimizing database queries.
2. Always load your model in top of the views so that this model can be loaded beforehand and is ready to accept inputs.
3. With this skeleton the model will be loaded only once in the server which is an optimized way of handling models in backend. For each different inputs, views will only call the pre loaded model there is no need to load the model for each different inputs.