DOING DATA SCIENCE FROM SCRATCH TASK BY TASK
I am on a mini-series of posts about creating an NLP application to help people improve their resumes. The core ideas were introduced in my initial post, and in my last post, I showed how to apply a markup to specific keywords. In this post, I will show you how to hook up a Python back-end and serve that user interface I had in my last post. I hope that if I can break down these tasks into short posts, I can successfully explain my process. As of the time of writing, I have made many such applications as proof of concepts. Are you ready to dive into the back-end world? Let’s do it!
Python back-end
Whilst sometimes I may seem a die-hard Python fan, the reality is that Python is just a tool in my toolbox. I often use Express with Node, and it really comes down to security and available SDK for the various external services such as Okta. Because I am only interested in proof of concepts, my life is less complicated than those who have to guarantee Data Privacy, Security, Performance and Resilience. Why we choose Python as a back-end language comes down, for me, to the machine learning Ecosystem. I never liked Django, but I know it is extremely popular. Flask has always been my go-to, and I never had a performance problem. Naturally, you need to deploy with Nginx and something like Gunicorn with Supervisor.
Hooking up a Single Page front end UI with Python has some steps. First, we need the Python back-end job, and then we need to make some changes to the Vue.js application. Ok, let’s code! I use Visual Code these days!
Flask
Let us discuss the Flask server side job
My approach to development is always an extreme Agile process. My emphasis is on working software at each iteration. An iteration can be minutes or hours, but generally, there is a time box of about 3 hours. Although it might not be obvious, my professional focus in more product management, and I prefer to leave the professionals to real product development.
Lines 1 through 3 in the web.py file deal with importing the libraries. Note we have Gensim already in scope. Line 5 is literally the most crucial in this script. Flask expects a static folder, but Vue and NPM build sets a Dist folder. We need to marry these two up.
Line 13 creates a route for the home page, and at this early stage, that just serves index.HTML, which is generated from the NPM build command. Line 19 runs the app, but please don’t ever do that in a web-facing context! Heed the warning well shown in Figure 1.
Now that the development web server is up and running let us turn our focus to adjust the Vue front end interface.
Vue
In case you just tuned in to the game at half time you can catch up by reading the previous article, and that will tell you how we got here! If you allow the analogy team Data Science Prototyping are up 2 at the interval!
Retrospectives are a big part of Agile, and it is a meaningful ceremony mostly! Generally, we sit down rather than STAND UP! Did you remember? I told you that Flask expects a static folder, but NPM build sets a Dist folder. We need to marry these two up, and we do that by introducing a configuration file for Vue.
Line 2, vue.config.js, defines the build output folder. Line 5 creates the static folder that the Flask requires. This is a critical step, and when omitted, we will just give the 404 error.
Now you need to bear in mind that we are dealing with two different technologies.
- Python is run-time interpreted.
- NPM needs to be built using transpilers, webpack and voodoo that is boring as hell but necessary. Web browser compatibility is worse than the Bog of Eternal Stench.
So the bottom line is
- Git add . ( stage the changes )
- Git commit -m ( wrap changes in a change object )
- Git push ( update the changes to the repo )
- NPM run build ( run voodoo and smell the bog )
- Python backend.py ( execute the server side )
Has to be your process, and it certainly is mine. So now we are ready to execute the hooked up process.
Run it
For this instalment, all that remains is to run the Python script. I already did the build and checked the output. Figure 2 shows the result.
There really isn’t anything new to see other than we are on port 5006 rather than the 8080 port for the Vue development server as shown in Figure 3.
However, I have now shown you how to create both a front-end application and a back-end server-side job. Gensim is already included, and I have my own NLP code from the initial post. In the next post, we will continue the journey.
- Add a better user interface to the front-end.
- Add some front-end API calls to process the resume and job description.
- Add some server-side API endpoints.
Now we are getting into the fun stuff. If you are asking yourself how this is relevant to Data Science well, you should accept that building an NLP model in Python is just not going to cut it. You need a User Interface, you need funding, and you need to convince others to invest both their cash and their family time in your project. Finally, if you are the project manager, product manager, or product owner, you need to build that backlog and deliver the project. You need to be aware of all the steps to bring an NLP solution to your consumers. It is so much less about Science and so much more about the lived realities of coding, marketing, design, privacy, security and a lot of hard work.
Come back and get the next instalment, where I will be adding those new features! Meanwhile, if you are interested in the development process and want to watch a video where I demonstrate how to make a product from scratch, you might enjoy my YouTube series. Building web services, API calls, user interfaces are all discussed in detail.
Share This Article
Towards Data Science is a community publication. Submit your insights to reach our global audience and earn through the TDS Author Payment Program.
Write for TDS
