Category Archives: Using flask to serve a machine learning model as a restful web service

Using flask to serve a machine learning model as a restful web service

By | 24.12.2020

GitHub is home to over 40 million developers working together to host and review code, manage projects, and build software together. If nothing happens, download GitHub Desktop and try again. If nothing happens, download Xcode and try again. If nothing happens, download the GitHub extension for Visual Studio and try again. A simple Flask application that can serve predictions machine learning model.

Make a http POST call with some data, and receive the prediction back using postman or python requests library. Returns an array of predictions given a JSON object representing independent variables. Here's a sample input:. Trains the model. This is currently hard-coded to be a random forest model that is run on a subset of columns of the titanic dataset. Note: Docker tag or id should be always specified in the end of the docker command to avoid issues. From This is for development purpose and will not work in a production environment outside of Docker Desktop for Mac.

Skip to content. Dismiss Join GitHub today GitHub is home to over 40 million developers working together to host and review code, manage projects, and build software together.

Productionalize Your Machine Learning Model Using Flask And Google App Engine

Sign up. Python Dockerfile Shell. Python Branch: master. Find file. Sign in Sign up. Go back. Launching Xcode If nothing happens, download Xcode and try again. Latest commit Fetching latest commit…. Steps for deploying ML model Install Flask and Docker Serialise your scikit-learn model this can be done using Pickle, or JobLib [optional] add column names list to scikit object ex: rf. The gateway is also reachable as gateway. You signed in with another tab or window. Reload to refresh your session.

You signed out in another tab or window. Aug 20, Aug 21, Feb 25, Apr 18, Update requirements. Mar 3, This small tutorial will help you understand how a trained machine learning model is used in production. But none of them explain what happens to your machine learning model after you train and optimize one at your local system in jupyter notebook or other IDE.

In a production environment, no one sits in front of the system giving input and checking the output of the model you have created. Setup and Requirements:.

using flask to serve a machine learning model as a restful web service

Also, please install below specified libraries:. Training a sample Machine Learning Model:. I am going to train a sample linear regression model on the Boston housing dataset both are available in the Scikit-learn library, at the end I am saving the trained model in a file using Joblib.

Below is the sample python code:. Creating a simple flask application:. Our app will do the below tasks:. Below is the code for the flask app.

You can also get it from GitHub, using this link. Testing our application local or dev environment :. After writing our application we will test it out in a local or development environment before moving it to production in Google Cloud Platform.

Just open the project directory in terminal and run below command:. The App that we have tested was deployed in the local system. To make our system accessible globally we need to deploy it in some server having a global URL for assessing our app. You can find the details for setting up google SDK here.

Below is the content of the deployment descriptor, app. Now create a directory structure as given below:. Finally, open the same directory folder in the terminal, and use the below command to deploy the app in Google cloud. After getting confirmation Google cloud SDK will copy all the required files in APP engine instance and if there are no errors, then we will get the global endpoint of our application. Testing our application Production Environment :.

In this article, we saw how a machine learning model is used in production. Although this is a very basic use case. But this will give you some idea of how the machine learning models are put into production in the cloud server inside different applications.The architecture exposed here can be seen as a way to go from proof of concept PoC to minimal viable product MVP for machine learning applications.

Python is not the first choice one can think of when designing a real-time solution.

using flask to serve a machine learning model as a restful web service

But as Tensorflow and Scikit-Learn are some of the most used machine learning libraries supported by Python, it is used conveniently in many Jupyter Notebook PoCs. What makes this solution doable is the fact that training takes a lot of time compared to predicting.

If you think of training as the process of watching a movie and predicting the answers to questions about it, then it seems quite efficient to not have to re-watch the movie after each new question. It should be really fast, whether the movie was complex or long.

First, a training pipeline is created to learn about the past data according to an objective function. Note that feature engineering done during training time should be carefully saved in order to be applicable to prediction. One usual problem among many others that can emerge along the way is feature scaling which is necessary for many algorithms.

Some careful adjustments should be thought of in advance so that the mapping function returns consistent outputs that will be correctly computed at prediction time. There is a major question to be addressed here. Why are we separating training and prediction to begin with? It is absolutely true that in the context of machine learning examples and courses, where all the data is known in advance including the data to be predicteda very simple way to build the predictor is to stack training and prediction data usually called a test set.

However, in real life systems, you usually have training data, and the data to be predicted comes in just as it is being processed.

Creating a Machine Learning Web API with Flask

In other words, you watch the movie at one time and you have some questions about it later on, which means answers should be easy and fast. Moreover, it is usually not necessary to re-train the entire model each time new data comes in since training takes time could be weeks for some image sets and should be stable enough over time. That is why training and predicting can be, or even should be, clearly separated on many systems, and this is also better reflecting how an intelligent system artificial or not learns.

Overfitting is particularly seen in datasets with many features, or with datasets with limited training data. In both cases, the data has too much information compared to what can be validated by the predictor, and some of them might not even be linked to the predicted variable. In this case, noise itself could be interpreted as a signal. A good way of controlling overfitting is to train on part of the data and predict on another part on which we have the ground truth.

Therefore the expected error on new data is roughly the measured error on that dataset, provided the data we train on is representative of the reality of the system and its future states.GitHub is home to over 40 million developers working together to host and review code, manage projects, and build software together. If nothing happens, download GitHub Desktop and try again. If nothing happens, download Xcode and try again. If nothing happens, download the GitHub extension for Visual Studio and try again.

Open the terminal. All the instructions are in there. Two attached files. One is a step-by-step notebookand the other is the Flask app that is going to serve the machine learning model to get predictions back. Skip to content. Dismiss Join GitHub today GitHub is home to over 40 million developers working together to host and review code, manage projects, and build software together.

Sign up. Build a flask app to server a machine learning model as a RESTful web service. Jupyter Notebook Python. Jupyter Notebook Branch: master. Find file. Sign in Sign up. Go back. Launching Xcode If nothing happens, download Xcode and try again. Latest commit Fetching latest commit….

Dependencies All included in the iPython notebook. Usage Open the terminal. You signed in with another tab or window. Reload to refresh your session.

You signed out in another tab or window. Serve a machine learning model using Flask.As a Python developer and data scientist, I have a desire to build web apps to showcase my work.

As much as I like to design the front-end, it becomes very overwhelming to take both machine learning and app development. So, I had to find a solution that could easily integrate my machine learning models with other developers who could build a robust web app better than I can. There is a clear division of labor here which is nice for defining responsibilities and prevents me from directly blocking teammates who are not involved with the machine learning aspect of the project.

Another advantage is that my model can be used by multiple developers working on different platforms, such as web or mobile. This article is intended especially for data scientists who do not have an extensive computer science background.

REST API & RESTful Web Services Explained - Web Services Tutorial

For this example, I put together a simple Naives Bayes classifier to predict the sentiment of phrases found in movie reviews. The reviews are divided into separate sentences and sentences are further divided into separate phrases.

All phrases have a sentiment score so that a model can be trained on which words lend a positive, neutral, or negative sentiment to a sentence. The majority of phrases had a neutral rating. At first, I tried to use a multinomial Naive Bayes classifier to predict one out of the 5 possible classes.

However, because the majority of the data had a rating of 2, the model did not perform very well. So, I limited the data to the extreme classes and trained the model to predict only negative or positive sentiment. It turned out that the multinomial Naive Bayes model was very effective at predicting positive and negative sentiment.

You can find a quick overview of the model training process in this Jupyter Notebook Walkthrough. After training the model in a Jupyter notebook, I transferred my code into Python scripts and created a class object for the NLP model. You can find the code in my Github repo at this link. You will also need to pickle or save your model so that you can quickly load the trained model into your API script.

The code block below contains a lot of Flask boilerplate and the code to load the classifier and vectorizer pickles. The parser will look through the parameters that a user sends to your API. For this example, we will be specifically looking for a key called query.

The query will be a phrase that a user will want our model to make a prediction on whether the phrase is positive or negative. GET will be the primary method because our objective is to serve predictions.This post aims to make you get started with putting your trained machine learning models into production using Flask API.

using flask to serve a machine learning model as a restful web service

Often times when working on a machine learning project, we focus a lot on Exploratory Data Analysis EDAFeature Engineering, tweaking with hyper-parameters etc. But we tend to forget our main goal, which is to extract real value from the model predictions. Deployment of machine learning models or putting models into production means making your models available to the end users or systems.

However, there is complexity in the deployment of machine learning models. I will be using linear regression to predict the sales value in the third month using rate of interest and sales of the first two months. The objective of a linear regression model is to find a relationship between one or more features independent variables and a continuous target variable dependent variable.

There are three fields which need to be filled by the user — rate of interest, sales in first month and sales in second month. I created a custom sales dataset for this project which has four columns — rate of interest, sales in first month, sales in second month and sales in third month.

Missing Data can occur when no information is provided for one or more items. I filled the rate column with zero and sales in first month with mean of that column if the value was not provided. I used linear regression as the machine learning algorithm. In simple words serializing is a way to write a python object on the disk that can be transferred anywhere and later de-serialized read back by a python script.

I converted the model which is in the form of a python object into a character stream using pickling. The idea is that this character stream contains all the information necessary to reconstruct the object in another python script.

The next part was to make an API which receives sales details through GUI and computes the predicted sales value based on our model. For this I de- serialized the pickled model in the form of python object. It displays the returned sales value in the third month. This article demonstrated a very simple way to deploy machine learning models. I used linear regression to predict sales value in the third month using rate of interest and sales in first two months.

One can use the knowledge gained in this blog to make some cool models and take them into production so that others can appreciate their work. Writing a simple Flask Web Application in 80 lines Sample tutorial for getting started with flask. In this course we will learn about…. Simple way to deploy machine learning models to cloud Deploy your first ML model to production with a simple tech stack.

Overview of Different Approaches to Deploying Machine Learning Models in Production - KDnuggets There are different approaches to putting models into productions, with benefits that can vary dependent on the….

These are some of my contacts details:. He is interested in data science, machine learning and their applications to real-world problems. Reposted with permission. By subscribing you accept KDnuggets Privacy Policy. Subscribe to KDnuggets News.By using our site, you acknowledge that you have read and understand our Cookie PolicyPrivacy Policyand our Terms of Service. The dark mode beta is finally here. Change your preferences any time. Stack Overflow for Teams is a private, secure spot for you and your coworkers to find and share information.

I am developing a web application which uses celery for task distribution and management. Web application also uses machine learning and deep learning algorithms to do predictions.

These predictive models are deployed on separate server as separate application and their predictive functions are integrated with celery as individual task. For example, X user wants to know forecast of stock price and submits query to web application. Web application will initiate celery tasks with X's query payload.

This increased our performance to 10 folds as compare to when we deployed RESTful endpoints for machine learning predictive models using Flask.

using flask to serve a machine learning model as a restful web service

For deep learning, we need to move to Tensorflow and integrate it with celery. After thorough research, it was concluded to use Tensorflow Serving and call predictive functions inside celery tasks on machine learning server.

Productionalize Your Machine Learning Model Using Flask And Google App Engine

Other approach, was to deploy TensorFlow models as separate end points using Sanic and rather than web application's celery submitting tasks directly to other server celery, now it will directly execute and call RESTful API Endpoint which will be asynchronous as well. What do you suggest, what would work best for us in this scenario? It really depends on the model's performance characteristics and your application's latency requirements.

In general, if you need low-latency, go for the REST API approach, if you need flexibility and something easy to implement for quick proof-of-concept, then go for the celery task approach. Additionally, I'd suggest trying out BentoMLit is a framework for high-performance and flexible model serving. In your case, it will make it a lot easier for the engineering team to experiment with those different deployments approaches, and decide which one works best for your specific use case.

Learn more. Ask Question. Asked 5 months ago. Active 11 days ago. Viewed times. Osama Rasheed. Osama Rasheed Osama Rasheed 87 11 11 bronze badges. Active Oldest Votes. Disclaimer: I'm the author of the BentoML project. Chaoyu Chaoyu 7 7 silver badges 16 16 bronze badges.

Sign up or log in Sign up using Google. Sign up using Facebook. Sign up using Email and Password. Post as a guest Name. Email Required, but never shown. The Overflow Blog. Featured on Meta. Feedback on Q2 Community Roadmap. Technical site integration observational experiment live on Stack Overflow.


thoughts on “Using flask to serve a machine learning model as a restful web service

Leave a Reply

Your email address will not be published. Required fields are marked *