Deploying A Pre-Trained Sklearn Model on Amazon SageMaker
Take Your Locally Trained Models To Production
I’ve written in the past about how you can train and deploy custom Sklearn and TensorFlow models on Amazon SageMaker. For certain use cases however you may have pre-trained models that you have trained elsewhere. In this article we’ll explore how to take your pre-trained model data and deploy it on SageMaker. In addition, for this example we will mostly work with the code in a local environment to make sure you don’t have too much heavy-lifting in the AWS console.
NOTE: For those of you new to AWS, make sure you make an account at the following link if you want to follow along. This article will assume a novice to intermediate level of knowledge with AWS and SageMaker. We will also not explore model building theory, the main focus of this article will be on deployment.
Table of Contents
- Setup
- Model Deployment Script
- Model Invocation
- Additional Resources & Conclusion
1. Setup
To start let us quickly locally train a Sklearn model that we can use for deployment. You can simply run the following Python script for setup.
The main key for this script is that we are using the joblib module to save the trained model. You should see this artifact display in your local directory.
It’s necessary to use joblib to save the model as that is the format that SageMaker is expecting for Sklearn models. Along with our local model script, one script that we can provide ahead of time is our inference script. This helps SageMaker understand how your input and output for your model serving will be configured. There are four default functions you can provide:
- model_fn: This will deserialize and load our joblib file.
- input_fn: You can pass in the format of data (json, csv, etc) that your model can expect for input.
- predict_fn: Our model prediction function.
-
output_fn: Processes the returned value from predict_fn and the type of response your endpoint will get.
The next part is to create an appropriate SageMaker IAM role within the AWS console. Here you want to give proper permissions for SageMaker: S3 Full Access, SageMaker Full Access, and ECR Full Access. This should be your main work within the AWS console for this example. Once you’ve created this role we can focus on building our model deployment script.
2. Model Deployment Script
To get started, have the following imports for our model deployment script. Make sure this script is in the same directory as your inference handler and model data (joblib).
Most of our orchestration with working with AWS services is done through the SDK, in this case the Python SDK: Boto3. We work with the Boto3 client for SageMaker to orchestrate our steps for model deployment.
Here we instantiate the clients for SageMaker and the S3 client where we will store our model data for SageMaker to access. The next step is the key portion, SageMaker needs model artifacts/data in a model.tar.gz format. To do this we will zip the local model artifact and inference.py script into a tar file.
Next for SageMaker to understand we need to put this model artifact in a S3 location.
Now we can focus on the three steps to build an endpoint on SageMaker.
For SageMaker Model Creation we need two features: model data and our container image. In this case we can retrieve the Sklearn image for inference directly from SageMaker using the SageMaker SDK.
Now we can provide the model data and image to create our SageMaker model.
Now we can take the SageMaker model and use it to create our Endpoint Configuration, here we can specify our instance type and count we want for our endpoint.
Now we can take this and create our endpoint which will take a few minutes to create successfully.
If you now run the script you will see an endpoint successfully created and also visible in your AWS console.
3. Model Invocation
We can now make a separate invoke file to test our endpoint with a sample point. We don’t want to add this to our main file because with each execution a new endpoint will be created. Grab your endpoint name and specify it in the following script and you will see execution.
4. Additional Resources & Conclusion
GitHub – RamVegiraju/Pre-Trained-Sklearn-SageMaker: Deploy a pre-trained Sklearn Model on Amazon…
For the entire code for the example access the link above. It’s very simple to deploy pre-trained models on SageMaker after understanding the format and structure of the model data. The main change that needs to occur is the image you retrieve using the SDK, this should match the framework that you are using. In the case that the container is not supported by SageMaker check out how to Bring Your Own Container.
Additional Resources
Multi-Model TensorFlow Endpoints with SageMaker
If you enjoyed this article feel free to connect with me on LinkedIn and subscribe to my Medium Newsletter. If you’re new to Medium, sign up using my Membership Referral.
Share This Article
Towards Data Science is a community publication. Submit your insights to reach our global audience and earn through the TDS Author Payment Program.
Write for TDS
