Data Science

👁 Image from Unsplash by Ben White

For frequent users of SageMaker Inference a common frustration is being able to debug endpoints quickly. Often times with SageMaker Endpoints you end up with a custom inference script that helps you control the pre and post processing of your model.

Initially when I first started with SageMaker I would re-deploy the endpoint every time I had to make a change to the inference script. This would be time consuming as I’d wait 2–3 minutes without understanding if inference would succeed or not. SageMaker Local Mode luckily helps simplify this process for us by pulling down the SageMaker containers to your local environment. With this article we’ll take a simple SKLearn model and deploy it using SageMaker Local Mode to show how useful it is for debugging your endpoints. Note that SageMaker Local Mode can also be utilized for preprocessing and training scripts as well, this article will only cover model serving/inference with the feature.

NOTE: For those of you new to AWS, make sure you make an account at the following link if you want to follow along. This article assumes relative comfort with AWS and SageMaker. Make sure to check out my additional resources section at the bottom if you want more guidance on getting started with SageMaker.

Setup

Before we can get started we have the usual SageMaker imports to get our environment ready.

The main import here is the SageMaker Python SDK, this is the package that supports Local Mode. This is a high level SDK that is a wrapper around the Boto3 Python SDK to simplify common processes and operations within SageMaker. To fully understand the difference between both SDKs please reference the following article.

Next we can create our Python Sklearn model, we’ll be training a Linear Regression model locally and generating a model.joblib file which contains the model artifacts/data for a sample regression problem.

The other script we provide with our generated model data is our inference script. This contains the inference handlers that will control the input and output of our model inference within the endpoint.

SageMaker expects these two pieces in our model.joblib and inference script to be tarred together in a model.tar.gz. We package these together and push it to a S3 Bucket for our Local Model to pick up on.

Now we have all the pieces we need to enable Local Mode.

SageMaker Local Mode Serving

First we create a SageMaker Local Session, this is essentially telling SageMaker we’re working in Local Mode.

👁 Verify Local Session (Screenshot by Author)

Next we configure our SKLearn SageMaker estimator with Local Mode enabled.

Here we pass in our inference script and then the path to our model artifacts with the packaged model data. This path can either be an S3 path or a file path in your local directory structure.

Let’s take a sample payload, in this case our model is expecting a 2D array so we create a numpy array with a sample data point from the test dataset.

Next we can perform a sample inference with our Local Mode Predictor. Note that the first time you run the following cell it will take a few minutes as the container image for SKLearn needs to be downloaded. Note that we are also specifying the instance type as local rather than an EC2 instance as we normally would.

What we see next is what we would normally have to wait to see while CloudWatch Logs are being generated during endpoint creation. We see our inference server starting up and running and we also see our sample prediction.

👁 Inference Server Starting (Screenshot by Author)

We also see our logging statements within our inference script and then also the output for our sample payload.

👁 Inference Logging (Screenshot by Author)

Let’s quickly break our inference script with a typo and see if Local Mode reflects the errors.

Here I change my predict_fn function to do "model.pred" instead of "model.predict" as expected for SKLearn models.

I rerun the cells that are creating my SKLearn estimator and performing inference with my updated script and I see the proper error reflected shortly afterwards.

👁 Error (Screenshot by Author)

Additional Resources & Conclusion

GitHub – RamVegiraju/SageMaker-Local-Mode: SM Local Mode Example Sklearn Deployment

You can find the code for the entire example in the link above. SageMaker Local Mode is a highly useful feature for quickly debugging your model training and inference as reflected in this article. For more local mode examples check out the AWS Samples repository. For further resources around getting started with SageMaker in general, check out my series here.

---

If you enjoyed this article feel free to connect with me on LinkedIn and subscribe to my Medium Newsletter. If you’re new to Medium, sign up using my Membership Referral.

Share This Article

• Share on Facebook
• Share on LinkedIn
• Share on X