 |
VOOZH | about |
|
VOOZH | about |
We’re so glad you’re here. You can expect all the best TNS content to arrive Monday through Friday to keep you on top of the news and at the top of your game.
Check your inbox for a confirmation email where you can adjust your preferences and even join additional groups.
Follow TNS on your favorite social media networks.
Become a TNS follower on LinkedIn.
Check out the latest featured and trending stories while you wait for your first TNS newsletter.
Amazon Web Services’ Lambda is one of the first serverless platforms in the industry. Since its launch in 2014, Amazon has added multiple features to make it the most mature Functions as a Service (FaaS). The platform supports various language runtimes, including Node.js, Python, Java, Ruby, C#, Go, and PowerShell. There is tight integration with mainstream AWS managed services that act as event sources to trigger Lambda functions.
Traditionally, serverless compute platforms and FaaS offerings such as AWS Lambda are associated with stateless functions. Since the functions are invoked and terminated based on events, there is no intrinsic persistence layer available. The state is always externalized by moving it to object storage, NoSQL database, in-memory database, or a relational database instance. It’s common to maintain state in Lambda functions by writing it to an object in an S3 bucket or a DynamoDB or RDS table.
But certain use cases such as machine learning inference demand a new approach. Downloading a large model from an Amazon S3 bucket increases the startup time, which results in latency. Some functions require external libraries that may be too large. Though AWS Lambda Layers’ concept addresses this problem, there is a limitation of 50MB (zipped, for direct upload), which defeats the purpose. Layers are static once they are deployed, which means the contents can be changed only by deploying a new layer.
In June 2020, AWS has added support for Amazon Elastic File System (EFS) for Lambda, enabling many exciting use cases.
This tutorial series covers all the aspects of using Amazon EFS with AWS Lambda to host the serverless machine learning API.
Amazon Elastic File System (EFS) provides a managed elastic NFS file system for AWS services and on-premises resources. It can scale to petabytes without disrupting applications, growing and shrinking automatically as the files or added and removed, eliminating the need to provision and manage capacity to accommodate growth.
Since EFS uses NFS v4, the industry standard for the shared file system, the file system can be easily attached to EC2 instances running Linux.
Amazon EFS exposes well-known access points that can be configured per application. EFS access points represent a flexible way to manage application access in NFS environments with increased scalability, security, and ease of use. One EFS file system can have multiple access points. Each access point can be configured with permissions associated with POSIX-compliant user id and group id. Combined with IAM, EFS file systems can have fine-grained security and access control.
It’s important to understand that Amazon EFS is available only within a VPC. Only those consumers within the same VPC can access the EFS file system. On-premises servers can mount EFS shares only after establishing connectivity through AWS Direct Connect or AWS VPN.
When an EFS file system is attached to an AWS Lambda function, it can access existing data and store data in it. This approach makes it possible to populate the filesystem with the dependencies and additional files that become available to all the Lambda instances.
The prerequisite for an AWS Lambda to access an EFS file system is that the function should be in the same VPC as the EFS. It should also have explicit permission to access the file system and create Elastic Network Interfaces (ENI) for the subnets of the VPC. Once these conditions are met, a Lambda function can read and write to the EFS file system.
The easiest way to populate the EFS file system accessed by a Lambda function is by mounting it to an EC2 instance. Using the standard NFS conventions, an EFS file system shows up in the /mnt directory.
When launching an EC2 instance from the AWS Console, it has an option to mount an existing EFS filesystem. The wizard automatically adds appropriate user script to permanently mount the file system by adding an entry in the /etc/fstab.
The powerful combination of EFS and Lambda functions can be used to host deep learning inference API in serverless mode. Since the size of a TensorFlow or PyTorch model may exceed the size limits of Lambda layers and /tmp directory, EFS comes in handy in storing the models.
The EFS storage backend for Lambda can also have all the dependencies such as OpenCV or PIL which are not only large but take time to install. A Lambda Python function can be pointed to an existing directory through the PYTHONPATH environment variable. The same file system will also have the fully trained model stored in a separate directory which is loaded by the function.
To populate the EFS file system with the Python modules and pre-trained model, we can use an Amazon EC2 instance or even a SageMaker Notebook. Both options give us the ability to mount the file system and add the dependencies either through a Python virtual environment or through the pip installer.
The below workflow highlights the steps involved in this approach:
This week The New Stack will feature a series of upcoming tutorials on this subject, where I will walk you through all the steps involved in hosting serverless machine learning inference API in AWS Lambda. Check in tomorrow for the next installment!
Janakiram MSV’s Webinar series, “Machine Intelligence and Modern Infrastructure (MI2)” offers informative and insightful sessions covering cutting-edge technologies. Sign up for the upcoming MI2 webinar at http://mi2.live.
