How Amazon EKS Auto Mode Simplifies Kubernetes Cluster Management (Part 1)

Amazon Web Services launched Amazon EKS Auto Mode at re:Invent 2024, and a new feature aims to simplify Kubernetes cluster management by automating key tasks, allowing users to focus on deploying and managing applications instead of grappling with infrastructure complexities.

This article provides a comprehensive overview of Elastic Kubernetes Service (EKS) Auto Mode, delves into its features and limitations, and explores its connection with Karpenter, AWS’s own cluster autoscaler.

What Is EKS Auto Mode?

EKS Auto Mode eliminates the need for users to manually configure and manage worker nodes, operating systems and core add-ons. Instead, AWS takes on the responsibility of these tasks, ensuring that clusters are production-ready and optimized for performance and cost-efficiency.

In a standard EKS cluster workflow, administrators manually configure virtual private cloud (VPC) and subnets, create node groups, specify worker node specifications, set up scaling policies, and manage node updates and patches while handling infrastructure maintenance. This approach requires significant manual intervention and technical expertise.

In contrast, the EKS Auto Mode workflow simplifies the process by allowing users to select cluster configuration. At the same time, AWS automatically provisions infrastructure, manages node scaling and optimization, handles security patches and updates and reduces operational overhead.

The key distinction is the level of automation: Standard EKS demands hands-on management, whereas EKS Auto Mode abstracts infrastructure complexities, enabling developers to concentrate more on application deployment and less on underlying infrastructure management.

EKS Auto Mode can be enabled on both new and existing EKS clusters, providing flexibility for users who want to adopt this new management approach. However, migrating from an existing cluster might present challenges, such as conflicts with existing software components and the need to be on Karpenter v1.1 or higher to avoid issues with NodePool and NodeClaim APIs.

Why Use EKS Auto Mode?

EKS Auto Mode offers a range of features designed to simplify Kubernetes management and enhance the user experience:

• Automated Infrastructure Management: Auto Mode automatically provisions and manages the necessary EC2 instances, storage and networking components required for your Kubernetes cluster. It selects the optimal instance types based on your workload requirements and dynamically scales resources to meet demand. This automation also includes the management and patching of core add-ons like CoreDNS, kube-proxy, and VPC CNI, as well as EBS CSI controllers and AWS load balancer controllers.
• Integrated Add-Ons: EKS Auto Mode includes managed versions of essential Kubernetes add-ons, such as Karpenter for autoscaling, the AWS Load Balancer Controller for managing load balancers and the Amazon EBS CSI driver for persistent storage. These add-ons are pre-configured and automatically updated by AWS, reducing the operational burden on users.
• Simplified Cluster Operations: Auto Mode simplifies cluster operations by automating tasks such as OS patching, security updates, and node lifecycle management. It ensures that your cluster is always running the latest software and security patches without manual intervention. This includes automatic updates for Kubernetes components and the operating system, performed by gracefully replacing nodes while respecting Kubernetes scheduling constraints.
• Enhanced Security: EKS Auto Mode utilizes Amazon Machine Images (AMIs) that are treated as immutable, enforcing locked-down software, read-only root file systems and SELinux mandatory access controls. Nodes are automatically rotated every 21 days to ensure they are running the latest security updates. It’s important to note that direct access to worker nodes is restricted, and users cannot SSH into them.
• Cost Optimization: Auto Mode optimizes compute costs by dynamically scaling resources and terminating unused instances. It also consolidates workloads onto other nodes to improve cost efficiency. Furthermore, it supports cost-saving features like Spot Instances, Savings Plans and Reserved Instances.
• Improved Performance: By automatically selecting the right instance types and optimizing resource allocation, Auto Mode ensures that your applications have the resources they need to perform optimally.
• Support for GPUs and Spot Instances: EKS Auto Mode supports GPU and Spot Instances, providing flexibility for users with varying workload requirements. However, it’s crucial to understand that Auto Mode does not allow mixing Spot and On-Demand instances in the same node group. Users must create separate node groups for each instance type, which can add complexity to the configuration.
• Simplified IAM Role Management: EKS Auto Mode automatically maps AWS IAM roles to Kubernetes permissions through EKS access entries, eliminating the need for manual configuration of aws-auth ConfigMaps or custom bindings. This simplifies the IAM role and policy management for worker nodes.

The Karpenter Konnection

Karpenter is an open-source cluster autoscaler for Kubernetes that provisions right-sized compute resources in response to changing application demands. It is a key component of EKS Auto Mode, providing efficient and cost-effective autoscaling capabilities.

Karpenter has emerged as a transformative open-source Kubernetes autoscaler designed to revolutionize cluster scaling and resource management. Developed by AWS and subsequently donated to the Cloud Native Computing Foundation, Karpenter provides a modern, high-performance approach to dynamically provisioning and optimizing Kubernetes infrastructure.

Unlike traditional autoscaling methods, Karpenter offers a just-in-time node provisioning strategy that directly interacts with cloud provider APIs, enabling rapid, intelligent scaling that can leverage features like spot instances and optimize resource efficiency. Its architecture enables flexible instance provisioning, addressing the complexity of cloud-native applications and supporting the rise in Kubernetes adoption.

EKS Auto Mode integrates Karpenter directly into the cluster, eliminating the need for manual installation and configuration. This integration simplifies autoscaling and allows users to leverage Karpenter’s features, such as intelligent instance selection, bin packing, and Spot Instance management.

However, there are some key differences and trade-offs to consider when comparing EKS Auto Mode with self-managed Karpenter:

• Cost vs. Control: EKS Auto Mode incurs a 12% management fee on top of EC2 instance costs, while self-managed Karpenter does not. However, Auto Mode offers greater automation and reduces operational overhead, saving time and resources.
• Customization: EKS Auto Mode provides limited customization options for Karpenter configurations, while self-managed Karpenter allows for more fine-grained control.
• Pod Limits: EKS Auto Mode limits 110 pods per node, which can lead to an increased node count compared to self-managed Karpenter, potentially impacting costs.

Ultimately, the choice between EKS Auto Mode and self-managed Karpenter depends on your specific needs and priorities. If you value automation and reduced operational overhead, Auto Mode might be the better choice. However, if you require more control over Karpenter configurations and want to avoid the management fee, self-managed Karpenter might be more suitable.

What Happens to EKS with Fargate?

AWS Fargate is a serverless compute engine that allows you to run containers without managing servers or clusters. EKS supports Fargate as a compute option, providing a fully managed serverless experience for Kubernetes workloads.

For a detailed explanation of EKS with Fargate, refer to my previous article. 

With EKS and Fargate, you can run your Kubernetes pods on Fargate without provisioning or managing EC2 instances. This simplifies cluster management and reduces operational overhead as AWS automatically scales, patches and secures the underlying infrastructure. The goal of EKS with Fargate and EKS Auto is to reduce the friction involved in configuring, scaling and managing the cluster infrastructure.

While EKS with Fargate offers a serverless experience, it has some limitations compared to EKS Auto Mode:

• Customization: EKS with Fargate provides limited customization options for the underlying infrastructure compared to EKS Auto Mode.
• Cost: Fargate can be more expensive than EC2 for certain workloads, especially those that require persistent storage or have specific networking requirements.
• Compatibility: Not all Kubernetes features are supported on Fargate, such as DaemonSets and privileged containers.

With the launch of EKS Auto Mode, it’s unclear if AWS will continue investing in EKS with Fargate. While EKS Auto Mode currently relies on EC2 instances for compute, it’s possible that Fargate will play a more significant role in evolving ECS, Amazon’s proprietary container management platform.

AWS might integrate Fargate more tightly with EKS Auto Mode, allowing users to choose between EC2 and Fargate as compute options. This would provide greater flexibility and cost optimization for different workloads.

The future of EKS with Fargate is centered on enhanced serverless capabilities, including improved GPU support, advanced security features, and more granular workload management.

Conclusion

EKS Auto Mode represents a significant step towards simplifying Kubernetes management on AWS and a shift towards a more serverless Kubernetes experience. By automating key tasks, integrating essential add-ons like Karpenter, and providing a managed environment, it allows users to focus on their applications rather than infrastructure complexities. This approach aligns with the trend seen in other managed Kubernetes offerings like GKE Autopilot and AKS Automatic, where cloud providers are taking on more responsibility for managing the underlying infrastructure.

However, EKS Auto Mode has its own distinct approach and limitations. The 12% management fee, restricted customization options, and potential debugging challenges are factors to consider when evaluating this new offering. Despite these limitations, the benefits of reduced operational overhead, enhanced security and cost optimization make EKS Auto Mode a compelling option for many Kubernetes users.

As EKS Auto Mode matures and AWS incorporates user feedback, it is likely to become an even more powerful and versatile tool for managing Kubernetes clusters on AWS. It will be interesting to see how this offering influences the adoption of serverless Kubernetes and shapes the future of Kubernetes management in the cloud.

EKS Auto Mode is currently available in all commercial AWS regions except China. Users can manage EKS Auto Mode clusters through various tools, including eksctl, AWS CLI, the AWS Management Console and their preferred IaC setups.