How To Run DeepSeek R1 on AWS Using Infrastructure as Code

This weekend, I changed my perspective on open source AI deployment. While scrolling through my social feeds, I noticed many posts about DeepSeek, a new open-source language model, causing a stir in the AI community. As someone who regularly deploys infrastructure for production environments, I was intrigued by DeepSeek’s promise of competitive performance at a fraction of the cost of major commercial models.

What caught my attention wasn’t just the benchmark numbers. However, DeepSeek’s 79.8% score on AIME 2024 mathematics tests is impressive, but rather the practical possibility of running these models on standard cloud infrastructure. I decided to put this to the test by deploying DeepSeek on AWS using Pulumi for infrastructure as code. Here’s what I learned from the experience.

Understanding DeepSeek’s Place in the AI Landscape

DeepSeek emerged from a Chinese AI startup founded in 2023. It brings something unique: High-performance language models released under the MIT license. While companies like OpenAI and Meta spend enormous resources on their models, DeepSeek achieves comparable results with significantly less investment.

In my testing, DeepSeek R1 demonstrated capabilities that make it particularly valuable for practical applications:

• Mathematics processing with 79.8% accuracy on AIME 2024 tests.
• Software engineering tasks with 49.2% accuracy on SWE-bench Verified.
• General knowledge handling with a 90.8% score on MMLU.

What makes this especially interesting for development teams is the availability of distilled versions with 1.5B to 70B parameters, allowing deployment on various hardware configurations, from local machines to cloud instances.

Deploying DeepSeek: A Practical Infrastructure Approach

After evaluating DeepSeek’s capabilities, I created a reproducible deployment process using Pulumi and AWS. The goal was to establish a GPU-powered environment that could efficiently handle the model while remaining cost-effective.

The deployment architecture I developed consists of three main components:

1. A GPU-enabled EC2 instance (g4dn.xlarge) for model hosting.
2. Ollama for model management and API compatibility.
3. Open WebUI for interaction and testing.

Here’s the real-world deployment process I developed, focusing on maintainability and scalability:

👁 Image

Prerequisites

Before embarking on our self-hosted DeepSeek model journey, ensure you have:

• An AWS account;
• Pulumi CLI installed;
• AWS CLI installed;
• A basic understanding of Ollama, a tool that simplifies running large language models (LLMs) on your hardware.

Creating the Infrastructure

To get started, I created a new Pulumi project:

pulumi new aws-typescript

I chose TypeScript for this example, but you can select any language you prefer.

After setting up the project, I deleted the sample code and replaced it with the following configurations.

Create an Instance Role With S3 Access

To download the NVIDIA drivers, I needed to create an instance role with S3 access (AmazonS3ReadOnlyAccess is enough here)

import * as pulumi from "@pulumi/pulumi";
import * as aws from "@pulumi/aws";
import * as fs from "fs";

const role = new aws.iam.Role("deepSeekRole", {
   name: "deepseek-role",
   assumeRolePolicy: JSON.stringify({
       Version: "2012-10-17",
       Statement: [
           {
               Action: "sts:AssumeRole",
               Effect: "Allow",
               Principal: {
                   Service: "ec2.amazonaws.com",
               },
           },
       ],
   }),
});


new aws.iam.RolePolicyAttachment("deepSeekS3Policy", {
   policyArn: "arn:aws:iam::aws:policy/AmazonS3ReadOnlyAccess",
   role: role.name,
});


const instanceProfile = new aws.iam.InstanceProfile("deepSeekProfile", {
   name: "deepseek-profile",
   role: role.name,

});

Create the Network

Next, I needed to create a VPC, subnet, Internet Gateway, and route table. This is all done with the following code snippet:

const vpc = new aws.ec2.Vpc("deepSeekVpc", {
   cidrBlock: "10.0.0.0/16",
   enableDnsHostnames: true,
   enableDnsSupport: true,
});
const subnet = new aws.ec2.Subnet("deepSeekSubnet", {
   vpcId: vpc.id,
   cidrBlock: "10.0.48.0/20",
   availabilityZone: pulumi.interpolate`${aws.getAvailabilityZones().then(it => it.names[0])}`, 
   mapPublicIpOnLaunch: true,
});

const internetGateway = new aws.ec2.InternetGateway("deepSeekInternetGateway", {
   vpcId: vpc.id,

});
const routeTable = new aws.ec2.RouteTable("deepSeekRouteTable", {
   vpcId: vpc.id,
   routes: [
       {
           cidrBlock: "0.0.0.0/0",
           gatewayId: internetGateway.id,
       },
   ],
});


const routeTableAssociation = new aws.ec2.RouteTableAssociation("deepSeekRouteTableAssociation", {
   subnetId: subnet.id,
   routeTableId: routeTable.id,
});


const securityGroup = new aws.ec2.SecurityGroup("deepSeekSecurityGroup", {
   vpcId: vpc.id,
   egress: [
       {
           fromPort: 0,
           toPort: 0,
           protocol: "-1",
           cidrBlocks: ["0.0.0.0/0"],
       },
   ],
   ingress: [
       {
           fromPort: 22,
           toPort: 22,
           protocol: "tcp",
           cidrBlocks: ["0.0.0.0/0"],
       },
       {
           fromPort: 3000,
           toPort: 3000,
           protocol: "tcp",
           cidrBlocks: ["0.0.0.0/0"],
       },
       {
           fromPort: 11434,
           toPort: 11434,
           protocol: "tcp",
           cidrBlocks: ["0.0.0.0/0"],
       },
   ],
});

Create the EC2 Instance

Finally, I can create the EC2 instance. For this, I need to create a SSH key pair and retrieve the Amazon Machine Images to use in our instances.

I also use a g4dn.xlarge, but you can change the instance type to any other instance type that supports GPU. You can find more information about the instance types.

If you need to create the key pair, run the following command:

openssl genrsa -out deepseek.pem 2048
openssl rsa -in deepseek.pem -pubout > deepseek.pub
ssh-keygen -f mykey.pub -i -mPKCS8 > deepseek.pem

const keyPair = new aws.ec2.KeyPair("deepSeekKey", {
   publicKey: pulumi.output(fs.readFileSync("deepseek.rsa", "utf-8")),
});

const deepSeekAmi = aws.ec2
   .getAmi({
       filters: [
           {
               name: "name",
               values: ["amzn2-ami-hvm-2.0.*-x86_64-gp2"],
           },
           {
               name: "architecture",
               values: ["x86_64"],
           },
       ],
       owners: ["137112412989"], // Amazon
       mostRecent: true,
   })
   .then(ami => ami.id);

const deepSeekInstance = new aws.ec2.Instance("deepSeekInstance", {
   ami: deepSeekAmi,
   instanceType: "g4dn.xlarge",
   keyName: keyPair.keyName,
   rootBlockDevice: {
       volumeSize: 100,
       volumeType: "gp3",
   },
   subnetId: subnet.id,
   vpcSecurityGroupIds: [securityGroup.id],
   iamInstanceProfile: instanceProfile.name,
   userData: fs.readFileSync("cloud-init.yaml", "utf-8"),
   tags: {
       Name: "deepSeek-server",
   },
});

export const amiId = deepSeekAmi;
export const instanceId = deepSeekInstance.id;
export const instancePublicIp = deepSeekInstance.publicIp;

Then, we configure the GPU instance with proper drivers and dependencies, install Ollama and run DeepSeek with this cloud config.

#cloud-config
users:
- default


package_update: true


packages:
- apt-transport-https
- ca-certificates
- curl
- openjdk-17-jre-headless
- gcc

runcmd:
- yum install -y gcc kernel-devel-$(uname -r)
- aws s3 cp --recursive s3://ec2-linux-nvidia-drivers/latest/ .
- chmod +x NVIDIA-Linux-x86_64*.run
- /bin/sh ./NVIDIA-Linux-x86_64*.run --tmpdir . --silent
- touch /etc/modprobe.d/nvidia.conf
- echo "options nvidia NVreg_EnableGpuFirmware=0" | sudo tee --append /etc/modprobe.d/nvidia.conf
- yum install -y docker
- usermod -a -G docker ec2-user
- systemctl enable docker.service
- systemctl start docker.service
- curl -s -L https://nvidia.github.io/libnvidia-container/stable/rpm/nvidia-container-toolkit.repo | sudo tee /etc/yum.repos.d/nvidia-container-toolkit.repo
- yum install -y nvidia-container-toolkit
- nvidia-ctk runtime configure --runtime=docker
- systemctl restart docker
- docker run -d --gpus=all -v ollama:/root/.ollama -p 11434:11434 --name ollama --restart always ollama/ollama
- sleep 120
- docker exec ollama ollama run deepseek-r1:7b
- docker exec ollama ollama run deepseek-r1:14b
- docker run -d -p 3000:8080 --add-host=host.docker.internal:host-gateway -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:main

Deploying the Infrastructure

With all configurations in place, we can deploy the infrastructure using:

pulumi up

This command provides a preview of the changes, allowing you to confirm before proceeding. Once confirmed, Pulumi creates the resources, and after some time, the EC2 instance is ready with DeepSeek R1 running.

Access the Ollama Web UI

To retrieve the public IP address of our EC2 instance, I used the following command to instruct Pulumi to print out the configuration:

pulumi stack output instancePublicIp
contains the code and configuration files
<yout-ip>

I then opened web UI with this address: http://<ip>:3000/

Use DeepSeek R1

Head to the dropdown in the upper right corner and select the model you want to use.

👁 Image

I selected deepseek-r1:14b to test my model.

👁 Image

Finally, I used the central chat box to begin using the model. My example prompt is: What are Pulumi’s benefits?

👁 Image

Cleaning up

After you are done experimenting with DeepSeek, I clean up the resources by running the following command:

pulumi destroy

Looking Forward

DeepSeek represents a significant step forward in accessible AI deployment. The combination of MIT licensing and competitive performance could make it a viable option for production environments.

For teams considering DeepSeek deployment, I recommend:

• Starting with the 7B model for a balanced performance/resource ratio.
• Using infrastructure as code (like Pulumi) for reproducible deployments.
• Implementing proper monitoring and scaling policies.
• Testing thoroughly with production-like workloads before deployment.

My GitHub repository contains the code and configuration files from this deployment, allowing others to build upon this foundation for their AI infrastructure needs.

This experience has shown me that enterprise-grade AI deployment is increasingly accessible to smaller teams. As we continue to see advances in model efficiency and deployment tools, the barrier to entry for production AI will continue to lower, opening new possibilities for innovation across the industry.

If you’re interested in exploring AI models or need a robust setup for your projects, consider trying DeepSeek with Pulumi. Remember, while the setup is straightforward, securing your instance and understanding the model’s capabilities are crucial steps before going live.