How to use Lambda to automatically download files uploaded to S3 to EC2 (Windows)
I want to use Lambda to automatically transfer files uploaded to S3 to EC2 (Windows). To do this, I have configured the following: Set s3:ObjectCreated:* as a trigger to invoke Lambda (Python). Start windows server 2022 and set IAM role to AdministratorAccess. Set IAM role of Lambda to AdministratorAccess.
These were implemented. However, when I execute the following in Python 3.7 and store the file in S3, it starts without error, but the file is not transferred to EC2.
Please give me some advice.
import logging
import boto3
LOGGER = logging.getLogger()
LOGGER.setLevel(logging.INFO)
ec2 = boto3.client('ec2')
ssm = boto3.client('ssm')
INSTANCE_ID = 'i-***'
def lambda_handler(event, context):
try:
#Get EC2 State
ec2_resp = ec2.describe_instances(InstanceIds=[INSTANCE_ID]),
ec2_state = ec2_resp['Reservations'][0]['Instances'][0]['State']['Name']
if not ec2_state == "running":
LOGGER.info('No EC2 is running')
return "END"
# RunPowerShellScript
command = r"aws s3 sync s3://***/ C:/Users/Administrator/Desktop/***/"
ssm.send_command(
InstanceIds=[INSTANCE_ID],
DocumentName="AWS-RunPowerShellScript",
Parameters={
"commands":[command],
"executionTimeout":["3600"]
},
)
except Exception as error:
LOGGER.error(error)
raise error
- Language
- English
asked 3 years ago3.4K views
- Newest
- Most votes
- Most comments
Why are you using Lambda in this case? I would send the S3 notification to SQS and have a process polling the queue on the EC2 instance. When it receives a new message, it download the file and process it.
- rePost-User-04930123 years ago
Thank you for your response. I was thinking of using Lambda to perform the above action and then continue to operate Window Server with Run Command by lambda.
I implemented similar solution in the past using S3 --> EventBridge --> Lambda function which invoked a SSM command
Example:
bucket_name = event['bucket_name']
s3_object = event['object_key']
instance_id = event['instance_id']
destination_path = event['destination']
command_string = f"Copy-S3Object -BucketName {bucket_name} -Key {s3_object} -LocalFile {destination_path}\{s3_object}"
cmd_id = ssm_client.send_command(
InstanceIds=[instance_id],
DocumentName="AWS-RunPowerShellScript",
Parameters={"commands": [command_string]},
CloudWatchOutputConfig={"CloudWatchOutputEnabled": True}
)
You will also need to verify that the EC2 instance has the proper IAM permissions assigned to access the S3 bucket and objects as well as the proper permissions for SSM management.
answered 3 years ago
Try Mountpoint for Amazon S3 - https://aws.amazon.com/blogs/aws/mountpoint-for-amazon-s3-generally-available-and-ready-for-production-workloads/
answered 3 years ago
- Behrens, Isaac EXPERT3 years ago
Mountpoint only support Linux today. You could use WSL or a container to run on Windows.
Relevant content
- Accepted Answer
asked 2 years ago
asked 3 years ago
