VOOZH about

URL: https://repost.aws/questions/QUGlVWzhWER6SJBPwaeN0QIg/removing-hudi-jars-from-emr-serverless-image

⇱ Removing Hudi jars from EMR Serverless Image | AWS re:Post

Skip to content

Removing Hudi jars from EMR Serverless Image

0

Hi Team,

We are using emr serverless image v7.0.0.0 for data ingestion activities. However, this image has hudi-related jars that have a few vulnerabilities. is it possible to remove these jars from the image, what would be the impact of eliminating them for normal spark-related jobs? right now we are not using hudi tables.

Topics
Analytics
Language
English

asked 2 years ago290 views

1 Answer
  • Newest
  • Most votes
  • Most comments
Are these answers helpful? Upvote the correct answer to help the community benefit from your knowledge.
0

Hello,

Thank you for writing on re:Post.

I see that you want to know about the vulnerabilities present in the Hudi jar that comes with EMR 7.0 Serverless image. I would like to inform you that EMR team is aware of the Vulnerabilities and working with Hudi to remove it from the upcoming fresh images.

However as you do not use Hudi, you can remove the Jars/using custom Images please follow document [+] to setup custom image and in the docker file remove dependency like

# Dockerfile
FROM public.ecr.aws/emr-serverless/spark/emr-7.0.0:latest

USER root

RUN rm -r /usr/lib/hudi/hudi-aws-bundle-0.14.0-amzn-1.jar \ 

# EMRS will run the image as hadoop
USER hadoop:hadoop

[+] https://docs.aws.amazon.com/emr/latest/EMR-Serverless-UserGuide/application-custom-image.html

You can test this on your setup. Feel free to reach back for further queries.

I hope I was able to address your query.

Thank you!

SUPPORT ENGINEER

answered 2 years ago

Relevant content