VOOZH about

URL: https://www.coursera.org/learn/packt-kafka-for-developers-data-contracts-using-schema-registry-ru4hb

⇱ Kafka for Developers - Data Contracts Using Schema Registry | Coursera


Kafka for Developers - Data Contracts Using Schema Registry

Keep adding new skills with 10,000+ programs for $239 (usually $399). Save now.

Kafka for Developers - Data Contracts Using Schema Registry

Included with

Ask Coursera

Gain insight into a topic and learn the fundamentals.
Intermediate level

Recommended experience

2 weeks to complete
at 10 hours a week
Flexible schedule
Learn at your own pace

Gain insight into a topic and learn the fundamentals.
Intermediate level

Recommended experience

2 weeks to complete
at 10 hours a week
Flexible schedule
Learn at your own pace

What you'll learn

  • Create and manage data contracts in Kafka using Schema Registry.

  • Implement AVRO schemas for scalable and efficient data serialization.

  • Build real-time Kafka applications using Gradle, Maven, and Spring Boot.

  • Ensure compatibility with schema evolution strategies.

Details to know

Shareable certificate

Add to your LinkedIn profile

Assessments

16 assignments

Taught in English

There are 15 modules in this course

Updated in May 2025.

This course now features Coursera Coach! A smarter way to learn with interactive, real-time conversations that help you test your knowledge, challenge assumptions, and deepen your understanding as you progress through the course. Unlock the power of data contracts in Kafka with this comprehensive course focusing on Schema Registry and AVRO serialization. You'll explore how to create robust data pipelines, ensuring compatibility and scalability across producer-consumer applications. By the end, you'll master tools and techniques that empower efficient data processing with seamless schema evolution. Start with the fundamentals of data serialization in Kafka, diving deep into popular formats like AVRO, Protobuf, and Thrift. Gradually, you'll build hands-on expertise by setting up Kafka in a local environment using Docker, creating custom AVRO schemas, and generating Java records for real-world applications. The course includes practical exercises, such as building an end-to-end Coffee Shop order service and exploring schema evolution strategies in Schema Registry. You'll also learn naming conventions, logical schema types, and compatibility strategies that ensure smooth upgrades in production environments. Designed for software developers and data engineers, this course assumes basic knowledge of Java and Kafka. Whether you're a beginner or looking to deepen your expertise in Kafka and Schema Registry, this course is your gateway to mastering data contracts.

In this module, we will set the foundation for the course by providing an overview of its objectives, structure, and prerequisites. You’ll gain a clear understanding of what to expect and how to prepare for success in this learning journey.

What's included

2 videos1 reading

2 videosTotal 4 minutes
  • Introduction3 minutes
  • Prerequisites1 minute
1 readingTotal 10 minutes
  • Full Course Resources10 minutes

In this module, we will dive into the intricacies of data contracts and serialization in Kafka. You’ll explore how serialization enhances Kafka's architecture and examine different serialization formats, such as AVRO, Protobuf, and Thrift, to understand their schema compatibility and applications.

What's included

2 videos1 assignment

2 videosTotal 9 minutes
  • Data Contract and Serialization in Kafka5 minutes
  • Serialization Formats4 minutes
1 assignmentTotal 15 minutes
  • Data Contract and Serialization in Kafka - Assessment15 minutes

In this module, we will introduce you to AVRO, one of the most popular serialization systems. You’ll explore the reasons behind its popularity and learn how to build a simple AVRO schema to get hands-on experience with its functionality.

What's included

2 videos1 assignment

2 videosTotal 9 minutes
  • Introduction to AVRO - What Is AVRO and Why AVRO?4 minutes
  • Build a Simple AVRO Schema4 minutes
1 assignmentTotal 15 minutes
  • Introduction to AVRO - A Data Serialization System - Assessment15 minutes

In this module, we will guide you through setting up a local Kafka environment using Docker Compose. You’ll practice producing and consuming messages with CLI tools and delve into AVRO serialization by leveraging the AVRO console producer and consumer for hands-on experience.

What's included

3 videos1 assignment

3 videosTotal 18 minutes
  • Set Up Kafka Broker and Zookeeper Using Docker Compose6 minutes
  • Producer and Consumer Messages Using CLI4 minutes
  • Produce and Consume Using AVRO Console Producer and Consumer8 minutes
1 assignmentTotal 15 minutes
  • Kafka Setup and Demo in Local Using Docker - Assessment15 minutes

In this module, we will set up the foundational components for a Greeting App using Gradle. You’ll learn how to configure the project for AVRO support and generate Java records from schema files, preparing the groundwork for seamless AVRO integration.

What's included

2 videos1 assignment

2 videosTotal 10 minutes
  • Base Project Setup for Greeting App3 minutes
  • Generate AVRO Java Records Using AVRO Schema Files7 minutes
1 assignmentTotal 15 minutes
  • Greeting App - Base AVRO Project Setup - Gradle - Assessment15 minutes

In this module, we will set up the Greeting App project using Maven as the build tool. You’ll learn how to configure Maven for AVRO support and generate Java records from schema files, ensuring the project is ready for AVRO-based serialization.

What's included

2 videos1 assignment

2 videosTotal 6 minutes
  • Base Project Setup for Greeting App - Maven3 minutes
  • Generate AVRO Java Records Using AVRO Schema Files - Maven3 minutes
1 assignmentTotal 15 minutes
  • Greeting App - Base AVRO Project Setup - Maven - Assessment15 minutes

In this module, we will guide you through building an AVRO-based Kafka producer and consumer using Java. You’ll learn how to implement serialization and deserialization for seamless data exchange within Kafka topics.

What's included

2 videos1 assignment

2 videosTotal 21 minutes
  • Let's Build AVRO Kafka Producer12 minutes
  • Let's Build AVRO Kafka Consumer9 minutes
1 assignmentTotal 15 minutes
  • Build AVRO Producer and Consumer in Java - Assessment15 minutes

In this module, we will build a real-time Coffee Shop Order Service using AVRO and Kafka. You’ll start with an application overview and progress through project setup, schema creation, and AVRO class generation. Finally, you’ll create producers and consumers to simulate and process coffee shop orders in a real-time streaming scenario.

What's included

8 videos1 assignment

8 videosTotal 47 minutes
  • Application Overview3 minutes
  • Project Setup for Coffee Shop - Gradle4 minutes
  • Project Setup for Coffee Shop - Maven4 minutes
  • Build a Coffee Order Schema Using AVRO14 minutes
  • Generating AVRO Classes Using Gradle2 minutes
  • Generating AVRO Classes Using Maven3 minutes
  • Build a Coffee Shop Order Producer11 minutes
  • Build a Coffee Shop Order Consumer5 minutes
1 assignmentTotal 15 minutes
  • Coffee Shop Order Service Using AVRO - A Real-Time Use Case - Assessment15 minutes

In this module, we will explore logical schema types in AVRO and their practical applications. You’ll learn to enhance the CoffeeOrder schema by adding logical types like timestamps, decimals, UUIDs, and dates to improve data accuracy and functionality in real-world scenarios.

What's included

4 videos1 assignment

4 videosTotal 23 minutes
  • Introduction to Logical Types in AVRO4 minutes
  • Add a Timestamp, Decimal Logical Type to the CoffeeOrder Schema13 minutes
  • Adding the UUID as Key for CoffeeOrder4 minutes
  • Date Logical Type2 minutes
1 assignmentTotal 15 minutes
  • Logical Schema Types in AVRO - Assessment15 minutes

In this module, we will delve into the inner workings of an AVRO record. You’ll uncover how data is stored, organized, and encoded, gaining insights into its efficiency and compatibility with evolving schemas.

What's included

1 video1 assignment

1 videoTotal 4 minutes
  • What's Inside an AVRO Record?4 minutes
1 assignmentTotal 15 minutes
  • AVRO Record- Under the Hood - Assessment15 minutes

In this module, we will explore the effects of schema changes in AVRO, focusing on the consumer application's failure to process data with an updated schema. You’ll gain practical insights into schema evolution challenges and why a schema registry is crucial for maintaining compatibility.

What's included

1 video1 assignment

1 videoTotal 4 minutes
  • Evolving the Schema - Consumer Fails to Read the New Schema4 minutes
1 assignmentTotal 15 minutes
  • Schema Changes in AVRO - Issues without Schema Registry - Assessment15 minutes

In this module, we will introduce you to the schema registry and its critical role in handling AVRO schemas effectively. You’ll learn how to publish and consume records using the schema registry, interact with its REST API, and work with "key" fields as AVRO records to enhance data management and compatibility.

What's included

4 videos1 assignment

4 videosTotal 28 minutes
  • Introduction to Schema Registry5 minutes
  • Publish and Consumer Record Using Schema Registry9 minutes
  • Schema Registry Internals and Interacting with Schema Registry Using REST Endpoint8 minutes
  • Publish and Consume "Key" as an AVRO Record7 minutes
1 assignmentTotal 15 minutes
  • Introduction to Schema Registry - Assessment15 minutes

In this module, we will focus on data evolution through the lens of schema changes managed by a schema registry. You’ll learn to update project configurations, explore compatibility types like backward, forward, and full, and understand the consequences of modifying AVRO schemas, enabling you to manage data evolution effectively.

What's included

7 videos1 assignment

7 videosTotal 55 minutes
  • Data Evolution and Schema Evolution5 minutes
  • Update the Code to Interact with Maven Local Repository - Gradle9 minutes
  • Update the Code to Interact with Maven Local Repository - Maven5 minutes
  • Deleting a Field in Schema - BACKWARD Compatibility11 minutes
  • Adding a New Field in Schema - FORWARD Compatibility12 minutes
  • Add/Delete Optional Fields - FULL Compatibility7 minutes
  • Modify Field Names - NONE Compatibility6 minutes
1 assignmentTotal 15 minutes
  • Data Evolution Using Schema Registry - Assessment15 minutes

In this module, we will cover different schema naming strategies in AVRO and their practical applications. You’ll create a schema for Coffee Update events and learn how to utilize RecordNameStrategy to manage schema versions while ensuring seamless data processing.

What's included

3 videos1 assignment

3 videosTotal 23 minutes
  • Different Types of Naming Strategies6 minutes
  • Coffee Update Event AVRO Schema4 minutes
  • Publish and Consume CoffeeOrder UpdateEvent Using RecordNameStrategy12 minutes
1 assignmentTotal 15 minutes
  • Schema Naming Strategies - Assessment15 minutes

In this module, we will create a coffee order service application using Spring Boot, Kafka, and Schema Registry. We'll cover every step, from setting up the project using Gradle or Maven to building RESTful endpoints for creating and updating coffee orders. Additionally, we'll explore how to configure Kafka producers and consumers to publish and process coffee order events, leveraging AVRO for structured data serialization.

What's included

10 videos3 assignments

10 videosTotal 71 minutes
  • Overview of the App1 minute
  • Setting Up the Base Project - Gradle4 minutes
  • Setting Up the Base Project - Maven5 minutes
  • Build the DTOs for CoffeeOrderService7 minutes
  • Build the POST Endpoint for the CoffeeOrderService - /coffee_orders6 minutes
  • Build the Service Layer to Map the DTO to AVRO Domain Object16 minutes
  • Configure the Kafka Producer Properties in Coffee Order Service4 minutes
  • Build Kafka Producer to Publish the CoffeeOrder Events9 minutes
  • Build the Coffee Order Consumer7 minutes
  • Build the PUT Endpoint for the CoffeeOrderService - PUT /coffee_orders/{id}12 minutes
3 assignmentsTotal 135 minutes
  • Build a Coffee Order Service Using Spring Boot and Schema Registry - Assessment15 minutes
  • Full Course Assessment60 minutes
  • Full Course Practice Assessment60 minutes

Instructor

Offered by

Why people choose Coursera for their career

👁 Image

Felipe M.

Learner since 2018
"To be able to take courses at my own pace and rhythm has been an amazing experience. I can learn whenever it fits my schedule and mood."
👁 Image

Jennifer J.

Learner since 2020
"I directly applied the concepts and skills I learned from my courses to an exciting new project at work."
👁 Image

Larry W.

Learner since 2021
"When I need courses on topics that my university doesn't offer, Coursera is one of the best places to go."
👁 Image

Chaitanya A.

"Learning isn't just about being better at your job: it's so much more than that. Coursera allows me to learn without limits."

Frequently asked questions

Yes, you can preview the first video and view the syllabus before you enroll. You must purchase the course to access content not included in the preview.

If you decide to enroll in the course before the session start date, you will have access to all of the lecture videos and readings for the course. You’ll be able to submit assignments once the session starts.

Once you enroll and your session begins, you will have access to all videos and other resources, including reading items and the course discussion forum. You’ll be able to view and submit practice assessments, and complete required graded assignments to earn a grade and a Course Certificate.

If you complete the course successfully, your electronic Course Certificate will be added to your Accomplishments page - from there, you can print your Course Certificate or add it to your LinkedIn profile.

This course is currently available only to learners who have paid or received financial aid, when available.

Yes. In select learning programs, you can apply for financial aid or a scholarship if you can’t afford the enrollment fee. If fin aid or scholarship is available for your learning program selection, you’ll find a link to apply on the description page.

Financial aid available,