VOOZH about

URL: https://dev.to/chen_debra_3060b21d12b1b0/deploying-apache-dolphinscheduler-319-cluster-with-mysql-instead-of-postgresql-a-practical-2mam

⇱ Deploying Apache DolphinScheduler 3.1.9 Cluster with MySQL Instead of PostgreSQL? A Practical Docker Compose Guide - DEV Community

👁 MySQL DS

Background

Today, we'll walk through one of the most requested deployment scenarios in the Apache DolphinScheduler community: deploying an Apache DolphinScheduler 3.1.9 cluster with MySQL as the metadata database using Docker Compose.

As many users know, the official Docker Compose deployment provided by DolphinScheduler uses PostgreSQL as the metadata repository by default. This is mainly because the GPLv2 license of MySQL is not fully compatible with Apache License 2.0, preventing the project from distributing an official MySQL-based package.

However, in real-world production environments, many organizations prefer MySQL due to its mature ecosystem, extensive tooling, operational familiarity, and widespread adoption across enterprises.

In this guide, we'll show how to adapt the official Docker Compose deployment and successfully run a DolphinScheduler cluster backed by MySQL metadata storage.

Pull Required Images 

docker pull apache/dolphinscheduler-master:3.1.9

docker pull apache/dolphinscheduler-worker:3.1.9

docker pull apache/dolphinscheduler-tools:3.1.9

docker pull apache/dolphinscheduler-api:3.1.9

docker pull apache/dolphinscheduler-alert-server:3.1.9

docker pull bitnami/zookeeper:3.7.1

Download the MySQL JDBC Driver 

wget https://downloads.mysql.com/archives/get/p/3/file/mysql-connector-j-8.0.33.zip

unzip -q mysql-connector-j-8.0.33.zip

cp mysql-connector-j-8.0.33/mysql-connector-j-8.0.33.jar .

Prepare Custom Images

Dockerfile for Master, Worker, API, and Alert Server 

# Based on the official DolphinScheduler image

ARG SERVICE=api

FROM apache/dolphinscheduler-${SERVICE}:3.1.9

# Copy the MySQL JDBC driver into the DolphinScheduler library directory

# DolphinScheduler loads JDBC drivers from the lib directory

COPY mysql-connector-j-8.0.33.jar /opt/dolphinscheduler/libs/

Dockerfile for Tools 

# Based on the official DolphinScheduler image

ARG SERVICE=tools

FROM apache/dolphinscheduler-${SERVICE}:3.1.9

# Copy the MySQL JDBC driver into the DolphinScheduler tools library directory

# DolphinScheduler loads JDBC drivers from the lib directory

COPY mysql-connector-j-8.0.33.jar /opt/dolphinscheduler/tools/libs/

Build Custom Images 

docker build --build-arg SERVICE=master -t apache/dolphinscheduler-master:3.1.9-mysql .

docker build --build-arg SERVICE=worker -t apache/dolphinscheduler-worker:3.1.9-mysql .

docker build --build-arg SERVICE=tools -t apache/dolphinscheduler-tools:3.1.9-mysql .

docker build --build-arg SERVICE=api -t apache/dolphinscheduler-api:3.1.9-mysql .

docker build --build-arg SERVICE=alert-server -t apache/dolphinscheduler-alert-server:3.1.9-mysql .

Update docker-compose.yaml

Disable the PostgreSQL service and add a MySQL service as the metadata database.

# Comment out the PostgreSQL service

# dolphinscheduler-postgresql:
# image: bitnami/postgresql:15.2.0
# ...

# MySQL Metadata Database Service

dolphinscheduler-mysql:
 image: mysql:8.0
 container_name: dolphinscheduler-mysql

 profiles:
 - all
 - schema

 environment:
 MYSQL_ROOT_PASSWORD: ${MYSQL_ROOT_PASSWORD:-root}
 MYSQL_DATABASE: ${MYSQL_DATABASE:-dolphinscheduler}

 volumes:
 - dolphinscheduler-mysql:/var/lib/mysql

 ports:
 - "3306:3306"

 # Expose MySQL to allow workers on other servers to connect

 healthcheck:
 test:
 [
 "CMD",
 "mysqladmin",
 "ping",
 "-h",
 "localhost",
 "-u",
 "${MYSQL_USERNAME:-root}",
 "-p${MYSQL_PASSWORD:-root}"
 ]

 interval: 5s
 timeout: 60s
 retries: 120

 networks:
 - dolphinscheduler

The remaining services (ZooKeeper, Schema Initializer, API, Alert Server, Master, Worker, Network, and Volume configurations) remain the same as the official Docker Compose deployment, with the following key modifications:

Schema Initializer Dependency 

depends_on:
 dolphinscheduler-mysql:
 condition: service_healthy

Master JVM Configuration 

environment:
 JAVA_OPTS: >
 -server
 -Duser.timezone=${SPRING_JACKSON_TIME_ZONE}
 -Xms8g
 -Xmx8g
 -Xmn4g
 -XX:+PrintGCDetails
 -Xloggc:gc.log
 -XX:+HeapDumpOnOutOfMemoryError
 -XX:HeapDumpPath=dump.hprof

Worker JVM Configuration 

environment:
 JAVA_OPTS: >
 -server
 -Duser.timezone=${SPRING_JACKSON_TIME_ZONE}
 -Xms8g
 -Xmx8g
 -Xmn4g
 -XX:+PrintGCDetails
 -Xloggc:gc.log
 -XX:+HeapDumpOnOutOfMemoryError
 -XX:HeapDumpPath=dump.hprof

Volume Configuration 

volumes:
 # Comment out PostgreSQL volume

 # dolphinscheduler-postgresql:

 dolphinscheduler-mysql:
 dolphinscheduler-zookeeper:
 dolphinscheduler-worker-data:
 dolphinscheduler-logs:
 dolphinscheduler-shared-local:

Update the .env File 

# Docker Hub Repository and Image Tag

HUB=apache
TAG=3.1.9

# MySQL Configuration

MYSQL_ROOT_PASSWORD=root
MYSQL_DATABASE=dolphinscheduler
MYSQL_USERNAME=root
MYSQL_PASSWORD=root

# DolphinScheduler Database Configuration

TZ=Asia/Shanghai

# Use MySQL as the metadata database

DATABASE=mysql

SPRING_JACKSON_TIME_ZONE=GMT+8

SPRING_DATASOURCE_URL=jdbc:mysql://dolphinscheduler-mysql:3306/${MYSQL_DATABASE}?useUnicode=true&characterEncoding=UTF-8&useSSL=false&serverTimezone=Asia/Shanghai&allowPublicKeyRetrieval=true

SPRING_DATASOURCE_USERNAME=${MYSQL_USERNAME}

SPRING_DATASOURCE_PASSWORD=${MYSQL_PASSWORD}

REGISTRY_ZOOKEEPER_CONNECT_STRING=dolphinscheduler-zookeeper:2181

MASTER_FETCH_COMMAND_NUM=10

Initialize the Database 

docker compose --profile schema up -d

Start the Entire Cluster 

docker compose --profile all up -d

Start the Worker Service 

docker compose up -d dolphinscheduler-worker

Start the Master Service 

docker compose up -d dolphinscheduler-master

Start the Alert Service 

docker compose up -d dolphinscheduler-alert

Start the API Service 

docker compose up -d dolphinscheduler-api

Restart All Services 

docker compose --profile all restart

Conclusion

Although Apache DolphinScheduler officially ships with a PostgreSQL-based Docker Compose deployment, many enterprises continue to standardize on MySQL for operational consistency and ecosystem compatibility.

By adding the MySQL JDBC driver, rebuilding the DolphinScheduler images, and adjusting the Docker Compose and environment configurations, you can quickly deploy a fully functional Apache DolphinScheduler 3.1.9 cluster powered by MySQL metadata storage.

This approach enables teams already invested in the MySQL ecosystem to integrate DolphinScheduler into their infrastructure with minimal friction while preserving the benefits of containerized deployment and cluster-based scheduling.

If your organization relies on MySQL as a strategic database platform, this solution provides a practical and production-friendly path to running Apache DolphinScheduler at scale.