Flink CDC is an Apache-2.0 streaming data integration tool built on Apache Flink that moves real-time and batch data between databases and downstream systems, and it is aimed at data engineers and platform teams who need change data capture pipelines without writing per-source job code.

What it is

Flink CDC is a distributed data integration tool for real-time data and batch data, built on top of Apache Flink. It is written in Java and released under the Apache-2.0 licence. The project prioritises efficient end-to-end data integration and adds functionality beyond raw capture, including full database synchronization, sharding table synchronization, schema evolution and data transformation. It exposes three API layers for different usage scenarios: the YAML API (Pipeline API), the SQL API (Table/SQL API), and a further layer described in the project documentation.

The concrete problem it solves is the cost of wiring change data capture by hand. Instead of writing and maintaining a separate job per source table, the YAML API lets users describe source, sink, routing, transformation and schema evolution rules in a single declarative file. Schema changes in the upstream database are handled through a configurable behaviour rather than a code change. The pipeline connectors cover Doris, Elasticsearch, Fluss, Hudi, Iceberg, Kafka, MaxCompute, MySQL, OceanBase, Oracle, Paimon, PostgreSQL and StarRocks, so the same pipeline definition can target different sinks across the ecosystem.

Key capabilities

• YAML API (Pipeline API) declares source, sink, transform, route and pipeline blocks in one file with zero code, submitted through the flink-cdc.sh CLI.
• Pipeline-level controls include pipeline.name, parallelism, and schema.change.behavior: evolve for schema evolution.
• Transform rules operate inline, with projections such as UPPER(product_name) as product_name and filters such as id > 10 AND order_id > 100.
• Route rules map source tables to sink tables, including wildcard patterns like app_db.\.* into ods_db.others.
• SQL API defines CDC sources with SQL DDL using 'connector' = 'mysql-cdc', deployable as a JAR under FLINK_HOME/lib/ and usable from the Flink SQL Client.
• Pipeline connectors are published as Maven artifacts under names such as flink-cdc-pipeline-connector-doris and flink-sql-connector-mysql-cdc.
• Covers both batch and streaming integration, with topic coverage spanning cdc, etl, elt, kafka, mysql, paimon and distributed execution.

Who uses it and how

• Data engineering teams syncing a whole MySQL database into an analytical store, using the documented MySQL-to-Doris pipeline as the reference pattern.
• Teams consolidating sharded tables, which the project lists as a first-class scenario alongside full database synchronization.
• Environments feeding Kafka, Paimon, Iceberg or Hudi, where the pipeline connector list matches the sink already in use.
• Flink SQL users who want CDC without leaving SQL, adding the connector JAR to FLINK_HOME/lib/ and running DDL in the Flink SQL Client.
• Operations where upstream schema changes are frequent, handled by the evolve behaviour rather than by pipeline rewrites.

Getting started

The project documentation points to a Quickstart Guide for setup, with two entry paths: describe the pipeline in a YAML file and submit it via flink-cdc.sh, or add the SQL connector JAR to FLINK_HOME/lib/ and define a CDC table with SQL DDL. Documentation is hosted at the project's stable nightlies site.

How it compares

This registry does not name comparable tools for this entry, so it stands alone here. Any assessment of alternatives would have to come from outside the facts provided.

When to use it — and when not to

A self-hoster must operate Apache Flink, since the tool is built on it and the SQL path expects a FLINK_HOME installation, so teams without Flink experience take on cluster work before the first pipeline runs. The project carries 119 open issues, and the README excerpt provided here is truncated mid-way through the SQL connector list, so the connector catalogue should be checked against the linked overview rather than assumed complete. Teams wanting a fully managed service, or a tool that does not require Flink underneath, should look elsewhere; the last push recorded is 2026-09-17.

source:
  type: mysql
  hostname: localhost
  port: 3306
  username: root
  password: 123456
  tables: app_db.\.*

sink:
  type: doris
  fenodes: 127.0.0.1:8030
  username: root
  password: ""

# Transform data on-the-fly
transform:
  - source-table: app_db.orders
    projection: id, order_id, UPPER(product_name) as product_name
    filter: id > 10 AND order_id > 100
    
# Route source tables to different sink tables
route:
  - source-table: app_db.orders
    sink-table: ods_db.ods_orders
  - source-table: app_db.shipments
    sink-table: ods_db.ods_shipments
  - source-table: app_db.\.*
    sink-table: ods_db.others

pipeline:
  name: Sync MySQL Database to Doris
  parallelism: 2
  schema.change.behavior: evolve  # Support schema evolution

CREATE TABLE mysql_binlog (
  id INT NOT NULL,
  name STRING,
  description STRING,
  weight DECIMAL(10,3),
  PRIMARY KEY(id) NOT ENFORCED
) WITH (
  'connector' = 'mysql-cdc',
  'hostname' = 'localhost',
  'port' = '3306',
  'username' = 'flinkuser',
  'password' = 'flinkpw',
  'database-name' = 'inventory',
  'table-name' = 'products'
);

SELECT id, UPPER(name), description, weight FROM mysql_binlog;

<dependency>
  <groupId>org.apache.flink</groupId>
  <artifactId>flink-connector-mysql-cdc</artifactId>
  <version>${flink-cdc.version}</version>
</dependency>