Flink CDC for YugabyteDB

How it Works and What it Took to Reach Technical Preview

July 9, 2026

Change data capture (CDC) is how modern data platforms stay in sync without batch jobs and nightly exports. You want every insert, update, and delete in your operational database to show up reliably and in order somewhere else: a warehouse, a search index, a lakehouse, or a second database. Flink CDC is a great way to achieve that, and YugabyteDB is now available as a source for it.

We recently built a YugabyteDB integration for Flink CDC. This fork of the Apache Flink CDC PostgreSQL connector talks to YugabyteDB’s logical replication stream and is now available as a Tech Preview.

This blog post by YugabyteDB Software Engineer intern Bakul Gupta discusses how Flink CDC works and the changes he made to the upstream connector to get it streaming correctly against YugabyteDB.

What is Flink CDC?

Flink CDC is a data integration framework built on Apache Flink. It started out as Ververica’s “CDC Connectors for Apache Flink” in 2021 and moved under the Apache Flink project in 2024.

The idea is straightforward. It captures row-level changes from a database and turns them into a continuous Flink stream that you can filter, join, transform, and send anywhere that Flink can write.

Flink CDC uses the Debezium PostgreSQL connector to read changes from the database’s logical replication stream. The difference between using Flink CDC and running Debezium on Kafka Connect lies in what happens next.

Instead of dropping change events into a Kafka topic, Flink CDC turns each change record into an entry in a continuous Flink stream that behaves like a live table. From there, you process the data with Flink SQL or the DataStream API and write it to any Flink sink: Kafka, Elasticsearch, Iceberg, a JDBC database, etc. You don’t even need Kafka or a separate Kafka Connect cluster to move data. Apache Flink alone is enough.

A Flink CDC source runs in two phases:

It takes a snapshot of the data already in the table.
It switches to streaming new changes as they happen, keeping track of its position so it can pick up where it left off after a restart.

Using YugabyteDB as the source is useful for several reasons:

Real-time propagation of YugabyteDB changes to Kafka, Elasticsearch, Iceberg, or a data lake.
Operational analytics, by continuously feeding a warehouse or lakehouse.
Event-driven architectures by turning row-level changes into streams.
Near-zero-downtime migrations off YugabyteDB, by combining the snapshot and the change stream in one pipeline.

How Flink CDC Works with YugabyteDB

The integration is a three-stage pipeline.

Change capture (source). The Flink postgres-cdc connector taps YugabyteDB’s logical replication to read every row-level change Architecture for CDC using PostgreSQL protocol | YugabyteDB Docs. A YugabyteDB-specific note: set the connector’s decoding plugin to pgoutput, because its default plugin isn’t supported.
Stream processing (Flink). The connector turns incoming change records into a continuous Flink stream that you can filter, transform, and join with Flink SQL or the DataStream API before sending it on.
Data sink (destination). The processed stream is continuously written to a downstream system via a Flink sink connector. For example, a JDBC sink writing into PostgreSQL, or a Kafka or Iceberg sink.

Bringing Flink CDC to YugabyteDB: The Required Changes

Since YugabyteDB is PostgreSQL-compatible, integrating with existing PostgreSQL tools such as Flink CDC is relatively straightforward. However, the upstream Postgres connector assumes a single PostgreSQL server and relies on a small number of PostgreSQL internals that require special handling in a distributed database. We forked the connector to address those differences while preserving PostgreSQL compatibility, ensuring the connector worked reliably with YugabyteDB without sacrificing functionality or test coverage.

Here is what had to change:

Knowing where to start streaming. After the first snapshot, the connector needs to know exactly where in the change log to begin reading. The upstream version asks the PostgreSQL server for its current log position using functions YugabyteDB doesn’t support (limitations). We changed it to read that position straight from the replication slot instead, so streaming picks up at the right spot with no missed or duplicated changes when it switches over.
Creating the replication slot. We adjusted how the slot is created so it no longer has to sit inside a transaction, which is what YugabyteDB expects.
Taking the first snapshot. The connector’s snapshot setup sent several commands to the database in a single batch, but YugabyteDB doesn’t allow that for one of them. We split the setup into separate steps, so the initial snapshot starts cleanly every time.

The connector’s test suite was also updated to match what YugabyteDB actually supports. For example, we dropped tests for PostgreSQL-only features like incremental snapshots and partition-root publishing, and switched the test data to explicit IDs so results stay predictable on a distributed database.

If you want the code-level detail, all of these changes are live in the YugabyteDB fork at github.com/yugabyte/flink-cdc.

Hardening for Tech Preview

Making the connector work was only half the job; ensuring it was stable enough for a Tech Preview was the other half. We ran it end-to-end and stress-tested it against real YugabyteDB clusters, hardened it for long-running, hands-off operation.

In addition, we validated the connector using our extensive stress-testing framework ‘A Behind-the-Scenes Look at Chaos Testing in YugabyteDB,’ which continuously verifies streaming correctness as changes are made to the database. The framework injects a wide range of failure scenarios (including network partitions, VM shutdowns and crashes, cluster restarts, and other operational disruptions) to ensure the connector continues streaming without any interruption and is resilient under real-world conditions.

The key to that stability is a forgiving restart strategy with regular checkpoints.

Flink saves its progress to durable storage on a schedule. If a job is interrupted for any reason, it restarts automatically and resumes from the last checkpoint instead of starting over. You can see those settings in the deployment below.

Deploying the Connector

The YugabyteDB fork lives at github.com/yugabyte/flink-cdc, with pre-built connector JARs on the Releases page and a ready-to-run Docker image.

The image is based on Flink 1.20.1 and bundles the postgres-cdc source connector, the JDBC sink connector, and the PostgreSQL driver under /opt/flink/lib/, so they are available to both the SQL client and your jobs right away.

You will need a running YugabyteDB cluster with YSQL logical replication enabled, a PostgreSQL instance to write into, and Docker.

Below is the full walkthrough for a pipeline that streams a table from YugabyteDB to PostgreSQL.

Prepare the source in YugabyteDB.
Connect with ysqlsh, create the table you want to capture, and add some sample rows, then create the publication and the logical replication slot:

CREATE TABLE shipments (
  shipment_id INT PRIMARY KEY,
  order_id INT,
  origin TEXT,
  destination TEXT,
  is_arrived BOOLEAN
);

INSERT INTO shipments VALUES
  (1001, 1, 'Beijing', 'Shanghai', TRUE),
  (1002, 2, 'New York', 'Los Angeles', FALSE),
  (1003, 3, 'Mumbai', 'Delhi', TRUE);

CREATE PUBLICATION dbz_publication FOR ALL TABLES;
SELECT * FROM pg_create_logical_replication_slot('flink', 'pgoutput');

Prepare the sink in PostgreSQL.
Create the matching destination table:

CREATE TABLE shipments (
  shipment_id INT PRIMARY KEY,
  order_id INT,
  origin TEXT,
  destination TEXT,
  is_arrived BOOLEAN
);

Create a docker-compose.yaml.
This brings up a Flink job manager and a task manager from the connector image. The restart strategy and checkpoint settings are what keep a long-running job resilient.:

services:
  jobmanager:
    image: quay.io/yugabyte/ybdb-flink-cdc:fl.3.5.yb.2026.1.0
    container_name: flink-jobmanager
    hostname: jobmanager
    ports: ["8081:8081", "6123:6123"]
    command: jobmanager
    volumes:
      - ./checkpoints:/opt/flink/checkpoints
    environment:
      FLINK_PROPERTIES: |-
        jobmanager.rpc.address: jobmanager
        restart-strategy.type: fixed-delay
        restart-strategy.fixed-delay.attempts: 800
        restart-strategy.fixed-delay.delay: 15 s
        state.checkpoints.dir: file:///opt/flink/checkpoints

  taskmanager:
    image: quay.io/yugabyte/ybdb-flink-cdc:fl.3.5.yb.2026.1.0
    container_name: flink-taskmanager
    hostname: taskmanager
    depends_on: [jobmanager]
    command: taskmanager
    volumes:
      - ./checkpoints:/opt/flink/checkpoints
    environment:
      FLINK_PROPERTIES: |-
        jobmanager.rpc.address: jobmanager
        taskmanager.numberOfTaskSlots: 4
        restart-strategy.type: fixed-delay
        restart-strategy.fixed-delay.attempts: 800
        restart-strategy.fixed-delay.delay: 15 s
        state.checkpoints.dir: file:///opt/flink/checkpoints

Start the Flink cluster.
```
docker compose up -d
```

Open the Flink SQL client.

docker compose exec jobmanager ./bin/sql-client.sh

Define the source and sink, and start the pipeline.
In the SQL client, turn on checkpointing, define a postgres-cdc source that points at YugabyteDB and a jdbc sink that points at PostgreSQL, then submit the job with a single INSERT:

SET 'execution.runtime-mode' = 'streaming';
SET 'execution.checkpointing.interval' = '60 s';
SET 'execution.checkpointing.timeout' = '10 min';

CREATE TABLE yb_shipments (
  shipment_id INT,
  order_id INT,
  origin STRING,
  destination STRING,
  is_arrived BOOLEAN,
  PRIMARY KEY (shipment_id) NOT ENFORCED
) WITH (
  'connector' = 'postgres-cdc',
  'hostname' = '<tserver-ip>',
  'port' = '5433',
  'username' = 'yugabyte',
  'password' = 'yugabyte',
  'database-name' = 'yugabyte',
  'schema-name' = 'public',
  'table-name' = 'shipments',
  'slot.name' = 'flink',
  'decoding.plugin.name' = 'pgoutput'
);

CREATE TABLE pg_shipments (
  shipment_id INT,
  order_id INT,
  origin STRING,
  destination STRING,
  is_arrived BOOLEAN,
  PRIMARY KEY (shipment_id) NOT ENFORCED
) WITH (
  'connector' = 'jdbc',
  'url' = 'jdbc:postgresql://<sink-host>:5432/postgres',
  'table-name' = 'shipments',
  'username' = 'your_user',
  'password' = 'your_password'
);

INSERT INTO pg_shipments SELECT * FROM yb_shipments;

Swap <tserver-ip> for a YugabyteDB tserver address that Flink can reach, and <sink-host> for your PostgreSQL host.
If your cluster requires TLS, add 'debezium.database.sslmode' = 'require' to the source table and point 'debezium.database.sslrootcert' at the CA certificate.
That last statement submits a continuous streaming job. Inserts, updates, and deletes on the YugabyteDB shipments table now flow through to PostgreSQL, and you can monitor this at http://localhost:8081.

Stream only new changes (optional).
To skip the initial snapshot and capture changes from this point forward, add 'debezium.snapshot.mode' = 'never' to the source table options.

Tech Preview Considerations

This is a Tech Preview, so it works, and it is documented, but there are still some limitations to consider:

No incremental snapshot. Leave scan.incremental.snapshot.enabled at its default of false. YugabyteDB doesn’t currently support it, and because checkpointing isn’t allowed during the initial snapshot, a long snapshot can time out. You can soften that with a generous checkpointing interval, a high tolerable-failed-checkpoints value, and a forgiving restart strategy.
No cross-table transactional guarantees end-to-end by default.
No automatic schema evolution. DDL changes have to be handled manually.
Exactly-once depends on the sink. The postgres-cdc source supports it, but a standard JDBC sink is at-least-once.
Primary keys are required. The workarounds for tables without one rely on incremental snapshots, which aren’t supported.
Use a unique slot.name per pipeline, so two jobs don’t fight over the same slot.

Best practices

Always define primary keys, and pick distributed keys so you don’t create hotspots.
Use pgoutput as the decoding plugin, and don’t enable incremental snapshot.
Set up checkpointing alongside a forgiving fixed-delay restart strategy.
Watch CDC lag, retention headroom, and checkpoint health as your main health signals.

Tech Preview Status and What’s Next

The YugabyteDB Flink CDC connector is available as a Tech Preview on releases 2025.2 and above. It works against real clusters today, with the snapshot-and-stream lifecycle, an end-to-end JDBC sink path, and the restart-based stability all in place.

As this feature heads toward general availability, the next priority is to broaden sink coverage and testing, and further refine resilience so the restart strategy becomes a safety net rather than something you have to lean on.

Conclusion

Flink CDC gives YugabyteDB a clean way to stream every change to the rest of your stack in real time, with or without Kafka. Getting here meant adapting the upstream Apache Flink CDC PostgreSQL connector to match how a distributed database actually behaves.

That involved reading the streaming position from the replication slot, dropping the single-server assumptions the connector was built on, splitting the snapshot setup that YugabyteDB can’t run in a single batch, and hardening the pipeline so it runs unattended for the long haul.

The final result is a robust connector that turns YugabyteDB into a proper Flink CDC source.