> ## Documentation Index
> Fetch the complete documentation index at: https://docs.zwiron.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Change Data Capture (CDC)

> Stream real-time inserts, updates, and deletes with Zwiron CDC via Postgres WAL or MySQL binlog.

**Change Data Capture (CDC)** is a sync mode that reads directly from a database's change log instead of querying tables. This gives you:

* **Real-time data** — changes land at the destination in seconds, not minutes
* **Deletes captured** — when a row is deleted at the source, it's deleted at the destination
* **Low source load** — no polling queries; the agent reads the log that the database is already writing

***

## How CDC works

Every major database maintains a log of changes (binlog, WAL, change tables). This log records every insert, update, and delete in order.

Zwiron's agent reads this log and applies the changes to the destination:

```
Source Database                    Agent                    Destination
─────────────                      ─────                    ───────────
INSERT row → written to log ──▶  reads log ──▶  INSERT at destination
UPDATE row → written to log ──▶  reads log ──▶  UPDATE at destination
DELETE row → written to log ──▶  reads log ──▶  DELETE at destination
```

The agent tracks its position in the log (called the **checkpoint**). If the agent restarts or the connection drops, it resumes from where it left off.

***

## Supported sources (live today)

| Database   | CDC mechanism            | Notes                                     |
| ---------- | ------------------------ | ----------------------------------------- |
| PostgreSQL | Logical replication slot | Requires `wal_level = logical`            |
| MySQL      | Binary log (binlog)      | Requires `binlog_format = ROW`            |
| MongoDB    | Change Streams           | Requires a replica set or sharded cluster |

Additional databases (MariaDB, SQL Server, Oracle, and others) are **on demand** — see [More connectors on demand](/connectors/coming-soon). Do not assume CDC is available for them until support enables the connector.

***

## Setting up CDC on PostgreSQL

### 1. Enable logical replication

In your `postgresql.conf`:

```
wal_level = logical
max_replication_slots = 5
max_wal_senders = 5
```

Restart Postgres after changing these settings.

### 2. Create a replication user

```sql theme={null}
CREATE USER zwiron_cdc WITH REPLICATION LOGIN PASSWORD 'your_password';
GRANT SELECT ON ALL TABLES IN SCHEMA public TO zwiron_cdc;
ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT SELECT ON TABLES TO zwiron_cdc;
```

### 3. Create a publication

Zwiron will automatically create a replication slot and publication when CDC is first enabled. Or you can create it manually:

```sql theme={null}
CREATE PUBLICATION zwiron_pub FOR ALL TABLES;
```

### 4. Use CDC in Zwiron

When creating a pipeline, select **CDC** as the sync mode. Zwiron will handle the rest.

***

## Setting up CDC on MySQL

### 1. Enable binary logging

In your `my.cnf`:

```ini theme={null}
[mysqld]
server-id = 1
log_bin = /var/log/mysql/mysql-bin.log
binlog_format = ROW
binlog_row_image = FULL
expire_logs_days = 7
```

Restart MySQL after changing these settings.

### 2. Create a replication user

```sql theme={null}
CREATE USER 'zwiron_cdc'@'%' IDENTIFIED BY 'your_password';
GRANT SELECT, RELOAD, SHOW DATABASES, REPLICATION SLAVE, REPLICATION CLIENT ON *.* TO 'zwiron_cdc'@'%';
FLUSH PRIVILEGES;
```

***

## Requirements for CDC

* **Primary key required** — every table you want to CDC must have a primary key. Zwiron uses this to identify which row to update or delete at the destination.
* **Replication must be enabled** — see the database-specific instructions above.
* **Log retention** — the database must retain the change log long enough for Zwiron to read it. If CDC falls too far behind (e.g. agent offline for a long time), it may need to perform a full re-sync.

***

## CDC vs. Incremental

|                      | Incremental     | CDC                       |
| -------------------- | --------------- | ------------------------- |
| Latency              | Minutes         | Seconds                   |
| Captures deletes     | No              | Yes                       |
| Source load          | Polling queries | Log reads (very low)      |
| Setup required       | None            | Replication config + user |
| Primary key required | No (optional)   | Yes                       |

Use Incremental if you just need efficiency. Use CDC if you need real-time data or need to capture deletes.

***

## Checkpointing

Zwiron tracks the agent's position in the change log (log sequence number, binlog position, or replication slot LSN). This checkpoint is saved after each batch of changes is successfully written to the destination.

If the agent restarts, it resumes from the last checkpoint — no data is lost or duplicated.

***

## What happens if CDC falls behind?

If the agent goes offline for an extended period and the database's change log has been truncated (older entries expired), Zwiron will detect the gap and perform an automatic full re-sync of the affected tables. You'll see this noted in the job logs.