Events

During catch-up and ongoing replication, Supabase ETL delivers events decoded from Postgres logical replication via write_events(). This page documents their Rust shapes and PostgreSQL semantics.

Event Overview

Data Modification Events

These events carry row data decoded from WAL after publication filtering. A source UPDATE can become a logical INSERT or DELETE when a row enters or leaves a publication's row filter.

Row Images

Data modification events use row-image helper types:

pub enum UpdatedTableRow {
    Full(TableRow),
    Partial(PartialTableRow),
}

pub enum OldTableRow {
    Full(TableRow),
    Key(TableRow),
}

TableRow is a complete dense row. Its values are ordered to match the replicated table-column order.

PartialTableRow is used when an update row is not complete. It exposes:

• total_columns(): the number of replicated columns in the table schema;
• values(): present values in replicated table-column order, excluding missing columns;
• missing_column_indexes(): zero-based replicated-column indexes for values ETL could not reconstruct.

OldTableRow::Full(row) contains a complete old row. OldTableRow::Key(row) contains only replica-identity columns, densely packed in replicated table-column order.

Insert

A new row was added to a table.

pub struct InsertEvent {
    pub commit_lsn: PgLsn,
    pub tx_ordinal: u64,
    pub replicated_table_schema: ReplicatedTableSchema,
    pub table_row: TableRow,
}

Update

An existing row was modified.

pub struct UpdateEvent {
    pub commit_lsn: PgLsn,
    pub tx_ordinal: u64,
    pub replicated_table_schema: ReplicatedTableSchema,
    pub updated_table_row: UpdatedTableRow,
    pub old_table_row: Option<OldTableRow>,
}

updated_table_row is the authoritative post-update payload:

• UpdatedTableRow::Full when ETL knows every replicated column value after decoding the update.
• UpdatedTableRow::Partial when PostgreSQL emitted UnchangedToast fields that ETL could not reconstruct safely.

old_table_row provides optional context for row matching, key changes, and reconstruction.

Unchanged Toast

PostgreSQL may encode an unchanged toasted value as UnchangedToast in the update's new tuple instead of resending the full column value. ETL can turn the update into UpdatedTableRow::Full only when that value can be recovered from the old-side row image:

• a FULL old row can recover unchanged toasted values for any replicated column;
• a key-only old row can recover unchanged toasted values only for replica-identity columns included in that key image;
• no old row means unchanged toasted values cannot be recovered from the event.

When any UnchangedToast field cannot be recovered, ETL emits UpdatedTableRow::Partial with known values and missing column indexes instead of pretending the unknown value is NULL or a replacement value.

Destinations that need complete replacement rows, full before/after comparison, or complete audit records should require REPLICA IDENTITY FULL for tables with toasted columns, or keep enough prior state to fill missing values themselves.

Old Row Mapping

ETL maps PostgreSQL pgoutput update tuple markers directly:

PostgreSQL chooses which old-side marker to emit from the table's replica identity:

For UPDATE, PostgreSQL only sends an old key image under DEFAULT or USING INDEX when it determines the old key must be logged. In practice, that happens when:

• any replica-identity column changed, or
• any replica-identity column contains external data, such as a toasted value that must be available from the old tuple.

That means non-identity updates under DEFAULT or USING INDEX often arrive with old_table_row = None. Under FULL, PostgreSQL sends a full old row for every published update. A FULL update with old_table_row = None is not a pgoutput shape.

When handling updates:

1. Treat updated_table_row as the post-update payload and handle UpdatedTableRow::Partial explicitly.
2. Treat OldTableRow::Key as replica-identity values, not necessarily the primary key. Use OldTableRow::Full for complete before-images.
3. If old_table_row is None, match or upsert from the new row according to the destination's keying model.

Delete

A row was removed from a table.

pub struct DeleteEvent {
    pub commit_lsn: PgLsn,
    pub tx_ordinal: u64,
    pub replicated_table_schema: ReplicatedTableSchema,
    pub old_table_row: Option<OldTableRow>,
}

For DELETE, valid PostgreSQL publications send an old-side image:

Important implications:

• Deletes do not carry a new row image. old_table_row is the delete payload.
• OldTableRow::Key(row) again means replica-identity columns only, not necessarily the table's primary key.
• Consumers that require full old rows for deletes need REPLICA IDENTITY FULL.
• The Rust field is optional at the event API boundary, but PostgreSQL pgoutput populates it for every published delete. Tables without usable replica identity fail at the source when they publish deletes.
• Destination implementations should decide up front whether key-only deletes are enough. If they are not, require REPLICA IDENTITY FULL for those tables.

Truncate

Decoded from WAL truncation, restricted to the published tables.

pub struct TruncateEvent {
    pub commit_lsn: PgLsn,
    pub tx_ordinal: u64,
    pub options: i8,
    pub truncated_tables: Vec<ReplicatedTableSchema>,
}

Transaction Events

These events delimit a committed source transaction in the logical stream. ETL uses pgoutput's non-streaming transaction protocol, so PostgreSQL knows the commit LSN and timestamp before sending Begin.

Begin

Synthesized by pgoutput to open the transaction's logical envelope, without a corresponding physical WAL BEGIN record.

pub struct BeginEvent {
    pub commit_lsn: PgLsn,
    pub tx_ordinal: u64,
    pub timestamp: i64,
    pub xid: u32,
}

commit_lsn comes from pgoutput's Begin.final_lsn: the start of the transaction's COMMIT WAL record. timestamp is the commit timestamp, in microseconds since PostgreSQL's epoch (2000-01-01), not the time SQL BEGIN executed.

Commit

Decoded from the transaction's COMMIT WAL record. Receiving this event does not by itself mean the destination writes are durable.

pub struct CommitEvent {
    pub commit_lsn: PgLsn,
    pub tx_ordinal: u64,
    pub flags: i8,
    pub end_lsn: PgLsn,
    pub timestamp: i64,
}

commit_lsn identifies the same COMMIT record as Begin.commit_lsn. end_lsn is the exclusive position after that record; it can equal the start of another transaction's COMMIT record. Resuming at that boundary can replay the other transaction, including changes whose WAL records precede it. timestamp is the commit timestamp in the same format as Begin.timestamp.

Schema Events

Relation

ETL-derived schema barrier carrying table schema and column-mask information for following data events. A repeated notification does not necessarily mean the schema changed.

pub struct RelationEvent {
    pub replicated_table_schema: ReplicatedTableSchema,
}

Pgoutput synthesizes RELATION metadata from the catalogs. A reconnect or catalog invalidation, including maintenance, can resend an unchanged schema. ETL can also synthesize a RelationEvent before an insert, update, or delete when a pending DDL snapshot has no following protocol relation. These events have no durable event position and do not consume a transaction ordinal.

The carried SnapshotId identifies a stored table-schema version, while the masks describe its replicated projection and row identity. Supported publication column-list changes receive a new snapshot even when the full table schema is unchanged. No-op DDL can also advance the snapshot without changing columns; receiving a protocol relation alone does not create a new version.

The source snapshot describes the table without a pipeline-specific column projection. Each pipeline uses its relation metadata to derive that projection, so publication column-list changes use the same logical schema-change path as table DDL. See why snapshots and masks are separate.

Destinations compare the snapshot and replication mask with applied metadata. An identical pair needs no schema transition. A newer snapshot must be validated even when its projected schema needs no physical DDL. An equal snapshot with a different replication mask is ambiguous and must be rejected. Preserve event order; relation delivery is not a checkpoint or deduplication key.

PostgreSQL pgoutput builds relation messages by walking the table descriptor in pg_attribute.attnum order and skipping columns that are not published. It sends tuple data in the same order as the relation message.

ETL builds replication and identity masks from relation-message column names, so the order is not needed to decide which columns are included. That name-based matching is sound because PostgreSQL live column names are unique within a table schema version. The order matters only after the masks are applied: stored table schemas are also ordered by attnum, so ReplicatedTableSchema::column_schemas() becomes a positional view that matches the tuple payloads exactly, even when a publication filters columns.

This requires the stored snapshot and relation metadata to describe the same schema state. A missing column can mean intentional publication filtering or an inconsistent snapshot; the mask alone cannot distinguish them. Schema-change handling remains in beta, with capture and relation-metadata cautions.

For DDL behavior, including add/drop/rename semantics, default and nullability handling, and operating cautions, see Schema Changes.

Begin/Commit Behavior

During initial sync, parallel Table Sync Workers may deliver Begin and Commit markers more than once. Ignore them if you do not need transaction markers; otherwise deduplicate them by commit_lsn plus tx_ordinal. Row writes must still be idempotent because ETL provides at-least-once delivery.

Understanding Event Positions

Transaction and data events include a commit LSN (Log Sequence Number) plus a transaction-local ordinal. Together, these fields provide stable ordering for idempotent destination writes. Relation events are not sequenced because they are connection-local protocol metadata.

commit_lsn and tx_ordinal

Key insight: Multiple events share the same commit_lsn when they belong to the same transaction. ETL uses commit_lsn plus tx_ordinal as the stable event position for ordered CDC writes.

Example

Consider a transaction that inserts two rows:

BEGIN;                    -- Transaction starts
INSERT INTO users ...;    -- commit_lsn: 0/16B3800, tx_ordinal: 1
INSERT INTO users ...;    -- commit_lsn: 0/16B3800, tx_ordinal: 2
COMMIT;                   -- Transaction commits at 0/16B3800

Both inserts have the same commit_lsn because they commit together. Their tx_ordinal values distinguish their order among the transaction events that carry event positions. Session-generated relation messages do not consume an ordinal.

Using LSNs

For ordering: For events that expose an event position, use commit_lsn to order transactions and tx_ordinal to order events within the same transaction. Preserve RelationEvent ordering directly as a schema barrier.

For idempotency: For events that expose one, use the event position, equivalent to (commit_lsn, tx_ordinal), together with the destination's table or row key when detecting replays. Do not use relation events as replay keys.

For checkpointing: Let ETL advance its worker checkpoint after the corresponding destination work is durable. A row's commit_lsn alone does not prove that every event in that transaction has been delivered: batches can split transactions. Completed-transaction progress uses CommitEvent.end_lsn, the exclusive position after the COMMIT record. Another transaction whose commit record starts exactly at that position remains replayable. Custom stores must implement the replication checkpoint contract; destinations must report durability through the supplied result handles.

Event Batching

ETL batches events before calling write_events(). A batch may contain events from multiple tables, multiple transactions, and mixed event types. Schema changes do not end a batch: because batching is driven by size and time, one batch may contain multiple Relation events and multiple schema changes.

Ordering requirement: Preserve per-table event order, including schema changes and truncates. Destinations may process independent work concurrently only when they preserve those barriers and the required durability ordering.