Schema Changes
How Supabase ETL handles DDL and evolving table schemas.
Schema-change support is in public beta. ETL automatically applies the simple column changes listed below when the destination can represent them safely. When it cannot, ETL either skips the destination DDL with a warning or rejects the change before applying it.
This page describes structural DDL compatibility. Row values are not validated against destination-specific value domains; see Row value handling.
Supported changes
| PostgreSQL change | Current behavior |
|---|---|
| Add or drop a replicated column | Applied automatically by BigQuery, ClickHouse, DuckLake, and Snowflake |
| Rename a replicated column | Applied automatically, subject to destination naming restrictions |
| Add, replace, or remove a column default | Supported literal defaults are translated on a best-effort basis |
Drop NOT NULL | Applied by BigQuery, ClickHouse, and DuckLake; Snowflake columns are already nullable |
Set NOT NULL | Detected, but not applied to existing destination columns |
| Change a type or type modifier | Detected, but physical type DDL is skipped with a warning |
| Publish an existing column | Added using the historical-value policy described below |
| Stop publishing a column | The destination column is dropped |
Iceberg is deprecated and rejects every newer schema instead of applying schema-change DDL.
Type changes require a resync
BigQuery, ClickHouse, DuckLake, and Snowflake currently leave the destination type unchanged after a PostgreSQL type change. Later schema changes or row writes may fail because the logical and physical schemas have diverged. Resynchronize the table after changing a replicated column type.
How schema changes reach a destination
ETL installs a PostgreSQL event trigger for published permanent tables. After a
supported ALTER TABLE or publication column-list change:
- PostgreSQL emits a transactional schema snapshot in WAL order.
- ETL stores the new versioned table schema.
- PostgreSQL sends a fresh
RELATIONmessage before following row events. - ETL sends the destination an ordered
Event::Relationcontaining the newReplicatedTableSchema. - The destination applies its supported DDL before writing following rows.
Relation is an ordered event, not a batch boundary. One write_events() call
can contain several relation and row events, which must be processed in order.
ETL captures the complete schema after PostgreSQL has applied the DDL instead
of replaying raw PostgreSQL SQL. It matches columns by PostgreSQL attnum, so a
rename keeps its logical identity while a drop and re-add becomes a drop plus
an add.
Ordered schema plans
ETL compares the previously applied logical snapshot with the new one and produces one destination-aware plan. Operations are ordered as follows:
- Drop columns that are no longer present.
- Apply renames, using a temporary name when needed to break a rename cycle.
- Add new columns.
- Apply type, nullability, and default alterations in order.
Every alteration includes adjacent before and after column schemas. A default
addition, removal, or replacement is represented by one Default alteration.
For a replacement, destinations that support existing-column defaults first
remove the old default idempotently, then set the new default only if it is
supported by the current destination renderer.
Mapped destination-name collisions fail before DDL. BigQuery and DuckLake use
ASCII-lowercase destination names; ClickHouse, Snowflake, and Iceberg preserve
source spelling. Snowflake requires QUOTED_IDENTIFIERS_IGNORE_CASE = FALSE so
quoted names remain stable.
Publication column changes
Changing the column list of a table already tracked by the running pipeline is
supported. Other live ALTER PUBLICATION changes are not.
When a publication starts exposing an existing column, ETL does not know that column's historical values in the destination. It therefore follows this policy:
- Add the destination column as nullable, even when PostgreSQL says
NOT NULL. - Keep existing destination rows
NULL. - Write the real value from future PostgreSQL row events.
- Install source default metadata only when the destination guarantees that it affects future writes without changing existing rows. BigQuery provides this guarantee; ClickHouse, DuckLake, and Snowflake do not.
- Reject the transition before DDL when the destination type cannot represent an unknown nullable value. BigQuery and ClickHouse therefore reject publication-added arrays and require a table resynchronization.
Stopping publication of a column drops the destination column and its stored history. Publishing it again creates a fresh nullable column; it does not restore the removed values. Resynchronize the table when exact history is required.
Only column-list changes for already tracked tables are supported while a pipeline is running. Use a controlled stop, restart, table resynchronization, or new pipeline for changes to:
- Publication table membership or
TABLES IN SCHEMA. - Row filters or published operation settings.
- Publication names or ownership.
publish_via_partition_rootand root/leaf identity.
Destination data for a table removed from a publication is not automatically deleted.
Defaults and historical rows
Destination defaults are best-effort metadata translations. PostgreSQL still sends evaluated values in future row events, so skipping destination default metadata does not lose replicated row values.
ETL supports deterministic literal defaults such as strings, numbers,
booleans, dates, times, timestamps, JSON, and UUIDs when the destination can
render them safely. Runtime or session-dependent expressions—including
now(), sequences, random(), current_user, and generated UUID functions—are
skipped with a warning.
ETL does not issue table-wide UPDATE, MERGE, CTAS/swap, or materialization
operations to backfill historical destination rows. The result depends on why
the column appeared:
| Column transition | Historical destination rows |
|---|---|
Physical PostgreSQL ADD COLUMN | The destination may expose a supported add-time default using its native metadata semantics, but ETL does not run a physical backfill |
| Publication reveals an existing column | Rows remain NULL; retroactive default metadata is omitted |
BigQuery is the publication-reveal exception for default metadata: ETL adds the
nullable column first and then uses SET DEFAULT,
which affects only future inserts.
Destination behavior
| Destination | Schema-change policy |
|---|---|
| BigQuery | Applies non-primary-key add, drop, rename, NOT NULL relaxation, and supported literal defaults. Publication-added scalars remain NULL for old rows and may receive a future-only default. Publication-added arrays are rejected. Primary-key definitions must remain stable. Type DDL is skipped. |
| ClickHouse | Applies add, drop, rename, NOT NULL relaxation, and supported literal defaults. Publication-added scalars are nullable and default-free; arrays are rejected because Nullable(Array(...)) is unsupported. Type DDL is skipped. ReplacingMergeTree also rejects primary-key changes. |
| DuckLake | Applies add, drop, rename, NOT NULL relaxation, and supported literal defaults transactionally. Publication-added columns are nullable and omit DuckLake initial defaults. ETL does not emit DEFAULT(NULL); some catalog clients display an omitted default as NULL. Type DDL is skipped. |
| Snowflake | Applies add, drop, rename, create-time defaults, and supported literal add-column defaults. Destination columns remain nullable. Publication-added columns omit defaults because Snowflake would populate old rows. Later existing-column default changes are skipped. Type DDL is skipped. |
| Iceberg | Deprecated. Accepts only an identical applied relation and rejects every newer logical schema, including publication-mask changes. |
See Destinations for maturity, setup requirements, and broader destination limitations.
Recovery and safety
Destination metadata records the target snapshot and replication mask. While a
change is Applying, it also records the previous snapshot and previous
replication mask as one logical recovery endpoint.
Initial table setup uses a separate Creating state. If setup is interrupted,
ETL retries the destination's idempotent write-path setup for that exact target
schema and records Applied only after the destination is ready for writes.
Iceberg also uses Creating to bracket its ETL-owned drop-and-recreate
implementation of source truncation, so an interrupted recreation can finish
before replayed writes continue.
Ancillary objects that are not required for writes, such as BigQuery's
customer-facing view, have their own idempotent reconciliation. This keeps
creation recovery separate from partially applied schema changes, which may
require operator recovery.
Applied metadata is authoritative. It records that ETL completed setup for
the destination table and logical schema, so a process restart or empty local
cache does not trigger creation DDL or structural repair. Destinations may make
read-only metadata calls needed to rebuild indispensable process-local state,
reopen ephemeral write channels, reconcile fully derived non-data-bearing
objects such as views, or clean up ETL-owned temporary artifacts left by a
completed operation. Those operations do not authorize table repair.
If the data-bearing table was deleted or modified outside ETL, ordinary
destination operations fail; resynchronize the table to establish a new,
linear lifecycle.
ETL does not infer missing recovery state:
- Previous snapshot and mask values must be stored together.
- Incomplete or stale recovery metadata fails during loading.
- An older relation, or an equal snapshot with a different replication mask, is rejected instead of rewinding destination DDL.
Recovery then follows each destination's DDL guarantees:
- DuckLake applies a plan transactionally and can retry unambiguous old or target states. Rename-cycle interruptions still require operator recovery.
- ClickHouse resumes only when the physical columns match a complete old or target endpoint. A partially applied DDL sequence requires operator recovery.
- BigQuery and Snowflake leave interrupted nontransactional DDL in
Applyingfor operator recovery rather than guessing what completed. - Iceberg rejects schema evolution.
Resynchronize the affected table when recovery cannot prove a safe endpoint.
Current limitations
- Table creation, removal, and rename are pipeline lifecycle operations, not live schema changes.
- General constraints, identity changes, replica-identity changes, and indexes are not portable destination DDL operations. Destinations may reject key changes required by their write model.
- A table-sync worker can fail closed if schema activity occurs during catch-up without a complete following relation before handover. Retry after schema activity settles or resynchronize the table.
- Destination default and add-column behavior is not a historical-data migration contract. Resynchronize when exact historical values matter.
Custom destinations
Custom destinations receive schema transitions through Event::Relation.
They should:
- Process the batch in order and flush writes using the previous schema.
- Compare the applied and incoming logical schemas.
- Validate the complete plan before recording
Applyingor issuing DDL. - Apply supported operations in the supplied order.
- Record
Appliedonly after following rows can safely use the new schema.
See Custom Implementations for the consumer contract and Events for relation-event details.