Destinations

Supabase ETL ships official destination implementations in the etl-destinations crate. Enable only the destination feature you need when embedding ETL or building the standalone replicator. ClickHouse is the easiest destination to start with locally: cargo x init starts it, and cargo x setup replicator configures it by default. BigQuery is the most mature cloud destination.

Status definitions

• Stable: The most mature built-in destination. Review its limitations and validate it against your production workload.
• In progress: Functional, but behavior, configuration, schema support, and operational requirements may still change. Test recovery and schema changes before production use.
• Deprecated: Retained for compatibility but no longer recommended for new deployments. Plan to move away from it.

Row value handling

ETL validates structural prerequisites such as primary keys, replica identity, column mappings, and schema transitions. It does not pre-validate each row value against destination-specific domains. When ETL can represent a value in the destination request or wire format, it sends that value and relies on the destination's native coercion, rounding, canonicalization, and rejection behavior. These semantics apply consistently to initial copy, inserts, and updates.

ETL returns a local error only when it cannot construct a faithful wire representation. Known destination differences do not add per-cell runtime validation. Test destination-specific value semantics that matter to your workload.

Publication column additions

An existing column newly exposed by a publication is added as nullable so old destination rows remain NULL. Defaults and array handling differ by destination. See Publication column changes for the complete policy.

Destination ownership

ETL assumes exclusive ownership of every data-bearing table it creates. Destination metadata is the authority for whether initial setup or a schema change completed; process-local caches do not replace that state. Once metadata is Applied, ETL does not recreate a missing table or repair external schema changes during ordinary writes or restart. Such changes are unsupported and typically cause the destination operation to fail. Use the table resynchronization lifecycle when a destination table must be rebuilt.

ETL may rebuild ephemeral client state and derived non-data-bearing objects, such as Snowflake streaming channels or BigQuery views. Direct destination truncation is also outside ETL's lifecycle: later behavior depends on the destination's offset and write semantics, and previously replicated rows are not restored automatically.

BigQuery

BigQuery uses the BigQuery Storage Write API and BigQuery change data capture to maintain destination tables. Use it when you already have a Google Cloud project. For a local first pipeline, prefer ClickHouse.

Table partitioning and clustering

Use table_options to select the physical layout for individual source tables. table_options, along with max_staleness_mins, is applied only when a table is created or recreated, never to a table that already exists. Restarting a pipeline does not modify an existing table, and changing either setting has no effect until the table is created again. A table is created, and so picks up the current settings, when it is replicated for the first time, when its replication state is reset, or when a source truncate makes ETL create a new physical table.

destination:
  big_query:
    # Project, dataset, and authentication fields omitted.
    table_options:
      tables:
        - table_id: 16384
          partition_by:
            kind: time_column
            column: created_at
            granularity: day
          cluster_by:
            - tenant_id
            - event_type

table_id is the source PostgreSQL table OID, matching the identifiers used by table_sync_copy; use the table IDs returned in the source publication response. Because the configuration follows the source table id rather than its name, renaming the source table does not detach its physical-layout settings, and destination metadata can continue pointing it at a different physical table. If a table is dropped and recreated, PostgreSQL assigns it a new OID, so its configuration must use the new table_id.

partition_by and cluster_by are independent. Set either one or both in a table entry: omit cluster_by for a partitioned-only table, or omit partition_by for a clustered-only table. To use BigQuery's default unpartitioned and unclustered layout, omit that table from tables. An entry with neither option is rejected because it has no effect.

Time-column partitioning supports replicated PostgreSQL date, timestamp, and timestamptz columns with hour, day, month, or year granularity; date columns do not support hour. Use kind: integer_range with column, start, end, and interval for an integer column, or kind: ingestion_time with granularity for ingestion-time partitioning. Clustering accepts one to four ordered, distinct replicated column names. BigQuery validates whether clustering column types are supported.

partition_by.column and each cluster_by entry use the source PostgreSQL column name; ETL applies the same flexible column names mapping to them as it does to the column itself, so a partitioning or clustering column does not need to be quoted or otherwise adjusted for the destination.

See the BigQuery documentation for partition expressions and clustering column requirements.

Flexible column names

BigQuery's flexible column names accept spaces, most punctuation, and non-ASCII Unicode characters. ETL passes any replicated PostgreSQL column name through to BigQuery for table creation, schema changes, and row writes; a column name is no longer required to be an ASCII identifier.

BigQuery documents column names as case-insensitive but does not specify the Unicode case-folding algorithm it uses for identifier resolution. ETL folds only ASCII letters when mapping a source column name to its destination name, matching the destination-name collision behavior used elsewhere for BigQuery. Name and name therefore collide locally and are treated as one destination column, while Ä and ä remain distinct. If BigQuery's own case-insensitivity considers such a non-ASCII pair equivalent, the table DDL is rejected instead of ETL silently merging the two columns.

Limitations

• Every replicated source table needs a primary key, and all primary-key columns must be included in the publication.
• PostgreSQL arrays containing NULL elements are not supported.
• BigQuery stores a top-level PostgreSQL NULL array as an empty array, so the two source values are indistinguishable in the destination.
• Numeric and JSON values follow BigQuery's native domain, rounding, and canonicalization behavior. BigQuery may transform or reject values that do not map exactly to its types.
• Supported schema changes are applied automatically, but PostgreSQL default expressions and backfill behavior are only supported where they map safely to BigQuery. See Schema Changes.
• BigQuery column renames are subject to BigQuery's row-access-policy and metadata limitations. Review BigQuery schema change behavior before renaming columns with tags, aspects, or policies.

ClickHouse

ClickHouse is the local default destination. cargo x init starts ClickHouse on http://localhost:8123 (etl / etl), and cargo x setup replicator writes config for that service. No cloud account is required.

The Standalone Replicator guide uses ClickHouse as its default example.

ClickHouse supports a current-state layout with ReplacingMergeTree and an append-only event-log layout with MergeTree.

engine is applied only when a table is created or recreated; restarting a pipeline or changing engine does not modify an existing table. ClickHouse has no way to alter a table's engine, so if engine no longer matches a table that already exists, ETL returns an error on write instead of silently applying the new setting. Drop and resynchronize the table, or revert engine to match it.

Limitations

• The default ReplacingMergeTree layout requires a source primary key with every primary-key column included in the publication, and requires ClickHouse 23.5 or newer.
• MergeTree preserves an append-only event log; it does not expose a current-state replica by itself.
• Source type changes are warned about, but physical type DDL is not currently applied or validated. Later RowBinary writes may fail or behave unpredictably; resynchronize the table after a source type change.
• ClickHouse Array columns are non-nullable. A top-level PostgreSQL NULL array cannot be encoded in RowBinary and fails the write; empty arrays and NULL array elements remain supported.
• Values that RowBinary can represent are sent without local ClickHouse domain checks and follow ClickHouse's native conversion or rejection behavior.
• Dotted nested subcolumns can be renamed only within the same parent, matching ClickHouse's documented RENAME COLUMN limitation.
• Tombstone cleanup and OPTIMIZE ... FINAL CLEANUP are operator-managed.

DuckLake

DuckLake writes through DuckDB to a file or PostgreSQL catalog and local or object storage.

Limitations

• Data storage URLs currently support file, s3, and gs schemes.
• Deployments must account for the required DuckDB extensions and the catalog, storage, and maintenance services they configure.
• Primary-key sorting requires source tables to have a primary key; other sorting modes do not add that requirement.
• DuckLake adds streaming columns as nullable. Supported literal defaults are stored as metadata; ETL does not emit DEFAULT(NULL) when no source default exists, although a catalog client may display an omitted default as NULL.

Snowflake

Snowflake uses direct Snowpipe Streaming and key-pair authentication.

Limitations

• Setup requires a Snowflake user and role with the documented warehouse, database, and schema privileges.
• The effective QUOTED_IDENTIFIERS_IGNORE_CASE parameter must be FALSE. Connectivity validation rejects other settings because ETL preserves quoted source column names exactly. See Snowflake identifier resolution.
• Schema evolution support is still in progress. Only defaults that ETL can translate safely at table creation or column addition are applied. Later existing-column default changes are skipped because Snowflake default provenance cannot be inferred safely.
• Validate channel recovery, committed offsets, and account-specific resource limits before production use.

Iceberg

The Apache Iceberg implementation is retained for compatibility and is not recommended for new deployments.

Limitations

• Schema-change DDL is not supported. A newer relation schema is rejected even when only a default, nullability, mapped-equivalent type, or publication mask changed. Existing tables must match the exact generated Iceberg schema.
• Apache Iceberg supports native schema evolution, but this deprecated ETL adapter does not implement it and does not translate PostgreSQL defaults.
• The destination is deprecated and is not receiving the same product investment as the active destination implementations.

Custom destinations and upstream support

Adding an official destination is a long-term maintenance commitment. It requires durable and idempotent writes, schema-evolution behavior, restart and recovery coverage, integration infrastructure, credential handling, ongoing dependency updates, and operational support. For that reason, maintainers are careful about accepting new destination implementations upstream. Open an issue or discussion before investing in an upstream implementation; acceptance is not guaranteed.

You do not need to contribute a destination upstream to use ETL. Implement the Destination trait in your own Rust project, wire it into Pipeline::new, and maintain it alongside your application. See Custom Implementations for a complete example.