Transforms

Transforms let you modify data in transit before it lands at the destination. They are configured per table, per column, directly on each pipeline — no separate ETL tool needed. Transforms are applied in the agent or hosted engine after data is read from the source and before it is written to the destination. Your source is never modified.

Column mapping

Every column from the source can be:

Rename (dest)

Write a column to the destination under a different name without changing the source.

Source column: cust_id → Destination column: customer_id
Source column: ts      → Destination column: created_at

Drop

Exclude a column entirely — it will not be written to the destination.

Drop: ssn, credit_card_number, internal_notes

Common use: strip PII before data reaches the warehouse.

Type cast

Change the Arrow type of a column at write time. Only safe (lossless) widening casts are allowed — Zwiron prevents lossy casts that would silently lose data.

Allowed cast groups	Examples
Integer widening	`int32` → `int64`, `int8` → `int32`
Float widening	`float32` → `float64`
Integer to float	`int64` → `float64`
Any to string	`int64` → `utf8`
Date/timestamp precision	`timestamp[s]` → `timestamp[us]`

Source: status int8 → Cast to: utf8 (for string-typed destinations)
Source: event_time timestamp[s] → Cast to: timestamp[us]

Hash

Replace a column’s value with its SHA-256 hash before writing. Use this to pseudonymize PII while preserving referential integrity.

Source: email → Hash algorithm: SHA-256 → Destination: hashed email string

Rows with the same source value will produce the same hash, so JOINs across tables still work.

Row filters

Filter which rows are written to the destination based on column values. Rows that don’t match the filter are dropped silently — they are never written.

Supported operators

Operator	Description
`=`	Equals
`!=`	Not equals
`>` `>=` `<` `<=`	Numeric/date comparison
`in`	Value is in a list
`not in`	Value is not in a list
`like`	String pattern match (`%` wildcard)
`not like`	String pattern does not match
`is null`	Column is null
`is not null`	Column is not null

Examples

Only sync active users:

column: status   op: =        value: "active"

Only sync records from the last year:

column: created_at   op: >=   value: "2025-01-01"

Exclude test accounts:

column: email   op: not like   value: "%@test.%"

Multiple filters

Multiple filter conditions are ANDed together by default. Set a condition’s or flag to OR it with the previous condition.

Computed columns

Add new columns to the destination that don’t exist in the source. Computed columns are derived from existing source columns using a built-in function — applied in the agent or hosted engine before the row is written. Some common examples:

Function	Example
`string_upper(col)`	`string_upper(country_code)` → `country_upper`
`coalesce(a, b)`	`coalesce(phone, mobile)` → `contact`
`sha256(col)`	`sha256(email)` → `hashed_email`
`json_extract(col, "$.path")`	`json_extract(metadata, "$.plan")` → `plan`
`year(ts)`	`year(created_at)` → `signup_year`
`if_else(cond, then, else)`	`if_else(is_trial, "trial", "paid")` → `plan_type`
`date_diff(a, b, unit)`	`date_diff(created_at, resolved_at, "hour")` → `resolution_hours`

Zwiron has over 90 built-in functions across math, string, date & time, conditional, aggregate, window, regex, JSON, and hashing categories.

Transform Function Reference

Browse every available function with descriptions and examples.

Flatten

For source tables with JSON or nested object columns, enable Flatten to automatically expand the top-level keys of a JSON column into individual destination columns.

Source column: metadata (JSON string)
Value: {"plan": "growth", "trial": false, "seats": 10}

After flatten:
  metadata.plan   = "growth"
  metadata.trial  = false
  metadata.seats  = 10

Where to configure transforms

Transforms are configured per table when creating or editing a pipeline:

Go to Jobs → select a job → Edit
Go to the Mapping tab
Select a table to configure its column mappings, filters, and computed columns
Save — changes take effect on the next run

Order of operations

For each row read from source, Zwiron applies transforms in this order:

Row filter — rows that don’t match are dropped
Column drop — excluded columns are removed
Hash — hashed columns are replaced with their hash
Type cast — column types are coerced
Column rename — column names are changed
Computed columns — new columns are derived
Flatten — JSON columns are expanded
Row is written to destination

Transforms and dbt

Zwiron’s transforms handle the data movement layer. For complex business logic — multi-table joins, window functions, aggregations — the standard pattern is:

Source → Zwiron → Raw schema in warehouse → dbt → Analytics schema

Zwiron delivers raw, clean data. dbt handles the business logic. This keeps both layers independent and testable.

Getting Started

Guides

Core Concepts

Pipelines

Connectors

Platform

Support

Column mapping

Rename (dest)

Drop

Type cast

Hash

Row filters

Supported operators

Examples

Multiple filters

Computed columns

Transform Function Reference

Flatten

Where to configure transforms

Order of operations

Transforms and dbt

​Column mapping

​Rename (dest)

​Drop

​Type cast

​Hash

​Row filters

​Supported operators

​Examples

​Multiple filters

​Computed columns

Transform Function Reference

​Flatten

​Where to configure transforms

​Order of operations

​Transforms and dbt

Column mapping

Rename (dest)

Drop

Type cast

Hash

Row filters

Supported operators

Examples

Multiple filters

Computed columns

Flatten

Where to configure transforms

Order of operations

Transforms and dbt