> ## Documentation Index
> Fetch the complete documentation index at: https://docs.zwiron.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Transforms

> Reshape, filter, hash, and compute columns in transit from source to destination — per table in Zwiron.

**Transforms** let you modify data in transit before it lands at the destination. They are configured per table, per column, directly on each pipeline — no separate ETL tool needed.

Transforms are applied **in the agent or hosted engine** after data is read from the source and before it is written to the destination. Your source is never modified.

***

## Column mapping

Every column from the source can be:

### Rename (dest)

Write a column to the destination under a different name without changing the source.

```
Source column: cust_id → Destination column: customer_id
Source column: ts      → Destination column: created_at
```

### Drop

Exclude a column entirely — it will not be written to the destination.

```
Drop: ssn, credit_card_number, internal_notes
```

Common use: strip PII before data reaches the warehouse.

### Type cast

Change the Arrow type of a column at write time. Only safe (lossless) widening casts are allowed — Zwiron prevents lossy casts that would silently lose data.

| Allowed cast groups      | Examples                            |
| ------------------------ | ----------------------------------- |
| Integer widening         | `int32` → `int64`, `int8` → `int32` |
| Float widening           | `float32` → `float64`               |
| Integer to float         | `int64` → `float64`                 |
| Any to string            | `int64` → `utf8`                    |
| Date/timestamp precision | `timestamp[s]` → `timestamp[us]`    |

```
Source: status int8 → Cast to: utf8 (for string-typed destinations)
Source: event_time timestamp[s] → Cast to: timestamp[us]
```

### Hash

Replace a column's value with its SHA-256 hash before writing. Use this to pseudonymize PII while preserving referential integrity.

```
Source: email → Hash algorithm: SHA-256 → Destination: hashed email string
```

Rows with the same source value will produce the same hash, so JOINs across tables still work.

***

## Row filters

Filter which rows are written to the destination based on column values. Rows that don't match the filter are dropped silently — they are never written.

### Supported operators

| Operator          | Description                         |
| ----------------- | ----------------------------------- |
| `=`               | Equals                              |
| `!=`              | Not equals                          |
| `>` `>=` `<` `<=` | Numeric/date comparison             |
| `in`              | Value is in a list                  |
| `not in`          | Value is not in a list              |
| `like`            | String pattern match (`%` wildcard) |
| `not like`        | String pattern does not match       |
| `is null`         | Column is null                      |
| `is not null`     | Column is not null                  |

### Examples

Only sync active users:

```
column: status   op: =        value: "active"
```

Only sync records from the last year:

```
column: created_at   op: >=   value: "2025-01-01"
```

Exclude test accounts:

```
column: email   op: not like   value: "%@test.%"
```

### Multiple filters

Multiple filter conditions are ANDed together by default. Set a condition's `or` flag to OR it with the previous condition.

***

## Computed columns

Add new columns to the destination that don't exist in the source. Computed columns are derived from existing source columns using a built-in function — applied in the agent or hosted engine before the row is written.

Some common examples:

| Function                      | Example                                                           |
| ----------------------------- | ----------------------------------------------------------------- |
| `string_upper(col)`           | `string_upper(country_code)` → `country_upper`                    |
| `coalesce(a, b)`              | `coalesce(phone, mobile)` → `contact`                             |
| `sha256(col)`                 | `sha256(email)` → `hashed_email`                                  |
| `json_extract(col, "$.path")` | `json_extract(metadata, "$.plan")` → `plan`                       |
| `year(ts)`                    | `year(created_at)` → `signup_year`                                |
| `if_else(cond, then, else)`   | `if_else(is_trial, "trial", "paid")` → `plan_type`                |
| `date_diff(a, b, unit)`       | `date_diff(created_at, resolved_at, "hour")` → `resolution_hours` |

Zwiron has over 90 built-in functions across math, string, date & time, conditional, aggregate, window, regex, JSON, and hashing categories.

<Card title="Transform Function Reference" icon="function" href="/concepts/transform-functions">
  Browse every available function with descriptions and examples.
</Card>

***

## Flatten

For source tables with JSON or nested object columns, enable **Flatten** to automatically expand the top-level keys of a JSON column into individual destination columns.

```
Source column: metadata (JSON string)
Value: {"plan": "growth", "trial": false, "seats": 10}

After flatten:
  metadata.plan   = "growth"
  metadata.trial  = false
  metadata.seats  = 10
```

***

## Where to configure transforms

Transforms are configured per table when creating or editing a pipeline:

1. Go to **Jobs** → select a job → **Edit**
2. Go to the **Mapping** tab
3. Select a table to configure its column mappings, filters, and computed columns
4. Save — changes take effect on the next run

***

## Order of operations

For each row read from source, Zwiron applies transforms in this order:

1. **Row filter** — rows that don't match are dropped
2. **Column drop** — excluded columns are removed
3. **Hash** — hashed columns are replaced with their hash
4. **Type cast** — column types are coerced
5. **Column rename** — column names are changed
6. **Computed columns** — new columns are derived
7. **Flatten** — JSON columns are expanded
8. Row is written to destination

***

## Transforms and dbt

Zwiron's transforms handle the data movement layer. For complex business logic — multi-table joins, window functions, aggregations — the standard pattern is:

```
Source → Zwiron → Raw schema in warehouse → dbt → Analytics schema
```

Zwiron delivers raw, clean data. dbt handles the business logic. This keeps both layers independent and testable.
