Databricks Connector

Use Databricks as a destination (load data into Delta tables on Unity Catalog) or a source (read existing tables). Zwiron connects through a Databricks SQL warehouse using the Databricks SQL driver (SQL warehouse / JDBC-style connectivity). Writes use Unity Catalog Volumes for Parquet staging, then COPY INTO — the lakehouse equivalent of Snowflake PUT + COPY.

Supported sync modes

As destination	Write modes
	Append, Upsert, Replace

As source	Full Refresh	Incremental
	✅	✅

When Databricks is the destination, source connectors that support CDC (PostgreSQL, MySQL, MongoDB, SQL Server, Oracle, and others) can stream changes into Databricks tables.

Connection details

Field	Required	Description
Host	Yes	Workspace server hostname (e.g. `dbc-xxxx.cloud.databricks.com` or `adb-xxx.azuredatabricks.net`)
Warehouse ID	Yes	SQL warehouse ID (Zwiron builds `/sql/1.0/warehouses/{id}` unless you override HTTP path)
Token	Yes	Personal access token or service principal token
Catalog	No	Unity Catalog name. Prefer a real UC catalog (e.g. `main`). Default in product: `hive_metastore` — Volume staging requires Unity Catalog; use a UC catalog for destination loads.
Schema	No	Target schema (default: `default`)
HTTP path	No	Override only if your warehouse uses a non-standard path

How to connect

Step 1 — SQL warehouse connection details

In Databricks, open SQL Warehouses.
Select (or create) a warehouse and start it if it is stopped.
Open Connection details.
Copy Server hostname. The HTTP path usually includes the warehouse ID (e.g. /sql/1.0/warehouses/<id>).

Step 2 — Access token

Create a token for a user or service principal that has the privileges below:

Settings → Developer → Access tokens → Generate new token, or
Create a service principal and generate a token for it (recommended for production).

Paste the token into Zwiron. Prefer a dedicated identity with least privilege — not a workspace admin personal token.

Step 3 — Connect in Zwiron

Go to Connections → Add Connection → Databricks
Fill in host, warehouse ID, token, Unity Catalog name, and schema
Click Test Connection
Save the connection

Privileges (Unity Catalog)

Grant the connecting identity at least:

Privilege	On	Purpose
`USE CATALOG`	Catalog	Resolve the three-level namespace
`USE SCHEMA`	Schema	Read/write within the schema
`CREATE SCHEMA`	Catalog	Optional — Zwiron can create the target schema if missing
`CREATE TABLE`	Schema	Create destination tables
`MODIFY`	Tables	Insert / merge / replace
`SELECT`	Tables	Read for source syncs and validation
`CREATE VOLUME`	Schema	Create the staging volume `__zwiron_staging`
`WRITE VOLUME` / `READ VOLUME`	Volume	PUT Parquet files and `COPY INTO`

Example (adjust names):

GRANT USE CATALOG ON CATALOG main TO `zwiron-sp`;
GRANT USE SCHEMA ON SCHEMA main.analytics TO `zwiron-sp`;
GRANT CREATE TABLE, CREATE VOLUME ON SCHEMA main.analytics TO `zwiron-sp`;
-- After tables exist:
GRANT SELECT, MODIFY ON TABLE main.analytics.orders TO `zwiron-sp`;

How Zwiron writes to Databricks

For destination loads, Zwiron:

Ensures the catalog.schema exists (CREATE SCHEMA IF NOT EXISTS)
Creates a Unity Catalog volume __zwiron_staging when needed
Writes Parquet files and PUTs them to /Volumes/{catalog}/{schema}/__zwiron_staging/...
Runs COPY INTO into the target table (or a temp table for upsert)
For upsert, merges into the target with primary keys, then cleans staging files

This path requires Unity Catalog volumes. Managed Delta tables in your catalog/schema are the default landing target.

Schema mapping

Source type	Databricks / Delta type
INTEGER / INT	INT
BIGINT	BIGINT
FLOAT	FLOAT
DOUBLE	DOUBLE
DECIMAL / NUMERIC	DECIMAL(p,s)
VARCHAR / TEXT / JSON	STRING
BOOLEAN	BOOLEAN
DATE	DATE
TIMESTAMP	TIMESTAMP
BYTES / BLOB	BINARY
ARRAY / MAP / STRUCT	STRING (JSON text)

Zwiron creates tables automatically when they do not exist and can evolve schemas (add/widen columns) according to job settings.

Cloud coverage

The same connection shape works on Databricks workspaces hosted on AWS, Azure, and GCP. Use the hostname and SQL warehouse from that cloud’s workspace.

Troubleshooting

Permission denied creating volume or COPY INTO

Confirm the identity has CREATE VOLUME on the schema and write access on __zwiron_staging. Legacy Hive-only setups without volume support cannot use the current staging path — use a Unity Catalog–enabled workspace.

Warehouse stopped / connection timeout

Start the SQL warehouse in Databricks, then retry Test Connection. Auto-stop warehouses must be running for the first sync.

Invalid access token

Regenerate the PAT or service principal token. Ensure the token belongs to an identity that can use the selected warehouse (CAN USE).

Catalog or schema not found

Create the catalog/schema in Unity Catalog, or grant CREATE SCHEMA so Zwiron can create the schema on connect. Check that Catalog in Zwiron matches the UC name exactly.

Connectors overview
PostgreSQL → Databricks pipelines
HubSpot → warehouse sync
Network access / egress IPs for hosted mode

Getting Started

Guides

Core Concepts

Pipelines

Connectors

Platform

Support

Databricks Connector

Supported sync modes

Connection details

How to connect

Step 1 — SQL warehouse connection details

Step 2 — Access token

Step 3 — Connect in Zwiron

Privileges (Unity Catalog)

How Zwiron writes to Databricks

Schema mapping

Cloud coverage

Troubleshooting

​Supported sync modes

​Connection details

​How to connect

​Step 1 — SQL warehouse connection details

​Step 2 — Access token

​Step 3 — Connect in Zwiron

​Privileges (Unity Catalog)

​How Zwiron writes to Databricks

​Schema mapping

​Cloud coverage

​Troubleshooting

​Related

Supported sync modes

Connection details

How to connect

Step 1 — SQL warehouse connection details

Step 2 — Access token

Step 3 — Connect in Zwiron

Privileges (Unity Catalog)

How Zwiron writes to Databricks

Schema mapping

Cloud coverage

Troubleshooting

Related