Databricks Integration

Kadoa can publish workflow output as read-only Delta tables and share them with your Databricks workspace through Delta Sharing. If you do not use Databricks, Kadoa can also provide an open Delta Sharing recipient token for compatible clients.

This connector is set up by Kadoa. It is separate from direct S3 access through the Cloud Storage connector.

Required Inputs

Choose the recipient mode that matches your environment.

Databricks Recipient

Use this mode if you have a Unity Catalog-enabled Databricks workspace.

Input	Description
Sharing identifier	The Databricks sharing identifier for the workspace that should receive the share
Workflows	All workflows by default, or a selected workflow list
Activity log	Enabled by default, unless you ask Kadoa to disable it

Your Databricks administrator can find the sharing identifier in the Databricks Delta Sharing recipient setup flow.

Token Recipient

Use this mode if you want to read the share with an open Delta Sharing client instead of a Databricks workspace.

Input	Description
Secure contact	The person or channel that should receive the one-time activation details
Workflows	All workflows by default, or a selected workflow list
Activity log	Enabled by default, unless you ask Kadoa to disable it

Token credentials are activation material. Kadoa does not store the credential file or bearer token in the connector configuration. Token lifetime is currently up to 365 days.

Setup Flow

Kadoa creates a Databricks connector for your team and selected workflows.
Kadoa provisions a dedicated recipient, share, schema, and storage path.
Kadoa verifies the provider-side share and grant setup before customer data is published.
After each successful workflow run, Kadoa stages and validates public data, publishes Delta tables, and updates the share.
For initial rollout, Kadoa publishes and verifies one live workflow delivery before running the full historical backfill.

Shared Tables

Each connector exposes physical Delta tables. Internal staging tables and raw provider tables are not shared.

Table	Purpose
`WF_<id>__V<n>`	Versioned workflow output for a specific schema version
`WF_<id>__LATEST`	Latest workflow output table for quick exploration
Workflow metadata tables	Workflow, schema version, and field mapping metadata
`ACTIVITY_LOG`	Customer-visible activity events, when activity sharing is enabled

Kadoa writes only public workflow fields into shared tables. Private fields, including fields whose names start with _, are excluded before publish.

Schema Changes

Kadoa keeps historical schema versions available. When a workflow schema changes, Kadoa publishes a new WF_<id>__V<n> table and updates WF_<id>__LATEST only after the version table and share checks pass. WF_<id>__LATEST follows the newest validated schema; older rows remain available in their WF_<id>__V<n> tables and metadata rows, and __LATEST may be rebuilt for the new schema. Use versioned tables for downstream models that need a stable schema. Use __LATEST for exploration or workloads that intentionally follow the newest schema.

Activity Log

Activity sharing is enabled by default. The activity table uses a fixed public column allowlist and does not include user emails, internal values, raw change payloads, or private workflow fields.

Backfill

For initial setup, Kadoa backfills historical workflow output for all scoped workflows by default. Activity history is also backfilled when activity sharing is enabled. Backfill uses the same delivery path as normal workflow runs, so data is staged, validated, published, and checked before being marked shared.

Freshness And Delivery State

New workflow runs are expected to appear after Kadoa stages, validates, and publishes the Delta tables. Delivery is asynchronous and can take longer during large backfills or schema changes. Kadoa tracks delivery state internally, including staging, schema, load, publish, share, and proof failures.

Disable And Revocation

Disabling a Databricks connector stops future deliveries. Revocation removes the recipient’s access to the share. Historical Delta data is not deleted by default.

Example Queries

Query the latest workflow output:

select *
from wf_<id>__latest
limit 100;

Query a fixed schema version:

select *
from wf_<id>__v1
limit 100;

Query activity events:

select occurred_at, event_title, resource_name, workflow_id, request_source
from activity_log
order by occurred_at desc;

Foundations

Security & Compliance

Build with UI

Build with Code

Integrations

Required Inputs

Databricks Recipient

Token Recipient

Setup Flow

Shared Tables

Schema Changes

Activity Log

Backfill

Freshness And Delivery State

Disable And Revocation

Example Queries

​Required Inputs

​Databricks Recipient

​Token Recipient

​Setup Flow

​Shared Tables

​Schema Changes

​Activity Log

​Backfill

​Freshness And Delivery State

​Disable And Revocation

​Example Queries

Required Inputs

Databricks Recipient

Token Recipient

Setup Flow

Shared Tables

Schema Changes

Activity Log

Backfill

Freshness And Delivery State

Disable And Revocation

Example Queries