Cloud Storage

Kadoa automatically pushes workflow results to your cloud storage after each run. This is ideal for data pipelines, warehousing, and archival.

Supported Providers

Amazon S3
Google Cloud Storage
Azure Blob Storage

Data Formats

Choose which formats to export:

Format	Description	Use Case
Parquet	Columnar, compressed	Analytics, Snowflake, Spark
JSONL	JSON Lines, one record per line	Streaming, log processing
JSON	JSON with metadata envelope	APIs, integrations
CSV	Comma-separated values	Excel, legacy systems

Parquet is recommended for analytics workloads. It’s compressed and optimized for columnar queries.

File Organization

Data is organized by team, workflow, and run:

s3://your-bucket/kadoa/{teamId}/{workflowId}/{runDatetimeSafe}-{runId}/
├── data.parquet
├── data.jsonl
├── data.json
└── data.csv

Path Variables

Variable	Description	Example
`{teamId}`	Your team UUID	`a1b2c3d4-5e6f-7a8b-9c0d-e1f2a3b4c5d6`
`{workflowId}`	Workflow identifier	`b2c3d4e5-6f7a-8b9c-0d1e-f2a3b4c5d6e7`
`{runId}`	Run identifier	`c3d4e5f6-7a8b-9c0d-1e2f-a3b4c5d6e7f8`
`{runDatetimeSafe}`	Filename-safe datetime	`2025-01-15_10-30-00Z`
`{runDatetime}`	ISO 8601 datetime	`2025-01-15T10:30:00.000Z`
`{runDate}`	Run date	`2025-01-15`

{runDatetimeSafe} replaces colons with hyphens and the T separator with an underscore, making it safe for systems that treat colons as illegal path characters (e.g., Windows filesystems, some S3 sync tools). Use {runDatetime} when you need a standard ISO 8601 timestamp and your tooling handles colons correctly.

S3 Setup

Submit a request through the Support Center to configure the data connector with:

Bucket name
Region
Access method (bucket policy or IAM credentials)
Desired export formats

Export Behavior

Automatic Push

After each workflow run completes successfully:

Data is converted to requested formats
Files are uploaded to your bucket
Export is logged for auditing

Retry Logic

Failed uploads are automatically retried:

3 attempts with exponential backoff (1s, 2s, 4s)
Failures are logged and can trigger alerts

Metadata

Each uploaded file includes S3 metadata:

x-kadoa-workflow-id: Workflow identifier
x-kadoa-job-id: Job identifier
x-kadoa-format: File format

Additional Fields

You can enrich exported rows with extra metadata columns. This is available for CSV, JSONL, and JSON formats (not Parquet). Each additional field has a custom name (the column header) and a value that can be:

A static string, e.g. Kadoa or production
A dynamic variable, e.g. {workflowId} or {runDate}, resolved at export time

Added fields appear as extra columns after the existing data columns. You can configure different additional fields per export format (CSV, JSONL, JSON).

Examples

Field Name	Value	Result
`source`	`Kadoa`	Every row gets `source = "Kadoa"`
`workflow`	`{workflowId}`	Every row gets `workflow = "b2c3d4e5-..."`
`workflowName`	`{workflowName}`	Every row gets `workflowName = "Product Monitor"`
`exportedAt`	`{runDatetime}`	Every row gets `exportedAt = "2025-01-15T10:30:00.000Z"`

Available Variables

Variable	Description	Example
`{teamId}`	Your team UUID	`a1b2c3d4-5e6f-7a8b-9c0d-e1f2a3b4c5d6`
`{workflowId}`	Workflow identifier	`b2c3d4e5-6f7a-8b9c-0d1e-f2a3b4c5d6e7`
`{workflowName}`	Workflow name	`Product Monitor`
`{runId}`	Run identifier	`c3d4e5f6-7a8b-9c0d-1e2f-a3b4c5d6e7f8`
`{runDate}`	Run date	`2025-01-15`
`{runDatetime}`	ISO 8601 datetime	`2025-01-15T10:30:00.000Z`

Use with Snowflake via Snowpipe

Cloud storage can feed a customer-managed Snowflake via Snowpipe pipeline. Kadoa also offers a native Snowflake integration that does not require customer-managed S3 or Snowpipe.

Kadoa → S3 → Snowpipe → Snowflake Table

Use with Other Data Warehouses

The same S3 data can feed other warehouses:

Warehouse	Integration Method
Snowflake	Snowpipe auto-ingestion
BigQuery	External Tables or Data Transfer
Redshift	COPY command or Spectrum
Databricks	Direct S3 access

Kadoa also offers a native Databricks integration through Delta Sharing. Use Cloud Storage when you want to own the storage path or ingestion pipeline; use Databricks when you want Kadoa to publish and share Delta tables directly.

Foundations

Security & Compliance

Build with UI

Build with Code

Integrations

Supported Providers

Data Formats

File Organization

Path Variables

S3 Setup

Export Behavior

Automatic Push

Retry Logic

Metadata

Additional Fields

Examples

Available Variables

Use with Snowflake via Snowpipe

Use with Other Data Warehouses

Next Steps

​Supported Providers

​Data Formats

​File Organization

​Path Variables

​S3 Setup

​Export Behavior

​Automatic Push

​Retry Logic

​Metadata

​Additional Fields

​Examples

​Available Variables

​Use with Snowflake via Snowpipe

​Use with Other Data Warehouses

​Next Steps

Supported Providers

Data Formats

File Organization

Path Variables

S3 Setup

Export Behavior

Automatic Push

Retry Logic

Metadata

Additional Fields

Examples

Available Variables

Use with Snowflake via Snowpipe

Use with Other Data Warehouses

Next Steps