Kadoa automatically pushes workflow results to your cloud storage after each run. This is ideal for data pipelines, warehousing, and archival.
Supported Providers
- Amazon S3
- Google Cloud Storage
- Azure Blob Storage
Choose which formats to export:
| Format | Description | Use Case |
|---|
| Parquet | Columnar, compressed | Analytics, Snowflake, Spark |
| JSONL | JSON Lines, one record per line | Streaming, log processing |
| JSON | JSON with metadata envelope | APIs, integrations |
| CSV | Comma-separated values | Excel, legacy systems |
Parquet is recommended for analytics workloads. It’s compressed and optimized for columnar queries.
File Organization
Data is organized by team, workflow, and run:
s3://your-bucket/kadoa/{teamId}/{workflowId}/{runDatetimeSafe}-{runId}/
├── data.parquet
├── data.jsonl
├── data.json
└── data.csv
Path Variables
| Variable | Description | Example |
|---|
{teamId} | Your team UUID | a1b2c3d4-5e6f-7a8b-9c0d-e1f2a3b4c5d6 |
{workflowId} | Workflow identifier | b2c3d4e5-6f7a-8b9c-0d1e-f2a3b4c5d6e7 |
{runId} | Run identifier | c3d4e5f6-7a8b-9c0d-1e2f-a3b4c5d6e7f8 |
{runDatetimeSafe} | Filename-safe datetime | 2025-01-15_10-30-00Z |
{runDate} | Run date | 2025-01-15 |
S3 Setup
Submit a request through the Support Center to configure the data connector with:
- Bucket name
- Region
- Access method (bucket policy or IAM credentials)
- Desired export formats
Export Behavior
Automatic Push
After each workflow run completes successfully:
- Data is converted to requested formats
- Files are uploaded to your bucket
- Export is logged for auditing
Retry Logic
Failed uploads are automatically retried:
- 3 attempts with exponential backoff (1s, 2s, 4s)
- Failures are logged and can trigger alerts
Each uploaded file includes S3 metadata:
x-kadoa-workflow-id: Workflow identifierx-kadoa-job-id: Job identifierx-kadoa-format: File format
Additional Fields
You can enrich exported rows with extra metadata columns. This is available for CSV, JSONL, and JSON formats (not Parquet).
Each additional field has a custom name (the column header) and a value that can be:
- A static string — e.g.
Kadoa or production
- A dynamic variable — e.g.
{workflowId} or {runDate}, resolved at export time
Added fields appear as extra columns after the existing data columns. You can configure different additional fields per export format (CSV, JSONL, JSON).
Examples
| Field Name | Value | Result |
|---|
source | Kadoa | Every row gets source = "Kadoa" |
workflow | {workflowId} | Every row gets workflow = "b2c3d4e5-..." |
exportedAt | {runDatetime} | Every row gets exportedAt = "2025-01-15T10:30:00.000Z" |
Available Variables
| Variable | Description | Example |
|---|
{teamId} | Your team UUID | a1b2c3d4-5e6f-7a8b-9c0d-e1f2a3b4c5d6 |
{workflowId} | Workflow identifier | b2c3d4e5-6f7a-8b9c-0d1e-f2a3b4c5d6e7 |
{runId} | Run identifier | c3d4e5f6-7a8b-9c0d-1e2f-a3b4c5d6e7f8 |
{runDate} | Run date | 2025-01-15 |
{runDatetime} | ISO 8601 datetime | 2025-01-15T10:30:00.000Z |
Use with Snowflake
Cloud storage is the foundation for Snowflake External Tables.
Kadoa → S3 → Snowflake External Table
Use with Other Data Warehouses
The same S3 data can feed other warehouses:
| Warehouse | Integration Method |
|---|
| Snowflake | External Tables |
| BigQuery | External Tables or Data Transfer |
| Redshift | COPY command or Spectrum |
| Databricks | Direct S3 access |
Next Steps
