Documentation Index
Fetch the complete documentation index at: https://docs.kadoa.com/llms.txt
Use this file to discover all available pages before exploring further.
Kadoa automatically pushes workflow results to your cloud storage after each run. This is ideal for data pipelines, warehousing, and archival.
Supported Providers
- Amazon S3
- Google Cloud Storage
- Azure Blob Storage
Choose which formats to export:
| Format | Description | Use Case |
|---|
| Parquet | Columnar, compressed | Analytics, Snowflake, Spark |
| JSONL | JSON Lines, one record per line | Streaming, log processing |
| JSON | JSON with metadata envelope | APIs, integrations |
| CSV | Comma-separated values | Excel, legacy systems |
Parquet is recommended for analytics workloads. It’s compressed and optimized for columnar queries.
File Organization
Data is organized by team, workflow, and run:
s3://your-bucket/kadoa/{teamId}/{workflowId}/{runDatetimeSafe}-{runId}/
├── data.parquet
├── data.jsonl
├── data.json
└── data.csv
Path Variables
| Variable | Description | Example |
|---|
{teamId} | Your team UUID | a1b2c3d4-5e6f-7a8b-9c0d-e1f2a3b4c5d6 |
{workflowId} | Workflow identifier | b2c3d4e5-6f7a-8b9c-0d1e-f2a3b4c5d6e7 |
{runId} | Run identifier | c3d4e5f6-7a8b-9c0d-1e2f-a3b4c5d6e7f8 |
{runDatetimeSafe} | Filename-safe datetime | 2025-01-15_10-30-00Z |
{runDatetime} | ISO 8601 datetime | 2025-01-15T10:30:00.000Z |
{runDate} | Run date | 2025-01-15 |
{runDatetimeSafe} replaces colons with hyphens and the T separator with an underscore, making it safe for systems that treat colons as illegal path characters (e.g., Windows filesystems, some S3 sync tools). Use {runDatetime} when you need a standard ISO 8601 timestamp and your tooling handles colons correctly.
S3 Setup
Submit a request through the Support Center to configure the data connector with:
- Bucket name
- Region
- Access method (bucket policy or IAM credentials)
- Desired export formats
Export Behavior
Automatic Push
After each workflow run completes successfully:
- Data is converted to requested formats
- Files are uploaded to your bucket
- Export is logged for auditing
Retry Logic
Failed uploads are automatically retried:
- 3 attempts with exponential backoff (1s, 2s, 4s)
- Failures are logged and can trigger alerts
Each uploaded file includes S3 metadata:
x-kadoa-workflow-id: Workflow identifierx-kadoa-job-id: Job identifierx-kadoa-format: File format
Additional Fields
You can enrich exported rows with extra metadata columns. This is available for CSV, JSONL, and JSON formats (not Parquet).
Each additional field has a custom name (the column header) and a value that can be:
- A static string — e.g.
Kadoa or production
- A dynamic variable — e.g.
{workflowId} or {runDate}, resolved at export time
Added fields appear as extra columns after the existing data columns. You can configure different additional fields per export format (CSV, JSONL, JSON).
Examples
| Field Name | Value | Result |
|---|
source | Kadoa | Every row gets source = "Kadoa" |
workflow | {workflowId} | Every row gets workflow = "b2c3d4e5-..." |
workflowName | {workflowName} | Every row gets workflowName = "Product Monitor" |
exportedAt | {runDatetime} | Every row gets exportedAt = "2025-01-15T10:30:00.000Z" |
Available Variables
| Variable | Description | Example |
|---|
{teamId} | Your team UUID | a1b2c3d4-5e6f-7a8b-9c0d-e1f2a3b4c5d6 |
{workflowId} | Workflow identifier | b2c3d4e5-6f7a-8b9c-0d1e-f2a3b4c5d6e7 |
{workflowName} | Workflow name | Product Monitor |
{runId} | Run identifier | c3d4e5f6-7a8b-9c0d-1e2f-a3b4c5d6e7f8 |
{runDate} | Run date | 2025-01-15 |
{runDatetime} | ISO 8601 datetime | 2025-01-15T10:30:00.000Z |
Use with Snowflake
Cloud storage is the foundation for Snowflake integration via Snowpipe.
Kadoa → S3 → Snowpipe → Snowflake Table
Use with Other Data Warehouses
The same S3 data can feed other warehouses:
| Warehouse | Integration Method |
|---|
| Snowflake | Snowpipe auto-ingestion |
| BigQuery | External Tables or Data Transfer |
| Redshift | COPY command or Spectrum |
| Databricks | Direct S3 access |
Next Steps
