# Answer an Assistant question
Source: https://docs.kadoa.com/api-reference/assistant/answer-question
post /v5/agent/answer
# Create or continue an Assistant session
Source: https://docs.kadoa.com/api-reference/assistant/create-or-continue-session
post /v5/agent
Creates a draft workflow-backed Assistant session. Explicit productType=realtime dispatches the Julie realtime lane after realtime notification settings are provisioned.
# Get Assistant pause state
Source: https://docs.kadoa.com/api-reference/assistant/get-pause-state
get /v5/agent/{sessionId}/pause-state
# Get Assistant strategy
Source: https://docs.kadoa.com/api-reference/assistant/get-strategy
get /v5/agent/data/{sessionId}/strategy
Returns strategy null when the session exists but no build strategy is available yet.
# Get Workflow Assistant timeline
Source: https://docs.kadoa.com/api-reference/assistant/get-workflow-timeline
get /v5/agent/workflows/{workflowId}/timeline
Returns user messages, Assistant messages, and clarification questions from the latest Lighthouse/Shelly or Julie customer thread. Internal and operations lanes are never selectable.
# Interrupt an Assistant session
Source: https://docs.kadoa.com/api-reference/assistant/interrupt-session
post /v5/agent/{sessionId}/interrupt
# Resume an Assistant session
Source: https://docs.kadoa.com/api-reference/assistant/resume-session
post /v5/agent/{sessionId}/resume
Resumes with the persona persisted on the session, preserving Lighthouse versus Julie behavior.
# Send a Workflow Assistant message
Source: https://docs.kadoa.com/api-reference/assistant/send-workflow-message
post /v5/agent/workflows/{workflowId}/assistant/messages
Bootstraps or reuses the workflow's customer Assistant session and enqueues update instructions without changing the workflow ID.
# Stop an Assistant session
Source: https://docs.kadoa.com/api-reference/assistant/stop-session
post /v5/agent/{sessionId}/stop
# Get session status
Source: https://docs.kadoa.com/api-reference/crawling/crawling-session-status
get /v4/crawl/{sessionId}/status
Get the current status of a crawling session
# Get session page content
Source: https://docs.kadoa.com/api-reference/crawling/get-crawled-page-meta
get /v4/crawl/{sessionId}/pages/{pageId}
Get the content of a specific page from a crawling session in HTML or Markdown format
# Get paginated session pages
Source: https://docs.kadoa.com/api-reference/crawling/get-crawled-pages
get /v4/crawl/{sessionId}/pages
Get a paginated list of pages for a crawling session
# Start crawl
Source: https://docs.kadoa.com/api-reference/crawling/start-crawling-session
post /v4/crawl/
Create a crawling configuration and start a session in one operation (equivalent to v4/crawl)
# Delete data quality rules for a field
Source: https://docs.kadoa.com/api-reference/data-quality/delete-field-rules
delete /v4/workflows/{workflowId}/schema-validation-rules/{fieldName}
Remove all data quality rules for a single schema field on a workflow
# Get data quality rules
Source: https://docs.kadoa.com/api-reference/data-quality/get-rules
get /v4/workflows/{workflowId}/schema-validation-rules
Retrieve the per-field data quality rules persisted on a workflow
# Upsert data quality rules
Source: https://docs.kadoa.com/api-reference/data-quality/upsert-rules
put /v4/workflows/{workflowId}/schema-validation-rules
Patch-merge the per-field data quality rules on a workflow; rules take effect on the next run
# Introduction
Source: https://docs.kadoa.com/api-reference/introduction
Kadoa is built as an API-first platform. Everything you can do on your Kadoa Dashboard is also possible with our flexible API.
## Quickstart
1. Generate your API key from [Settings > General](https://www.kadoa.com/settings)
2. Create your first workflow on [Kadoa.com](https://www.kadoa.com/add)
3. Fetch the data by using our API.
## Authentication
All API endpoints are authenticated using your workspace API key.
```json theme={null}
"headers": {
"x-api-key": "tk-your-api-key"
}
```
**First time using the API?** You need to generate your API key first:
1. Go to [Settings > General](https://www.kadoa.com/settings)
2. Click **Generate API Key**
3. Copy and securely store your key (it starts with `tk-`)
# Get all available locations
Source: https://docs.kadoa.com/api-reference/locations/get-all-locations
get /v4/locations
# Get all data changes
Source: https://docs.kadoa.com/api-reference/monitoring/get-all-changes
get /v4/changes
# Get data change by ID
Source: https://docs.kadoa.com/api-reference/monitoring/get-change-by-id
get /v4/changes/{changeId}
# Create a new notification channel
Source: https://docs.kadoa.com/api-reference/notifications/create-notification-channel
post /v5/notifications/channels
# Create a new notification settings
Source: https://docs.kadoa.com/api-reference/notifications/create-notification-settings
post /v5/notifications/settings
# Delete a notification channel
Source: https://docs.kadoa.com/api-reference/notifications/delete-notification-channel
delete /v5/notifications/channels/{channelId}
# Delete a notification settings
Source: https://docs.kadoa.com/api-reference/notifications/delete-notification-settings
delete /v5/notifications/settings/{settingsId}
# Retrieve a specific event type details
Source: https://docs.kadoa.com/api-reference/notifications/get-event-type
get /v5/notifications/event-types/{eventType}
# Retrieve all available notification event types
Source: https://docs.kadoa.com/api-reference/notifications/get-event-types
get /v5/notifications/event-types
# Retrieve a specific notification channel
Source: https://docs.kadoa.com/api-reference/notifications/get-notification-channel
get /v5/notifications/channels/{channelId}
# Retrieve all notification channels
Source: https://docs.kadoa.com/api-reference/notifications/get-notification-channels
get /v5/notifications/channels
# Retrieve a specific notification settings
Source: https://docs.kadoa.com/api-reference/notifications/get-notification-setting
get /v5/notifications/settings/{settingsId}
# Retrieve all notification settings
Source: https://docs.kadoa.com/api-reference/notifications/get-notification-settings
get /v5/notifications/settings
# Test a notification settings with mock data
Source: https://docs.kadoa.com/api-reference/notifications/test-notifications
post /v5/notifications/test
# Update a notification channel
Source: https://docs.kadoa.com/api-reference/notifications/update-notification-channel
put /v5/notifications/channels/{channelId}
# Update a notification settings
Source: https://docs.kadoa.com/api-reference/notifications/update-notification-settings
put /v5/notifications/settings/{settingsId}
# Create new schema
Source: https://docs.kadoa.com/api-reference/schemas/create-schema
post /v4/schemas/
Create a new data schema with specified fields and entity type
# Delete schema
Source: https://docs.kadoa.com/api-reference/schemas/delete-schema
delete /v4/schemas/{schemaId}
Delete a schema and all its revisions
# List all schemas
Source: https://docs.kadoa.com/api-reference/schemas/get-all-schemas
get /v4/schemas/
Retrieve all schemas accessible by the authenticated user
# Get schema by ID
Source: https://docs.kadoa.com/api-reference/schemas/get-schema-by-id
get /v4/schemas/{schemaId}
Retrieve a specific schema by its unique identifier
# Update existing schema
Source: https://docs.kadoa.com/api-reference/schemas/update-schema
put /v4/schemas/{schemaId}
Update schema metadata or create a new revision with updated fields
# Create support issue
Source: https://docs.kadoa.com/api-reference/support/create-support-issue
post /v4/support/issues
Create a new support ticket linked to a workflow or data change
# List support issues
Source: https://docs.kadoa.com/api-reference/support/get-support-issues
get /v4/support/issues
Retrieve all support issues for the current customer
# Get issue states
Source: https://docs.kadoa.com/api-reference/support/get-support-states
get /v4/support/states
Retrieve the available states for support issues
# Update support issue
Source: https://docs.kadoa.com/api-reference/support/update-support-issue
patch /v4/support/issues/{issueId}
Update the priority or status of an existing support issue
# Apply template update
Source: https://docs.kadoa.com/api-reference/templates/apply-update
post /v4/templates/{templateId}/apply
Apply a template version's settings to linked workflows
# Create template
Source: https://docs.kadoa.com/api-reference/templates/create-template
post /v4/templates/
Create a new template for reusing prompts, schemas, and settings across workflows
# Create template version
Source: https://docs.kadoa.com/api-reference/templates/create-version
post /v4/templates/{templateId}/versions
Publish a new version of a template with prompt, schema, validation, and notification settings
# Delete template
Source: https://docs.kadoa.com/api-reference/templates/delete-template
delete /v4/templates/{templateId}
Archive a template (soft delete)
# Get template
Source: https://docs.kadoa.com/api-reference/templates/get-template
get /v4/templates/{templateId}
Retrieve a template by ID, including its version history
# Link workflows to template
Source: https://docs.kadoa.com/api-reference/templates/link-workflows
post /v4/templates/{templateId}/link
Link one or more workflows to a template
# List template schemas
Source: https://docs.kadoa.com/api-reference/templates/list-schemas
get /v4/templates/{templateId}/schemas
Retrieve all schemas linked to a template
# List templates
Source: https://docs.kadoa.com/api-reference/templates/list-templates
get /v4/templates/
Retrieve all templates for the current team
# List template workflows
Source: https://docs.kadoa.com/api-reference/templates/list-workflows
get /v4/templates/{templateId}/workflows
Retrieve all workflows linked to a template
# Preview schema change
Source: https://docs.kadoa.com/api-reference/templates/preview-schema-change
post /v4/templates/{templateId}/preview-schema-change
Preview the data a schema change would produce for one linked workflow, without applying the template or running the workflow
# Save workflow as template
Source: https://docs.kadoa.com/api-reference/templates/save-from-workflow
post /v4/templates/from-workflow
Create a new template from an existing workflow's configuration
# Unlink workflows from template
Source: https://docs.kadoa.com/api-reference/templates/unlink-workflows
post /v4/templates/{templateId}/unlink
Unlink one or more workflows from a template
# Update template
Source: https://docs.kadoa.com/api-reference/templates/update-template
put /v4/templates/{templateId}
Update a template's name or description
# Create a variable
Source: https://docs.kadoa.com/api-reference/variables/create-a-variable
post /v4/variables/
Create a new key-value variable. Keys must be unique within the team/user scope. Variables can be referenced in workflow prompts using @variableKey syntax.
# Delete a variable
Source: https://docs.kadoa.com/api-reference/variables/delete-a-variable
delete /v4/variables/{variableId}
Permanently delete a variable by its ID. A secret bound to a connector returns 409 Conflict.
# Get a variable
Source: https://docs.kadoa.com/api-reference/variables/get-a-variable
get /v4/variables/{variableId}
Retrieve a specific variable by its ID.
# List all variables
Source: https://docs.kadoa.com/api-reference/variables/list-all-variables
get /v4/variables/
Retrieve all variables in the current team/user scope (max 100).
# Update a variable
Source: https://docs.kadoa.com/api-reference/variables/update-a-variable
patch /v4/variables/{variableId}
Update an existing variable. At least one field (key, value, or dataType) must be provided.
# Create a new workflow
Source: https://docs.kadoa.com/api-reference/workflows/create-a-new-workflow
post /v4/workflows
Create a workflow from natural-language instructions, an existing schema, an inline schema, raw-data output options, or a published template.
# Create workflow data export
Source: https://docs.kadoa.com/api-reference/workflows/create-workflow-data-export
get /v4/workflows/{workflowId}/data/export
Materializes the full result set as CSV, JSON, or Parquet to object storage and returns a self-authenticating
URL that can be opened by external clients (Claude Desktop, Claude for Excel, code execution sandboxes,
browsers) without an Authorization header. The URL is HMAC-signed and bound to the exportId, workflowId,
format and expiry.
# Create a workflow trigger
Source: https://docs.kadoa.com/api-reference/workflows/create-workflow-trigger
post /v4/workflows/{workflowId}/triggers
Create a trigger that fires when the source workflow emits an event.
The target must share the source ownership scope (same team for team
workflows, or the same user for personal workflows) and must not be a
realtime workflow. Superadmins are subject to the same scope check on
the public endpoint.
# Delete a workflow
Source: https://docs.kadoa.com/api-reference/workflows/delete-a-workflow
delete /v4/workflows/{workflowId}
# Delete a trigger
Source: https://docs.kadoa.com/api-reference/workflows/delete-workflow-trigger
delete /v4/workflows/{workflowId}/triggers/{triggerId}
# Download workflow data export
Source: https://docs.kadoa.com/api-reference/workflows/download-workflow-data-export
get /v4/workflows/{workflowId}/data/exports/{exportId}
Streams a file previously materialized via GET /v4/workflows/{workflowId}/data/export.
Sets Content-Disposition: attachment so browsers trigger a download.
# Get a list of workflows
Source: https://docs.kadoa.com/api-reference/workflows/get-all-workflows
get /v4/workflows
Retrieves a list of workflows with pagination and search capabilities
# Get the workflow run history
Source: https://docs.kadoa.com/api-reference/workflows/get-the-workflow-history
get /v4/workflows/{workflowId}/history
# Get workflow by ID
Source: https://docs.kadoa.com/api-reference/workflows/get-workflow-by-id
get /v4/workflows/{workflowId}
Retrieves detailed information about a specific workflow. This endpoint requires authentication and proper
team access permissions.
# Get workflow data by ID
Source: https://docs.kadoa.com/api-reference/workflows/get-workflow-data-by-id
get /v4/workflows/{workflowId}/data
# Get workflow quality metrics
Source: https://docs.kadoa.com/api-reference/workflows/get-workflow-quality-metrics
get /v4/workflows/{workflowId}/quality-metrics
Returns the data quality metric results for a workflow run, plus the run's total data
quality issue count.
Metrics are only available once the run has finished, before that the endpoint returns
`{ "results": null, "issueCount": null }`.
Each metric result carries a `status`:
- `OK`: the metric was computed, see `data` for it's details.
- `PENDING`: the metric has not been computed yet.
- `NOT_APPLICABLE`: the metric cannot be computed for this workflow, see `message`.
- `ERROR`: the metric computation failed, see `message`.
# Get a specific trigger
Source: https://docs.kadoa.com/api-reference/workflows/get-workflow-trigger
get /v4/workflows/{workflowId}/triggers/{triggerId}
# List destinations delivering this workflow's results
Source: https://docs.kadoa.com/api-reference/workflows/list-workflow-connectors
get /v4/workflows/{workflowId}/connectors
Returns the team's destinations that deliver this workflow after each run. Only deliverable destinations are listed (enabled, verified, not revoked).
# List workflow triggers
Source: https://docs.kadoa.com/api-reference/workflows/list-workflow-triggers
get /v4/workflows/{workflowId}/triggers
Get the triggers where the specified workflow is the source (`triggers`)
and the triggers where it is the target (`incoming`). Outgoing rows are
always returned; each carries `targetWorkflowName` when the caller can
access the target. Incoming rows are returned only when the caller can
access the source workflow, and each carries `sourceWorkflowName`.
`count` is the outgoing trigger count.
# Pause a workflow
Source: https://docs.kadoa.com/api-reference/workflows/pause-a-workflow
put /v4/workflows/{workflowId}/pause
# Resume or unpause a workflow
Source: https://docs.kadoa.com/api-reference/workflows/resume-a-workflow
put /v4/workflows/{workflowId}/resume
Unpauses a paused workflow, or resumes a preview/error workflow.
# Run a workflow
Source: https://docs.kadoa.com/api-reference/workflows/run-a-workflow
put /v4/workflows/{workflowId}/run
# Schedule a workflow
Source: https://docs.kadoa.com/api-reference/workflows/schedule-a-workflow
put /v4/workflows/{workflowId}/schedule
# Update workflow metadata
Source: https://docs.kadoa.com/api-reference/workflows/update-workflow-metadata
put /v4/workflows/{workflowId}/metadata
# Update a trigger
Source: https://docs.kadoa.com/api-reference/workflows/update-workflow-trigger
patch /v4/workflows/{workflowId}/triggers/{triggerId}
Update trigger properties (eventType, enabled)
# Change Detection
Source: https://docs.kadoa.com/docs/change-detection
How Kadoa detects and classifies data changes between workflow runs
To enable change detection, two things are **required**:
1. Your workflow must be scheduled (hourly, daily, etc.) or set to real-time mode. Configure scheduling in the [UI](/docs/ui/workflows/sources) or via [Code](/docs/sdk/workflows/create).
2. Your workflow must subscribe to the data change event. Without this subscription, change diffs will not be delivered.
Make sure both conditions are met for change tracking to function.
## Workflow Modes
| Mode | How It Works | Availability |
| - | - | - |
| **Scheduled** | Compares data between runs (hourly, daily, weekly) | All plans |
| **Real-time** | Continuously monitors for changes | All plans |
Both modes use the same change detection algorithm described below.
## Algorithm Overview
The change detection process follows these matching rules:
1. **Exact Matches**: Objects that are completely identical (with whitespace normalization) are unchanged.
2. **Key Matches**: When key fields are configured, objects with the same key are compared for updates.
3. **Leftovers**: Remaining objects are classified as added or removed.
## Matching Methods
### Key Field Matching
When **key fields** are configured in your schema, objects are matched based on these fields. **ALL key fields must match** for objects to be considered the same:
```json theme={null}
// Previous run
{
"id": "12345",
"title": "Product A",
"price": "$100"
}
// Current run
{
"id": "12345", // Key field matches
"title": "Product A Pro", // Changed
"price": "$150" // Changed
}
```
**Result**: `changed` - ID matches, so same object with updates
**Key Field Behavior**:
* Key fields defined with `isKey: true` in your [schema](/docs/workflows/schemas)
* Common key fields: `id`, `url`, `link`, `sku`
* Key fields should use scalar `STRING`, `NUMBER`, or `LINK` types
* If ANY key field changes, objects treated as `removed` + `added` (not `changed`)
* At least one key field is required to classify a record as `changed`
### Matching Without Key Fields
Without key fields, Kadoa only matches records that are exactly equal. A modified record is reported as one removed record and one added record because there is no reliable identity connecting the two versions.
```json theme={null}
// Previous run
{
"name": "John Doe",
"age": "25",
"city": "New York"
}
// Current run
{
"name": "John Doe", // Same
"age": "26", // Changed
"city": "New York" // Same
}
```
**Result**: one `removed` record and one `added` record. To receive a single `changed` record instead, configure `id` or another stable field with `isKey: true`.
## Change Types
| Type | Description |
| - | - |
| **Changed** | Object matched by configured key fields but some fields differ |
| **Added** | Object exists in current run but no match in previous run |
| **Removed** | Object existed in previous run but no match in current run |
## How Changes Appear
### In the UI
View detected changes directly in your workflow's run history:
1. Open your workflow
2. Click the **History** tab on the right-hand side
3. Find a run with data changes
4. Click on the **Diff** component to view changes
The diff view displays:
* Change type indicators (added, removed, changed)
* Visual diff highlighting for modified fields
* Timestamps for when changes were detected
![Data Changes View]()
See [Data Delivery](/docs/destinations/overview) for notification channels, APIs, and payload formats.
## Method Comparison
| Method | When Used | Logic | Best For |
| - | - | - | - |
| **Key Field Matching** | Set `isKey: true` in [schema](/docs/workflows/schemas) | ALL key fields must match exactly | Updates to existing records |
| **Exact Matching** | No key fields configured | Entire records must match | Added and removed records |
## Setup Data Change
### Fields
Select which fields from your [schema](/docs/workflows/schemas) to monitor. Only changes to selected fields will trigger notifications. By default, Kadoa monitor all fields.
### Change Types
Choose which types of changes to track:
| Type | Description |
| - | - |
| **Changed** | Existing record was modified; requires at least one schema key field |
| **Added** | New record appeared |
| **Removed** | Record no longer exists |
### Conditions
Add conditions to filter notifications based on field values. Combine multiple conditions using logical operators.
**Logical Operators:**
* **Any** (OR): Notify if any condition matches
* **All** (AND): Notify only if all conditions match
**Available Operators:**
| Operator | Description | Example |
| - | - | - |
| `contains` | Field value contains text | title contains "sale" |
| `does not contain` | Field value excludes text | title does not contain "expired" |
| `equals` | Exact match | status equals "active" |
| `does not equal` | Does not match | status does not equal "draft" |
| `greater than` | Numeric comparison | price greater than 100 |
| `less than` | Numeric comparison | stock less than 10 |
| `before` | Date is before value | published before 2025-01-01 |
| `after` | Date is after value | published after 2025-01-01 |
| `in the last` | Date within N days | published in the last 7 days |
| `is empty` | Field has no value | description is empty |
| `is not empty` | Field has a value | price is not empty |
### Date Filtering
Use `before`, `after`, or `in the last` operators to filter changes by date. Useful for monitoring news feeds or content where you only care about recent updates.
Your data must include a date field for date filtering to work.
**Supported Date Formats:**
* ISO: `2025-01-15` (recommended)
* US/EU: `01/15/2025`, `15/01/2025`
* Month names: `Jan 15, 2025`
* Timestamps: `2025-01-15T10:30:00Z`
## Retrieve Historical Changes
View change history for your monitoring workflows.
1. Open your workflow
2. Click the **History** tab
3. Select a run to view its changes
![Change History]()
The `/changes` endpoint returns all historical data changes. [View full API reference →](/api-reference/monitoring/get-all-changes)
```bash theme={null}
GET https://api.kadoa.com/v4/changes
```
Filter by workflow and date range:
```bash theme={null}
GET https://api.kadoa.com/v4/changes?startDate=2024-11-01T00:00:00Z&endDate=2024-11-30T23:59:59Z
```
### Response Format
Each change includes a `differences` array:
```json theme={null}
{
"differences": [
{
"type": "changed",
"fields": [
{
"key": "title",
"previousValue": "Old Title",
"value": "New Title"
}
]
}
]
}
```
**`differences` structure:**
* `type`: Change type (`added`, `removed`, or `changed`)
* `fields`: All fields of the object
**Field change properties:**
* `key`: Field name
* `value`: Current value
* `previousValue`: Previous value (only for `changed` type)
# Compliance Center
Source: https://docs.kadoa.com/docs/compliance-center
View compliance rules and access audit tools for your team's workflows
The Compliance Center gives your team visibility into the compliance rules applied to your workflows and quick access to audit tools.
To open it, go to **Settings** → **Compliance** in your dashboard sidebar.
For compliance concepts, certifications, and our policy, see [Compliance Overview](/docs/security/compliance).
## Quick Access
The Compliance Center provides shortcuts to key audit tools:
| Tool | Description |
| - | - |
| **Workflow History** | Opens a static snapshot of all workflows for inventory audits |
| **Activity Log** | Opens a time-series record of user actions and workflow runs |
## Compliance Rules
The Compliance Center displays the [automated compliance rules](/docs/security/compliance#automated-compliance-rules) currently applied to your team's workflows, along with their current status (enabled or disabled).
![Compliance rules in the Compliance Center]()
Compliance rules are configured by Kadoa on your behalf. To enable, disable, or update rules, please [contact support](https://www.kadoa.com/contact).
## Compliance Information
When enabled, a custom compliance reminder is displayed to users when they create a new workflow. This helps ensure team members are aware of your organization's data sourcing policies before starting any extraction.
![Compliance reminder shown during workflow creation]()
## Approval Flow
When Compliance Review is enabled, all new workflows require approval from a Compliance Officer before they can be activated:
1. A user creates a new workflow
2. The workflow enters a **Pending Approval** state during preview
3. The assigned Compliance Officer receives a notification
4. The officer reviews the workflow configuration and preview data
5. The officer approves or rejects the workflow with feedback
6. If approved, the workflow becomes active
![Compliance approval flow]()
## Reporting & Audit Log
The Compliance Center provides quick access to two reporting tools:
### Workflow History
A static snapshot of all workflows, useful for inventory audits. Includes:
* Data sources accessed or attempted
* User who created/modified workflows
* Version history and configuration changes
* Number of records extracted
### Activity Log
A time-series record of all platform activity. You can filter by:
* Workflow
* User
* Date range
* Event type
Export filtered data for compliance reviews from the [Activity Log](https://www.kadoa.com/activity-log).
## Compliance Officer Role
Admins can assign the **Compliance Officer** role to team members:
1. Go to **Settings** → **Members**
2. Click the **...** menu next to the member you want to update
3. Select **Edit member**
4. Change their role to **Compliance Officer**
5. Click **Save**
Available roles: Member, Admin, Compliance Officer, Viewer.
Compliance Officers have:
* Read-only access to all workflows and execution logs
* Access to compliance reports and audit trails
* Ability to approve or reject workflows (when Compliance Review is enabled)
## Learn More
* [Compliance Overview](/docs/security/compliance)
* [Team Management](/docs/ui/team-management)
* [Security Introduction](/docs/security/introduction)
# Data Quality
Source: https://docs.kadoa.com/docs/data-quality
Per-field rules and platform metrics that surface quality issues in your workflow data
Data quality in Kadoa has two layers that run at the end of every workflow run:
* **Rules** are per-field checks you define for each schema field (e.g. the format of a string). Kadoa automatically generates rules that make sense for your data, but you can customize them at any time.
* **Platform metrics** are built-in metrics that Kadoa computes automatically (e.g. completeness).
## Rules
Each schema field can have its own validation rules, specific to the field's data type.
### Type
The type defines which rules are available. The default matches the schema's data type, but you can cast values to the desired type for validation.
### Presence
Presence sets the minimum percentage of rows where the field must have a value. If fewer rows than the target have a value, the field is flagged.
### Uniqueness
Uniqueness sets the minimum percentage of values that must be unique in the column. If fewer rows than the target are unique, the field is flagged.
### String fields
Strings can be validated against one of three mutually-exclusive format kinds:
* **Free text**: Any value is allowed. Optionally match a character set (e.g. "Alphanumeric") and cap min and/or max length.
* **Defined format**: Match a pattern preset (e.g. "URL"), or add a custom regular expression.
* **Option from a list**: Match an options preset (e.g. "Language (2-letter)") or add a custom list of allowed values.
Selecting "Customize" on any preset prefills the custom regex or list so you can take it as a starting point and tweak it.
Independently of the format, strings can bound their **HTML element count**: the number of HTML elements the value contains (`Foo
bar` counts 1). Set a minimum of 1 to require raw HTML markup, or a maximum of 0 to forbid it.
### Number fields
* **Maximum decimal places**: The maximum allowed number of decimal places (e.g. "0" for whole numbers).
* **Range**: The minimum and/or maximum allowed values.
## Platform metrics
Platform metrics are built-in and require no configuration.
### Completeness
Compares the actual row count against the expected row count for the run, and reports:
* Actual vs expected row count
* Any qualifying signals detected from the source (such as pagination information)
* Whether the result was capped by a configured limit
### Suspicious values
Inspects each row for values that do not fit with the rest of the row (e.g. implausible or out-of-domain values) and reports how many columns have any flagged values, with examples.
## When changes take effect
* Validation runs at the end of every workflow run.
* Rule edits are saved immediately, but only apply on the **next** run.
* Workflows linked to a template inherit their rules from the template and are read-only. Unlink the workflow from its template to edit rules directly.
## Learn more
* [Configure rules in the UI →](/docs/ui/data-quality)
* [Manage rules with the SDK →](/docs/sdk/data-quality/overview)
* [API Reference →](/api-reference/data-quality/get-rules)
# Amazon S3
Source: https://docs.kadoa.com/docs/destinations/amazon-s3
Deliver data from completed runs to your S3 bucket
Kadoa configures the destination for you through the [Support Center](/docs/support-center).
## Before you begin
* An AWS account with permission to create a bucket and edit its bucket policy.
* Kadoa's AWS principal ARN, provided by Support. Your bucket policy grants access without sharing AWS keys.
* A bucket in any AWS region. Kadoa uses SSE-S3 encryption by default.
If your bucket requires SSE-KMS, contact Support before you request the destination.
## Prepare the bucket
Create an S3 bucket in your preferred AWS region.
Add this bucket policy. Replace ``, ``, and `` with your values.
Use a prefix with no leading or trailing slash, for example `kadoa`.
```json theme={null}
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": { "AWS": "" },
"Action": ["s3:PutObject", "s3:DeleteObject", "s3:AbortMultipartUpload", "s3:ListBucket"],
"Resource": ["arn:aws:s3::://*", "arn:aws:s3:::"]
}
]
}
```
## Request the destination
Send these details through the [Support Center](/docs/support-center):
| Detail | Example |
| - | - |
| Bucket name | `` |
| Region | `` |
| Prefix | `kadoa` |
| Encryption | SSE-S3 (default) or SSE-KMS |
| Formats | Parquet, JSONL, JSON, CSV |
| Workflow health files | On (default) or off |
## Verify the first delivery
After a workflow run completes, list the delivered files:
```bash theme={null}
aws s3 ls "s3:////" --recursive
```
* Confirm one file per selected format exists for that run.
* Confirm each data file's row count matches the same run in the dashboard.
* Run the workflow again. With the [default path template](/docs/destinations/data-destinations#file-layout-and-path-template), confirm that the new run has a new folder.
## What's next
Load files from your bucket with a Snowpipe that you manage.
See delivery timing, formats, file paths, and workflow health files.
Deliver completed-run data to a Cloud Storage bucket.
# How data destinations work
Source: https://docs.kadoa.com/docs/destinations/data-destinations
What Kadoa delivers, when, and where to see it
A destination delivers data from completed runs to storage you own or a data platform you use. Destinations belong to your workspace and can deliver all of its workflows or a selected set. Kadoa configures every destination for you through the [Support Center](/docs/support-center).
## When Kadoa delivers
Kadoa delivers data to Amazon S3, Google Cloud Storage, and SFTP immediately after a workflow run completes. Data is available in Snowflake and Databricks about 5 minutes after completion.
## Formats
Amazon S3, Google Cloud Storage, and SFTP support these file formats.
| Format | Use case |
| - | - |
| Parquet | Compressed columnar data for analytics and warehouse loads |
| JSONL | One JSON record per line for streaming and line-based processing |
| JSON | A JSON document with a metadata envelope for APIs |
| CSV | Comma-separated values for spreadsheets and systems that require flat files |
## File layout and path template
The default path template is:
```text theme={null}
{teamId}/{workflowId}/{runDatetimeSafe}-{runId}/data
```
If you set a prefix, Kadoa adds it before this path. For each run, Kadoa writes one file for each selected format. With the default template, each run has a unique path. This prevents overwrites of files from other runs.
| Token | Description | Example |
| - | - | - |
| `{teamId}` | Your team UUID | `a1b2c3d4-5e6f-7a8b-9c0d-e1f2a3b4c5d6` |
| `{workflowId}` | Workflow identifier | `b2c3d4e5-6f7a-8b9c-0d1e-f2a3b4c5d6e7` |
| `{runId}` | Run identifier | `c3d4e5f6-7a8b-9c0d-1e2f-a3b4c5d6e7f8` |
| `{runDatetimeSafe}` | Run finish time in UTC without colons, safe for filenames | `2025-01-15_10-30-00Z` |
| `{runDatetime}` | Run finish time in UTC in ISO 8601 format | `2025-01-15T10:30:00.000Z` |
| `{runDate}` | Run finish date in UTC | `2025-01-15` |
| `{runStartDatetimeSafe}` | Run start time in UTC without colons, safe for filenames | `2025-01-14_23-30-00Z` |
| `{runStartDatetime}` | Run start time in UTC in ISO 8601 format | `2025-01-14T23:30:00.000Z` |
| `{runStartDate}` | Run start date in UTC | `2025-01-14` |
The default template uses the run finish time. To group exports by the run start date, use a start token in your path template:
```text theme={null}
{teamId}/{workflowId}/{runStartDate}/{runId}/data
```
## Additional fields
Additional fields add source labels or workflow details to every exported record. They are available in CSV, JSONL, and JSON files, but not Parquet.
Each field has a name you choose and a fixed value or a variable. Kadoa replaces variables with the details of the workflow or run.
| Field name | Configured value | Example value in the file |
| - | - | - |
| `source` | `Kadoa` | `Kadoa` |
| `workflowName` | `{workflowName}` | `Product Monitor` |
| `runTimestamp` | `{runDatetime}` | `2025-01-15T10:30:00.000Z` |
In additional fields, `{runDate}` and `{runDatetime}` use the run start time. The JSON metadata envelope's `executedAt` also uses the start time.
Supported variables:
* `{teamId}`
* `{workflowId}`
* `{workflowName}`
* `{runId}`
* `{runDate}`
* `{runDatetime}`
## Workflow health files
Health files let you monitor workflow status from your Amazon S3, Google Cloud Storage, or SFTP destination.
They are enabled by default for Amazon S3 and Google Cloud Storage, and disabled for SFTP. Kadoa can enable or disable health files for your destination.
When enabled, Kadoa writes one JSON file for each of these [events](/docs/notifications):
* `workflow_finished`: a scheduled workflow finishes.
* `workflow_failed`: a scheduled or realtime workflow fails.
* `workflow_recovered`: a realtime workflow recovers.
File path:
```
/workflow-health/date=2025-01-15/.json
```
```json theme={null}
{
"schemaVersion": 1,
"eventId": "",
"eventType": "workflow_failed",
"status": "failed",
"eligibleAt": "2025-01-15T10:30:00.000Z",
"workflow": { "id": "", "name": "Product Monitor" },
"runId": ""
}
```
`runId` is present only when the event belongs to a run.
## View your destinations
See which destinations deliver a workflow's results without contacting support:
* **In the dashboard**: the **Deliver** section of the [Pipeline](/docs/ui/workflows/pipeline) panel lists connected destinations first, or open the workflow's **Download** dropdown.
* **Via the API**: [List workflow destinations](/api-reference/workflows/list-workflow-connectors) returns each destination's name, provider, target (bucket URI, SFTP host, or warehouse share), and file formats.
Only deliverable destinations are listed (enabled, verified, not revoked).
## Request, change, or disable a destination
To request a new destination for a workflow, open the workflow's [Pipeline](/docs/ui/workflows/pipeline) panel, choose **Deliver**, click **Add** under Destinations, and click **Set up** next to the destination. The [Support Center](/docs/support-center) request opens prefilled with the workflow and the details we need; fill them in and submit.
For a change or to disable a destination, submit a Support Center request that names the destination, what you want to change, and the new value.
## Glossary
**Destination:** The configuration that sends completed-run data to your storage or data platform.
**Delivery:** One transfer or publication for one run.
**Run:** One execution of a workflow.
**Workflow:** The configured process that collects and structures your data.
**Team:** The Kadoa workspace that owns workflows and destinations.
# Databricks
Source: https://docs.kadoa.com/docs/destinations/databricks
Query data from completed runs through Delta Sharing
Kadoa configures the destination for you through the [Support Center](/docs/support-center).
## Before you begin
* A Unity Catalog-enabled workspace with compute for queries.
* Permission to create a catalog from a share: metastore admin, or `CREATE CATALOG` with `USE PROVIDER` or provider ownership.
Get your sharing identifier in the workspace where you will query the data:
```sql theme={null}
select current_metastore();
```
Send the full value, including the cloud and region: `::`.
You need a compatible Delta Sharing client and a contact who can receive credentials securely.
A Databricks workspace is not required.
## Request the destination
Send these details through the [Support Center](/docs/support-center):
| Detail | Value |
| - | - |
| Access method | Databricks workspace or open Delta Sharing client |
| Sharing identifier | Workspace only: `::` |
| Credential contact | Open client only: contact for the activation link |
| Activity log | On (default) or off |
Activity sharing includes available historical events.
## Import the share
Open **Catalog** > gear icon > **OpenSharing** > **Shared with me**.
Older workspaces call it **Delta Sharing**.
Select the Kadoa provider and the share whose name starts with `KADOA_SHARE_`.
A new share can take up to one minute to appear.
Select **Mount to catalog** > **Create a new catalog**.
Enter a name such as `kadoa`. Select **Create**.
Give readers `USE CATALOG` on the catalog, `USE SCHEMA` on its schemas, and `SELECT` on the shared tables.
Open the activation link. Select **Download Credential File**. You can download the file only once. Kadoa does not store it.
Store the file securely. Restrict access to authorized readers.
Tokens expire within 365 days of creation. Contact Support before expiry.
If you lose the file, request a replacement from Support.
For Python, install `delta-sharing`. List the shared tables using the path to your credential file:
```python theme={null}
import delta_sharing
profile = ""
client = delta_sharing.SharingClient(profile)
print(client.list_all_tables())
```
Use the returned share, schema, and table names to read a table:
```python theme={null}
df = delta_sharing.load_as_pandas(
f"{profile}#.."
)
```
For large tables, use a [Spark client](https://docs.databricks.com/aws/en/opensharing/read-data-open#apache-spark-read-shared-data) instead of loading the table into pandas.
## Verify the first delivery
Allow about 5 minutes after a workflow run completes.
1. Find your `workflow_id` in `WORKFLOW_RUNS`. Select the delivered run and note its `job_id`.
2. Match its `workflow_id`, `schema_version`, and `schema_fingerprint` in `WORKFLOW_SCHEMA_VERSIONS`. Read `versioned_table_name`.
3. Read that table with a `job_id` filter. Compare the row count with the same run in the Kadoa dashboard.
In Databricks, select your catalog and the schema whose name starts with `TEAM_`. Run this query:
```sql theme={null}
select count(*) as row_count
from
where job_id = '';
```
With the Python client, use the earlier example to read `WORKFLOW_RUNS` and `WORKFLOW_SCHEMA_VERSIONS`.
Then load the versioned table and count the run's rows:
```python theme={null}
print(df.loc[df["job_id"] == ""].shape[0])
```
## Query the shared tables
| Table | Contains |
| - | - |
| `WORKFLOW_RUNS` | Delivered runs, schema versions, row counts, and timestamps |
| `WORKFLOW_SCHEMA_VERSIONS` | Workflow IDs and their generated table names |
| `WORKFLOW_SCHEMA_FIELDS` | Field names, types, and positions in `field_index` |
| `WF___V` | Workflow data for one schema version |
| `WF___LATEST` | Workflow data for the latest schema version |
Get exact table names from `WORKFLOW_SCHEMA_VERSIONS`. Do not substitute a workflow ID into a table name.
Each data table can contain multiple runs. Filter by `job_id` to query one run. Older schema versions remain available in versioned tables.
Other tables include `ACTIVITY_LOG` when enabled and `TPL___LATEST` for shared template data.
## What's next
Query shared secure views.
Review destination behavior.
# Email
Source: https://docs.kadoa.com/docs/destinations/email
Receive email notifications when workflow events occur
## Setup
```typescript title="Node SDK" theme={null}
import { KadoaClient } from "@kadoa/node-sdk";
const client = new KadoaClient({ apiKey: "YOUR_API_KEY" });
// Use your account email
await client.notification.setupForWorkflow({
workflowId: "YOUR_WORKFLOW_ID",
events: ["workflow_finished", "workflow_failed"],
channels: { EMAIL: true },
});
// Or specify custom recipients
await client.notification.setupForWorkflow({
workflowId: "YOUR_WORKFLOW_ID",
events: ["workflow_data_change"],
channels: {
EMAIL: {
name: "team-alerts",
recipients: ["team@company.com", "alerts@company.com"],
},
},
});
```
```python title="Python SDK" theme={null}
from kadoa_sdk import KadoaClient, KadoaClientConfig
from kadoa_sdk.notifications import SetupWorkflowNotificationSettingsRequest
client = KadoaClient(KadoaClientConfig(api_key="YOUR_API_KEY"))
# Use your account email
client.notification.setup_for_workflow(
SetupWorkflowNotificationSettingsRequest(
workflow_id="YOUR_WORKFLOW_ID",
events=["workflow_finished", "workflow_failed"],
channels={"EMAIL": True},
)
)
# Or specify custom recipients
client.notification.setup_for_workflow(
SetupWorkflowNotificationSettingsRequest(
workflow_id="YOUR_WORKFLOW_ID",
events=["workflow_data_change"],
channels={
"EMAIL": {
"name": "team-alerts",
"recipients": ["team@company.com", "alerts@company.com"],
}
},
)
)
```
For API configuration, see the [API reference](/api-reference/notifications).
### Dashboard Setup
1. Go to **Notifications** in the sidebar
2. Click **Add Channel** → **Email**
3. Add a single or group of recipient email addresses
4. Save changes
![Email Channel UI]()
![Email Channel Group UI]()
## Email Templates
Each event type has a dedicated email template with relevant information.
![Data change email]()
![Workflow finished email]()
![Workflow failed email]()
![Validation anomaly email]()
## System Emails
These emails are sent automatically and cannot be configured.
Sent to the workflow author when preview data is ready for review.
![Sample ready email]()
# Google Cloud Storage
Source: https://docs.kadoa.com/docs/destinations/google-cloud-storage
Deliver data from completed runs to a Cloud Storage bucket
Kadoa configures the destination for you through the [Support Center](/docs/support-center).
## Before you begin
* A Google Cloud project with permission to create a bucket and edit its IAM policy.
* Kadoa's service account email, provided by Support. Your bucket IAM policy grants access without sharing keys.
* An authenticated Google Cloud CLI (`gcloud`).
## Prepare the bucket
Create a dedicated bucket and keep it private.
Replace both placeholders, then run this command.
```bash theme={null}
gcloud storage buckets add-iam-policy-binding "gs://" \
--member="serviceAccount:" \
--role="roles/storage.objectUser"
```
Kadoa needs `roles/storage.objectUser` to create, list, overwrite, and delete objects. `roles/storage.objectCreator` is not enough.
## Request the destination
Send these details through the [Support Center](/docs/support-center):
Use a prefix with no leading or trailing slash, for example `kadoa`.
| Detail | Example |
| - | - |
| Bucket name | `` |
| Prefix | `kadoa` |
| Formats | Parquet, JSONL, JSON, CSV |
| Workflow health files | On (default) or off |
## Verify the first delivery
After a workflow run completes, list the delivered files with your authenticated `gcloud` account:
```bash theme={null}
gcloud storage ls "gs:////" --recursive
```
* Confirm one file per selected format exists for that run.
* Confirm each data file's row count matches the same run in the dashboard.
* Run the workflow again. With the [default path template](/docs/destinations/data-destinations#file-layout-and-path-template), confirm that the new run has a new folder.
## What's next
Load files from your bucket with a Snowpipe that you manage.
See delivery timing, formats, file paths, and workflow health files.
Deliver completed-run data to an S3 bucket.
# Google Sheets
Source: https://docs.kadoa.com/docs/destinations/google-sheets
## Overview
Fetch data from the Kadoa API and add it to a Google Sheet using Google Apps Script.
This workaround is a temporary solution until Kadoa releases an official
Google Sheets destination.
## Prerequisites
Before you begin, make sure you have the following:
* A Google account
* A Google Sheet where you want to store the data
* Access to the Kadoa API and your API key
## Step-by-Step Guide
1. In your Google Sheet, go to "Extensions" -> "Apps Script" to open the Google Apps Script editor.
2. Replace the contents of the script editor with the following code:
```javascript theme={null}
function fetchDataFromKadoa() {
var sheet = SpreadsheetApp.getActiveSpreadsheet().getActiveSheet();
var workflowId = "YOUR_WORKFLOW_ID"; // Add Kadoa workflow ID
var apiKey = "YOUR_API_KEY"; // Add Kadoa API key
var limit = 50; // Choose record limit
var page = 1; // Starting page
var baseUrl = `https://api.kadoa.com/v4/workflows/${workflowId}/data`;
var headers = {
"x-api-key": apiKey,
};
var options = {
method: "get",
headers: headers,
};
var allData = [];
do {
var url = `${baseUrl}?limit=${limit}&page=${page}`;
var response = UrlFetchApp.fetch(url, options);
var responseData = JSON.parse(response.getContentText());
// Extract the data array from the response
var data = responseData.data;
var pagination = responseData.pagination;
if (data.length === 0) {
Logger.log("No data found.");
break;
}
// Add current page data to allData array
allData = allData.concat(data);
// Check if there are more pages
page++;
} while (page <= pagination.totalPages);
if (allData.length === 0) {
Logger.log("No data found across all pages.");
return;
}
// Get column headers dynamically from the first object in the data array
var headers = Object.keys(allData[0]);
// Write headers to the sheet
sheet.getRange(1, 1, 1, headers.length).setValues([headers]);
// Write data to the sheet
var rowData = [];
allData.forEach(function (item) {
var row = [];
headers.forEach(function (header) {
row.push(item[header] || ""); // Add a fallback if some data is missing
});
rowData.push(row);
});
// Write the data starting from the second row
sheet.getRange(2, 1, rowData.length, headers.length).setValues(rowData);
}
```
3. Replace `'YOUR_WORKFLOW_ID'` with your Kadoa workflow ID and `'YOUR_API_KEY'` with your Kadoa API key.
4. Save the script and run the `fetchDataFromKadoa` function either by clicking the play button in the toolbar or by going to "Run" -> "Run function" -> "fetchDataFromKadoa".
5. If this is your first time running a script accessing external APIs, authorize the script to access external services.
6. Check your Google Sheet. The data from the Kadoa API should now be populated.
## Scheduling Automatic Updates (Optional)
To keep your Google Sheet data in sync with Kadoa, you can set up a trigger to run the script on a schedule:
1. In the script editor, go to "Edit" > "Current project's triggers".
2. Click "Add Trigger" and configure the desired frequency.
3. Save the trigger.
The script will now automatically fetch and update the data in your Google Sheet based on the specified schedule.
# Overview
Source: https://docs.kadoa.com/docs/destinations/overview
Choose where Kadoa sends your data
Kadoa delivers the data of completed runs to storage and data platforms you own, sends you events, and answers requests from your code.
## Data delivery
Deliver files to an S3 bucket you own.
Deliver files to a Cloud Storage bucket you own.
Query read-only secure views in Snowflake.
Query read-only Delta tables through Delta Sharing.
Deliver files to an SFTP server you host.
Compare scope, formats, timing, and status.
Azure Blob Storage is available on request through the [Support Center](/docs/support-center).
## Events & streaming
Send an HTTP POST to your endpoint when events occur.
Stream updates to live dashboards.
## Programmatic access
Retrieve data on demand with pagination.
Node.js and Python libraries.
Retrieve data from a terminal or CI/CD job.
Give AI assistants access to Kadoa data.
Spreadsheet workflows.
## Notifications
For the same events in your code, use [Webhooks](/docs/destinations/webhooks) or [WebSockets](/docs/destinations/websockets).
Alerts and notifications by email.
Team notifications in Slack.
## Formats by method
Amazon S3, Google Cloud Storage, and SFTP deliver files in the formats below. Snowflake and Databricks provide tables from completed runs for you to query.
| Format | Available in |
| - | - |
| JSON | REST API, SDK, CLI, MCP, Webhooks, WebSockets, Amazon S3, Google Cloud Storage, SFTP |
| Parquet | Amazon S3, Google Cloud Storage, SFTP |
| JSONL | Amazon S3, Google Cloud Storage, SFTP |
| CSV | REST API, CLI, Amazon S3, Google Cloud Storage, SFTP |
| Tables | Snowflake, Databricks |
# REST API
Source: https://docs.kadoa.com/docs/destinations/rest-api
Pull data from Kadoa on-demand using the REST API
The REST API gives you complete control over when and how you retrieve your extracted data. Perfect for batch processing, scheduled jobs, or on-demand access.
## Basic Usage
### Get Latest Data
Retrieve the most recent data from a workflow:
[View full API reference →](/api-reference/workflows/get-workflow-data-by-id)
```bash theme={null}
GET https://api.kadoa.com/v4/workflows/{workflowId}/data
```
```json Response theme={null}
{
"workflowId": "workflow-123",
"runId": "run-456",
"executedAt": "2024-01-15T10:30:00Z",
"data": [
{
"id": "123",
"title": "Product Name",
"price": 99.99,
"extractedAt": "2024-01-15T10:30:00Z"
}
],
"pagination": {
"totalCount": 150,
"page": 1,
"totalPages": 6,
"limit": 25
}
}
```
The total row count is `pagination.totalCount`. There is no `hasMore` flag; to check whether further pages exist, compare `pagination.page < pagination.totalPages`.
**Key field deduplication**: If your workflow schema marks one or more fields as key fields (`isKey: true`), Kadoa automatically deduplicates results so there is at most one record per unique key combination. Key fields should be scalar `STRING`, `NUMBER`, or `LINK` fields. Records where any key field is missing or empty are treated as distinct and are not merged.
## Pagination and Filtering
### Handle Large Datasets
Use pagination for efficient data retrieval:
```bash theme={null}
GET https://api.kadoa.com/v4/workflows/{workflowId}/data?page=1&limit=100
```
### Query Parameters
| Parameter | Type | Default | Description |
| - | - | - | - |
| `page` | integer | `1` | Page number |
| `limit` | integer | `25` | Rows per page. Set to `0` to stream all rows without paging |
| `sortBy` | string | — | Field name to sort by |
| `order` | string | `asc` | Sort order: `asc` or `desc` |
| `filters` | string | — | JSON-encoded array of filter objects (see below) |
| `runId` | string | — | Retrieve data from a specific historical run instead of the latest |
| `format` | string | `json` | Response format: `json` or `csv` |
### Filtering
Pass a URL-encoded JSON array to `filters`. Each entry specifies a field, an operator, and a value:
```bash theme={null}
GET https://api.kadoa.com/v4/workflows/{workflowId}/data?limit=50&filters=[{"field":"jobTitle","operator":"CONTAINS","value":"Manager"},{"field":"postedDate","operator":"AFTER","value":"2024-01-01"}]
```
Available operators:
| Operator | Description |
| - | - |
| `EQUALS` / `NOT_EQUALS` | Exact match |
| `CONTAINS` / `NOT_CONTAINS` | Substring match (case-insensitive) |
| `STARTS_WITH` / `ENDS_WITH` | Prefix / suffix match |
| `GREATER_THAN` / `LESS_THAN` / `GREATER_THAN_OR_EQUAL` / `LESS_THAN_OR_EQUAL` | Numeric or date comparison |
| `IN` / `NOT_IN` | Value must (or must not) be in an array: `"value": ["Sales","Marketing"]` |
| `IS_NULL` / `IS_NOT_NULL` | Field presence check |
| `IS_EMPTY` / `IS_NOT_EMPTY` | Null or empty string check |
| `BEFORE` / `AFTER` | Date field comparison |
| `WITHIN_LAST_DAYS` | Date field within the last N days: `"value": 7` |
## Data Formats
### JSON (Default)
Standard JSON format, perfect for modern applications:
```json theme={null}
{
"data": [...],
"pagination": {...}
}
```
### CSV Format
Add `?format=csv` to receive a CSV file instead of JSON:
```bash theme={null}
GET https://api.kadoa.com/v4/workflows/{workflowId}/data?format=csv
```
All pagination and filter parameters apply. For large exports that would exceed the response timeout, use `download=link`:
```bash theme={null}
GET https://api.kadoa.com/v4/workflows/{workflowId}/data?format=csv&download=link
```
This materializes the file in object storage and returns a `downloadPath` you can fetch separately.
### Parquet option
Create a signed URL for workflow data as a typed Parquet file:
[View full API reference →](/api-reference/workflows/create-workflow-data-export)
```bash theme={null}
GET https://api.kadoa.com/v4/workflows/{workflowId}/data/export?format=parquet
```
Use Parquet when you want to load Kadoa output directly into analytical tools such as DuckDB, Spark, Polars, Snowflake, BigQuery, or pandas. The export endpoint materializes the requested result set and returns a signed `url`. Pass `runId` to export a specific historical run:
```bash theme={null}
curl "https://api.kadoa.com/v4/workflows/{workflowId}/data/export?format=parquet&runId={runId}" \
-H "x-api-key: YOUR_API_KEY"
```
Filtering, sorting, and row selection use the same query parameters as CSV and JSON exports. For direct streaming of the complete run artifact, use `GET /v4/workflows/{workflowId}/data/parquet`.
Typed columns are preserved where the workflow schema provides native types, including booleans, numbers, dates, timestamps, JSON-compatible objects, and arrays. Older workflow runs may still reflect the type information available when that run was produced.
## Error Handling
### Common HTTP Status Codes
| Code | Meaning | Action |
| - | - | - |
| 200 | Success | Process data normally |
| 400 | Bad Request | Check query parameters |
| 401 | Unauthorized | Verify API key |
| 404 | Not Found | Check workflow ID |
| 429 | Rate Limited | Wait and retry |
| 500 | Server Error | Contact support |
## API Reference
For complete API documentation, see:
* [Get workflow data by ID](/api-reference/workflows/get-workflow-data-by-id)
* [Authentication](/api-reference/introduction)
# SDK
Source: https://docs.kadoa.com/docs/destinations/sdk
Access and retrieve extracted data using the Kadoa SDK
## Basic Usage
Retrieve your data using the SDK's built-in methods.
```typescript Node SDK theme={null}
const result = await client.extraction.run({
urls: ["https://sandbox.kadoa.com/ecommerce"],
name: "Product Extraction",
});
// Data is included in the result
console.log(result.data); // Array of extracted items
console.log(result.pagination); // { page, limit, totalCount, totalPages }
```
```python Python SDK theme={null}
from kadoa_sdk import ExtractionOptions, KadoaClient, KadoaClientConfig
client = KadoaClient(KadoaClientConfig(api_key="YOUR_API_KEY"))
result = client.extraction.run(
ExtractionOptions(
urls=["https://sandbox.kadoa.com/ecommerce"],
name="Product Extraction",
)
)
# Data is included in the result
print(result.data) # Array of extracted items
print(result.pagination) # { page, limit, totalCount, totalPages }
```
## Fetch Data
If you have an existing workflow, you can fetch its data directly using the workflow ID:
```typescript Node SDK theme={null}
// Simplest way to fetch workflow data
const data = await client.extraction.fetchData({
workflowId: workflowId,
});
console.log(data.data);
```
```python Python SDK theme={null}
from kadoa_sdk import FetchDataOptions, KadoaClient, KadoaClientConfig
client = KadoaClient(KadoaClientConfig(api_key="YOUR_API_KEY"))
# Simplest way to fetch workflow data
data = client.extraction.fetch_data(
FetchDataOptions(workflow_id="YOUR_WORKFLOW_ID")
)
print(data.data)
```
For more control, you can specify all available options:
```typescript Node SDK theme={null}
const data = await client.extraction.fetchData({
workflowId: workflowId,
page: 1,
limit: 10,
});
console.log(data.data); // Array of extracted items
console.log(data.pagination); // { page, limit, totalCount, totalPages }
```
```python Python SDK theme={null}
from kadoa_sdk import FetchDataOptions, KadoaClient, KadoaClientConfig
client = KadoaClient(KadoaClientConfig(api_key="YOUR_API_KEY"))
data = client.extraction.fetch_data(
FetchDataOptions(
workflow_id="YOUR_WORKFLOW_ID",
page=1,
limit=10,
)
)
print(data.data)
print(data.pagination)
```
## Pagination
Handle large datasets efficiently using the SDK's pagination methods:
```typescript Node SDK theme={null}
// Option 1: Iterate page by page
for await (const page of client.extraction.fetchDataPages({
workflowId: workflowId,
})) {
console.log("Page data:", page.data);
console.log("Page number:", page.pagination.page);
}
// Option 2: Get everything at once
const allData = await client.extraction.fetchAllData({
workflowId: workflowId,
});
console.log("All data:", allData);
```
```python Python SDK theme={null}
import asyncio
from kadoa_sdk import FetchDataOptions, KadoaClient, KadoaClientConfig
client = KadoaClient(KadoaClientConfig(api_key="YOUR_API_KEY"))
workflow_id = "YOUR_WORKFLOW_ID"
# Option 1: Iterate page by page
async def iterate_pages() -> None:
async for page in client.extraction.fetch_data_pages(
FetchDataOptions(workflow_id=workflow_id)
):
print("Page data:", page.data)
if page.pagination:
print("Page number:", page.pagination.page)
asyncio.run(iterate_pages())
# Option 2: Get everything at once
all_data = client.extraction.fetch_all_data(FetchDataOptions(workflow_id=workflow_id))
print("All data:", all_data)
```
## Response Format
Data returned by the SDK follows this structure:
```json theme={null}
{
data: [
{
// Your extracted fields
title: "Product Name",
price: "$99.99",
inStock: true
}
],
pagination: {
page: 1,
limit: 50,
totalCount: 150,
totalPages: 3
}
}
```
```json theme={null}
response_format = {
"data": [
{
"title": "Product Name",
"price": "$99.99",
"inStock": True,
}
],
"pagination": {"page": 1, "limit": 50, "totalCount": 150, "totalPages": 3},
}
print(response_format)
```
# SFTP
Source: https://docs.kadoa.com/docs/destinations/sftp
Deliver data from completed runs to your SFTP server
Kadoa configures the destination for you through the [Support Center](/docs/support-center).
## Before you begin
Your SFTP server must support:
* Public internet access without a source-IP allowlist. Kadoa has no fixed SFTP IP address.
* Password or SSH private-key login.
* One of these host-key algorithms: `ssh-ed25519`, `ecdsa-sha2-nistp256`, `rsa-sha2-256`, or `rsa-sha2-512`.
* Renaming files within the base directory. Kadoa uploads each file with a `.part` suffix and renames it when the upload is complete. Kadoa uses `posix-rename@openssh.com` when the server offers it, and the standard SFTP rename otherwise.
If your server requires a source-IP allowlist, use [Amazon S3](/docs/destinations/amazon-s3) or [Google Cloud Storage](/docs/destinations/google-cloud-storage).
## Prepare the server
Create a dedicated account on your SFTP server.
Create a directory such as `/incoming/kadoa`.
Use an absolute path as the SFTP account sees it. The path cannot be `/`.
Allow the account to create subdirectories and read, write, rename, and delete files within the base directory.
Get the host-key algorithm and SHA-256 fingerprint from your server administrator.
For an OpenSSH server with an Ed25519 host key, run this command on the server:
```bash theme={null}
ssh-keygen -lf /etc/ssh/ssh_host_ed25519_key.pub
```
Record the full fingerprint, including `SHA256:`. This example uses the `ssh-ed25519` algorithm.
Confirm the fingerprint with your server administrator through a separate channel from the one used for the server address. Kadoa checks the fingerprint before authentication. If it does not match, Kadoa stops the connection.
## Add your credentials
A team Owner or Admin must create the [Secret Variables](/docs/workflows/secrets).
Never send passwords, private keys, or passphrases through email, chat, or a ticket.
In **Team settings**, open **Variables**. Choose one login method:
* **Password:** Store the password in a Secret Variable such as `SFTP_PASSWORD`.
* **SSH private key:** Authorize the matching public key for the SFTP account. Kadoa-managed secret fields support single-line values only. Store the private key in your Google Secret Manager project. Create a Secret Variable such as `SFTP_PRIVATE_KEY` that references it.
For an encrypted private key, store its passphrase in a separate Secret Variable such as `SFTP_KEY_PASSPHRASE`.
## Request the destination
Send these non-secret details through the [Support Center](/docs/support-center):
| Detail | Example |
| - | - |
| Host | `sftp.example.com` |
| Port | `22` |
| Username | `kadoa-delivery` |
| Authentication method | Password or SSH private key |
| Credential variable | `SFTP_PASSWORD` or `SFTP_PRIVATE_KEY` |
| Passphrase variable, for encrypted keys | `SFTP_KEY_PASSPHRASE` |
| Host-key algorithm | `ssh-ed25519` |
| Host-key fingerprint | `SHA256:<43_CHARACTERS>` |
| Base directory | `/incoming/kadoa` |
| Formats | Parquet, JSONL, JSON, CSV |
| Workflow health files | Off (default) or on |
## Verify the first delivery
After a workflow run completes, open the base directory in your SFTP client.
* Confirm that each selected format has one data file for the run.
* Ignore data files that end in `.part`. These files are incomplete and can remain after an interrupted delivery.
* Confirm that each data file's row count matches the same run in the dashboard.
* Run the workflow again. With the [default path template](/docs/destinations/data-destinations#file-layout-and-path-template), confirm that the new run has a new directory.
See [How data destinations work](/docs/destinations/data-destinations) for formats, file paths, and workflow health files.
# Slack
Source: https://docs.kadoa.com/docs/destinations/slack
Receive workflow notifications in Slack
Send [notification events](/docs/notifications) such as data changes, workflow health, and data quality issues to a Slack channel.
## Connect a channel
1. Create an [incoming webhook](https://api.slack.com/messaging/webhooks) in your Slack workspace for the channel you want to post to.
2. In Kadoa, go to **Notifications** in the sidebar and click **Add Channel** → **Slack**.
3. Enter the channel name and paste the webhook URL, then save.
![Slack channel setup]()
Leave **Channel ID** empty unless Kadoa support has set up a managed Slack app for your workspace.
Then subscribe the channel to events at the workspace or workflow level. See [Notifications](/docs/notifications) for the available events and configuration options.
# Snowflake
Source: https://docs.kadoa.com/docs/destinations/snowflake
Query data from completed runs through read-only Snowflake views
Kadoa configures the destination for you through the [Support Center](/docs/support-center).
## Before you begin
* A Snowflake role with `IMPORT SHARE` and `CREATE DATABASE` privileges. `ACCOUNTADMIN` has both.
* Access to a Snowflake warehouse for queries.
Find your account details:
```sql theme={null}
select
current_organization_name() as organization_name,
current_account_name() as account_name,
current_region() as snowflake_region;
```
If the query fails, open the account selector in Snowsight and select **View account details**.
Copy the **Account/Server URL** instead.
## Request the destination
Send these details or your Account/Server URL through the [Support Center](/docs/support-center):
| Detail | Example |
| - | - |
| Organization name | `` |
| Account name | `` |
| Region | `` |
| Activity log | Off (default) or on |
Historical activity events are not included.
Kadoa confirms availability for your region. Accounts in Kadoa's region use a direct share. Other regions or clouds require a private listing.
## Import the share
For a direct share, complete these steps:
In Snowsight, open **Data sharing** > **External sharing** > **Shared with you**.
Under **Ready to Get**, select the share whose name starts with `KADOA_SHARE_`.
Enter a database name such as `KADOA`.
Choose the roles that can query it.
Select **Get Data**.
In your new database, find the schema whose name starts with `TEAM_`.
Replace `` with that schema name.
If you chose another database name, replace `KADOA` too:
```sql theme={null}
select verification_nonce
from KADOA..connector_verification;
```
Send the returned code to Support. Kadoa then publishes data to the same database and schema.
For a private listing, open **Shared with you** > **Privately shared listings**.
Select the listing whose name starts with `KADOA_LISTING_`. Complete the [listing import steps](https://docs.snowflake.com/en/collaboration/consumer-listings-access).
Support provides the verification instructions for your listing.
## Verify the first delivery
Allow about 5 minutes after a workflow run completes.
Select your warehouse, database, and Kadoa schema in Snowsight.
Find the delivered run and its data view:
```sql theme={null}
select r.job_id, r.schema_version, r.row_count, r.shared_at, v.view_name
from workflow_runs r
join workflow_schema_versions v
on r.workflow_id = v.workflow_id
and r.schema_version = v.schema_version
and r.schema_fingerprint = v.schema_fingerprint
where r.workflow_id = ''
order by r.shared_at desc
limit 20;
```
Use the returned `view_name` and `job_id` to count that run's rows:
```sql theme={null}
select count(*) as row_count
from
where job_id = '';
```
Confirm that `job_id` and the row count match the same run in the Kadoa dashboard.
## Query the shared views
| View | Contains |
| - | - |
| `WORKFLOW_RUNS` | Delivered runs, schema versions, row counts, and timestamps |
| `WORKFLOW_SCHEMA_VERSIONS` | Workflow IDs and their generated view names |
| `WORKFLOW_SCHEMA_FIELDS` | Field names, types, and positions in `field_order` |
| `WF___V` | Workflow data for one schema version |
| `WF___LATEST` | Workflow data for the latest schema version |
Get exact view names from `WORKFLOW_SCHEMA_VERSIONS`. Do not substitute a workflow ID into a view name.
Each data view can contain multiple runs. Filter by `job_id` to query one run. Versioned views keep their schema when a new version appears.
Optional views include `ACTIVITY_LOG` and `TPL___LATEST` for template data.
## Load cloud storage exports with Snowpipe
You can also load files from [Amazon S3](/docs/destinations/amazon-s3) or [Google Cloud Storage](/docs/destinations/google-cloud-storage) with a Snowpipe you manage.
This option uses bucket delivery instead of the native Snowflake share.
1. Complete a workflow run so a sample data file exists.
2. Limit the pipe to one workflow's path and one format, such as Parquet. Exclude other formats and workflow health files.
3. Configure storage access and automatic loading with Snowflake's [S3 guide](https://docs.snowflake.com/en/user-guide/data-load-snowpipe-auto-s3) or [GCS guide](https://docs.snowflake.com/en/user-guide/data-load-snowpipe-auto-gcs).
See [file paths](/docs/destinations/data-destinations#file-layout-and-path-template) for the default layout.
If Kadoa hosts your S3 bucket, send Support the pipe's `notification_channel` value to configure event notifications.
## What's next
Set up bucket delivery.
Use Delta Sharing.
See delivery timing, formats, and file paths.
# Webhooks
Source: https://docs.kadoa.com/docs/destinations/webhooks
Receive HTTP notifications when workflow events occur
Your webhook endpoint must be publicly accessible.
## Setup
```typescript title="Node SDK" theme={null}
import { KadoaClient } from "@kadoa/node-sdk";
const client = new KadoaClient({ apiKey: "YOUR_API_KEY" });
await client.notification.setupForWorkflow({
workflowId: "YOUR_WORKFLOW_ID",
events: ["workflow_data_change"],
channels: {
WEBHOOK: {
name: "my-webhook",
webhookUrl: "https://api.example.com/webhooks/kadoa",
httpMethod: "POST",
},
},
});
```
```python title="Python SDK" theme={null}
from kadoa_sdk import KadoaClient, KadoaClientConfig
from kadoa_sdk.notifications import SetupWorkflowNotificationSettingsRequest
client = KadoaClient(KadoaClientConfig(api_key="YOUR_API_KEY"))
client.notification.setup_for_workflow(
SetupWorkflowNotificationSettingsRequest(
workflow_id="YOUR_WORKFLOW_ID",
events=["workflow_data_change"],
channels={
"WEBHOOK": {
"name": "my-webhook",
"webhookUrl": "https://api.example.com/webhooks/kadoa",
"httpMethod": "POST",
}
},
)
)
```
For API configuration, see the [API reference](/api-reference/notifications).
### Dashboard Setup
1. Go to **Notifications** in the sidebar
2. Click **Add Channel** → **Webhook**
3. Enter your endpoint URL
4. Save changes
![Webhook setup]()
## Authentication
Add custom headers for webhook authentication:
```json theme={null}
// POST /v5/notifications/channels
{
"channelType": "WEBHOOK",
"name": "Authenticated Webhook",
"config": {
"webhookUrl": "https://api.example.com/webhooks/kadoa",
"httpMethod": "POST",
"auth": {
"type": "HEADER",
"headers": {
"Authorization": "Bearer your-secret-token",
"X-Custom-Header": "custom-value"
}
}
}
}
```
## Request Format
Every webhook request includes:
| Header | Description |
| - | - |
| `Content-Type` | `application/json` |
| `X-Kadoa-Event` | Event type (e.g., `workflow_data_change`) |
## Payload Reference
All payloads follow this structure:
```json theme={null}
{
"eventType": "event_name",
"timestamp": "2025-01-15T10:30:00Z",
"data": { /* event-specific data */ }
}
```
```json title="workflow_data_change" theme={null}
{
"eventType": "workflow_data_change",
"timestamp": "2025-01-15T10:30:00Z",
"data": {
"id": "change_123",
"workflowId": "wf_123",
"data": [
{ "id": "record-1", "name": "Product A", "price": 29.99 }
],
"differences": [
{
"type": "changed",
"fields": [
{ "key": "price", "value": 29.99, "previousValue": 24.99 }
]
},
{
"type": "added",
"fields": [
{ "key": "id", "value": "record-2" },
{ "key": "name", "value": "New Product" }
]
}
],
"url": "https://monitored-page.com",
"createdAt": "2025-01-15T10:30:00Z",
"metadata": {
"workflowName": "Product Monitor",
"tags": ["pricing"]
}
}
}
```
```json title="workflow_finished" theme={null}
{
"eventType": "workflow_finished",
"timestamp": "2025-01-15T10:30:00Z",
"data": {
"id": "wf_123",
"jobId": "job_456",
"source": "scheduler"
}
}
```
```json title="workflow_failed" theme={null}
{
"eventType": "workflow_failed",
"timestamp": "2025-01-15T10:30:00Z",
"data": {
"workflowId": "wf_123",
"workflowName": "Product Monitor",
"source": "Product Monitor",
"sourceUrl": "https://monitored-page.com",
"url": "https://monitored-page.com",
"reason": "Access blocked by bot protection",
"action": "No action needed. We'll email you when the workflow recovers."
}
}
```
```json title="workflow_recovered" theme={null}
{
"eventType": "workflow_recovered",
"timestamp": "2025-01-15T10:45:00Z",
"data": {
"workflowId": "wf_123",
"workflowName": "Product Monitor",
"source": "Product Monitor",
"sourceUrl": "https://monitored-page.com",
"url": "https://monitored-page.com"
}
}
```
```json title="workflow_validation_anomaly_change" theme={null}
{
"eventType": "workflow_validation_anomaly_change",
"timestamp": "2025-01-15T10:30:00Z",
"data": {
"validationId": "val_123",
"workflowId": "wf_123",
"jobId": "job_456",
"anomaliesCountTotal": 15,
"anomaliesChangeTotal": 10,
"anomaliesByRule": [
{ "ruleName": "missing_values", "count": 8, "change": 5 }
],
"previousJobId": "job_455",
"workflowName": "Product Monitor",
"createdAt": "2025-01-15T10:30:00Z"
}
}
```
`workflow_failed` fields: required `workflowId`, `source`, `reason`, `action`; optional `workflowName`, `sourceUrl`, `url`.
`workflow_recovered` fields: required `workflowId`, `workflowName`, `source`; optional `sourceUrl`, `url`.
| Event | Description |
| - | - |
| `workflow_data_change` | Monitored data changes. Difference types: `changed`, `added`, `removed` |
| `workflow_finished` | Workflow completes successfully |
| `workflow_failed` | Workflow fails. Includes `reason` and a human-readable `action`. |
| `workflow_recovered` | Workflow recovers after failure. Sent via `workflow_failed` subscription; suppressed if `workflow_finished` is also configured. |
| `workflow_validation_anomaly_change` | Validation anomalies change |
## Error Handling
Kadoa retries failed webhook deliveries with exponential backoff:
| Attempt | Delay |
| - | - |
| 1st retry | 1 second |
| 2nd retry | 2 seconds |
| 3rd retry | 4 seconds |
Your endpoint should return a `2xx` status code to acknowledge receipt.
# WebSockets
Source: https://docs.kadoa.com/docs/destinations/websockets
Receive real-time event notifications via WebSockets
Connect to Kadoa's WebSocket server for instant event notifications. Kadoa broadcasts events to your connected clients.
WebSockets are Kadoa's lowest-latency notification channel, but they should be treated as a best-effort realtime transport rather than an exactly-once delivery system. For critical workflows, make your event handling idempotent and use `event.id` as a dedupe key.
## Setup
```typescript title="Node SDK" theme={null}
import { KadoaClient } from "@kadoa/node-sdk";
const client = new KadoaClient({ apiKey: "YOUR_API_KEY" });
const realtime = await client.connectRealtime();
realtime.onEvent((event) => {
console.log("Event:", event.type, event.message);
});
realtime.onConnection((connected) => {
console.log("Connected:", connected);
});
realtime.onError((error) => {
console.error("Error:", error);
});
```
```python title="Python SDK" theme={null}
import asyncio
from kadoa_sdk import KadoaClient, KadoaClientConfig
async def main() -> None:
client = KadoaClient(KadoaClientConfig(api_key="YOUR_API_KEY"))
realtime = await client.connect_realtime()
realtime.on_event(lambda event: print("Event:", event))
realtime.on_connection(lambda connected, reason=None: print("Connected:", connected))
realtime.on_error(lambda error: print("Error:", error))
try:
await asyncio.Event().wait()
finally:
await realtime.close_async()
asyncio.run(main())
```
`workflow_failed` message fields: required `workflowId`, `source`, `reason`, `action`; optional `workflowName`, `sourceUrl`, `url`.
`workflow_recovered` message fields: required `workflowId`, `workflowName`, `source`; optional `sourceUrl`, `url`.
For API configuration, see the [API reference](/api-reference/notifications).
### Dashboard Setup
1. Go to **Notifications** in the sidebar
2. Click **Add Channel** → **WebSocket**
3. Select the **Enable Websocket real-time streaming** checkbox.
![WebSocket setup]()
## Connect to both EU and US
Keep two WebSocket connections open concurrently, one to each regional endpoint:
| Connection | Endpoint |
| - | - |
| EU | `wss://realtime-eu.kadoa.com` |
| US | `wss://realtime-us.kadoa.com` |
`wss://realtime.kadoa.com` routes to the nearest healthy region. Using it alongside
the US endpoint does not guarantee connections to two different regions.
1. Use the same team's API key for both connections. For each connection, request
a token with `POST https://api.kadoa.com/v4/oauth2/token`, passing the key in
the `x-api-key` header. The response contains `access_token` and `team_id`.
2. Connect to each regional endpoint with `?access_token=` appended.
Keep API keys, tokens, and authenticated URLs out of logs.
3. Send the following message on **both** sockets, using the returned `team_id`:
```json theme={null}
{ "action": "subscribe", "channel": "" }
```
4. Confirm each socket receives `subscribe.ack`. Its `region` should be
`europe-west3` for EU and `us-east1` for US. Treat acknowledgments, heartbeats,
and `control.draining` messages separately from business events.
5. Both sockets receive the same team feed. Process the first arrival of each
event and deduplicate subsequent copies by `event.id` across both connections.
If separate processes consume the sockets, they need shared deduplication or
idempotent downstream processing.
6. Reconnect each socket independently to its regional endpoint, obtaining a fresh
token and subscribing again. Track the latest `_cursor` separately for each
connection, including duplicate deliveries, and send it as `lastCursor` when
resubscribing. Follow the drain and heartbeat handling below while keeping the
other connection active.
To verify the setup, disconnect EU and confirm US continues receiving events.
Restore EU, then repeat for US. Both regions share upstream event infrastructure;
this protects against a connection or regional serving failure, not every possible
pipeline outage.
## Reconnects and Redeploys
During an infrastructure drain or redeploy, Kadoa sends a `control.draining` message before closing the socket. This is an expected lifecycle event, not a workflow failure and not a permanent outage.
Clients should reconnect automatically when they receive `control.draining`:
1. Detect `control.draining`.
2. If it includes `retryAfterMs`, wait roughly that long. Otherwise use a short reconnect delay.
3. Open a new WebSocket connection.
4. Re-subscribe to your team channel after the new connection is open.
Kadoa also sends heartbeat messages. If your client stops receiving heartbeats for too long, close the stale socket and reconnect using the same flow.
After the drain notice, Kadoa will close the old socket with close code `1001` and reason `Server shutting down`. Treat that close as the final shutdown of the old connection, not as a workflow error.
### Reliability Notes
* WebSockets are optimized for low-latency notifications, not for guaranteed exactly-once delivery.
* A reconnecting client can still observe a brief gap during deploys, drains, or network interruptions.
* Custom clients can reduce that gap by persisting the latest event `_cursor` and re-subscribing with `lastCursor`.
* Even with overlap-aware reconnects, clients should treat `event.id` as an idempotency key and dedupe repeated deliveries.
### Advanced Custom Client Behavior
Kadoa may send an additive control-plane message before closing a socket:
```json theme={null}
{
"type": "control.draining",
"connectionId": "12345-1711111111111-7",
"retryAfterMs": 500,
"deadlineAt": "2026-03-22T14:05:30.000Z",
"resumeSupported": true
}
```
Business events may include an optional `_cursor`:
```json theme={null}
{
"eventType": "workflow_finished",
"id": "event-uuid",
"timestamp": "2025-01-15T10:30:00.000Z",
"_cursor": "1742-0",
"message": {}
}
```
If you run a custom client and want lower-disruption reconnects, store the latest `_cursor` and re-subscribe with:
```json theme={null}
{
"action": "subscribe",
"channel": "",
"lastCursor": "1742-0"
}
```
This improves continuity, but it is still not a promise of zero-loss or exactly-once delivery.
```python title="Python reconnect example" theme={null}
import json
import threading
from collections.abc import Mapping
import requests
import websocket
API_KEY = "YOUR_API_KEY"
PUBLIC_API_URI = "https://api.kadoa.com"
WSS_API_URI = "wss://realtime.kadoa.com"
RECONNECT_DELAY_SECONDS = 5
CONNECTION_DRAINING_CLOSE_CODE = 1001
team_id: str | None = None
last_cursor: str | None = None
def fetch_access_token() -> tuple[str, str]:
response = requests.post(
f"{PUBLIC_API_URI}/v4/oauth2/token",
headers={"Content-Type": "application/json", "x-api-key": API_KEY},
timeout=10,
)
response.raise_for_status()
data: Mapping[str, object] = response.json()
access_token = data["access_token"]
current_team_id = data["team_id"]
if not isinstance(access_token, str) or not isinstance(current_team_id, str):
raise ValueError("Token response must include string access_token and team_id values")
return access_token, current_team_id
def connect() -> None:
access_token, current_team_id = fetch_access_token()
def on_open(ws: websocket.WebSocketApp) -> None:
global team_id
team_id = current_team_id
payload = {"action": "subscribe", "channel": current_team_id}
if last_cursor:
payload["lastCursor"] = last_cursor
ws.send(json.dumps(payload))
def on_message(ws: websocket.WebSocketApp, message: str) -> None:
global last_cursor
payload: Mapping[str, object] = json.loads(message)
if payload.get("type") == "heartbeat":
return
if payload.get("type") == "control.draining":
retry_after_ms = payload.get("retryAfterMs")
retry_after_seconds = (
retry_after_ms / 1000
if isinstance(retry_after_ms, (int, float))
else RECONNECT_DELAY_SECONDS
)
threading.Timer(retry_after_seconds, connect).start()
return
cursor = payload.get("_cursor")
if isinstance(cursor, str):
last_cursor = cursor
print("event", payload["eventType"], payload)
def on_close(
ws: websocket.WebSocketApp,
close_status_code: int | None,
close_msg: str | None,
) -> None:
print(f"closed code={close_status_code} reason={close_msg}")
if close_status_code == CONNECTION_DRAINING_CLOSE_CODE:
print("old draining socket closed")
def on_error(ws: websocket.WebSocketApp, error: object) -> None:
print("websocket error", error)
socket = websocket.WebSocketApp(
f"{WSS_API_URI}?access_token={access_token}",
on_open=on_open,
on_message=on_message,
on_close=on_close,
on_error=on_error,
)
socket.run_forever()
connect()
```
## Event Handling
Filter events by type:
```typescript theme={null}
realtime.onEvent((event) => {
switch (event.type) {
case "workflow_finished":
console.log("Workflow completed:", event.message.id);
break;
case "workflow_data_change":
console.log("Data changed:", event.message.differences);
break;
case "workflow_failed":
console.log("Workflow failed:", event.message.reason);
break;
case "workflow_recovered":
console.log("Workflow recovered:", event.message.workflowId);
break;
}
});
```
## Event Format
All events follow this structure:
```json theme={null}
{
"eventType": "event_name",
"id": "event-uuid",
"timestamp": "2025-01-15T10:30:00.000Z",
"_cursor": "1742-0",
"message": { /* event-specific data */ }
}
```
```json title="workflow_data_change" theme={null}
{
"eventType": "workflow_data_change",
"id": "2df91fbd-74c1-4d11-91aa-50030393574b",
"timestamp": "2025-01-15T10:30:00.000Z",
"message": {
"id": "change_123",
"workflowId": "wf_123",
"data": [
{ "id": "record-1", "name": "Product A", "price": 29.99 }
],
"differences": [
{
"type": "changed",
"fields": [
{ "key": "price", "value": 29.99, "previousValue": 24.99 }
]
}
],
"url": "https://monitored-page.com",
"createdAt": "2025-01-09T10:00:00Z"
}
}
```
```json title="workflow_finished" theme={null}
{
"eventType": "workflow_finished",
"id": "event-uuid",
"timestamp": "2025-01-15T10:30:00.000Z",
"message": {
"id": "wf_123",
"jobId": "job_456",
"source": "scheduler"
}
}
```
```json title="workflow_failed" theme={null}
{
"eventType": "workflow_failed",
"id": "event-uuid",
"timestamp": "2025-01-15T10:30:00.000Z",
"message": {
"workflowId": "wf_123",
"workflowName": "Product Monitor",
"source": "Product Monitor",
"sourceUrl": "https://monitored-page.com",
"url": "https://monitored-page.com",
"reason": "Access blocked by bot protection",
"action": "No action needed. We'll email you when the workflow recovers."
}
}
```
```json title="workflow_recovered" theme={null}
{
"eventType": "workflow_recovered",
"id": "event-uuid",
"timestamp": "2025-01-15T10:45:00.000Z",
"message": {
"workflowId": "wf_123",
"workflowName": "Product Monitor",
"source": "Product Monitor",
"sourceUrl": "https://monitored-page.com",
"url": "https://monitored-page.com"
}
}
```
```json title="workflow_validation_anomaly_change" theme={null}
{
"eventType": "workflow_validation_anomaly_change",
"id": "event-uuid",
"timestamp": "2025-01-15T10:30:00.000Z",
"message": {
"validationId": "val_123",
"workflowId": "wf_123",
"jobId": "job_456",
"anomaliesCountTotal": 15,
"anomaliesChangeTotal": 10,
"anomaliesByRule": [
{ "ruleName": "missing_values", "count": 8, "change": 5 }
],
"previousJobId": "job_455",
"workflowName": "Product Monitor",
"createdAt": "2025-01-15T10:30:00Z"
}
}
```
# Introduction
Source: https://docs.kadoa.com/docs/introduction
Kadoa automates web scraping at scale with AI. Our AI agents extract, transform, and integrate data from any website.
Effortless, reliable, compliant.
## Start building
Describe what you want; the Assistant builds the workflow.
Python and Node.js SDKs, or the REST API.
Connect the MCP server and use Kadoa from ChatGPT, Claude, Cursor, and other AI tools.
## Kadoa basics
The core unit: a source, a schema, and a schedule.
Per-field validation rules on every run.
Get only what changed, as it changes.
Reuse a prompt, schema, and notifications.
Route events to email, Slack, and webhooks.
Snowflake, cloud storage, Google Sheets, and more.
# Notifications
Source: https://docs.kadoa.com/docs/notifications
Get notified when workflow events occur
Notifications alert you when workflows complete, fail, or detect data changes. You can receive them via email, webhooks, or WebSockets.
## Configuration Levels
Configure notifications at two levels:
* **Workspace-Level**: Default notifications for all workflows unless overridden
* **Workflow-Level**: Apply to a specific workflow and override workspace settings
## Event Types
### Events You Can Subscribe To
| Event | Dashboard label | Description |
| - | - | - |
| `workflow_finished` | Workflow finished | Workflow completed successfully |
| `workflow_failed` / `workflow_recovered` | Workflow health | Workflow failed or recovered ([details below](#workflow-health)) |
| `workflow_data_change` | Data changes | Data changes detected ([Change Detection Rules](/docs/change-detection)) |
| `workflow_validation_anomaly_change` | Data quality issues | New data quality issues |
These same events can also start another workflow — see [Triggers](/docs/ui/workflows/triggers).
### Workflow Health
Subscribing to **Workflow health** gives you failure and recovery notifications for your workflows.
* **Failure**: When a run fails, you receive a `workflow_failed` event with the error reason and a suggested action.
* **Recovery**: When the workflow succeeds after a failure, you receive a `workflow_recovered` event.
If you also subscribe to **Workflow finished**, recovery notifications are replaced by the `workflow_finished` event for scheduled workflows, so you won't receive both for the same run.
For real-time monitors, `workflow_recovered` is always sent when the monitor comes back online, regardless of other subscriptions.
### System Events
These events are automatic and can't be configured:
| Event | Description |
| - | - |
| `workflow_sample_finished` | Preview data ready. Sent to workflow author only via **email**. |
| `workflow_export_completed` | Data exported to [cloud storage](/docs/destinations/amazon-s3). Fires on failure. |
## Notification Channels
| Channel | Description |
| - | - |
| **Email** | Sent to your account email or custom recipients ([details](/docs/destinations/email)) |
| **Slack** | Send notifications to Slack channels ([details](/docs/destinations/slack)) |
| **Webhook** | For custom destinations ([details](/docs/destinations/webhooks)) |
| **WebSocket** | Real-time data streaming ([details](/docs/destinations/websockets)) |
Create a channel before subscribing to events. One channel can receive multiple event types.
Payload structure varies by channel. Please see each channel's documentation for format details and examples.
## Data Change Configuration
For `workflow_data_change` events, filter what triggers notifications:
* **Fields**: Select specific fields to track (all by default)
* **Conditions**: Filter notifications based on criteria
* **Change Type**: Select what type of change you want to track
* **Updated**: Additions, modifications, and removals. Classifying a modification as an update requires at least one [schema key field](/docs/workflows/schemas); without a key, it is reported as one removal and one addition.
* **Added**: New records only
* **Removed**: Deleted records only
The `workflow_data_change` is triggered after the data changes are detected according to [Change Detection Rules](/docs/change-detection).
## Notification Content
Choose what notifications contain. Set workspace defaults in [Settings → Notifications](https://www.kadoa.com/settings?section=notifications) and override them per workflow.
| Block | Event types | Default |
| - | - | - |
| **AI summary** | Data changes | Off |
| **Changed records** | Data changes | On |
| **CSV attachment** | Data changes, Workflow finished | Off |
![Workspace notification content]()
CSV files are attached to the email when small enough; larger exports are delivered as a download link. The link is always included.
## Templates
Notifications can be included as part of a [template](/docs/templates). When a workflow is linked to a template that includes notifications, the notification settings are managed through the template.
## Managing Notifications
* [UI Docs](/docs/ui/notifications)
* [SDK Docs](/docs/sdk/notifications/overview)
Do you need help in setting up notifications? Contact [support team](https://www.kadoa.com/contact?type=support).
# Observability
Source: https://docs.kadoa.com/docs/observability
Reliability, throughput, and incident metrics for every workflow in your workspace
The [**Observability**](https://www.kadoa.com/observability) page in the left sidebar shows reliability, throughput, and incidents for every workflow in your workspace.
Pick a range of 30, 90, or 365 days. Headline numbers compare against the previous period, and **Export CSV** downloads the per-workflow table.
This page covers your workflows. For Kadoa platform availability, see [status.kadoa.com](https://status.kadoa.com).
## Success rate
One bar per day across the selected range.
* **Scheduled workflows**: the share of runs that completed successfully.
* **Real-time monitors**: monitor uptime. Run counts stay blank because polls are not runs.
* **Grey bar**: no data for that day.
![Success rate]()
The table lists every workflow with its daily trend, run counts, and success rate. If your workspace has real-time monitors, an **Avg reload (ms)** column shows their average polling cycle time.
## Records delivered
Cumulative rows delivered by completed runs, broken down per workflow with the average per run. Workflows that delivered nothing are listed too.
![Records delivered]()
## Tickets
Tickets are created automatically when a workflow fails, or manually by your team in the [Support Center](/docs/support-center).
![Tickets]()
The distribution chart shows how long resolved tickets took. Click a ticket ID to open it.
## Mean time to recover
Average time between a workflow's first failure and its next successful run.
![Mean time to recover]()
## Turnaround time
How long each recently created workflow took to get up and running, split into three phases:
| Phase | Measures |
| - | - |
| **Setup** | Creating the workflow until its first successful preview |
| **Approval** | How long the preview waited for your team's approval |
| **Runtime** | Median duration of a completed production run |
![Turnaround time]()
## Send this data to your own stack
You can mirror everything on this page into your own observability tooling or data warehouse.
### Events
Subscribe to the [workflow events](/docs/notifications#event-types) you care about, including run failures and data changes.
Push them with [webhooks](/docs/destinations/webhooks) or [websockets](/docs/destinations/websockets), or pull them with the [REST API](/docs/destinations/rest-api). Example payloads and available fields are in the [payload reference](/docs/destinations/webhooks#payload-reference).
### Aggregated metrics
We can deliver the aggregated metrics shown on this page to your cloud storage bucket on a schedule, ready to load into Snowflake, BigQuery, Databricks, or Redshift. See [Amazon S3](/docs/destinations/amazon-s3) for the ingestion patterns, and request setup through the [Support Center](/docs/support-center).
## Learn more
* [Notifications](/docs/notifications)
* [Data quality](/docs/data-quality)
* [Support Center](/docs/support-center)
# Command Line Interface (CLI)
Source: https://docs.kadoa.com/docs/sdk/cli
Manage Kadoa workflows from the terminal
The Kadoa CLI offers similar features to the [MCP Server](/docs/sdk/mcp) — workflow, monitor, notification, variable, template, and change operations available from your terminal, scripts, and CI/CD pipelines. It wraps the [Node SDK](/docs/sdk/introduction) and runs on Node 18+.
## Prerequisites
* A [Kadoa account](https://kadoa.com/signup)
* The Kadoa CLI installed (see [Installation](/docs/sdk/introduction#cli))
## Authentication
The CLI authenticates with OAuth. A `tk-...` API key is not a CLI token. `kadoa login` opens your browser, signs you in via Kadoa SSO, and stores the access + refresh tokens in `~/.kadoa/config.json` (mode `0600`). Tokens refresh automatically; you stay logged in until you `kadoa logout`.
For non-interactive use, provide both `KADOA_ACCESS_TOKEN` and `KADOA_REFRESH_TOKEN` so the CLI can refresh the OAuth session.
```bash theme={null}
# Interactive OAuth login
kadoa login
# Sign out (removes ~/.kadoa/config.json)
kadoa logout
```
Check your current auth status, active team, and role:
```bash theme={null}
kadoa whoami
# Email: john@acme.com
# Name: John Doe
# Auth: OAuth (config)
# Token: valid
# Active team: Acme Corp [ADMIN]
```
## Quick Start
Authenticate via your browser — no API key required.
```bash theme={null}
kadoa login
```
Describe what to extract in natural language and point it at a URL.
```bash theme={null}
kadoa create "Extract product names and prices" --url https://sandbox.kadoa.com/ecommerce
```
Trigger a run, then read the extracted data.
```bash theme={null}
kadoa run
kadoa data
```
Stream the full dataset to a file.
```bash theme={null}
kadoa export --format csv --out products.csv
```
## Commands
### Global Options
| Option | Description |
| - | - |
| `--json` | Force JSON output |
| `-V, --version` | Show version |
| `-h, --help` | Show help |
### Auth & Team
| Command | Description |
| - | - |
| `kadoa login` | OAuth login (saves credentials to `~/.kadoa/config.json`) |
| `kadoa logout` | Remove stored credentials |
| `kadoa whoami` | Show current user, team, and role |
| `kadoa team list` | List your teams and see which is active |
| `kadoa team switch [team]` | Switch active team by name or ID |
### Workflows
| Command | Description |
| - | - |
| `kadoa create --url ` | Create a prompt-driven workflow. Repeat `--url` for multi-URL workflows on the same domain. |
| `kadoa list` | List workflows. Flags: `--state `, `--limit `, `--format ` |
| `kadoa get ` | Get workflow details |
| `kadoa run ` | Run a workflow. `--limit ` caps records. |
| `kadoa pause ` | Pause an ACTIVE workflow so it stops running on its schedule |
| `kadoa approve ` | Approve and activate a workflow (also resumes a paused one) |
| `kadoa update ` | Update workflow config. Flags: `--name`, `--url`, `--prompt`, `--schedule`, `--limit` |
| `kadoa delete ` | Delete a workflow. Use `--force` to skip the confirmation. |
`kadoa create` accepts the full flag set: `--name`, `--entity`, `--description`, `--tag`, `--limit`, `--schedule`, `--cron`, plus `--notify-email/--notify-webhook/--notify-slack-channel/--notify-websocket` to wire notifications at creation time.
The `list` and `get` `status` field shows the computed [workflow status](/docs/workflows/lifecycle), such as Complete, Running, Failed, Paused, or Scheduled. Although the filter flag is named `--state`, it filters this computed status rather than the API response's persisted `state` field.
### Data
| Command | Description |
| - | - |
| `kadoa data ` | Paginated read. Flags: `--limit `, `--page `, `--format ` |
| `kadoa export ` | Export the full dataset via signed URL. Flags: `--format `, `--out `, `--run-id `, `--filters `, `--sort-by `, `--order ` |
`export` is the right choice for large workflows — it streams the full dataset rather than paginating.
### Real-time Monitors
Real-time monitors watch a page and alert on changes. Each detected change is captured in the `changes` log.
| Command | Description |
| - | - |
| `kadoa monitor create [prompt] --url ` | Create a real-time monitor. At least one `--notify-*` channel is required. |
| `kadoa changes` | List detected changes. Flags: `--workflow `, `--start-date `, `--end-date `, `--skip `, `--limit `, `--format` |
| `kadoa change ` | Show full snapshot and diff for a single detected change |
### Notifications
Notifications are split into **channels** (where alerts are delivered: email, webhook, Slack, WebSocket) and **settings** (which events route to which channels).
| Command | Description |
| - | - |
| `kadoa notifications channels list` | List channels. `--workflow ` filters to a workflow. |
| `kadoa notifications channels get ` | Show details for a single channel |
| `kadoa notifications channels create --type ` | Create a channel (see flags below) |
| `kadoa notifications channels delete ` | Delete a channel. `--force` skips the prompt. |
| `kadoa notifications settings list` | List event-to-channel mappings. `--workflow `, `--event ` filter. |
| `kadoa notifications settings delete ` | Delete a setting (channels are kept) |
| `kadoa notifications configure --event ` | One-step setup: create/reuse a channel and wire it to an event |
`notifications` is aliased as `notify`. `channels create` flags depend on `--type`:
* `EMAIL` → `--recipient ` (repeatable; omit for account default), `--name `
* `WEBHOOK` → `--url `, `--method `, `--auth-bearer ` or `--auth-basic `
* `SLACK` → `--slack-channel-id ` or `--slack-channel-name ` (OAuth) or `--slack-webhook ` (legacy)
* `WEBSOCKET` → no extra flags
`configure` mirrors the [MCP `configure_notifications` tool](/docs/sdk/mcp#notifications) — accepts `--workflow`, repeatable `--event` (or `--event all`), and the same `--notify-*` flags as `kadoa create`.
### Variables
Variables are key/value pairs you reference in workflow prompts as `@key`.
| Command | Description |
| - | - |
| `kadoa vars list` | List variables in the active team |
| `kadoa vars get ` | Show a single variable |
| `kadoa vars create --key --value ` | Create a variable. `--type ` |
| `kadoa vars update ` | Update key, value, or type |
| `kadoa vars delete ` | Delete a variable. `--force` skips the prompt. |
`vars` is aliased as `variables`.
### Templates
Templates are reusable, versioned workflow configurations (prompt + schema + notifications).
| Command | Description |
| - | - |
| `kadoa templates list` | List templates in the active team |
| `kadoa templates get ` | Show a template with all published versions |
| `kadoa templates create --name ` | Create an empty template. `--description ` |
| `kadoa templates update ` | Update name or description |
| `kadoa templates delete ` | Archive a template (existing workflows are unaffected). `--force` skips the prompt. |
| `kadoa templates version ` | Publish a new version. `--prompt `, `--schema-id ` or `--schema-entity ` |
| `kadoa templates save ` | Save a workflow as a new template (`--name`, `--description`) or as a new version of an existing one (`--template-id`) |
| `kadoa templates schemas ` | List schemas associated with a template |
### Shell Completion
Tab-complete subcommands, flags, and workflow IDs:
```bash theme={null}
# Zsh — add to ~/.zshrc
eval "$(kadoa completion zsh)"
# Bash — add to ~/.bashrc
eval "$(kadoa completion bash)"
```
Zsh shows workflow names inline: `kadoa get ` displays `abc123 -- My Workflow`.
## Output Formatting
The CLI auto-detects the right format: tables in interactive terminals, JSON when piping or redirecting. Override with `--json` or `--format`.
```bash theme={null}
kadoa list --json # Force JSON
kadoa data --format csv # CSV output
kadoa list --json | jq '.[].id' # Pipe to jq
kadoa data --format csv > data.csv # Save to file
```
## CI/CD
Supply OAuth tokens via environment variables and the CLI runs non-interactively:
```yaml theme={null}
# GitHub Actions
env:
KADOA_ACCESS_TOKEN: ${{ secrets.KADOA_ACCESS_TOKEN }}
KADOA_REFRESH_TOKEN: ${{ secrets.KADOA_REFRESH_TOKEN }}
steps:
- run: |
kadoa run $WORKFLOW_ID --json
kadoa export $WORKFLOW_ID --format csv --out output.csv
```
## Permissions
The commands you can run depend on your team role. Viewers and Compliance Officers have read-only access — write operations return a 403 with a hint to contact your team admin. The permission matrix matches the [MCP Server tool permissions](/docs/sdk/mcp#tools).
Each command also accepts `--help` for the full flag list, e.g. `kadoa create --help`.
# Data Quality
Source: https://docs.kadoa.com/docs/sdk/data-quality/overview
Read and edit the per-field data quality rules of a workflow using the SDK
Data quality rules are per-field checks that run at the end of every workflow run. Kadoa generates a default set for each workflow. With the SDK you read the current rules, replace the rules of specific fields, and remove the rules of a field.
For the available rule kinds and how each one is evaluated, see [Data Quality](/docs/data-quality).
## Prerequisites
* Kadoa account with API key
* SDK installed: `npm install @kadoa/node-sdk` or `pip install kadoa-sdk`
* A workflow with a schema. Rules are keyed by schema field name.
## Set rules for fields
Fields in the request replace their existing rules. Fields you leave out keep theirs. Every rule carries `editedBy` and `editedAt`, so the dashboard can show who changed it. API clients send `editedBy: "user"`. Send `editedAt` as UTC ending in `Z`.
```typescript Node SDK theme={null}
import { KadoaClient } from "@kadoa/node-sdk";
const client = new KadoaClient({ apiKey: "YOUR_API_KEY" });
const workflowId = "WORKFLOW_ID";
// Every rule records who set it and when
const edited = {
editedBy: "user",
editedAt: new Date().toISOString(),
} as const;
const rules = await client.dataQuality.upsertRules(workflowId, {
title: {
kind: "STRING",
presence: { target: 100, ...edited },
maxLength: { value: 120, ...edited },
},
link: {
kind: "STRING",
format: {
kind: "FORMAT",
source: { kind: "PRESET", preset: "url" },
...edited,
},
},
});
console.log(Object.keys(rules)); // ["title", "link"]
```
```python Python SDK theme={null}
from datetime import datetime, timezone
from kadoa_sdk import KadoaClient, KadoaClientConfig
client = KadoaClient(KadoaClientConfig(api_key="YOUR_API_KEY"))
workflow_id = "WORKFLOW_ID"
# Every rule records who set it and when. The API expects UTC ending in "Z"
edited = {
"editedBy": "user",
"editedAt": datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ"),
}
rules = client.data_quality.upsert_rules(
workflow_id,
{
"title": {
"kind": "STRING",
"presence": {"target": 100, **edited},
"maxLength": {"value": 120, **edited},
},
"link": {
"kind": "STRING",
"format": {
"kind": "FORMAT",
"source": {"kind": "PRESET", "preset": "url"},
**edited,
},
},
},
)
print(list(rules)) # ['title', 'link']
```
```bash REST API theme={null}
curl -X PUT "https://api.kadoa.com/v4/workflows/WORKFLOW_ID/schema-validation-rules" \
-H "x-api-key: YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"title": {
"kind": "STRING",
"presence": { "target": 100, "editedBy": "user", "editedAt": "2026-09-22T10:00:00Z" },
"maxLength": { "value": 120, "editedBy": "user", "editedAt": "2026-09-22T10:00:00Z" }
},
"link": {
"kind": "STRING",
"format": {
"kind": "FORMAT",
"source": { "kind": "PRESET", "preset": "url" },
"editedBy": "user",
"editedAt": "2026-09-22T10:00:00Z"
}
}
}'
```
The response is the merged ruleset of the whole workflow. Rule edits apply on the next run.
### Rule shapes
The `kind` of a field selects its rule shape and casts the field's values to that type before evaluation.
| Kind | Rules |
| - | - |
| `STRING` | `presence`, `uniqueness`, `minLength`, `maxLength`, `minHtmlElements`, `maxHtmlElements`, `format` |
| `NUMBER` | `presence`, `uniqueness`, `minimum`, `maximum`, `maxDecimalPlaces` |
| `DATE` | `presence`, `uniqueness`, `minimum`, `maximum` (a fixed `YYYY-MM-DD` date or a preset relative to the run date) |
| `OBJECT` | `presence`, `uniqueness`, `additionalProperties`, and `properties` with the rules of each sub-field |
| `ARRAY` | `presence`, `uniqueness`, `minItems`, `maxItems`, and `items` with the rules applied to every element |
| `OTHER` | `presence`, `uniqueness` |
`presence` and `uniqueness` take a `target` of 0, 20, 40, 60, 80, or 100 percent. A string `format` is one of three kinds: `FREE_TEXT` with a charset preset, `FORMAT` with a pattern preset or a custom regular expression, and `LIST` with a list preset or custom values. The Node SDK exports these shapes as types, starting from `DataQualityRules`. The Python SDK takes and returns them as plain dicts.
## Read the rules of a workflow
```typescript Node SDK theme={null}
import { KadoaClient } from "@kadoa/node-sdk";
const client = new KadoaClient({ apiKey: "YOUR_API_KEY" });
const workflowId = "WORKFLOW_ID";
// null when the workflow has no rules
const rules = await client.dataQuality.getRules(workflowId);
for (const [field, fieldRules] of Object.entries(rules ?? {})) {
console.log(field, fieldRules.kind, fieldRules.presence?.target);
}
```
```python Python SDK theme={null}
from kadoa_sdk import KadoaClient, KadoaClientConfig
client = KadoaClient(KadoaClientConfig(api_key="YOUR_API_KEY"))
workflow_id = "WORKFLOW_ID"
# None when the workflow has no rules
rules = client.data_quality.get_rules(workflow_id)
for field, field_rules in (rules or {}).items():
print(field, field_rules["kind"], field_rules.get("presence", {}).get("target"))
```
```bash REST API theme={null}
curl -X GET "https://api.kadoa.com/v4/workflows/WORKFLOW_ID/schema-validation-rules" \
-H "x-api-key: YOUR_API_KEY"
```
## Remove the rules of one field
```typescript Node SDK theme={null}
import { KadoaClient } from "@kadoa/node-sdk";
const client = new KadoaClient({ apiKey: "YOUR_API_KEY" });
const workflowId = "WORKFLOW_ID";
// Other fields keep their rules
const remaining = await client.dataQuality.deleteFieldRules(
workflowId,
"link",
);
console.log(Object.keys(remaining)); // ["title"]
```
```python Python SDK theme={null}
from kadoa_sdk import KadoaClient, KadoaClientConfig
client = KadoaClient(KadoaClientConfig(api_key="YOUR_API_KEY"))
workflow_id = "WORKFLOW_ID"
# Other fields keep their rules
remaining = client.data_quality.delete_field_rules(workflow_id, "link")
print(list(remaining)) # ['title']
```
```bash REST API theme={null}
curl -X DELETE "https://api.kadoa.com/v4/workflows/WORKFLOW_ID/schema-validation-rules/link" \
-H "x-api-key: YOUR_API_KEY"
```
## Limits
* A rule tree nests at most 4 levels deep and holds at most 200 rules.
* Workflows linked to a template inherit their rules from the template. Editing them returns `409` with code `TEMPLATE_CONTROLLED_FIELD`. Unlink the workflow from its template first.
* Edits do not change the results of the current run. They apply from the next run on.
## Learn more
* [Data quality concepts and rule kinds](/docs/data-quality)
* [Edit rules in the UI](/docs/ui/data-quality)
* [API reference](/api-reference/data-quality/get-rules)
# Getting Started
Source: https://docs.kadoa.com/docs/sdk/introduction
Create and run your first extraction using SDK, CLI, MCP Server, or REST API
## Prerequisites
* Create a [Kadoa account](https://kadoa.com/signup)
* Generate your API key from [Settings > General](https://kadoa.com/settings)
## Choose Your Approach
TypeScript/Node.js and Python SDKs for type-safe development
Language-agnostic HTTP API for any platform
Manage workflows from the terminal and CI/CD pipelines
Use Kadoa from ChatGPT, Claude, Cursor, and other AI assistants
## Installation
### SDK
```bash theme={null}
npm install @kadoa/node-sdk
```
```bash theme={null}
yarn add @kadoa/node-sdk
```
```bash theme={null}
pnpm add @kadoa/node-sdk
```
```bash theme={null}
bun add @kadoa/node-sdk
```
```bash theme={null}
uv add kadoa-sdk
```
```bash theme={null}
pip install kadoa-sdk
```
You can view the Kadoa SDK source code on [GitHub](https://github.com/kadoa-org/kadoa-sdks).
### CLI
```bash theme={null}
npm install -g @kadoa/cli
```
```bash theme={null}
pnpm add -g @kadoa/cli
```
```bash theme={null}
npx @kadoa/cli
```
The CLI uses OAuth. Sign in interactively, or provide both OAuth tokens so a non-interactive session can refresh. A `tk-...` API key is not a CLI token.
```bash theme={null}
# Interactive OAuth
kadoa login
# Non-interactive OAuth
export KADOA_ACCESS_TOKEN=YOUR_ACCESS_TOKEN
export KADOA_REFRESH_TOKEN=YOUR_REFRESH_TOKEN
```
Enable tab completions for workflow IDs, subcommands, and flags:
```bash theme={null}
# Add to ~/.zshrc
eval "$(kadoa completion zsh)"
```
Zsh shows workflow names inline: `kadoa get ` displays `abc123 -- My Workflow`.
```bash theme={null}
# Add to ~/.bashrc
eval "$(kadoa completion bash)"
```
See the [full CLI reference](/docs/sdk/cli) for all commands.
### MCP Server
The remote MCP server at `https://mcp.kadoa.com/mcp` requires no local install or API key; connect from your MCP client and sign in with your Kadoa account via OAuth. See [MCP Server setup](/docs/sdk/mcp) for per-client instructions.
For local use via stdio:
```bash theme={null}
npx -y @kadoa/mcp
```
## Quick Start
```typescript Node SDK theme={null}
import { KadoaClient } from '@kadoa/node-sdk';
const client = new KadoaClient({
apiKey: 'YOUR_API_KEY'
});
const result = await client.extraction.run({
urls: ["https://sandbox.kadoa.com/ecommerce"],
name: "My First Extraction",
limit: 10,
});
console.log(result.data);
```
```python Python SDK theme={null}
from kadoa_sdk import ExtractionOptions, KadoaClient, KadoaClientConfig
client = KadoaClient(KadoaClientConfig(api_key="YOUR_API_KEY"))
result = client.extraction.run(
ExtractionOptions(
urls=["https://sandbox.kadoa.com/ecommerce"],
name="My First Extraction",
limit=10,
)
)
print(result.data)
```
```bash CLI theme={null}
kadoa create "Extract product names and prices" \
--url https://sandbox.kadoa.com/ecommerce \
--name "My First Extraction"
```
```text MCP Server theme={null}
> "Extract product names and prices from https://sandbox.kadoa.com/ecommerce"
```
```bash REST API theme={null}
# 1. Create workflow with schema definition
curl -X POST https://api.kadoa.com/v4/workflows \
-H "x-api-key: YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"urls": ["https://sandbox.kadoa.com/ecommerce"],
"name": "My First Extraction",
"entity": "Product",
"fields": [
{
"name": "title",
"dataType": "STRING",
"description": "Product name"
},
{
"name": "price",
"dataType": "MONEY",
"description": "Product price"
}
]
}'
# Response:
# {
# "success": true,
# "workflowId": "507f1f77bcf86cd799439011"
# }
# 2. Check workflow status
curl -X GET https://api.kadoa.com/v4/workflows/507f1f77bcf86cd799439011 \
-H "x-api-key: YOUR_API_KEY"
# 3. Get the data (once extraction is complete)
curl -X GET https://api.kadoa.com/v4/workflows/507f1f77bcf86cd799439011/data \
-H "x-api-key: YOUR_API_KEY"
```
First-time setup may take longer while Kadoa prepares the workflow. Synchronous SDK calls wait up to 30 minutes by default.
That's it! With the SDK, data is automatically extracted. With the API, you define the schema and then retrieve your structured data.
## Define What to Extract
For more control, specify exactly what fields you want:
```typescript Node SDK theme={null}
const workflow = await client
.extract({
urls: ["https://sandbox.kadoa.com/ecommerce"],
name: "Product Monitor",
extraction: (builder) =>
builder
.entity("Product")
.field("name", "Product name", "STRING", {
example: "MacBook Pro",
})
.field("price", "Price in USD", "MONEY")
.field("inStock", "Is available", "BOOLEAN"),
})
.create();
const result = await workflow.run({ limit: 10 });
const response = await result.fetchData({});
console.log(response.data);
```
```python Python SDK theme={null}
from kadoa_sdk import (
ExtractOptions,
FieldOptions,
KadoaClient,
KadoaClientConfig,
RunWorkflowOptions,
)
client = KadoaClient(KadoaClientConfig(api_key="YOUR_API_KEY"))
extract_options = ExtractOptions(
urls=["https://sandbox.kadoa.com/ecommerce"],
name="Product Monitor",
extraction=lambda builder: builder.entity("Product")
.field("name", "Product name", "STRING", FieldOptions(example="MacBook Pro"))
.field("price", "Price in USD", "MONEY")
.field("inStock", "Is available", "BOOLEAN"),
)
workflow = client.extract(extract_options).create()
print(f"Workflow created: {workflow.workflow_id}")
result = workflow.run(RunWorkflowOptions(limit=10))
response = result.fetch_data({})
print(response.data)
```
```bash CLI theme={null}
kadoa create "Extract product name, price, and availability" \
--url https://sandbox.kadoa.com/ecommerce \
--name "Product Monitor" \
--entity "Product"
```
```text MCP Server theme={null}
> "Extract product name, price in USD, and availability from https://sandbox.kadoa.com/ecommerce"
```
```javascript REST API theme={null}
// POST https://api.kadoa.com/v4/workflows
{
"urls": ["https://sandbox.kadoa.com/ecommerce"],
"name": "Product Monitor",
"entity": "Product",
"fields": [
{
"name": "name",
"dataType": "STRING",
"description": "Product name"
},
{
"name": "price",
"dataType": "MONEY",
"description": "Price in USD"
},
{
"name": "inStock",
"dataType": "BOOLEAN",
"description": "Is available"
}
]
}
```
## Next Steps
* [Working with Schemas →](/docs/sdk/schemas/overview) - Learn about inline and reusable schemas
* [Variables →](/docs/sdk/variables/overview) - Manage reusable key-value pairs for workflow prompts
* [Templates →](/docs/sdk/templates/overview) - Create reusable, versioned workflow configurations
* [Create Workflows →](/docs/sdk/workflows/create) - Write prompts, define schemas, and configure scheduling
* [Manage Workflows →](/docs/sdk/workflows/manage) - List, retrieve, pause, and delete workflows
* [Access Data →](/docs/destinations/sdk) - Retrieve extracted data with pagination and real-time updates
# MCP Server
Source: https://docs.kadoa.com/docs/sdk/mcp
The Kadoa MCP server gives any compatible AI model or agent a standardized, secure way to work with Kadoa via the [Model Context Protocol](https://modelcontextprotocol.io). It has tools for creating, running, and managing workflows, fetching and exporting extracted data, and configuring notifications, variables, and templates, all through natural language.
Kadoa’s MCP server follows the authenticated remote [MCP spec](https://modelcontextprotocol.io/specification/2025-03-26), so the server is centrally hosted and managed.
For help getting started after setup, see some of our [example prompts](#common-use-cases).
## Setup
### General
The server is centrally hosted and managed by Kadoa, using Streamable HTTP as its transport. There is nothing to install and no API key to manage.
There are two ways to connect:
* **Personal connection** (`https://mcp.kadoa.com/mcp`): You sign in with your own Kadoa account, and every tool call runs as you, under your role and active team. Best for individual use.
* **Service connector** (`https://service.mcp.kadoa.com/mcp`): A workspace admin generates OAuth credentials once, and applications or organization members connect as a shared workspace service account, with no Kadoa login. Best for organization-wide rollouts and app contexts where nobody can sign in interactively. See [MCP Service Connector →](/docs/sdk/mcp-service-connector).
| | Personal connection | Service connector |
| - | - | - |
| Who signs in | Each user | Nobody; OAuth client ID and secret |
| Acts as | You (your role and team) | Shared workspace service account |
| Set up by | Each user | A workspace admin, once |
| Best for | Individual use | Organizations, apps, automations |
For a personal connection, add the server in your client: a browser opens so you can sign in with your Kadoa account and grant access. The connection then stays authorized; if a session expires, your client prompts you to sign in again.
With a personal connection, the MCP server acts as you: every tool runs under your Kadoa role and active team, and each action is recorded in the workflow's [history](/api-reference/workflows/get-the-workflow-history).
Instructions for specific clients follow below.
### Claude
**Team, Enterprise (Claude.ai)**
A Claude organization owner can roll Kadoa out to everyone at once: add a custom connector for `https://mcp.kadoa.com/mcp` under **Admin settings → Connectors** (members sign in with their own Kadoa accounts), or use the [service connector](/docs/sdk/mcp-service-connector) so members connect without Kadoa accounts.
Individual members can also add it themselves under **Settings → Connectors → Add custom MCP** with the same URL.
**Free, Pro (Claude.ai)**
1. Go to **Settings → Connectors → Add custom MCP**
2. Enter the URL: `https://mcp.kadoa.com/mcp`
3. Sign in with your Kadoa account via OAuth
### Claude Code
```bash theme={null}
claude mcp add kadoa --transport http https://mcp.kadoa.com/mcp
```
Then run `/mcp` in a Claude Code session to go through the authentication flow.
### Claude Desktop
Add the server to your Claude Desktop config file (create it if it doesn't exist), then restart Claude Desktop:
```json theme={null}
// macOS: ~/Library/Application Support/Claude/claude_desktop_config.json
// Linux: ~/.config/Claude/claude_desktop_config.json
// Windows: %APPDATA%\Claude\claude_desktop_config.json
{
"mcpServers": {
"kadoa": {
"type": "http",
"url": "https://mcp.kadoa.com/mcp"
}
}
}
```
### ChatGPT
1. Go to **Settings → Connectors → Add MCP server**
2. Enter the URL: `https://mcp.kadoa.com/mcp`
3. Choose OAuth authentication and sign in with your Kadoa account
### Codex
```bash theme={null}
codex mcp add kadoa --url https://mcp.kadoa.com/mcp
```
Then run `codex mcp login kadoa` to go through the authentication flow. If this is your first remote MCP server in Codex, enable the rmcp client in `~/.codex/config.toml` first:
```toml theme={null}
[features]
experimental_use_rmcp_client = true
```
### Cursor
```json theme={null}
// .cursor/mcp.json
{
"mcpServers": {
"kadoa": {
"type": "http",
"url": "https://mcp.kadoa.com/mcp"
}
}
}
```
### Gemini CLI
```bash theme={null}
gemini mcp add -t http kadoa https://mcp.kadoa.com/mcp
```
### Others
Any MCP-compatible client can connect with the following settings:
* **URL:** `https://mcp.kadoa.com/mcp`
* **Transport:** Streamable HTTP
* **Auth:** OAuth (sign in with your Kadoa account)
## Tools
The tools you can use depend on your team role. Members, Admins, and Owners can use every tool except `approve_workflow`, which is reserved for Compliance Officers. Viewers and Compliance Officers otherwise have read-only access. If a tool is not available for your role, you'll receive an error message with instructions to contact your team admin.
The columns below show access for the **Member / Admin / Owner**, **Viewer**, and **Compliance Officer** roles.
### Workflows
| Tool | Description | Member / Admin / Owner | Viewer | Compliance Officer |
| - | - | :-: | :-: | :-: |
| `create_workflow` | Create an agentic navigation workflow from a prompt | | | |
| `list_workflows` | List all workflows with status | | | |
| `get_workflow` | Get detailed workflow information including prompt, schedules, and notification config | | | |
| `update_workflow` | Update workflow configuration and schema | | | |
| `delete_workflow` | Delete a workflow | | | |
| `approve_workflow` | Approve and activate a workflow | | | |
| `pause_workflow` | Pause an active workflow so it stops running on its schedule | | | |
| `request_workflow_update` | Ask the Kadoa Assistant to update an existing workflow without replacing it | | | |
To find workflows that need action, call `list_workflows` with `statusFilters: ["group:attention"]`. It includes workflows waiting for an Assistant answer, unfinished setup, or preview approval.
### Kadoa Assistant
| Tool | Description | Member / Admin / Owner | Viewer | Compliance Officer |
| - | - | :-: | :-: | :-: |
| `get_workflow_assistant` | Get Assistant lifecycle state and any pending clarification question | | | |
| `get_workflow_assistant_timeline` | Read paginated customer-visible Assistant messages and questions | | | |
| `answer_workflow_assistant_question` | Answer the current clarification question and request resume | | | |
| `interrupt_workflow_assistant` | Safely interrupt Assistant work without pausing the workflow schedule | | | |
| `resume_workflow_assistant` | Resume inactive or interrupted Assistant work | | | |
| `stop_workflow_assistant` | Stop active Assistant work without deleting the workflow | | | |
| `get_workflow_strategy` | Get the latest customer-safe build strategy when available | | | |
| `get_workflow_history` | List configuration revision history with field-level diffs | | | |
### Realtime monitoring
| Tool | Description | Member / Admin / Owner | Viewer | Compliance Officer |
| - | - | :-: | :-: | :-: |
| `create_realtime_monitor` | Create an asynchronous realtime monitor after notification settings are persisted | | | |
### Runs and extracted data
| Tool | Description | Member / Admin / Owner | Viewer | Compliance Officer |
| - | - | :-: | :-: | :-: |
| `list_workflow_runs` | List execution run history; filter by outcome and paginate | | | |
| `run_workflow` | Execute a workflow | | | |
| `fetch_data` | Get extracted data from a workflow (paginated, with filters and sorting) | | | |
| `export_data` | Export the full extracted dataset as a signed URL, ideal for large workflows | | | |
### Detected changes
| Tool | Description | Member / Admin / Owner | Viewer | Compliance Officer |
| - | - | :-: | :-: | :-: |
| `list_changes` | List detected data changes across real-time monitoring workflows | | | |
| `get_change` | Get detailed information about a specific detected change | | | |
### Notifications
| Tool | Description | Member / Admin / Owner | Viewer | Compliance Officer |
| - | - | :-: | :-: | :-: |
| `list_notification_channels` | List notification channels for a workflow or workspace | | | |
| `create_notification_channel` | Create an email, webhook, Slack, or WebSocket notification channel | | | |
| `delete_notification_channel` | Delete a notification channel | | | |
| `list_notification_settings` | List event-to-channel notification mappings | | | |
| `configure_notifications` | Set up notifications for workflow events in one step | | | |
| `delete_notification_setting` | Remove a notification event-to-channel mapping | | | |
### Teams and identity
| Tool | Description | Member / Admin / Owner | Viewer | Compliance Officer |
| - | - | :-: | :-: | :-: |
| `whoami` | Show current user details and team memberships | | | |
| `team_list` | List all teams you belong to and see which is active | | | |
| `team_switch` | Switch the active team by name or ID | | | |
`whoami` identifies the current account and team. In Kadoa Assistant sessions, it reports the selected team by name and role rather than an internal identifier.
### Activity
| Tool | Description | Member / Admin / Owner | Viewer | Compliance Officer |
| - | - | :-: | :-: | :-: |
| `list_activity` | List team-wide Activity Log events, with filters and optional change details | | | |
Use `list_activity` to answer team-wide questions such as who changed a workflow, which workflows failed, or what happened this week. Events are newest first. Filter by time, workflow, user, event type, resource type, or interface. Use `get_workflow_history` instead when you need configuration revisions and field-level diffs for one workflow.
### Observability
| Tool | Description | Member / Admin / Owner | Viewer | Compliance Officer |
| - | - | :-: | :-: | :-: |
| `get_observability` | Get workflow health, throughput, recovery, and attention items for the active team | | | |
`get_observability` reports a 30-day window by default. Ask for a shorter or longer window when you need it, up to 365 days. It returns a ranked attention list for down and degraded workflows. Add `includeWorkflows` or `includeDailySeries` only when you need the larger result.
### Variables
| Tool | Description | Member / Admin / Owner | Viewer | Compliance Officer |
| - | - | :-: | :-: | :-: |
| `create_variable` | Create a key-value variable (STRING, NUMBER, BOOLEAN, or JSON) | | | |
| `list_variables` | List all variables in the current team scope | | | |
| `get_variable` | Get a specific variable by ID | | | |
| `update_variable` | Update a variable's key, value, or data type | | | |
| `delete_variable` | Delete a variable (two-step confirmation) | | | |
### Templates
| Tool | Description | Member / Admin / Owner | Viewer | Compliance Officer |
| - | - | :-: | :-: | :-: |
| `list_templates` | List all templates in the current team | | | |
| `get_template` | Get a template by ID, including all published versions | | | |
| `create_template` | Create a reusable template for workflow configurations | | | |
| `update_template` | Update a template's name or description | | | |
| `delete_template` | Delete a template (two-step confirmation) | | | |
| `create_template_version` | Publish a version with prompt, schema, and notifications | | | |
| `save_workflow_as_template` | Create a template from an existing workflow's configuration | | | |
| `list_template_schemas` | List schemas associated with a template | | | |
| `link_template_to_workflows` | Link existing workflows to a template | | | |
| `apply_template_version` | Roll out a template version to linked workflows | | | |
| `list_template_workflows` | List workflows linked to a template | | | |
| `unlink_template_from_workflows` | Unlink workflows from a template, keeping their configuration | | | |
The `status` field returned by `list_workflows` and `get_workflow` shows the computed [workflow status](/docs/workflows/lifecycle), such as Complete, Running, Failed, Paused, or Scheduled. The tool's `state` filter uses these computed status values rather than the API response's persisted `state` field.
### Scheduling
The `create_workflow` tool accepts an `interval` parameter for recurring extractions (defaults to `ONLY_ONCE`). Use `CUSTOM` with cron expressions for fine-grained control. See [Scheduling →](/docs/sdk/workflows/scheduling) for all available intervals and examples.
### Fetching and exporting data
Two tools retrieve extracted data: use `fetch_data` for paginated reads (default 50 rows, max 500) with `filters`, `sortBy`, and `order`; use `export_data` for the full dataset. It returns a self-authenticating signed URL ideal for Claude for Excel ('Get Data from Web') or pandas/duckdb code execution.
### Workflow history
`get_workflow_history` returns configuration revision history: who changed a workflow and which fields changed. `list_workflow_runs` is different: it returns execution outcomes, timing, record counts, and errors. Use its `status` filter (`success`, `failed`, or `in_progress`) and pagination when diagnosing runs.
### Updating workflows with the Assistant
Use `request_workflow_update` to change an existing workflow in place. Then use `get_workflow_assistant` to inspect lifecycle state and any pending question. Answer a pending question with `answer_workflow_assistant_question`; do not use `resume_workflow_assistant` for a question that still needs an answer. Interrupt, resume, and stop control Assistant work, not the workflow schedule.
`create_realtime_monitor` is asynchronous and requires at least one notification channel. It returns the workflow, session, thread, and job IDs for follow-up. See [Kadoa Assistant](/docs/sdk/workflows/assistant) for the full lifecycle.
## Common use cases
Once connected, you manage the full workflow lifecycle through natural conversation. Copy an example prompt into your client to explore what's possible; see the [tools](#tools) above for the full surface:
> Create a workflow that extracts product name, price, and rating from [https://sandbox.kadoa.com/ecommerce](https://sandbox.kadoa.com/ecommerce), run it, and show me the first results.
> Set up a workflow that scrapes competitor prices from [https://sandbox.kadoa.com/ecommerce](https://sandbox.kadoa.com/ecommerce) daily at 6:00, and email me when a run finishes or fails.
> Show me all changes my real-time monitors detected this week and summarize what moved.
> Export the full 'Product Monitor' dataset and analyze price trends by category.
> Update the 'Product Monitor' schema to add a rating field, then pause the old 'Legacy Scraper' workflow.
> My Product Monitor is waiting for a question. Show me what it needs, then answer it: use the US store.
> Save 'Product Monitor' as a template called 'Product Scraper', then create a workflow from it for each of these five store URLs.
> Which workflows need attention, which ones are failing, and what changed across my team this week?
> List my teams, switch to Acme Corp, and show me which workflows are failing there.
### Changing a template
A published template version is an immutable snapshot. To change shared prompt, schema, validation, notifications, or frequency, create a new version and apply it to the workflows that should adopt it. `update_template` changes only a template's name or description. See [Templates](/docs/sdk/templates/overview) for the rollout flow.
## Troubleshooting
### I don't have access to Kadoa
* Verify the MCP server is configured in the correct config file
* Restart your MCP client after configuration changes
* Re-authenticate via OAuth if prompted
### Authentication failed
* Your OAuth session may have expired, so re-authenticate when prompted by your MCP client
* If switching teams, verify the team exists with `team_list` before using `team_switch`
### `update_workflow` returns "No approval received"
This message comes from your MCP client, not from Kadoa. Claude (and some other clients) gate certain tool calls behind a one-time approval prompt, and `update_workflow` is often the first tool in a session to hit it. The call never reaches the Kadoa MCP server, so nothing on the Kadoa side (roles, team permissions, API access) is involved.
* Retry the call. The gate usually clears for the rest of the session after the first prompt.
* If it persists, look for an approval prompt in your client and click through it, or set the Kadoa connector's tools to "Always allow" in your client's connector settings.
* Read-only tools (`get_workflow`, `fetch_data`) and other write tools (`run_workflow`, `delete_workflow`) are not affected, so if those work and only `update_workflow` fails, this is the cause.
# MCP Service Connector
Source: https://docs.kadoa.com/docs/sdk/mcp-service-connector
The MCP service connector connects applications and organization members to the Kadoa MCP server through a shared workspace service account for programmatic access. A workspace admin generates OAuth client credentials once and everyone who connects through them acts as the service account, with no individual Kadoa accounts or login prompts.
For individual access using your own Kadoa account, see [MCP Server setup →](/docs/sdk/mcp#setup).
## Setup
1. In Kadoa, open **Settings → General → MCP service connector**.
2. Select **Generate credentials** and copy the MCP server URL, OAuth client ID, and OAuth client secret. The secret is shown once.
3. A Claude organization owner opens **Admin settings → Connectors → Add custom connector**.
4. Enter `https://service.mcp.kadoa.com/mcp`, then paste the OAuth client ID and secret under Advanced settings.
5. In the connector's tool settings, allow only the tools that organization members should use.
6. Organization members open **Settings → Connectors**, find the custom connector, and select **Connect**.
## How it works
The connector is an OAuth client (a client ID and secret) registered with Kadoa for your workspace. When an organization member selects **Connect** in Claude, the OAuth handshake completes silently against `https://service.mcp.kadoa.com/mcp`, with no Kadoa login or consent screen. The resulting session acts as your workspace's shared service account.
* The service connector uses a separate hostname from the personal connector at `https://mcp.kadoa.com/mcp`, so users can configure both side by side.
* All members share the same service-account identity and team, and team switching is unavailable.
* Each workspace has at most one active connector.
The service connector acts as the workspace service account: every tool call runs under the shared service account and its team, and workflow [history](/api-reference/workflows/get-the-workflow-history) records the service account, not the individual member who ran the action. If you need per-user attribution, use the [personal connection](/docs/sdk/mcp#setup).
## Supported clients
Claude (Team and Enterprise custom connectors) is supported today. Other MCP clients need their OAuth redirect URI registered with Kadoa first. [Contact support](https://www.kadoa.com/contact?type=support) to enable one.
## Manage access
* The client secret is shown once, at generation or rotation. Store it securely.
* To rotate credentials, select **Rotate secret** in Kadoa, replace the secret in Claude's connector settings, and reconnect. Rotation immediately invalidates tokens issued with the previous secret.
* To stop all access immediately, select **Revoke connector**.
* To limit what members can do, restrict the allowed tools in the MCP client's connector settings.
Never paste a Kadoa workspace API key into any MCP client. The service connector's OAuth credentials are the only credentials that belong there.
# Notifications
Source: https://docs.kadoa.com/docs/sdk/notifications/overview
Get workflow notifications programmatically
Notifications alert you when workflows complete, fail, or detect data changes. You can receive them via email, webhooks, or WebSockets. The SDK handles channel management automatically.
For event types and channel details, see [Notifications](/docs/notifications).
## Prerequisites
* Kadoa account with API key
* SDK installed: `npm install @kadoa/node-sdk` or `uv add kadoa-sdk`
## Quick Start
```typescript Node SDK theme={null}
import { KadoaClient } from "@kadoa/node-sdk";
const client = new KadoaClient({ apiKey: "YOUR_API_KEY" });
await client.notification.setupForWorkflow({
workflowId: "YOUR_WORKFLOW_ID",
events: ["workflow_data_change"],
channels: {
WEBHOOK: {
name: "my-webhook",
webhookUrl: "https://your-server.com/webhook",
httpMethod: "POST",
},
},
});
```
```python Python SDK theme={null}
from kadoa_sdk import KadoaClient, KadoaClientConfig
from kadoa_sdk.notifications import SetupWorkflowNotificationSettingsRequest
client = KadoaClient(KadoaClientConfig(api_key="YOUR_API_KEY"))
client.notification.setup_for_workflow(
SetupWorkflowNotificationSettingsRequest(
workflow_id="YOUR_WORKFLOW_ID",
events=["workflow_data_change"],
channels={
"WEBHOOK": {
"name": "my-webhook",
"webhookUrl": "https://your-server.com/webhook",
"httpMethod": "POST",
}
},
)
)
```
```text MCP Server theme={null}
> "Set up a webhook notification on my 'Product Monitor' workflow that triggers on data changes, sending to https://your-server.com/webhook"
```
The SDK reuses existing channels with the same name. Running setup multiple times won't create duplicates.
## Channel Types
### Email
```typescript Node SDK theme={null}
// Use account email
await client.notification.setupForWorkflow({
workflowId: workflowId,
events: ["workflow_finished", "workflow_failed"],
channels: { EMAIL: true },
});
// Or specify recipients
await client.notification.setupForWorkflow({
workflowId: workflowId,
events: ["workflow_finished"],
channels: {
EMAIL: {
name: "team-notifications",
recipients: ["team@example.com"],
},
},
});
```
```python Python SDK theme={null}
from kadoa_sdk import KadoaClient, KadoaClientConfig
from kadoa_sdk.notifications import SetupWorkflowNotificationSettingsRequest
client = KadoaClient(KadoaClientConfig(api_key="YOUR_API_KEY"))
# Use account email
client.notification.setup_for_workflow(
SetupWorkflowNotificationSettingsRequest(
workflow_id="YOUR_WORKFLOW_ID",
events=["workflow_finished", "workflow_failed"],
channels={"EMAIL": True},
)
)
# Or specify recipients
client.notification.setup_for_workflow(
SetupWorkflowNotificationSettingsRequest(
workflow_id="YOUR_WORKFLOW_ID",
events=["workflow_finished"],
channels={
"EMAIL": {
"name": "team-notifications",
"recipients": ["team@example.com"],
}
},
)
)
```
### Slack
```typescript Node SDK theme={null}
await client.notification.setupForWorkflow({
workflowId: workflowId,
events: ["workflow_failed"],
channels: {
SLACK: {
name: "team-notifications",
slackChannelId: "C1234567890",
slackChannelName: "alerts",
webhookUrl: "https://hooks.slack.com/services/YOUR/WEBHOOK",
},
},
});
```
```python Python SDK theme={null}
from kadoa_sdk import KadoaClient, KadoaClientConfig
from kadoa_sdk.notifications import SetupWorkflowNotificationSettingsRequest
client = KadoaClient(KadoaClientConfig(api_key="YOUR_API_KEY"))
client.notification.setup_for_workflow(
SetupWorkflowNotificationSettingsRequest(
workflow_id="YOUR_WORKFLOW_ID",
events=["workflow_failed"],
channels={
"SLACK": {
"name": "team-notifications",
"slackChannelId": "C1234567890",
"slackChannelName": "alerts",
"webhookUrl": "https://hooks.slack.com/services/YOUR/WEBHOOK",
}
},
)
)
```
### Webhook
```typescript Node SDK theme={null}
await client.notification.setupForWorkflow({
workflowId: workflowId,
events: ["workflow_data_change", "workflow_finished"],
channels: {
WEBHOOK: {
name: "api-destination",
webhookUrl: "https://api.example.com/webhooks/kadoa",
httpMethod: "POST",
},
},
});
```
```python Python SDK theme={null}
from kadoa_sdk import KadoaClient, KadoaClientConfig
from kadoa_sdk.notifications import SetupWorkflowNotificationSettingsRequest
client = KadoaClient(KadoaClientConfig(api_key="YOUR_API_KEY"))
client.notification.setup_for_workflow(
SetupWorkflowNotificationSettingsRequest(
workflow_id="YOUR_WORKFLOW_ID",
events=["workflow_finished"],
channels={
"WEBHOOK": {
"name": "api-destination",
"webhookUrl": "https://api.example.com/webhooks/kadoa",
"httpMethod": "POST",
}
},
)
)
```
### WebSocket
Receive events in real-time
For reconnect, drain, and reliability semantics, see [WebSockets](/docs/destinations/websockets).
```typescript Node SDK theme={null}
const client = new KadoaClient({ apiKey });
const realtime = await client.connectRealtime();
// Subscribe to all events
realtime.onEvent((event) => {
console.log("Event received:", event.type, event.message);
});
// Filter events by type
realtime.onEvent((event) => {
if (event.type === "workflow_finished") {
console.log("Workflow completed:", event.message);
}
});
// Handle errors
realtime.onError((error) => {
console.error("WebSocket error:", error);
});
realtime.close();
```
```python Python SDK theme={null}
import asyncio
from collections.abc import Mapping
from kadoa_sdk import KadoaClient, KadoaClientConfig
async def main() -> None:
client = KadoaClient(KadoaClientConfig(api_key="YOUR_API_KEY"))
realtime = await client.connect_realtime()
# Subscribe to all events
realtime.on_event(lambda event: print("Event received:", event["type"], event["message"]))
# Filter events by type
def handle_filtered(event: Mapping[str, object]) -> None:
if event["type"] == "workflow_finished":
print("Workflow completed:", event["message"])
realtime.on_event(handle_filtered)
# Handle errors
realtime.on_error(lambda error: print("WebSocket error:", error))
try:
await asyncio.Event().wait()
finally:
await realtime.close_async()
asyncio.run(main())
```
## Subscribe to All Events
```typescript theme={null}
await client.notification.setupForWorkflow({
workflowId: workflowId,
events: "all",
channels: { EMAIL: true },
});
```
## Reusing Channels
Reference existing channels by ID
```typescript Node SDK theme={null}
const workflowId = "YOUR_WORKFLOW_ID";
// List channels
const channels = await client.notification.channels.listChannels({});
const webhookChannel = channels.find((channel) => channel.name === "my-webhook");
if (!webhookChannel) {
throw new Error("Webhook channel not found");
}
if (!webhookChannel.id) {
throw new Error("Webhook channel has no ID");
}
await client.notification.setupForWorkflow({
workflowId,
events: ["workflow_data_change"],
channels: {
WEBHOOK: { channelId: webhookChannel.id },
},
});
```
```python Python SDK theme={null}
from kadoa_sdk import KadoaClient, KadoaClientConfig
from kadoa_sdk.notifications import SetupWorkflowNotificationSettingsRequest
client = KadoaClient(KadoaClientConfig(api_key="YOUR_API_KEY"))
# List channels
channels = client.notification.channels.list_channels()
webhook_channel = next((c for c in channels if c.name == "my-webhook"), None)
if webhook_channel is None:
raise RuntimeError("Webhook channel not found")
client.notification.setup_for_workflow(
SetupWorkflowNotificationSettingsRequest(
workflow_id="YOUR_WORKFLOW_ID",
events=["workflow_data_change"],
channels={
"WEBHOOK": {"channelId": webhook_channel.id}
},
)
)
```
## Template-Managed Notifications
When a workflow is linked to a [template](/docs/templates) that includes notifications, the notification settings are managed through the template. Attempting to modify notifications on a template-linked workflow returns a `409` error. Unlink the workflow first, or update the template and apply the change.
## Setup Notifications via API
### Step 1: Create Notification Channels
First, create the channels you want to use for notifications. See [Email](/docs/destinations/email), [Webhooks](/docs/destinations/webhooks) and [WebSockets](/docs/destinations/websockets) for channel creation examples.
### Step 2: Subscribe to Events
Subscribe to events by sending a POST request to `/v5/notifications/settings`.
#### Workspace-Level Subscription
Apply notifications to all workflows:
```javascript Request theme={null}
// POST /v5/notifications/settings
{
"eventType": "workflow_finished",
"eventConfiguration": {},
"enabled": true,
"channelIds": [""]
}
```
```json Response theme={null}
{
"data": {
"settings": {
"id": "settings-123",
"eventType": "workflow_finished",
"eventConfiguration": {},
"enabled": true,
"channelIds": [""]
}
}
}
```
#### Workflow-Level Subscription
Apply notifications to a specific workflow:
```javascript Request theme={null}
// POST /v5/notifications/settings
{
"workflowId": "",
"eventType": "workflow_data_change",
"eventConfiguration": {},
"enabled": true,
"channelIds": [""]
}
```
```json Response theme={null}
{
"data": {
"settings": {
"id": "settings-456",
"workflowId": "",
"eventType": "workflow_data_change",
"eventConfiguration": {},
"enabled": true,
"channelIds": [""]
}
}
}
```
### Step 3: Update Existing Settings
To add more channels to existing settings, use a PUT request:
```javascript Request theme={null}
// PUT /v5/notifications/settings/
{
"channelIds": ["existing-channel-id", "new-channel-id"]
}
```
```json Response theme={null}
{
"data": {
"settings": {
"id": "",
"workflowId": "YOUR_WORKFLOW_ID",
"eventType": "workflow_data_change",
"eventConfiguration": {},
"enabled": true,
"channelIds": ["existing-channel-id", "new-channel-id"]
}
}
}
```
For payload formats, see [Email](/docs/destinations/email), [Webhooks](/docs/destinations/webhooks), and [WebSockets](/docs/destinations/websockets).
# Schemas
Source: https://docs.kadoa.com/docs/sdk/schemas/overview
Create and manage reusable schemas programmatically using the SDK
For an overview of schema concepts and available data types, see [Schemas](/docs/workflows/schemas).
## Prerequisites
* Kadoa account with API key
* SDK installed: `npm install @kadoa/node-sdk` or `uv add kadoa-sdk`
## Working with Schemas
Define the structure of data you want to extract using the builder API:
```typescript Node SDK theme={null}
const extraction = await client
.extract({
urls: ["https://sandbox.kadoa.com/ecommerce"],
name: "Product Extraction",
extraction: (builder) =>
builder
.entity("Product")
.field("title", "Product name", "STRING", { example: "Laptop" })
.field("price", "Product price", "MONEY")
.field("inStock", "Availability", "BOOLEAN")
.field("rating", "Star rating 1-5", "NUMBER"),
})
.create();
```
```python Python SDK theme={null}
from kadoa_sdk import ExtractOptions, FieldOptions, KadoaClient, KadoaClientConfig
client = KadoaClient(KadoaClientConfig(api_key="YOUR_API_KEY"))
extract_options = ExtractOptions(
urls=["https://sandbox.kadoa.com/ecommerce"],
name="Product Extraction",
extraction=lambda builder: builder.entity("Product")
.field("title", "Product name", "STRING", FieldOptions(example="Laptop"))
.field("price", "Product price", "MONEY")
.field("inStock", "Availability", "BOOLEAN")
.field("rating", "Star rating 1-5", "NUMBER"),
)
extraction = client.extract(extract_options).create()
print(f"Extraction created successfully: {extraction}")
```
### Reusable Schemas
For consistent data extraction across multiple workflows, you can create and manage schemas separately using the Schema Management API. Schemas can also be bundled into [templates](/docs/sdk/templates/overview) along with a prompt and notification settings.
## Schema Management API
The Schema Management API allows you to create, retrieve, and delete schemas programmatically. Saved schemas can be reused across multiple extractions, ensuring consistent data structure.
### When to Use Saved Schemas
Use saved schemas when you:
* Extract the same data structure from multiple websites
* Want to maintain consistent field definitions across workflows
* Need to programmatically manage schema lifecycle
* Share schemas across different parts of your application
For one-off extractions, inline schema definitions (shown above) are simpler and don't require separate schema management.
If you need to apply the same schema, prompt, and notifications to multiple workflows, consider using [templates](/docs/sdk/templates/overview) instead. Templates bundle all three into a versioned configuration.
## Create a Schema
```typescript Node SDK theme={null}
const schema = await client.schema.createSchema({
name: schemaName,
entity: "Product",
fields: [
{
name: "title",
description: "Product name",
fieldType: "SCHEMA",
dataType: "STRING",
example: "iPhone 15 Pro",
},
{
name: "price",
description: "Product price",
fieldType: "SCHEMA",
dataType: "MONEY",
},
{
name: "inStock",
description: "Availability",
fieldType: "SCHEMA",
dataType: "BOOLEAN",
},
{
name: "rating",
description: "Star rating",
fieldType: "SCHEMA",
dataType: "NUMBER",
},
],
});
console.log("Schema created:", schema.id);
```
```python Python SDK theme={null}
from kadoa_sdk import KadoaClient, KadoaClientConfig
from kadoa_sdk.schemas import (
CreateSchemaRequest,
DataField,
FieldExample,
SchemaField,
)
client = KadoaClient(KadoaClientConfig(api_key="YOUR_API_KEY"))
# Create field objects
fields = [
SchemaField(
actual_instance=DataField(
name="title",
description="Product name",
fieldType="SCHEMA",
dataType="STRING",
example=FieldExample(actual_instance="iPhone 15 Pro"),
)
),
SchemaField(
actual_instance=DataField(
name="price",
description="Product price",
fieldType="SCHEMA",
dataType="MONEY",
)
),
SchemaField(
actual_instance=DataField(
name="inStock",
description="Availability",
fieldType="SCHEMA",
dataType="BOOLEAN",
)
),
SchemaField(
actual_instance=DataField(
name="rating",
description="Star rating",
fieldType="SCHEMA",
dataType="NUMBER",
)
),
]
# Create schema request
create_request = CreateSchemaRequest(
name="Product Schema",
entity="Product",
fields=fields,
)
schema = client.schema.create_schema(create_request)
print("Schema created:", schema.id)
```
## Get a Schema
Retrieve an existing schema by ID:
```typescript Node SDK theme={null}
const schema = await client.schema.getSchema(schemaId);
console.log(schema.name); // 'Product Schema'
console.log(schema.entity); // 'Product'
console.log(schema.schema); // Array of field definitions
```
```python Python SDK theme={null}
from kadoa_sdk import KadoaClient, KadoaClientConfig
client = KadoaClient(KadoaClientConfig(api_key="YOUR_API_KEY"))
schema_id = "YOUR_SCHEMA_ID"
schema = client.schema.get_schema(schema_id)
print(schema.name)
print(schema.entity)
print(schema.var_schema) # Array of field definitions
```
## Delete a Schema
Remove a schema when it's no longer needed:
```typescript Node SDK theme={null}
await client.schema.deleteSchema(schemaId);
```
```python Python SDK theme={null}
from kadoa_sdk import KadoaClient, KadoaClientConfig
client = KadoaClient(KadoaClientConfig(api_key="YOUR_API_KEY"))
schema_id = "YOUR_SCHEMA_ID"
client.schema.delete_schema(schema_id)
```
Deleting a schema does not affect existing workflows or extractions that were created using it.
## Update a Schema
Modify an existing schema's name, entity, or fields:
```typescript Node SDK theme={null}
const updated = await client.schema.updateSchema(schemaId, {
name: updatedName,
entity: "Product",
fields: [
{
name: "title",
description: "Product name",
fieldType: "SCHEMA",
dataType: "STRING",
example: "MacBook Pro",
},
{
name: "price",
description: "Product price in USD",
fieldType: "SCHEMA",
dataType: "MONEY",
},
{
name: "sku",
description: "Stock keeping unit",
fieldType: "SCHEMA",
dataType: "STRING",
example: "Stock keeping unit",
},
],
});
console.log("Schema updated:", updated.id);
```
```python Python SDK theme={null}
from kadoa_sdk import KadoaClient, KadoaClientConfig
from kadoa_sdk.schemas import (
DataField,
FieldExample,
SchemaField,
UpdateSchemaRequest,
)
client = KadoaClient(KadoaClientConfig(api_key="YOUR_API_KEY"))
schema_id = "YOUR_SCHEMA_ID"
update_request = UpdateSchemaRequest(
name="Updated Product Schema",
entity="Product",
fields=[
SchemaField(
actual_instance=DataField(
name="title",
description="Product name",
fieldType="SCHEMA",
dataType="STRING",
example=FieldExample(actual_instance="MacBook Pro"),
)
),
SchemaField(
actual_instance=DataField(
name="price",
description="Product price in USD",
fieldType="SCHEMA",
dataType="MONEY",
)
),
SchemaField(
actual_instance=DataField(
name="sku",
description="Stock keeping unit",
fieldType="SCHEMA",
dataType="STRING",
example=FieldExample(actual_instance="SKU-123"),
)
),
],
)
updated = client.schema.update_schema(schema_id, update_request)
print("Schema updated:", updated.id)
```
## Use a Saved Schema
Reference a saved schema in your extraction:
```typescript Node SDK theme={null}
const extraction = await client
.extract({
urls: ["https://sandbox.kadoa.com/ecommerce"],
name: "Product Extraction",
extraction: () => ({ schemaId }),
})
.create();
const result = await extraction.run();
```
```python Python SDK theme={null}
from kadoa_sdk import ExtractOptions, KadoaClient, KadoaClientConfig
client = KadoaClient(KadoaClientConfig(api_key="YOUR_API_KEY"))
schema_id = "YOUR_SCHEMA_ID"
extract_options = ExtractOptions(
urls=["https://sandbox.kadoa.com/ecommerce"],
name="Product Extraction - PY-SCHEMAS-005",
extraction=lambda _: {"schemaId": schema_id},
)
extraction = client.extract(extract_options).create()
run_result = extraction.run()
print("Extraction completed:", run_result)
```
## Field Types
Schemas support four types of fields:
1. **Regular fields** - Structured data extraction (shown above)
2. **Classification fields** - Categorize content into predefined labels
3. **Source fields** - Capture values Kadoa already holds, such as the page URL or the full page HTML
4. **Function fields** - Fields Kadoa computes for you, such as a stable unique ID
### Available Data Types
For regular fields, specify the `dataType`:
`STRING` • `NUMBER` • `BOOLEAN` • `DATE` • `DATETIME` • `MONEY` • `IMAGE` • `LINK` • `OBJECT` • `ARRAY`
[See data type details and examples →](/docs/workflows/schemas#data-types)
## Classification Fields
Categorize extracted content into predefined labels:
```typescript Node SDK theme={null}
const schema = await client.schema.createSchema({
name: schemaName,
entity: "Article",
fields: [
{
name: "title",
description: "Article headline",
fieldType: "SCHEMA",
dataType: "STRING",
example: "Breaking News",
},
{
name: "category",
description: "Article category",
fieldType: "CLASSIFICATION",
categories: [
{ title: "Technology", definition: "Tech news and updates" },
{ title: "Business", definition: "Business and finance" },
{ title: "Sports", definition: "Sports coverage" },
],
},
],
});
```
```python Python SDK theme={null}
from kadoa_sdk import KadoaClient, KadoaClientConfig
from kadoa_sdk.schemas import (
Category,
ClassificationField,
CreateSchemaRequest,
DataField,
FieldExample,
SchemaField,
)
client = KadoaClient(KadoaClientConfig(api_key="YOUR_API_KEY"))
# Create categories
categories = [
Category(title="Technology", definition="Tech news and updates"),
Category(title="Business", definition="Business and finance"),
Category(title="Sports", definition="Sports coverage"),
]
# Create fields
fields = [
SchemaField(
actual_instance=DataField(
name="title",
description="Article headline",
fieldType="SCHEMA",
dataType="STRING",
example=FieldExample(actual_instance="Breaking News"),
)
),
SchemaField(
actual_instance=ClassificationField(
name="category",
description="Article category",
fieldType="CLASSIFICATION",
categories=categories,
)
),
]
# Create schema request
create_request = CreateSchemaRequest(
name="Article Schema",
entity="Article",
fields=fields,
)
schema = client.schema.create_schema(create_request)
print("Schema with classification created:", schema.id)
```
## Function Fields
Functions add fields whose values Kadoa computes for you instead of extracting them from the page. A function field is a regular `SCHEMA` field with an output `dataType`, a top-level `mapper`, and optional `mapperConfig`. Configuration is flat: for example, `mapper: "categoryClassifier"` uses `mapperConfig: { categoryName: "generic_english" }`. Classification retains its own field shape (`fieldType: CLASSIFICATION`), covered above. The previous `params.mapper` representation remains accepted for compatibility.
For example, add a stable unique ID to each record:
```typescript Node SDK theme={null}
const schema = await client.schema.createSchema({
name: schemaName,
entity: "Product",
fields: [
{
name: "title",
description: "Product name",
fieldType: "SCHEMA",
dataType: "STRING",
example: "Laptop",
},
{
name: "recordId",
description: "Unique ID",
fieldType: "SCHEMA",
dataType: "STRING",
mapper: "stableUniqueId",
},
],
});
console.log("Schema created:", schema.id);
```
Functions vary by workspace; the API rejects fields yours isn't enabled for.
## Source Fields
To keep raw page content, add a regular `SCHEMA` field with a `params.sourceKey` instead of an extraction. Supported source keys: `link` (page URL), `allExtracts.fullHtml`, and `allExtracts.fullMarkdown`.
```typescript Node SDK theme={null}
const schema = await client.schema.createSchema({
name: schemaName,
entity: "Article",
fields: [
{
name: "pageUrl",
description: "Page URL",
fieldType: "SCHEMA",
dataType: "LINK",
params: { sourceKey: "link" },
},
{
name: "fullHtml",
description: "Full page HTML",
fieldType: "SCHEMA",
dataType: "STRING",
params: { sourceKey: "allExtracts.fullHtml" },
},
],
});
console.log("Schema created:", schema.id);
```
# Templates
Source: https://docs.kadoa.com/docs/sdk/templates/overview
Create and manage reusable workflow templates programmatically using the SDK
Templates let you define reusable configurations (prompt, schema, data validation rules, notification settings, and run frequency) that can be linked to multiple workflows and versioned over time.
## Prerequisites
* Kadoa account with API key
* SDK installed: `npm install @kadoa/node-sdk`
Templates are currently available in the Node SDK. Python SDK support is coming soon.
## When to Use Templates
Use templates when you:
* Run the same extraction logic across multiple websites
* Want to roll out prompt or schema changes to many workflows at once
* Need versioned configurations with the ability to roll back
* Share standardized extraction setups across your team
For one-off extractions, inline schema definitions are simpler and don't require template management.
## Create a Template
```typescript Node SDK theme={null}
const template = await client.template.create({
name: "Job Listing",
description: "Extracts job postings with title, company, and location",
});
console.log("Template created:", template.id);
```
```bash REST API theme={null}
curl -X POST "https://api.kadoa.com/v4/templates" \
-H "x-api-key: YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"name": "Job Listing",
"description": "Extracts job postings with title, company, and location"
}'
```
```text MCP Server theme={null}
> "Create a template called 'Job Listing' for extracting job postings with title, company, and location"
```
## List Templates
```typescript Node SDK theme={null}
const templates = await client.template.list();
for (const template of templates) {
console.log(`${template.id}: ${template.name}`);
}
```
```bash REST API theme={null}
curl -X GET "https://api.kadoa.com/v4/templates" \
-H "x-api-key: YOUR_API_KEY"
```
```text MCP Server theme={null}
> "List my templates"
```
## Get a Template
Retrieve a template by ID, including all published versions:
```typescript Node SDK theme={null}
const template = await client.template.get("TEMPLATE_ID");
console.log(template.name);
console.log(template.description);
console.log(template.versions); // Array of published versions
```
```bash REST API theme={null}
curl -X GET "https://api.kadoa.com/v4/templates/TEMPLATE_ID" \
-H "x-api-key: YOUR_API_KEY"
```
```text MCP Server theme={null}
> "Show me the details of the Job Listing template"
```
## Update a Template
Modify a template's name or description:
```typescript Node SDK theme={null}
const updated = await client.template.update("TEMPLATE_ID", {
name: "Updated Job Listing",
description: "Now includes salary range",
});
console.log("Template updated:", updated.id);
```
```bash REST API theme={null}
curl -X PUT "https://api.kadoa.com/v4/templates/TEMPLATE_ID" \
-H "x-api-key: YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"name": "Updated Job Listing",
"description": "Now includes salary range"
}'
```
```text MCP Server theme={null}
> "Update the Job Listing template description to 'Now includes salary range'"
```
## Delete a Template
Deleting a template archives it. Existing linked workflows keep their current configuration but are unlinked.
```typescript Node SDK theme={null}
await client.template.delete("TEMPLATE_ID");
```
```bash REST API theme={null}
curl -X DELETE "https://api.kadoa.com/v4/templates/TEMPLATE_ID" \
-H "x-api-key: YOUR_API_KEY"
```
```text MCP Server theme={null}
> "Delete the Job Listing template"
```
## Versioning
Templates support versioning so you can iterate on configurations and roll back if needed.
### Publish a New Version
Each version can include prompt, schema fields, data validation rules, and notification settings:
```typescript Node SDK theme={null}
const version = await client.template.createVersion("TEMPLATE_ID", {
prompt: "Extract job listings including salary information",
schemaEntity: "JobListing",
schemaFields: [
{
name: "title",
description: "Job title",
fieldType: "SCHEMA",
dataType: "STRING",
},
{
name: "company",
description: "Company name",
fieldType: "SCHEMA",
dataType: "STRING",
},
{
name: "salary",
description: "Salary range",
fieldType: "SCHEMA",
dataType: "STRING",
},
],
});
console.log("Published version:", version.version);
```
```bash REST API theme={null}
curl -X POST "https://api.kadoa.com/v4/templates/TEMPLATE_ID/versions" \
-H "x-api-key: YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"prompt": "Extract job listings including salary information",
"schemaEntity": "JobListing",
"schemaFields": [
{
"name": "title",
"description": "Job title",
"fieldType": "SCHEMA",
"dataType": "STRING"
},
{
"name": "company",
"description": "Company name",
"fieldType": "SCHEMA",
"dataType": "STRING"
},
{
"name": "salary",
"description": "Salary range",
"fieldType": "SCHEMA",
"dataType": "STRING"
}
]
}'
```
```text MCP Server theme={null}
> "Add a new version to the Job Listing template with a salary field"
```
### Version Parameters
| Parameter | Description |
| - | - |
| `prompt` | User prompt to copy into linked workflows |
| `schemaFields` | Inline schema fields (mutually exclusive with `schemaId`) |
| `schemaId` | Reference an existing saved schema (mutually exclusive with `schemaFields`) |
| `schemaEntity` | Entity name for the schema |
| `dataValidation` | Validation config and rules (`regex`, `custom_sql`, or `llm`) |
| `notifications` | Notification event-to-channel mappings |
## Create a Workflow from a Template
A workflow created from a template inherits the selected version's prompt, schema, validation rules, notifications, and frequency. Optionally pass workflow-specific instructions: they are added to the template prompt and remain attached to that workflow when a later template version is applied.
The public field is `userPrompt` in REST and the SDKs, and `prompt` in MCP. `additionalInstructions` is internal and should not be sent by clients.
```typescript Node SDK theme={null}
const workflow = await client.workflow.create({
urls: ["https://example.de/jobs"],
templateId: "TEMPLATE_ID",
userPrompt: "Only include German-language listings and skip archived roles",
});
```
```python Python SDK theme={null}
from kadoa_sdk.workflows import CreateWorkflowInput
workflow = client.workflow.create(
CreateWorkflowInput(
urls=["https://example.de/jobs"],
template_id="TEMPLATE_ID",
user_prompt="Only include German-language listings and skip archived roles",
)
)
```
```bash REST API theme={null}
curl -X POST "https://api.kadoa.com/v4/workflows" \
-H "x-api-key: YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"urls": ["https://example.de/jobs"],
"templateId": "TEMPLATE_ID",
"userPrompt": "Only include German-language listings and skip archived roles"
}'
```
```text MCP Server theme={null}
> "Create a workflow from the Job Listing template for https://example.de/jobs. Only include German-language listings and skip archived roles."
```
Omitting `templateVersion` resolves the latest published version once at request time. When the selected template has no prompt, `userPrompt` becomes the complete workflow prompt.
## List Linked Workflows
```typescript Node SDK theme={null}
const workflows = await client.template.getLinkedWorkflows("TEMPLATE_ID");
for (const workflow of workflows) {
console.log(`${workflow.workflowId}: ${workflow.workflowName} (version ${workflow.templateVersion})`);
}
```
```bash REST API theme={null}
curl -X GET "https://api.kadoa.com/v4/templates/TEMPLATE_ID/workflows" \
-H "x-api-key: YOUR_API_KEY"
```
```text MCP Server theme={null}
> "Show me the workflows linked to the Job Listing template"
```
## Save from Workflow
Create a template from an existing workflow's configuration. Pass `name` to create a new template, or `templateId` to add a new version to an existing one:
```typescript Node SDK theme={null}
// Create a new template from a workflow
const newTemplate = await client.template.createFromWorkflow({
workflowId: "WORKFLOW_ID",
name: "Product Scraper",
description: "Created from existing product extraction workflow",
});
console.log("Template created:", newTemplate.templateId);
console.log("Version:", newTemplate.version);
```
```bash REST API theme={null}
curl -X POST "https://api.kadoa.com/v4/templates/from-workflow" \
-H "x-api-key: YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"workflowId": "WORKFLOW_ID",
"name": "Product Scraper",
"description": "Created from existing product extraction workflow"
}'
```
```text MCP Server theme={null}
> "Save my 'Product Monitor' workflow as a new template called 'Product Scraper'"
```
To add as a new version to an existing template:
```typescript Node SDK theme={null}
const newVersion = await client.template.createFromWorkflow({
workflowId: "WORKFLOW_ID",
templateId: "EXISTING_TEMPLATE_ID",
});
console.log("New version added:", newVersion.version);
```
## List Template Schemas
View the schemas associated with a template's versions:
```typescript Node SDK theme={null}
const schemas = await client.template.listSchemas("TEMPLATE_ID");
for (const schema of schemas) {
console.log(`${schema.schemaId}: ${schema.schemaName} (${schema.entity})`);
}
```
```bash REST API theme={null}
curl -X GET "https://api.kadoa.com/v4/templates/TEMPLATE_ID/schemas" \
-H "x-api-key: YOUR_API_KEY"
```
```text MCP Server theme={null}
> "Show me the schemas for the Job Listing template"
```
# Variables
Source: https://docs.kadoa.com/docs/sdk/variables/overview
Create and manage reusable variables programmatically using the SDK
Variables are key-value pairs that can be referenced in workflow prompts using `@variableKey` syntax. When a workflow runs, Kadoa replaces variable references with the actual values.
For an overview of variable concepts, data types, and usage in prompts, see [Variables](/docs/workflows/variables).
## Prerequisites
* Kadoa account with API key
* SDK installed: `npm install @kadoa/node-sdk`
Variables are currently available in the Node SDK. Python SDK support is coming soon.
## Create a Variable
```typescript Node SDK theme={null}
const variable = await client.variable.create({
key: "target_url",
value: "https://example.com/products",
dataType: "STRING",
});
console.log("Variable created:", variable.id);
```
```bash REST API theme={null}
curl -X POST "https://api.kadoa.com/v4/variables" \
-H "x-api-key: YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"key": "target_url",
"value": "https://example.com/products",
"dataType": "STRING"
}'
```
```text MCP Server theme={null}
> "Create a variable called target_url with value https://example.com/products"
```
### Data Types
| Type | Description | Example |
| - | - | - |
| `STRING` | Plain text (default) | `"ESG reports"` |
| `NUMBER` | Numeric values | `"25"` |
| `BOOLEAN` | True/false | `"true"` |
| `JSON` | Structured JSON data | `'{"zip": "10001"}'` |
Values are always passed as strings, regardless of data type.
## List Variables
```typescript Node SDK theme={null}
const variables = await client.variable.list();
for (const variable of variables) {
console.log(`${variable.key}: ${variable.value} (${variable.dataType})`);
}
```
```bash REST API theme={null}
curl -X GET "https://api.kadoa.com/v4/variables" \
-H "x-api-key: YOUR_API_KEY"
```
```text MCP Server theme={null}
> "List my variables"
```
## Get a Variable
```typescript Node SDK theme={null}
const variable = await client.variable.get("VARIABLE_ID");
console.log(variable.key);
console.log(variable.value);
console.log(variable.dataType);
```
```bash REST API theme={null}
curl -X GET "https://api.kadoa.com/v4/variables/VARIABLE_ID" \
-H "x-api-key: YOUR_API_KEY"
```
```text MCP Server theme={null}
> "Show me the target_url variable"
```
## Update a Variable
Workflows referencing this variable via `@key` will use the new value on their next run.
```typescript Node SDK theme={null}
const updated = await client.variable.update("VARIABLE_ID", {
value: "https://example.com/products/v2",
});
console.log("Variable updated:", updated.key);
```
```bash REST API theme={null}
curl -X PATCH "https://api.kadoa.com/v4/variables/VARIABLE_ID" \
-H "x-api-key: YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"value": "https://example.com/products/v2"
}'
```
```text MCP Server theme={null}
> "Update the target_url variable to https://example.com/products/v2"
```
## Delete a Variable
Deleting a variable is permanent. Workflows referencing it via `@key` will no longer resolve the value.
```typescript Node SDK theme={null}
await client.variable.delete("VARIABLE_ID");
```
```bash REST API theme={null}
curl -X DELETE "https://api.kadoa.com/v4/variables/VARIABLE_ID" \
-H "x-api-key: YOUR_API_KEY"
```
```text MCP Server theme={null}
> "Delete the target_url variable"
```
## Using Variables in Prompts
Reference variables in workflow prompts with `@variableKey`:
```typescript Node SDK theme={null}
// Create a variable
await client.variable.create({
key: "search_term",
value: "ESG reports",
dataType: "STRING",
});
// Use it in a workflow prompt; Kadoa replaces @search_term at runtime
const workflow = await client
.extract({
urls: ["https://sandbox.kadoa.com/ecommerce"],
name: "Variable-Driven Search",
userPrompt: "Search for @search_term and extract all matching results",
extraction: (builder) =>
builder
.entity("Result")
.field("title", "Result title", "STRING", {
example: "Annual ESG report",
})
.field("url", "Result URL", "LINK", {
example: "https://example.com/reports/annual-esg-report",
}),
})
.create();
```
# Kadoa Assistant
Source: https://docs.kadoa.com/docs/sdk/workflows/assistant
Build, update, and monitor workflows with the Kadoa Assistant
The [Kadoa Assistant](/docs/ui/getting-started) builds workflows from instructions and can update an existing workflow. Assistant requests are asynchronous: save the returned workflow, session, and thread IDs, then check the Assistant when you need its current state or a pending question.
The Assistant is available in the Node SDK, REST API, and [MCP Server](/docs/sdk/mcp). It is not currently available in the Python SDK.
## Create a realtime monitor
Use a realtime monitor when you want continuous change detection and alerts. It is a separate workflow mode: do not convert a scheduled workflow to realtime by changing its interval.
Realtime monitor creation requires at least one notification channel. The API persists notification settings before it starts the Assistant, so a successful response includes the workflow, session, and thread to use for follow-up.
```typescript Node SDK theme={null}
import { KadoaClient } from "@kadoa/node-sdk";
const client = new KadoaClient({ apiKey: "YOUR_API_KEY" });
const created = await client.assistant.createRealtimeWorkflow({
instructions: "Monitor product prices on https://example.com/products and alert me when they change.",
notificationChannelIds: ["NOTIFICATION_CHANNEL_ID"],
});
console.log(created.workflowId, created.sessionId, created.threadId, created.jobId);
```
```bash REST API theme={null}
curl -X POST "https://api.kadoa.com/v5/agent" \
-H "x-api-key: YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"prompt": "Monitor product prices on https://example.com/products and alert me when they change.",
"productType": "realtime",
"notificationChannelIds": ["NOTIFICATION_CHANNEL_ID"]
}'
```
See [create or continue an Assistant session](/api-reference/assistant/create-or-continue-session) for the complete API contract.
```text MCP Server theme={null}
> Monitor prices on https://example.com/products and notify me by email when they change.
```
For scheduled and one-time extractions, notifications are optional. See [Notifications](/docs/sdk/notifications/overview) for configuring channels and [Change Detection](/docs/change-detection) for how changes are identified.
## Update an existing workflow
Send a change request to the workflow Assistant instead of deleting and recreating the workflow. The request keeps the workflow ID and either reuses or starts its customer-facing Assistant session.
```typescript Node SDK theme={null}
const accepted = await client.assistant.requestWorkflowUpdate("WORKFLOW_ID", {
instructions: "Include the product SKU and keep paginating until there are no more products.",
});
console.log(accepted.sessionId, accepted.threadId, accepted.jobId);
```
```bash REST API theme={null}
curl -X POST "https://api.kadoa.com/v5/agent/workflows/WORKFLOW_ID/assistant/messages" \
-H "x-api-key: YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{"prompt":"Include the product SKU and keep paginating until there are no more products."}'
```
See the [send a workflow message API reference](/api-reference/assistant/send-workflow-message).
## Check status and answer questions
An Assistant can pause to ask for clarification. Read its pause state, then answer the current question with the returned session, question, and optional thread IDs. To find the workflows waiting on you, filter the workflow list with `statusFilters=group:attention`, as shown in [Find workflows that need attention](/docs/sdk/workflows/manage#find-workflows-that-need-attention). Workflows with an open support ticket are listed under `group:support` instead, even when a question is pending.
```typescript Node SDK theme={null}
const state = await client.assistant.getPauseState("SESSION_ID");
if (state.pendingQuestion) {
await client.assistant.answerQuestion({
sessionId: "SESSION_ID",
threadId: state.pendingQuestion.threadId,
questionId: state.pendingQuestion.questionId,
answers: { "0": "Use the US store." },
});
}
```
Answering a question requests its resume. `resume()` is for an inactive or interrupted session, not for a pending question. `interrupt()` and `stop()` control Assistant work only; they do not pause or resume the workflow schedule.
* [Get pause state →](/api-reference/assistant/get-pause-state)
* [Answer a question →](/api-reference/assistant/answer-question)
* [Interrupt a session →](/api-reference/assistant/interrupt-session)
* [Resume a session →](/api-reference/assistant/resume-session)
* [Stop a session →](/api-reference/assistant/stop-session)
## Read the conversation and strategy
Use the timeline to show customer-visible messages and clarification questions. Pass `nextCursor` from one response as `cursor` to read older entries. The timeline excludes internal and operations activity.
```typescript Node SDK theme={null}
const timeline = await client.assistant.getTimeline("WORKFLOW_ID", { limit: 50 });
for (const item of timeline.items) {
console.log(item.time, item.kind);
}
if (timeline.pagination.hasMore) {
const older = await client.assistant.getTimeline("WORKFLOW_ID", {
cursor: timeline.pagination.nextCursor ?? undefined,
});
console.log(older.items);
}
```
When a build has produced a customer-safe strategy, you can retrieve its approach, data source, and realtime interval. A `null` strategy means the session exists but has not produced one yet.
```typescript Node SDK theme={null}
const strategy = await client.assistant.getStrategy("SESSION_ID");
```
* [Get the Assistant timeline →](/api-reference/assistant/get-workflow-timeline)
* [Get the latest strategy →](/api-reference/assistant/get-strategy)
## Next steps
* [Manage workflows](/docs/sdk/workflows/manage)
* [Configure notifications](/docs/sdk/notifications/overview)
* [Use the MCP Server](/docs/sdk/mcp)
# Create Workflows
Source: https://docs.kadoa.com/docs/sdk/workflows/create
Create and manage workflows programmatically using SDK, CLI, MCP Server, or REST API
## Overview
Create workflows programmatically using the Kadoa SDK, CLI, MCP Server, or REST API:
* Create workflows with natural language prompts
* Use existing schemas or define custom ones
* Configure monitoring and scheduling options
## Prerequisites
Before you begin, you'll need:
* A Kadoa account
* Your [API key](https://kadoa.com/settings)
* For SDK: `npm install @kadoa/node-sdk` or `yarn add @kadoa/node-sdk` or `uv add kadoa-sdk`
## Authentication
```typescript Node SDK theme={null}
import { KadoaClient } from '@kadoa/node-sdk';
const client = new KadoaClient({
apiKey: 'YOUR_API_KEY'
});
const status = await client.status();
console.log(status);
console.log(status.user);
```
```python Python SDK theme={null}
import asyncio
from kadoa_sdk import KadoaClient, KadoaClientConfig
client = KadoaClient(KadoaClientConfig(api_key="YOUR_API_KEY"))
status = asyncio.run(client.status())
print(status)
print(status.user)
```
```bash CLI theme={null}
# Interactive OAuth login (saves to ~/.kadoa/config.json)
kadoa login
# Or provide both OAuth tokens for non-interactive refresh
export KADOA_ACCESS_TOKEN=YOUR_ACCESS_TOKEN
export KADOA_REFRESH_TOKEN=YOUR_REFRESH_TOKEN
# Check auth status
kadoa whoami
```
A `tk-...` API key authenticates SDK and local MCP requests, but it is not a CLI token.
```text MCP Server theme={null}
The remote server at mcp.kadoa.com/mcp uses OAuth, so no API key is needed.
For local stdio mode, set KADOA_API_KEY in your client config or run `kadoa login` (shared with the CLI).
```
```javascript REST API theme={null}
const headers = {
"x-api-key": "YOUR_API_KEY",
"Content-Type": "application/json"
};
```
## Extraction Methods
Choose how you want to extract data from websites:
### Auto Mode
The simplest way of creating a workflow is by simply passing a URL. Kadoa will automatically suggest a data schema and extract all relevant data from the page.
```typescript Node SDK theme={null}
// SDK: AI automatically detects and extracts data
const result = await client.extraction.run({
urls: ["https://sandbox.kadoa.com/ecommerce"]
});
console.log(result.data);
```
```python Python SDK theme={null}
from kadoa_sdk import ExtractionOptions, KadoaClient, KadoaClientConfig
client = KadoaClient(KadoaClientConfig(api_key="YOUR_API_KEY"))
# SDK: AI automatically detects and extracts data
result = client.extraction.run(
ExtractionOptions(
urls=["https://sandbox.kadoa.com/ecommerce"]
)
)
print(result.data)
```
```text MCP Server theme={null}
> "Extract all products from https://sandbox.kadoa.com/ecommerce"
```
```javascript REST API theme={null}
// API: Describe the extraction in plain language
// POST https://api.kadoa.com/v4/workflows
{
"urls": ["https://sandbox.kadoa.com/ecommerce"],
"name": "Auto Product Extraction",
"userPrompt": "Extract all product information including name, price, availability, and any other relevant details"
}
// Or define a minimal schema:
{
"urls": ["https://sandbox.kadoa.com/ecommerce"],
"name": "Auto Product Extraction",
"entity": "Product",
"fields": [
{
"name": "data",
"dataType": "STRING",
"description": "Extract all product information"
}
]
}
```
### Custom Schema
Define exactly what fields you want to extract for precise control:
```typescript Node SDK theme={null}
const workflow = await client
.extract({
urls: ["https://sandbox.kadoa.com/ecommerce"],
name: "Structured Product Extraction",
extraction: (builder) =>
builder
.entity("Product")
.field("title", "Product name", "STRING", {
example: "iPhone 15 Pro",
})
.field("price", "Price in USD", "MONEY")
.field("inStock", "Availability", "BOOLEAN")
.field("rating", "Rating 1-5", "NUMBER")
.field("releaseDate", "Launch date", "DATE"),
})
.create();
const result = await workflow.run({ limit: 10 });
// Use destructuring for cleaner access
const { data } = await result.fetchData({});
console.log(data);
```
```python Python SDK theme={null}
from kadoa_sdk import (
ExtractOptions,
FieldOptions,
KadoaClient,
KadoaClientConfig,
RunWorkflowOptions,
)
client = KadoaClient(KadoaClientConfig(api_key="YOUR_API_KEY"))
workflow = (
client.extract(
ExtractOptions(
urls=["https://sandbox.kadoa.com/ecommerce"],
name="Structured Product Extraction",
extraction=lambda builder: builder.entity("Product")
.field("title", "Product name", "STRING", FieldOptions(example="iPhone 15 Pro"))
.field("price", "Price in USD", "MONEY")
.field("inStock", "Availability", "BOOLEAN")
.field("rating", "Rating 1-5", "NUMBER")
.field("releaseDate", "Launch date", "DATE"),
)
)
.create()
)
result = workflow.run(RunWorkflowOptions(limit=10))
# Use destructuring for cleaner access
response = result.fetch_data({})
print(response.data)
```
```text MCP Server theme={null}
> "Extract product title, price in USD, availability, rating, and release date from https://sandbox.kadoa.com/ecommerce"
```
```javascript REST API theme={null}
// POST https://api.kadoa.com/v4/workflows
{
"urls": ["https://sandbox.kadoa.com/ecommerce"],
"name": "Structured Product Extraction",
"entity": "Product",
"fields": [
{
"name": "title",
"dataType": "STRING",
"description": "Product name"
},
{
"name": "price",
"dataType": "MONEY",
"description": "Price in USD"
},
{
"name": "inStock",
"dataType": "BOOLEAN",
"description": "Availability"
},
{
"name": "rating",
"dataType": "NUMBER",
"description": "Rating 1-5"
},
{
"name": "releaseDate",
"dataType": "DATE",
"description": "Launch date"
}
]
}
```
**Available Data Types:** `STRING`, `NUMBER`, `BOOLEAN`, `DATE`, `DATETIME`, `MONEY`, `IMAGE`, `LINK`, `OBJECT`, `ARRAY`
[See all data types →](/docs/workflows/schemas#data-types)
### Classification
Automatically categorize content into predefined classes:
```typescript Node SDK theme={null}
const workflow = await client
.extract({
urls: ["https://sandbox.kadoa.com/news"],
name: "Article Classifier",
extraction: (builder) =>
builder
.entity("Article")
.field("title", "Headline", "STRING", {
example: "Tech Company Announces New Product",
})
.field("content", "Article text", "STRING", {
example: "The article discusses the latest innovations...",
})
.classify("sentiment", "Content tone", [
{ title: "Positive", definition: "Optimistic tone" },
{ title: "Negative", definition: "Critical tone" },
{ title: "Neutral", definition: "Balanced tone" },
])
.classify("category", "Article topic", [
{ title: "Technology", definition: "Tech news" },
{ title: "Business", definition: "Business news" },
{ title: "Politics", definition: "Political news" },
]),
})
.create();
//Note: 'limit' here is limiting number of extracted records not fetched
const result = await workflow.run({ limit: 10, variables: {} });
console.log(result.jobId);
const data = result.fetchData({ limit: 10 });
console.log(data);
```
```python Python SDK theme={null}
from kadoa_sdk import (
ExtractOptions,
FieldOptions,
KadoaClient,
KadoaClientConfig,
RunWorkflowOptions,
)
client = KadoaClient(KadoaClientConfig(api_key="YOUR_API_KEY"))
workflow = (
client.extract(
ExtractOptions(
urls=["https://sandbox.kadoa.com/news"],
name="Article Classifier",
extraction=lambda builder: builder.entity("Article")
.field(
"title",
"Headline",
"STRING",
FieldOptions(example="Tech Company Announces New Product"),
)
.field(
"content",
"Article text",
"STRING",
FieldOptions(example="The article discusses the latest innovations..."),
)
.classify(
"sentiment",
"Content tone",
[
{"title": "Positive", "definition": "Optimistic tone"},
{"title": "Negative", "definition": "Critical tone"},
{"title": "Neutral", "definition": "Balanced tone"},
],
)
.classify(
"category",
"Article topic",
[
{"title": "Technology", "definition": "Tech news"},
{"title": "Business", "definition": "Business news"},
{"title": "Politics", "definition": "Political news"},
],
),
)
)
.create()
)
# Note: 'limit' here is limiting number of extracted records not fetched
result = workflow.run(RunWorkflowOptions(limit=10, variables={}))
print(result.job_id)
data = result.fetch_data({"limit": 10})
print(data)
```
```javascript REST API theme={null}
// API example coming soon
```
## Prompts
Every workflow is driven by a prompt: a plain-language description of what to extract and how to get there. The AI agent handles navigation, pagination, clicking, and forms automatically. For details, see [Prompts](/docs/workflows/prompts).
Navigation is handled automatically when you provide a prompt.
### Using a Prompt
For multi-step or complex extractions, write a prompt in plain language. The AI agent follows your prompt to navigate, interact, and extract:
```typescript Node SDK theme={null}
const workflow = await client
.extract({
urls: ["https://sandbox.kadoa.com/ecommerce"],
name: "Product Catalog",
userPrompt: "Extract all products. Click 'Next' to continue through pagination. For each product, open the detail page and extract the full description and specifications.",
extraction: (builder) =>
builder
.entity("Product")
.field("title", "Product name", "STRING", {
example: "Wireless Headphones",
})
.field("price", "Product price", "MONEY")
.field("description", "Full description", "STRING", {
example: "Over-ear headphones with active noise cancellation",
})
.field("specifications", "Technical specs", "STRING", {
example: "Bluetooth 5.3, 30-hour battery life",
}),
})
.create();
const result = await workflow.run({ limit: 50 });
```
```python Python SDK theme={null}
from kadoa_sdk import (
ExtractOptions,
FieldOptions,
KadoaClient,
KadoaClientConfig,
RunWorkflowOptions,
)
client = KadoaClient(KadoaClientConfig(api_key="YOUR_API_KEY"))
workflow = (
client.extract(
ExtractOptions(
urls=["https://sandbox.kadoa.com/ecommerce"],
name="Product Catalog",
user_prompt="Extract all products. Click 'Next' to continue through pagination. For each product, open the detail page and extract the full description and specifications.",
extraction=lambda builder: builder.entity("Product")
.field("title", "Product name", "STRING", FieldOptions(example="Wireless Headphones"))
.field("price", "Product price", "MONEY")
.field(
"description",
"Full description",
"STRING",
FieldOptions(example="Over-ear headphones with active noise cancellation"),
)
.field(
"specifications",
"Technical specs",
"STRING",
FieldOptions(example="Bluetooth 5.3, 30-hour battery life"),
),
)
)
.create()
)
result = workflow.run(RunWorkflowOptions(limit=50))
```
```text MCP Server theme={null}
> "Extract all products from https://sandbox.kadoa.com/ecommerce including title, price, description, and specs. Navigate through pagination and open detail pages."
```
```javascript REST API theme={null}
// POST https://api.kadoa.com/v4/workflows
{
"urls": ["https://sandbox.kadoa.com/ecommerce"],
"name": "Product Catalog",
"userPrompt": "Extract all products. Click 'Next' to continue through pagination. For each product, open the detail page and extract the full description and specifications.",
"entity": "Product",
"fields": [
{ "name": "title", "dataType": "STRING", "description": "Product name" },
{ "name": "price", "dataType": "MONEY", "description": "Product price" },
{ "name": "description", "dataType": "STRING", "description": "Full description" },
{ "name": "specifications", "dataType": "STRING", "description": "Technical specs" }
]
}
```
[Learn how to write effective prompts →](/docs/workflows/prompts#writing-good-prompts)
Complex multi-step extractions can take significantly longer to complete (usually around an hour). We recommend avoiding waiting for results synchronously.
## Writing Prompts
Write clear, step-by-step prompts for complex extraction tasks. The AI agent follows your prompt autonomously, handling clicks, forms, pagination, and file downloads.
[Learn how to write effective prompts →](/docs/workflows/prompts#writing-good-prompts)
## Create from a Template
If your team has [templates](/docs/templates) set up, you can create a workflow and link it to a template. The template's prompt, schema, and notification settings are applied to the workflow.
```text MCP Server theme={null}
> "Create a workflow from the 'Job Listing' template for https://example.com/careers"
```
```javascript REST API theme={null}
// Step 1: Create the workflow as usual
// POST https://api.kadoa.com/v4/workflows
{
"urls": ["https://example.com/careers"],
"name": "Example Careers"
}
// Step 2: Link it to a template (applies template config)
// POST https://api.kadoa.com/v4/templates/template-123/link
{
"workflowIds": ["workflow-456"]
}
```
Linking applies the template's latest version. The template-managed parts become read-only on the workflow. See [Templates](/docs/sdk/templates/overview) for full details.
## Next Steps
* [Manage Workflows →](/docs/sdk/workflows/manage) - List, retrieve, pause, and delete workflows
* [Schedule & Run Workflows →](/docs/sdk/workflows/scheduling) - Configure intervals, manual execution, and check status
* [Writing Prompts →](/docs/workflows/prompts) - Write effective prompts for complex extraction tasks
* [Working with Schemas →](/docs/sdk/schemas/overview) - Create and manage reusable schemas
* [Templates →](/docs/sdk/templates/overview) - Reuse configurations across workflows
* [Data Delivery →](/docs/destinations/overview) - Retrieve extracted data
* [API Reference →](/api-reference/workflows/create-a-new-workflow)
# Manage Workflows
Source: https://docs.kadoa.com/docs/sdk/workflows/manage
List, retrieve, and manage existing workflows
## List Workflows
```typescript Node SDK theme={null}
const workflows = await client.workflow.list({ limit: 100 });
for (const workflow of workflows) {
console.log(`${workflow.id}: ${workflow.name}`);
}
```
```python Python SDK theme={null}
from kadoa_sdk import KadoaClient, KadoaClientConfig
from kadoa_sdk.workflows import ListWorkflowsRequest
client = KadoaClient(KadoaClientConfig(api_key="YOUR_API_KEY"))
workflows = client.workflow.list(ListWorkflowsRequest(limit=100))
for workflow in workflows:
print(f"{workflow.id}: {workflow.name}")
```
```bash REST API theme={null}
curl -X GET "https://api.kadoa.com/v4/workflows" \
-H "x-api-key: YOUR_API_KEY"
```
## Find workflows that need attention
Use `group:attention` for Assistant questions and previews to review. Workflows with open support cases appear in `group:support`, even when awaiting your reply. See [all groups](/docs/workflows/lifecycle#filtering-workflows).
`awaitingUserInput` contains the wait's start time and optional question text.
```typescript Node SDK theme={null}
import { WorkflowStatusFilter } from "@kadoa/node-sdk";
const needsAttention = await client.workflow.list({
statusFilters: [WorkflowStatusFilter.Attention],
});
for (const workflow of needsAttention) {
const question = workflow.awaitingUserInput?.prompt;
console.log(workflow.name, question ?? "waiting on preview approval");
}
```
```bash REST API theme={null}
curl -X GET "https://api.kadoa.com/v4/workflows?statusFilters=group:attention" \
-H "x-api-key: YOUR_API_KEY"
```
```text MCP Server theme={null}
> What needs my attention in Kadoa?
```
[Reply to the Assistant](/docs/sdk/workflows/assistant) or [review the preview](/docs/ui/workflows/human-in-the-loop). Check `availableActions.approve` before approving. An incomplete update can block approval even in this group.
`statusFilters` is available in the Node SDK, the REST API, and the MCP server. The Python SDK does not expose it yet, so call `GET /v4/workflows?statusFilters=group:attention` directly from Python.
## Get Workflow
```typescript Node SDK theme={null}
const workflow = await client.workflow.get('YOUR_WORKFLOW_ID');
```
```python Python SDK theme={null}
from kadoa_sdk import KadoaClient, KadoaClientConfig
client = KadoaClient(KadoaClientConfig(api_key="YOUR_API_KEY"))
workflow = client.workflow.get("YOUR_WORKFLOW_ID")
```
```bash REST API theme={null}
curl -X GET "https://api.kadoa.com/v4/workflows/YOUR_WORKFLOW_ID" \
-H "x-api-key: YOUR_API_KEY"
```
## Pause & Resume
Only `ACTIVE` workflows can be paused.
```typescript Node SDK theme={null}
// Pause
await client.workflow.pause('YOUR_WORKFLOW_ID');
// Resume
await client.workflow.resume('YOUR_WORKFLOW_ID');
```
```python Python SDK theme={null}
from kadoa_sdk import KadoaClient, KadoaClientConfig
client = KadoaClient(KadoaClientConfig(api_key="YOUR_API_KEY"))
# Pause
client.workflow.pause("YOUR_WORKFLOW_ID")
# Resume
client.workflow.resume("YOUR_WORKFLOW_ID")
```
## Update
```typescript Node SDK theme={null}
await client.workflow.update("YOUR_WORKFLOW_ID", {
name: "New Name",
updateInterval: "DAILY",
});
```
```python Python SDK theme={null}
from kadoa_sdk import KadoaClient, KadoaClientConfig
from kadoa_sdk.workflows import UpdateWorkflowRequest
client = KadoaClient(KadoaClientConfig(api_key="YOUR_API_KEY"))
client.workflow.update(
"YOUR_WORKFLOW_ID",
UpdateWorkflowRequest(name="New Name", update_interval="DAILY"),
)
```
```bash REST API theme={null}
curl -X PUT "https://api.kadoa.com/v4/workflows/YOUR_WORKFLOW_ID/metadata" \
-H "x-api-key: YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{"name": "New Name", "updateInterval": "DAILY"}'
```
## Run History
Workflow run history is the execution record for one workflow. It includes outcome, start and finish times, record counts, and public errors.
The Node SDK and MCP Server can filter execution history by `success`, `failed`, or `in_progress` and paginate it.
```typescript Node SDK theme={null}
const history = await client.workflow.listWorkflowRuns("WORKFLOW_ID", {
page: 1,
limit: 25,
status: "failed",
});
for (const run of history.workflowRuns) {
console.log(run.id, run.state, run.startedAt, run.errors);
}
```
```text MCP Server theme={null}
> Show the last 25 failed runs for my Product Monitor and summarize the errors.
```
```bash REST API theme={null}
curl "https://api.kadoa.com/v4/workflows/WORKFLOW_ID/history?page=1&limit=25&status=failed" \
-H "x-api-key: YOUR_API_KEY"
```
See [workflow run history API reference](/api-reference/workflows/get-the-workflow-history) for the complete response.
## Delete
This permanently deletes the workflow and all its data.
```typescript Node SDK theme={null}
await client.workflow.delete('YOUR_WORKFLOW_ID');
```
```python Python SDK theme={null}
from kadoa_sdk import KadoaClient, KadoaClientConfig
client = KadoaClient(KadoaClientConfig(api_key="YOUR_API_KEY"))
client.workflow.delete("YOUR_WORKFLOW_ID")
```
```bash REST API theme={null}
curl -X DELETE "https://api.kadoa.com/v4/workflows/YOUR_WORKFLOW_ID" \
-H "x-api-key: YOUR_API_KEY"
```
## Templates
Workflows can be linked to [templates](/docs/templates) that manage their prompt, schema, and notification settings. When linked, those fields become read-only on the workflow.
See [Templates](/docs/sdk/templates/overview) for linking, unlinking, and applying template updates via the API.
## Fetch Data
```typescript Node SDK theme={null}
const workflows = await client.workflow.list({ limit: 100 });
for (const workflow of workflows) {
if (!workflow.id) {
continue;
}
const data = await client.extraction.fetchAllData({
workflowId: workflow.id,
});
console.log(`${workflow.name}: ${data.length} records`);
}
```
```python Python SDK theme={null}
from kadoa_sdk import FetchDataOptions, KadoaClient, KadoaClientConfig
from kadoa_sdk.workflows import ListWorkflowsRequest
client = KadoaClient(KadoaClientConfig(api_key="YOUR_API_KEY"))
workflows = client.workflow.list(ListWorkflowsRequest(limit=100))
for workflow in workflows:
data = client.extraction.fetch_all_data(
FetchDataOptions(workflow_id=workflow.id)
)
print(f"{workflow.name}: {len(data)} records")
```
```bash REST API theme={null}
curl -X GET "https://api.kadoa.com/v4/workflows/YOUR_WORKFLOW_ID/data?limit=50&page=2" \
-H "x-api-key: YOUR_API_KEY"
```
# Schedule & Run Workflows
Source: https://docs.kadoa.com/docs/sdk/workflows/scheduling
Configure scheduling, run workflows manually, and check execution status
## Prerequisites
* A Kadoa account with API key
* SDK installed: `npm install @kadoa/node-sdk` or `uv add kadoa-sdk`
* An existing workflow ([create one first](/docs/sdk/workflows/create))
## Scheduling Options
Configure when your workflow runs:
```typescript Node SDK theme={null}
const workflow = await client
.extract({
urls: ["https://sandbox.kadoa.com/ecommerce/pagination"],
name: "Scheduled Extraction",
extraction: (builder) =>
builder
.entity("Product")
.field("title", "Product name", "STRING", { example: "Sample" }),
})
.setInterval({
schedules: ["0 9 * * MON-FRI", "0 18 * * MON-FRI"],
})
.create();
// Workflow runs automatically on schedule
console.log("Scheduled workflow:", workflow.workflowId);
```
```python Python SDK theme={null}
from kadoa_sdk import ExtractOptions, FieldOptions, KadoaClient, KadoaClientConfig
client = KadoaClient(KadoaClientConfig(api_key="YOUR_API_KEY"))
workflow = (
client.extract(
ExtractOptions(
urls=["https://sandbox.kadoa.com/ecommerce/pagination"],
name="Scheduled Extraction",
extraction=lambda builder: builder.entity("Product").field(
"title", "Product name", "STRING", FieldOptions(example="Sample")
),
)
)
.set_interval({"schedules": ["0 9 * * MON-FRI", "0 18 * * MON-FRI"]})
.create()
)
# Workflow runs automatically on schedule
print("Scheduled workflow:", workflow.workflow_id)
```
```bash REST API theme={null}
curl -X POST https://api.kadoa.com/v4/workflows \
-H "x-api-key: YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"urls": ["https://sandbox.kadoa.com/ecommerce/pagination"],
"name": "Scheduled Extraction",
"interval": "CUSTOM",
"schedules": ["0 9 * * MON-FRI", "0 18 * * MON-FRI"],
"entity": "Product",
"fields": [{"name": "title", "dataType": "STRING", "description": "Product name"}]
}'
```
```text MCP Server theme={null}
> "Create a workflow to extract products from sandbox.kadoa.com/ecommerce/pagination, scheduled at 9am and 6pm on weekdays"
```
## Available Intervals
| Interval | Description |
| - | - |
| `ONLY_ONCE` | Run once |
| `EVERY_10_MINUTES` | Every 10 minutes |
| `HALF_HOURLY` | Every 30 minutes |
| `HOURLY` | Every hour |
| `THREE_HOURLY` | Every 3 hours |
| `SIX_HOURLY` | Every 6 hours |
| `TWELVE_HOURLY` | Every 12 hours |
| `EIGHTEEN_HOURLY` | Every 18 hours |
| `DAILY` | Once per day |
| `TWO_DAY` | Every 2 days |
| `THREE_DAY` | Every 3 days |
| `WEEKLY` | Once per week |
| `BIWEEKLY` | Every 2 weeks |
| `TRIWEEKLY` | Every 3 weeks |
| `FOUR_WEEKS` | Every 4 weeks |
| `MONTHLY` | Once per month |
| `REAL_TIME` | [Real time monitoring](/docs/sdk/workflows/assistant#create-a-realtime-monitor) |
| `CUSTOM` | Use cron expressions |
## Manual Execution
Run workflows on demand:
```typescript Node SDK theme={null}
const workflow = await client.workflow.get(workflowId);
console.log(`Current workflow state: ${workflow.displayState}`);
const result = await client.workflow.runWorkflow(workflowId, {
limit: 10,
});
console.log(`Workflow scheduled with runId: ${result.jobId}`);
```
```python Python SDK theme={null}
from kadoa_sdk import KadoaClient, KadoaClientConfig, RunWorkflowOptions
client = KadoaClient(KadoaClientConfig(api_key="YOUR_API_KEY"))
workflow_id = "YOUR_WORKFLOW_ID"
workflow = client.workflow.get(workflow_id)
print(f"Current workflow state: {workflow.display_state}")
result = client.workflow.run_workflow(
workflow_id,
input=RunWorkflowOptions(limit=10),
)
print(f"Workflow scheduled with runId: {result.job_id}")
```
```bash REST API theme={null}
curl -X PUT https://api.kadoa.com/v4/workflows/YOUR_WORKFLOW_ID/run \
-H "x-api-key: YOUR_API_KEY" \
-H "Content-Type: application/json"
```
```text MCP Server theme={null}
> "Run my 'Scheduled Extraction' workflow with a limit of 10 records"
```
```json Response theme={null}
{
"success": true,
"jobId": "job_abc123"
}
```
## Checking Workflow Status
Poll the workflow status to know when extraction is complete:
```typescript Node SDK theme={null}
const extraction = await client
.extract({
urls: ["https://sandbox.kadoa.com/ecommerce/pagination"],
name: "Paginated Extraction",
userPrompt: "Extract all products, paginating through all pages",
extraction: (builder) =>
builder
.entity("Product")
.field("title", "Product name", "STRING", {
example: "Sennheiser HD 6XX",
})
.field("price", "Product price", "MONEY"),
})
.create();
const result = await extraction.run({ limit: 10 });
// Fetch a single page with pagination info
const page = await result.fetchData({ page: 1, limit: 5 });
console.log("Page data:", page.data);
console.log("Pagination:", page.pagination);
// Or get all data at once
const allData = await result.fetchAllData({});
console.log("All data:", allData);
```
```python Python SDK theme={null}
from kadoa_sdk import (
ExtractOptions,
FieldOptions,
KadoaClient,
KadoaClientConfig,
RunWorkflowOptions,
)
client = KadoaClient(KadoaClientConfig(api_key="YOUR_API_KEY"))
extraction = (
client.extract(
ExtractOptions(
urls=["https://sandbox.kadoa.com/ecommerce/pagination"],
name="Paginated Extraction",
user_prompt="Extract all products, paginating through all pages",
extraction=lambda builder: builder.entity("Product")
.field(
"title",
"Product name",
"STRING",
FieldOptions(example="Sennheiser HD 6XX"),
)
.field("price", "Product price", "MONEY"),
)
)
.create()
)
result = extraction.run(RunWorkflowOptions(limit=10))
# Fetch a single page with pagination info
page = result.fetch_data({"page": 1, "limit": 5})
print("Page data:", page.data)
print("Pagination:", page.pagination)
# Or get all data at once
all_data = result.fetch_all_data({})
print("All data:", all_data)
```
```bash REST API theme={null}
# 1. Run workflow
curl -X PUT https://api.kadoa.com/v4/workflows/YOUR_WORKFLOW_ID/run \
-H "x-api-key: YOUR_API_KEY"
# 2. Poll status until complete
curl -X GET https://api.kadoa.com/v4/workflows/YOUR_WORKFLOW_ID \
-H "x-api-key: YOUR_API_KEY"
# 3. Once state is "COMPLETED", fetch data
curl -X GET https://api.kadoa.com/v4/workflows/YOUR_WORKFLOW_ID/data \
-H "x-api-key: YOUR_API_KEY"
```
```json Response (Status) theme={null}
{
"id": "507f1f77bcf86cd799439011",
"name": "My Workflow",
"state": "ACTIVE",
"lastRun": {
"id": "run-123",
"state": "IN_PROGRESS",
"startedAt": "2024-01-15T10:00:00Z",
"completedAt": null
}
}
```
**Workflow States:**
* `IN_PROGRESS` - Extraction is running
* `COMPLETED` - Data is ready to retrieve
* `FAILED` - Extraction failed (check errors field)
## Proxy Locations
Specify geographic location for extraction:
```typescript Node SDK theme={null}
const workflow = await client
.extract({
urls: ["https://sandbox.kadoa.com/magic"],
name: "Geo-located Extraction",
extraction: (builder) =>
builder
.entity("Product")
.field("title", "Title", "STRING", { example: "example" }),
})
.setLocation({
type: "manual",
isoCode: "US",
})
.create();
```
```python Python SDK theme={null}
from kadoa_sdk import ExtractOptions, FieldOptions, KadoaClient, KadoaClientConfig
client = KadoaClient(KadoaClientConfig(api_key="YOUR_API_KEY"))
workflow = (
client.extract(
ExtractOptions(
urls=["https://sandbox.kadoa.com/magic"],
name="Geo-located Extraction",
extraction=lambda builder: builder.entity("Product").field(
"title", "Title", "STRING", FieldOptions(example="example")
),
)
)
.set_location({"type": "manual", "isoCode": "US"})
.create()
)
```
```bash REST API theme={null}
curl -X POST https://api.kadoa.com/v4/workflows \
-H "x-api-key: YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"urls": ["https://sandbox.kadoa.com/magic"],
"location": {
"type": "manual",
"isoCode": "US"
}
}'
```
**Available locations:**
* `US` - United States
* `GB` - United Kingdom
* `DE` - Germany
* `NL` - Netherlands
* `CA` - Canada
* `auto` - Automatic selection
## Bypass Preview Mode
Skip manual review and activate workflows immediately:
```typescript Node SDK theme={null}
const workflow = await client
.extract({
urls: ["https://sandbox.kadoa.com/magic"],
name: "Direct Activation",
extraction: (builder) =>
builder
.entity("Product")
.field("title", "Title", "STRING", { example: "example" }),
})
.bypassPreview() // Skip review step
.create();
// Workflow is immediately active
```
```python Python SDK theme={null}
from kadoa_sdk import ExtractOptions, FieldOptions, KadoaClient, KadoaClientConfig
client = KadoaClient(KadoaClientConfig(api_key="YOUR_API_KEY"))
workflow = (
client.extract(
ExtractOptions(
urls=["https://sandbox.kadoa.com/magic"],
name="Direct Activation",
extraction=lambda builder: builder.entity("Product").field(
"title", "Title", "STRING", FieldOptions(example="example")
),
)
)
.bypass_preview() # Skip review step
.create()
)
# Workflow is immediately active
```
```bash REST API theme={null}
curl -X POST https://api.kadoa.com/v4/workflows \
-H "x-api-key: YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"bypassPreview": true,
"urls": ["https://sandbox.kadoa.com/magic"],
"entity": "Product",
"fields": [{"name": "title", "dataType": "STRING", "description": "Title"}]
}'
```
## Next Steps
* [Create Workflows →](/docs/sdk/workflows/create)
* [Writing Prompts →](/docs/workflows/prompts)
* [Configure Notifications →](/docs/sdk/notifications/overview)
* [API Reference →](/api-reference/workflows/schedule-a-workflow)
# Overview
Source: https://docs.kadoa.com/docs/security/compliance
Learn more about our built-in compliance layer
Kadoa provides built-in compliance features such as approval flow, compliance rules, and auditing. Our compliance layer ensures that organizational guidelines and rules around data extraction are automatically enforced.
## Compliance Process
Depending on your internal compliance proesses, we recommend the following controls:
* **Compliance training**: Ensure all authorized Kadoa users complete compliance training and understand your organization’s data sourcing policies.
* **Automated compliance checks**: Leverage Kadoa's built-in compliance layer to automatically enforce restrictions.
* **Regular reporting and auditing**: Assign the compliance officer user role to enable regular auditing across teams.
## Automated Compliance Rules
Your compliance team can configure automated checks tailored to your specific organizational requirements. Kadoa supports the following customizable compliance rules:
| Rule Type | Description |
| - | - |
| **Source Blacklisting** | Explicitly prevent extracting or accessing prohibited sources or domains. |
| **Captcha & Explicit TOS Enforcement** | Prevent automated circumvention of explicit Terms of Service pop-ups and automated Captcha bypassing. |
| **Sensitive Data (PII) Detection** | Automatically detects attempts to extract common forms of Personally Identifiable Information (PII) and can be configured to redact, flag, or block such data. |
| **robots.txt Check** | Automatically checks `robots.txt` directives for all web interactions, promoting ethical data sourcing from websites. |
| **Custom Restrictions** | Configure fine-grained custom restrictions. |
Violations immediately trigger clear and descriptive warnings or errors, preventing accidental non-compliance.
We also allow you to add custom compliance hints/reminders on the screen where users configure a new workflow.
## Setting Up Compliance
To view which rules are active for your team and access audit tools, see [Compliance Center](/docs/compliance-center).
## Our Compliance Policy
Beyond the configurable rules within the platform, Kadoa operates under a comprehensive set of internal guidelines for ethical data sourcing.
You can review our company's official Web Scraping Compliance Policy, along with our AI Policy, SOC 2 report, and all other compliance documents, in our **[Trust Center](https://trust.kadoa.com)**.
Everything you build on Kadoa is strictly proprietary and confidential. Your competitive edge is sacred to us.
## Compliance Guidance
Our compliance team can help with specific regulations, custom rules, or audit requirements.
Contact us at [compliance@kadoa.com](mailto:compliance@kadoa.com)
# Introduction
Source: https://docs.kadoa.com/docs/security/introduction
Learn more about our data security practices and compliance measures.
## Overview
Kadoa is built with best-in-class security practices to keep your data safe and secure at every layer.
This includes state-of-the-art encryption, safe and reliable infrastructure partners, and independently verified security controls.
Request our Data Processing Agreement (DPA) for specific details.
## Shared responsibility
Under our shared responsibility model, Kadoa secures the components that we control, including the application layer, underlying platform, and cloud infrastructure.
This includes protecting against threats targeting these components through security controls, monitoring, and incident response.
Customers are responsible for how they use Kadoa.
This includes configuring workspace security and access, determining what data they extract, managing API keys and destinations, and monitoring their audit log.
## Governance
Kadoa establishes policies and controls, monitors compliance with those controls, and proves the security and compliance to third-party auditors.
Our policies are based on the following foundational principles:
Access should be limited to only those with a legitimate business needs,
based on the principle of least privilege.
Security controls should be applied consistently across all areas of the
enterprise.
Security controls should be implemented and layered according to the
principle of defense-in-depth.
The implementation of controls should be iterative, continuously improving
effectiveness and decreasing friction.
## Data Protection
All data stores are encrypted at rest.
Kadoa uses TLS 1.3 or higher everywhere data is transmitted over networks.
Kadoa backs-up all production data using a point-in-time approach. Backups are
persisted for 30 days, and are globally replicated for resiliency against
regional disasters.
## Data Isolation
Kadoa implements strict data isolation to ensure complete privacy between teams and customers:
* **Complete Team Separation**: Each team's data is fully isolated. No data is ever shared or accessible between teams
* **API Scope Enforcement**: API keys are strictly scoped to individual teams and cannot access data from other teams
* **Isolated Processing**: Workflow executions happen in isolated environments with team-level boundaries
* **Database-Level Security**: Row-level security policies and team identifiers ensure data separation at the storage layer
## Data Regions
When starting on ouer Enterprise Plan, you have the option to select the region where you want your data to be stored. The available options are:
* United States
* European Union
## Certifications
Kadoa is SOC 2 certified. This certification validates our security controls, data handling practices, and operational procedures through independent third-party auditing.
Visit our **[Trust Center](https://trust.kadoa.com)** to review certifications, request access to compliance documents, and download security policies.
For additional questions, contact [security@kadoa.com](mailto:security@kadoa.com).
## Product Security
### Penetration testing
Kadoa engages with third-party firms to conduct penetration testing at least annually.
All areas of the Kadoa product and cloud infrastructure are in-scope for these assessments, and source code is fully available to testers to maximize effectiveness and coverage.
### Vulnerability scanning
Kadoa uses multiple vulnerability monitoring techniques including code-level scanning, dependency scanning, and security reviews to identify and remediate vulnerabilities.
Vulnerabilities are prioritized based on severity and risk, and are remediated according to a schedule.
### Prompt injection
Kadoa uses large language models (LLMs) to build web data pipelines. Because web content is untrusted, a page could contain text crafted to hijack the model and make it ignore its task. This is known as *indirect prompt injection*, and it is an inherent, not-fully-solvable risk of any system that applies LLMs to external content.
We treat it as inevitable and our goal is to ensure that if it does happen, it cannot cause harm. We do this with defense-in-depth, where each layer limits the blast radius of the layer above it:
* **Limited tool access**: our models are constrained to web scraping tasks. They have no tools to call external systems, send data anywhere, or change your configuration, so instructions hidden in a page have nothing to act on.
* **Isolated execution**: all workflow execution runs in isolated, containerized sandbox environments with boundaries kept as tight as possible, at both the workflow and team level.
* **Least privilege**: each workflow runs with only the access it needs, so an injection can never do more than the workflow itself is permitted to do.
* **Continuous testing**: prompt injection is in scope for our regular third-party penetration tests, and we run ongoing internal adversarial testing to catch model regressions over time.
The most damaging prompt injection scenario requires three things at once: access to sensitive data, exposure to untrusted content, and a way to send data out. Untrusted content is unavoidable for a web data platform, so we prevent the other two and keep even a successful injection contained.
## Responsible Disclosure
To report a vulnerability, please reach out to [security@kadoa.com](mailto:security@kadoa.com)
# IP ACL
Source: https://docs.kadoa.com/docs/security/ip-allowlist
Restrict access to your Kadoa workspace to a trusted set of IP addresses.
## Overview
IP ACL (IP Access Control List) restricts access to your Kadoa workspace so that
authenticated requests are only accepted from IP addresses you trust. Once
enabled, a request carrying a valid session or API key is still rejected if it
does not originate from an allowed address.
IP ACL applies across the API, the custom services API, and the realtime and
event streams, so the same trusted set of addresses governs every way your team
connects to Kadoa.
IP ACL is available on the Enterprise plan and can be managed by team Admins
and Owners.
## How it works
The list holds one or more IPv4 addresses or CIDR ranges. A single address such
as `203.0.113.10` covers one host; a range such as `203.0.113.0/24` covers a
whole subnet. Add the public egress addresses your team and integrations connect
from — your office network, VPN, or a static NAT gateway.
IP ACL is scoped to your organization, so a single list protects every team in
your workspace.
## Modes
You control enforcement with a three-way switch in the dashboard:
| Mode | Behavior |
| - | - |
| **Disabled** | IP ACL is inactive. All authenticated requests are accepted. |
| **Audit** | Requests are never blocked, but any request that would be denied is recorded in your Activity log. Use this to validate your list before enforcing. |
| **Enabled** | Requests from addresses not on the list are rejected. |
We recommend starting in **Audit** mode. Let your team work normally for a day
or two, review the Activity log for any would-be denials, add any missing
addresses, and then switch to **Enabled**.
## Setting it up
1. Go to **Settings → IP ACL** in the dashboard.
2. Add the IP addresses or CIDR ranges your team connects from. The current IP
you are connecting from is shown so you can confirm it is covered.
3. Choose a mode. To prevent accidental lockout, you cannot switch to
**Enabled** unless your current IP is already on the list.
4. Save.
Kadoa will not let you enable enforcement, or remove the last entry covering
your own address, if doing so would block your current connection. You can
still prune a redundant entry as long as another entry still covers your IP.
If you do get locked out, contact [support@kadoa.com](mailto:support@kadoa.com)
to restore access.
## Auditing
Both **Audit** and **Enabled** modes record activity you can review in your
team's Activity log:
* Denied (or would-be-denied) connections appear as **Access denied · IP ACL**,
with the user and the originating IP address.
* Changes to the list itself, and changes to the mode.
This gives you an ongoing record of where your workspace is being accessed from.
## Limitations
* IPv4 addresses and CIDR ranges are supported. IPv6 is not yet supported.
* IP ACL governs authenticated API and stream access. Sign-in itself is not
gated — a blocked address may complete sign-in, but the resulting session is
rejected on its first request to a Kadoa service.
For questions about configuring IP ACL for your organization, contact your
account team or [support@kadoa.com](mailto:support@kadoa.com).
# SCIM
Source: https://docs.kadoa.com/docs/security/scim
SCIM, or System for Cross-domain Identity Management allows for the automation of user provisioning for your Kadoa workspace.
## Overview
With SCIM enabled, user accounts can be automatically created, updated, and suspended based on your IDP settings.
This eliminates the need for manual account management in Kadoa.
This integration helps ensure that your team's access stays in sync with your organization’s directory.
## Requirements
* Kadoa team with Enterprise plan
* SCIM token (provided by Kadoa, one per team)
* SSO configured for your team
* Admin access to your identity provider
## Supported Identity Providers
### Microsoft Entra
1. In your enterprise application, set Provisioning Mode to `Automatic`
2. Enter the SCIM endpoint URL: `https://api.kadoa.com/scim/v2`
3. Add the SCIM token for authentication
4. Test the connection and save
### Other Providers
Okta and OneLogin support is available upon request. Contact your account team for provider-specific configuration guides.
## What syncs via SCIM
SCIM synchronizes the following user properties:
* Email address
* First and last name
* Active/suspended status
* Team membership
## Which SCIM version is supported?
* SCIM 2.0 ([RFC 7643](https://datatracker.ietf.org/doc/html/rfc7643)) is supported
* SCIM 1.1 is **not** supported
# Single Sign-On (SSO)
Source: https://docs.kadoa.com/docs/security/sso
Configure SAML-based Single Sign-On for centralized identity management
## Overview
Once SSO is enabled, members on SSO-approved domains must log in via SSO by default. You can allow other login types for users on other domains.
Members can login via your identity provider's website or by clicking the option to Continue with SSO on the login page.
## Configuration Overview
### Step 1: Request SSO Setup
Contact your Kadoa account team to initiate SSO configuration. We'll provide:
* SAML metadata URL or XML
* Entity ID
* SSO URL
### Step 2: Configure Your Identity Provider
1. Create a new SAML application in your IdP
2. Enter the Kadoa-provided SAML details
3. Configure attribute mappings:
* Email (required)
* First Name (recommended)
* Last Name (recommended)
4. Assign users or groups to the application
## Just-In-Time (JIT) Provisioning
When a user logs in via SSO for the first time:
* A new Kadoa account is automatically created
* Profile information is populated from SAML attributes
* Default team membership is assigned
* User role defaults to Member (configurable by admins)
# Support Center
Source: https://docs.kadoa.com/docs/support-center
Submit support requests and track issues directly from the dashboard
1. Click **Support** in the left sidebar navigation
2. View all your existing support requests
3. Click **New Issue** to submit a new request
![Support Center]()
## Submitting a Support Request
* **Select the affected workflow** from your existing workflows
* **Specify the issue type**: wrong data, missing fields, wrong website, scraper not working...
* **Optionally pause the workflow** while the issue is being investigated
* Provide details:
1. **Title**: Brief summary of the issue
2. **Description**: Detailed explanation with specific examples
3. **Priority**: Urgent, High, Medium, or Low
4. **Attachments**: Screenshots or files that help illustrate the issue
![New Issue Form]()
## Tracking Your Issues
The Support Center shows all your submitted issues with:
* **Status**: Triage, Todo, In Progress, Done, Canceled...
* **Priority level**
* **Last updated timestamp**
* **Issue ID** for reference
## API Access
You can also manage support issues programmatically using the [Support API](/api-reference/support/get-support-issues). The API lets you list, create, and update support issues, as well as retrieve available issue states.
## Other Ways to Get Help
* **Documentation**: Browse our [docs](/docs/introduction) for guides and API reference
* **Compliance questions**: Contact [compliance@kadoa.com](mailto:compliance@kadoa.com)
# Templates
Source: https://docs.kadoa.com/docs/templates
Reusable, versioned configurations that standardize prompt, schema, data quality, notifications, and frequency across workflows
Templates let you define a reusable configuration (prompt, schema, data quality rules, notifications, and run frequency) and apply it to multiple workflows. When you update the template, you can push changes to all linked workflows at once.
A template can include any combination of these parts:
| Part | What it controls |
| - | - |
| **Prompt** | The extraction instructions for the AI agent |
| **Schema** | The field definitions and entity name |
| **Data quality** | The per-field validation rules |
| **Notifications** | Event subscriptions and notification channels |
| **Frequency** | How often linked workflows run |
Each part is optional. For example, a template can define only a schema, or a schema and notifications without a prompt.
## Versioning
Templates are versioned. Every time you save changes, a new version is published. Versions are immutable. Once published, a version cannot be edited.
This means you can:
* Roll out updates to linked workflows selectively
* Roll back to a previous version if needed
* See exactly what changed between versions
* Preview the data impact of a new version on a linked workflow before applying it
## Linking Workflows
When you link a workflow to a template, the template's configuration is applied to the workflow. The configuration managed by the template becomes **read-only** on the workflow, and you can manage it through the template UI instead.
To modify a template-managed field on a workflow, either update the template and apply the change, or unlink the workflow first.
Unlinking a workflow preserves its current configuration. The workflow keeps the prompt, schema, and notifications it had. They become editable again from the workflow itself.
## Managing Templates
Create and manage templates visually in the dashboard.
Create and manage templates programmatically with the REST API or MCP Server.
## Learn More
* [Schemas](/docs/workflows/schemas): Field definitions and data types used in template schemas
* [Notifications](/docs/notifications): Event types and channels used in template notifications
* [API Reference](/api-reference/templates/list-templates)
# AI personalization
Source: https://docs.kadoa.com/docs/ui/ai-personalization
Give your workspace's AI agents standing instructions they follow on every new workflow
You can use Markdown to customize your agents with **AI personalization**. They read it when they start working on a new workflow and follow the instructions it contains.
It's especially useful for rules that apply across all your workflows, for example how your team names fields, which date format your warehouse expects, which sources are off limits, and who to ask when something is ambiguous.
## Where to set it
Go to **Settings > General** and edit the **AI personalization** field, then click **Save**.
![The AI personalization field in workspace settings, holding four house rules]()
Anyone in the workspace can edit it. Renaming the workspace still requires an Admin or Owner.
The field takes Markdown and holds up to 8,000 characters.
## When it takes effect
The text is read once, when a workflow's [Assistant](/docs/ui/assistant) session is created. Every workflow has one long-lived session, so:
* Workflows and real-time monitors you create after saving an edit follow the new text.
* Workflows that already exist keep the instructions their session started with. Reopening the chat resumes that session, so it does not pick up the edit. To apply new rules to an existing workflow, say them in the chat or put them in the workflow [prompt](/docs/workflows/prompts).
Personalization applies to the whole workspace, so everyone's agents follow it. If you belong to several workspaces, each one has its own text.
## What to write
Write it like onboarding notes for a new colleague. Short, concrete rules beat long prose.
```markdown theme={null}
## Output conventions
- Dates are ISO 8601 (YYYY-MM-DD), always UTC.
- Prices are numeric, no currency symbol, with a separate `currency` field (ISO 4217).
- Use snake_case for every field name.
## Our sources
- "The portal" means partners.acme-internal.com. It needs the login stored in the `portal_login` variable.
- Never extract personal contact details from job postings. Company-level fields only.
## How to work with us
- If a field is missing on more than half the rows, ask before guessing a fallback.
- Prefer failing a run loudly over delivering partial data.
```
A few more examples of instructions that pay off:
| Goal | Instruction |
| - | - |
| Consistent naming | "Name the primary identifier `external_id`, never `id`." |
| Regional defaults | "Unless told otherwise, extract the German (de-DE) version of a site." |
| Compliance boundary | "Do not extract user reviews or any author names from review sites." |
| Escalation rule | "If a source starts blocking us, ask before switching to a different source." |
## What not to put here
* **Secrets.** Store credentials as [variables](/docs/workflows/variables) or [secrets](/docs/workflows/secrets) and reference them by name.
* **Instructions for one workflow.** Those belong in that workflow's [prompt](/docs/workflows/prompts) or in a [template](/docs/ui/templates).
* **Schema definitions.** Define fields in the workflow [schema](/docs/ui/workflows/schemas), where they are validated per run.
## Next steps
Build and debug workflows by chatting
Write per-workflow extraction instructions
Share prompt and schema settings across workflows
Store reusable values and credentials
# Kadoa Assistant
Source: https://docs.kadoa.com/docs/ui/assistant
Change and debug your workflows by chatting in natural language
The **Kadoa Assistant** builds your workflows and keeps maintaining them. Once a workflow exists, open the chat any time to change how it works or debug a run that came out wrong. For the first build see [Getting Started](/docs/ui/getting-started); for writing the extraction prompt see [Prompts](/docs/workflows/prompts).
## Open the Assistant
On a workflow built with the Assistant, open it and click **AI assistant**. The chat opens with the workflow in context, so you can get straight to the problem.
Each workflow has one Assistant chat. If a teammate already opened it, you join the same conversation.
## Debug with the Assistant
Debugging is a conversation. Describe what is wrong, and the Assistant investigates, makes the change, and reruns a sample so you can confirm the fix before anything goes live.
Say what is wrong and what you expected, for example *"The salary field is empty on most rows, but the postings show a salary near the title."*
The Assistant looks into the workflow, adjusts the extraction, schema, navigation, or transformation, and reruns the preview. It tells you what it changed as it works.
![The Kadoa Assistant working through its plan and narrating each step]()
Check the updated rows.
![The review state: a table of sample rows with Approve, Rerun, and Report problem actions, and a note that no quota is used until you approve]()
If the sample looks right, approve to apply the change. If not, reply with what is still off and the Assistant reruns. Repeat as many times as you need.
The Assistant can investigate and fix issues across the whole workflow:
| If you see... | Ask the Assistant to... |
| - | - |
| A field that is empty or wrong | Find the value on the page and extract it correctly |
| Rows missing or pagination stopping early | Keep paginating until there are no more results |
| A run that looks stale or returned too little | Look into why and fix it |
| Output in the wrong format | Normalize a field, for example dates to `YYYY-MM-DD` |
| A source that started returning blocked or empty pages | Investigate the access issue and adapt |
| The wrong fields, schedule, alerts, or monitored fields | Change the schema, [schedule](/docs/ui/workflows/scheduling), [notifications](/docs/ui/notifications), or [monitoring](/docs/change-detection) |
For automatic checks on every run instead of spot-fixing, add [Data Quality](/docs/ui/data-quality) rules.
## What to say
Brief the Assistant like a colleague: lead with the symptom and the result you expect, and name the field, the page, and what the correct value should look like. The same advice as [writing a good prompt](/docs/workflows/prompts) applies.
| Vague | Specific |
| - | - |
| "The data is broken." | "Since yesterday the `price` field is null on every row. The source still shows a price on each product page." |
| "Get more data." | "Open each fund profile and extract name, strategy, AUM, and inception date. Continue until there are no more results." |
| "Fix the dates." | "`published_date` comes back as free text like 'June 3, 2026'. Normalize it to YYYY-MM-DD." |
Type `@` in the chat to reference a saved [variable](/docs/workflows/variables) or [template](/docs/ui/templates), for example a login value or a known source pattern.
Rules you repeat on every workflow belong in [AI personalization](/docs/ui/ai-personalization) instead. The Assistant reads them when it starts on a new workflow.
## When the Assistant can't fix it
The Assistant tells you and automatically hands the issue to [Kadoa support](/docs/support-center). For how failed runs are classified and auto-recovered, see [Error Handling](/docs/workflows/error-handling).
## Next steps
Write prompts that get the best results
Review and approve data before it goes live
Understand failures and recovery
Validate every run with per-field rules
Give your workspace's agents standing instructions
# Data Explorer
Source: https://docs.kadoa.com/docs/ui/data-explorer
Query a workflow's data with SQL or a plain-English question
Data Explorer lets you query the data of a workflow from the dashboard. Use it to spot-check results, count distinct values, find duplicates or empty fields, and filter rows before you export.
Open a workflow and click **Explore** above the data table.
## Ask
Type a question and click **Run** (or press Cmd+Enter). Kadoa writes a query from your question, runs it, and shows the result in the data grid. While the box is empty, it cycles through example questions built from the workflow's columns.
Examples:
* count of rows per brand, largest first
* rows where price is empty
* average price per category, highest first
* rows per day, newest first
Click **SQL** to see and edit the query that ran.
If a question cannot be answered from the columns in the workflow, Kadoa says so instead of guessing.
## SQL
Click **SQL**, write a query, and click **Run** or press Cmd+Enter. All rows are in one table called `data`. Column names, the table name and keywords autocomplete as you type; press Tab to accept.
Data Explorer uses [DuckDB SQL](https://duckdb.org/docs/sql/introduction). A few useful queries:
```sql theme={null}
-- Total rows and distinct keys
SELECT COUNT(*) AS rows, COUNT(DISTINCT url) AS distinct_urls FROM data;
-- Duplicate keys
SELECT url, COUNT(*) AS n FROM data GROUP BY 1 HAVING COUNT(*) > 1;
-- Rows with an empty value
SELECT * FROM data WHERE title IS NULL OR title = '';
-- Rows per day
SELECT CAST(posted_date AS DATE) AS day, COUNT(*) FROM data GROUP BY 1 ORDER BY 1 DESC;
-- Column statistics (min, max, null %, distinct count)
SUMMARIZE data;
```
## Results
Results replace the data grid. The banner shows the result count and the number of rows in the run. Click **Back to all data** to return, or the download button to save every row of the result as CSV.
## Limits
* Queries run on the latest published run. Select a run in **History** to query that run instead.
* Queries are read-only. One `SELECT` per run; `WITH` clauses are fine.
* Up to 1,000 rows are shown. Add a filter or aggregate to narrow larger results.
* Queries run in your browser. The run's data is downloaded once per session; very large runs take longer to load.
* Ask sends the schema and five sample rows to the model, never the full dataset.
* Chrome and Edge are supported.
# Export & Access Data
Source: https://docs.kadoa.com/docs/ui/data-export
View and export your extracted data from the Kadoa dashboard
## View Data
All extracted data is available in your [Kadoa Dashboard](https://kadoa.com/dashboard):
1. Navigate to your workflow
2. Browse records in the table view
3. Filter and sort to find specific items
## Export Data
1. Open a completed workflow run
2. Click **Download** (top right)
3. Select format:
* **CSV** - Table rows for spreadsheets
* **JSON** - Includes metadata (`workflowId`, `runId`, `executedAt`) plus your data
**JSON structure:**
```json theme={null}
{
"workflowId": "68e7fc4befac26b8f917755f",
"runId": "68e7fc4cefac26b8f9177560",
"executedAt": "2025-10-09T18:17:49.209Z",
"data": [
{
"title": "Product Name",
"price": "$99.99",
"inStock": true,
"extractedAt": "2024-01-15T10:30:00Z"
}
]
}
```
## Delivered to destinations
If your team has [data destinations](/docs/destinations/data-destinations), the **Download** dropdown also lists the destinations receiving this workflow's results after each run, with provider and target.
The **Deliver** section of the [Pipeline](/docs/ui/workflows/pipeline) panel shows the same destinations with their file formats, and lets you request new destinations such as Snowflake, Databricks, Amazon S3, Google Cloud Storage, or SFTP.
![Download menu listing file formats and synced data destinations]()
# Data Quality
Source: https://docs.kadoa.com/docs/ui/data-quality
View per-field validation rules and platform metrics for a workflow, and edit rules from the dashboard
The **Data quality** tab in the workflow sidebar shows the quality state for the currently selected run and is also where you edit per-field rules.
## Workflow Data Quality
1. Go to [Workflows](https://www.kadoa.com/dashboard) and click on a workflow
2. In the right sidebar, open the **Data quality** section.
![Workflow data quality sidebar section]()
The section has two areas, top to bottom:
* **Rules**: One row per schema field that supports validation, with the configured checks summarized.
* **Platform metrics**: Built-in metrics Kadoa computes automatically.
While the section is open, the data rows on the left are filtered to only show rows with data quality issues.
## Rules
Each row represents one schema field:
* The status dot is green when all of that field's rules passed on the current run, red when any rule failed, and absent when no rules are configured.
* The chips after the field name summarize the configured checks, for example **Presence ≥ 80%**, **Format url**. Failing chips turn red.
### Edit a field's rules
Click any field row to open the dialog to edit rules. You can read about the different kind of available rules [in the getting started guide](/docs/data-quality).
![Workflow data quality edit dialog]()
Rule changes take effect on the **next** run. The currently displayed results are unaffected until you trigger another run.
## Platform metrics
Kadoa currently offers the following built-in platform metrics. You can click any card to open a detail view.
* **Completeness**: reports whether we found as many rows as the page suggests. The detail view shows the contributing signals.
* **Suspicious values**: reports values that do not fit with the rest of the row. The detail view shows the flagged columns with example values.
## Template-managed workflows
When a workflow is linked to a template that owns its data quality rules, rules can not be edited in the workflow.
To change rules, edit the template (and publish a new version), or unlink the workflow from its template.
## Learn more
* [Data quality concepts →](/docs/data-quality)
* [API reference →](/api-reference/data-quality/get-rules)
# Getting Started
Source: https://docs.kadoa.com/docs/ui/getting-started
Build a web dataset from a single prompt with the Kadoa Assistant
## Create your first workflow
Describe what you want in plain language and the **Kadoa Assistant** builds the workflow: it finds the source, proposes a schema, builds and tests the extraction, and shows you a sample to approve. Open the [Kadoa Dashboard](https://kadoa.com/dashboard) and click **Create your first workflow**.
Building and previewing is **free**. You only use quota once you approve and run on the full dataset. See [Usage & Billing](/docs/usage).
![The Kadoa workflow setup prompt with a prompt editor and configuration controls]()
Describe the task in plain language, for example *"Get me all job postings from sandbox.kadoa.com/careers"*. Include a source URL when you have one; the Assistant can also help find the source.
The setup prompt includes optional controls for:
* **Product type**:
* **Workflow**: Extract a dataset once or on a schedule.
* **Real-time**: Monitor a website continuously. Select at least one notification channel before you start, unless a selected template provides notifications.
* [**Templates**](/docs/ui/templates): Start with a reusable prompt, schema, data quality rules, notifications, or frequency.
* [**Notifications**](/docs/ui/notifications): Choose existing channels or create one without leaving the prompt.
* [**Tags**](/docs/workspaces): Organize the workflow with workspace tags.
* [**Variables**](/docs/workflows/variables): Insert reusable values into the prompt.
Real-time monitoring is available on request. [Contact support](https://www.kadoa.com/contact?type=support) to enable it for your workspace.
You can also use the prompt shortcuts directly: `@` for notification channels, `#` for tags, and `$` for variables.
The Assistant confirms the source, writes the extraction code, and runs a preview, narrating each step. The result is a **deterministic pipeline**: it extracts your data the same way on every run, not a fresh AI guess each time.
![The Kadoa Assistant working through its setup plan step by step]()
Check the preview data. It is free and uses no quota until you approve.
![Preview data generated by the Assistant, shown as a structured table]()
Tell the Assistant what to fix, for example *"add a salary field"* or *"pull location from the detail page"*, and it reruns the preview. Nothing is charged until you approve. For automatic checks on every run, add [Data Quality](/docs/ui/data-quality) rules.
The workflow produces the full dataset. Add [scheduling](/docs/ui/workflows/scheduling), [change detection](/docs/change-detection), and [notifications](/docs/ui/notifications) any time.
![A completed workflow showing the full extracted dataset, with download and run controls and the data quality panel]()
Keep chatting to change fields, navigation, or the schedule, or edit the **Sources**, **Schema**, **Notifications**, and **Tags** panels directly. Templates keep their managed settings read-only until you update or unlink the template.
## Example
Prompt: *"Get me all job postings from sandbox.kadoa.com/careers"*. The Assistant proposes a five-field schema and returns structured data:
```json theme={null}
[
{
"title": "Graphic Designer",
"detailUrl": "https://sandbox.kadoa.com/careers/career/graphic-designer",
"shortDescription": "Create designs for our digital platforms.",
"postedDate": "2024-01-15"
},
{
"title": "Data Analyst",
"detailUrl": "https://sandbox.kadoa.com/careers/career/data-analyst",
"shortDescription": "Interpret data and analyze results to drive business decisions.",
"postedDate": "2023-10-20"
}
]
```
[Export and access your data →](/docs/ui/data-export) · [Is web scraping allowed?](/docs/security/compliance)
## Next steps
Adjust or add the URLs a workflow extracts from
Customize the fields the Assistant proposed
Validate every run with per-field rules
Review and approve data before it goes live
# Notifications
Source: https://docs.kadoa.com/docs/ui/notifications
Get notified when workflow events occur in the Kadoa Dashboard
Notifications alert you when workflows complete, fail, or detect data changes. You can receive them via email, webhooks, or WebSockets.
Please look at event types and notifications channels before getting started, see [Notifications](/docs/notifications).
## Workspace Notifications
Set default notifications for all workflows:
1. Go to [Settings → Notifications](https://www.kadoa.com/settings?section=notifications)
2. Click **Add notification**
3. Select an **event type**
4. Choose or create a **channel**
5. Click **Save**
![Workspace notification settings]()
### Notification Content
Below the workspace notifications, toggle what each event's notifications contain: an AI summary of the change, the changed records, and a CSV attachment. These are workspace defaults; workflows inherit them unless overridden.
![Workspace notification content]()
## Workflow Notifications
Set notifications for a specific workflow in either place:
* **During creation:** In the setup prompt, open **Notifications**, select one or more channels, or choose **Create channel**. The selection is applied to data-change notifications when the workflow is created.
* **After creation:** Open your workflow, click the **Notifications** tab, and configure its event subscriptions.
To configure notifications after creation:
1. Open your workflow
2. Click the **Notifications** tab
3. Click **Add notification**
4. Select an **event type**
5. Choose or create a **channel**
6. Click **Save**
![Workflow notification settings]()
Workflow settings override workspace settings for the same event type.
## Data Change Notifications
Filter what triggers `workflow_data_change` alerts:
1. Select **Data changes** as event type
2. Configure tracking:
* **Fields**: Select fields to monitor (all by default)
* **Conditions**: Add filter criteria
* **Change type**: Choose tracking for Updated, Added, or Removed data. (all by default)
* **Notification content**: Choose what the notification contains (AI summary, changed records, CSV attachment); workflows inherit the workspace defaults unless overridden
3. Choose a **channel**
4. Click **Save**
![Workflow data changes settings]()
## Template-Managed Notifications
When a workflow is linked to a [template](/docs/ui/templates) that includes notifications, the notification settings are managed through the template. To change them, edit the template or unlink the workflow.
## Testing
Verify your configuration works:
1. Go to the **Notifications** page
2. Select an event type
3. Click **Test**
4. Check your channel for the test notification
![Test notifications button]()
# Team Management
Source: https://docs.kadoa.com/docs/ui/team-management
Add members, manage invitations, and assign roles in your team workspace
## Add Team Members
Only Admins and Owners can add and remove team members.
To invite a new team member:
1. Open **Settings** from the sidebar
2. Click **Add members**
3. Enter the email address of the person you want to invite
4. Select their role (Member, Admin, Viewer, or Compliance Officer)
5. Click **Send invitation**
![]()
![]()
![]()
**What happens next:**
* If the person has a Kadoa account, they receive an invitation notification
* If they're new to Kadoa, they receive an email to create an account and join your team
* You can resend or cancel pending invitations from **Settings**
## Manage Invitations
From **Settings**, you can:
* **Resend invitations** that recipients missed
* **Cancel invitations** that are no longer needed
* **View invitation status** to see who hasn't joined yet
## Change Member Roles
1. Go to **Settings**
2. Find the team member in the list
3. Click on their current role
4. Select the new role from the dropdown
5. Confirm the change
## Remove Team Members
1. Go to **Settings**
2. Find the team member in the list
3. Click the remove button (trash icon)
4. Confirm the removal
Removing a team member does not delete their workflows. Workflows remain accessible to the team.
## Roles and Permissions
For a detailed breakdown of what each role can do, see [Workspaces → Roles and Permissions](/docs/workspaces#roles-and-permissions).
| Role | Create Workflows | Edit All Workflows | Manage Members | Access Compliance |
| - | - | - | - | - |
| Member | ✅ | Own only | ❌ | ❌ |
| Admin | ✅ | ✅ | ✅ | Limited |
| Owner | ✅ | ✅ | ✅ | Limited |
| Viewer | ❌ | Read-only | ❌ | ❌ |
| Compliance Officer | ❌ | Read-only | ❌ | ✅ |
## Learn More
* [Workspaces Overview →](/docs/workspaces)
* [SSO Configuration →](/docs/security/sso)
* [SCIM Provisioning →](/docs/security/scim)
# Templates
Source: https://docs.kadoa.com/docs/ui/templates
Create and manage reusable workflow configurations in the Kadoa Dashboard
Templates let you define a reusable configuration (prompt, schema, data quality rules, notifications, and frequency) and apply it to multiple workflows. You can pick a template directly from the [Kadoa Assistant](/docs/ui/getting-started) prompt box to pre-fill a new workflow. For an overview, see [Templates](/docs/templates).
## Create a Template
1. Go to [Templates](https://www.kadoa.com/templates) and click **Create template**
2. Enter a **name** for your template
3. Configure the parts you want to include:
* **Prompt**: Enter extraction instructions
* **Schema**: Define fields and entity name using the schema editor
* **Data quality**: Define per-field validation rules
* **Notifications**: Add event subscriptions and choose notification channels
* **Frequency**: Set how often linked workflows run
4. Click **Save changes** to publish the first version
![Template editor with prompt, schema, and notifications tabs]()
You don't need to fill in all parts. A template can include any combination of prompt, schema, data quality, notifications, and frequency.
## Save a Workflow as a Template
You can create a template from an existing workflow's configuration:
1. Open the workflow you want to use as a starting point
2. Click the **actions menu** and select **Save as template**
3. Choose which parts to include (prompt, schema, notifications)
4. Enter a name and click **Save**
The workflow is automatically linked to the new template.
## Edit a Template
1. Go to [Templates](https://www.kadoa.com/templates) and click on a template
2. Make your changes in the **Prompt**, **Schema**, or **Notifications** tabs
3. Click **Save changes**
Each save publishes a new version. Previous versions are preserved in the **History** tab.
## Duplicate a Template
1. Go to [Templates](https://www.kadoa.com/templates) and open the **actions menu** on a template
2. Select **Duplicate** and optionally enter a name (defaults to the original name with a Copy suffix)
3. Click **Duplicate**
![Template actions menu with Duplicate]()
The copy is independent and starts fresh at version 1: the latest published version's prompt, schema, data quality rules, notifications, and frequency are carried over, but version history and linked workflows are not.
## Apply Template Changes to Linked Workflows
When you publish a new version, linked workflows don't update automatically; you decide when to roll it out.
1. Open the template and switch to the **Linked workflows** tab to see everything connected
2. Click **Link workflows** to pick existing workflows to link, or use the version controls on a linked workflow to roll out a new version
3. Review the affected workflows in the confirmation dialog and confirm
![Linked workflows tab on a template]()
Before applying, switch the dialog to **Preview data changes** to see how each field will change. Fields are grouped into **Added**, **Updated**, and **Removed**. Workflows with previous runs show existing data for unchanged and removed fields, plus grounded simulations for new and changed fields. Without a previous run, source-backed values remain empty.
![Preview data changes dialog]()
A few things worth knowing:
* **Active real-time monitors are skipped.** Pause them first if you want them in the batch.
* The parts managed by the template stay read-only on each linked workflow. To change them, edit the template and apply, or unlink the workflow first.
## Delete a Template
1. Go to [Templates](https://www.kadoa.com/templates)
2. Select the template(s) you want to delete
3. Click **Delete**
Deleting a template unlinks all associated workflows. Their configurations are preserved.
## Use a template when creating a workflow
When creating a new workflow, select a template from the setup prompt to pre-fill its available configuration:
1. Open [Create workflow](https://kadoa.com/workflow/new)
2. Open the **Template** selector and choose a template
3. Review the attached prompt, schema, data quality, notification, and frequency settings
4. Complete the prompt and create the workflow
The new workflow is automatically linked to the template. Template-managed settings remain read-only on the workflow until you update or unlink the template.
![Template picker in the workflow creation wizard]()
# Preview approval and data verification
Source: https://docs.kadoa.com/docs/ui/workflows/human-in-the-loop
When you review a preview and when Kadoa checks a run
You approve previews. Kadoa handles data verification.
## Approve a preview
The [Assistant](/docs/ui/getting-started) builds your workflow and generates a preview.
1. When **Preview ready** appears, check the fields and values.
2. If changes are needed, ask the Assistant and wait for the updated preview.
3. When the data is correct, select **Approve** to start the workflow.
If the settings changed after the preview ran, **Approve** starts a new preview with the current settings instead. Approve that preview once it finishes.
Preview data is free. No quota is used until you approve.
During updates, approval is disabled and the previous preview stays visible. If **Preview update incomplete** appears, select **Report problem**. Kadoa needs to refresh the preview before you can approve it.
## Data verification
**Verifying data** means a run is held for Kadoa review before delivery. The banner explains why. No review is needed from you.
Automated [data quality checks](/docs/data-quality) cover field rules, completeness, and suspicious values. Agents investigate flagged results against the source and attempt fixes.
Runs that pass the required checks can be approved automatically. If issues remain or human approval is required, a Kadoa specialist reviews the run before delivery.
During repairs, an update banner replaces the review banner. Previously delivered data stays available.
See [Workflow lifecycle](/docs/workflows/lifecycle) for all states and API fields.
# Pipeline
Source: https://docs.kadoa.com/docs/ui/workflows/pipeline
See how a workflow extracts, transforms, and delivers data
The **Pipeline** tab in the workflow sidebar shows a workflow's three steps:
* **Extract**: how the source is fetched, such as a direct API call or a browser scrape.
* **Transform**: the workflow's transformation, if any, plus built-in function fields.
* **Deliver**: where each run's results go. Downloads, API and SDK access, and the destinations Kadoa delivers to.
## Extract
The **Extract** section describes how the source is fetched: the strategy, the approach (direct API or browser), the URL, the request shape, and the selectors the workflow relies on.
![Pipeline panel on the Extract tab showing strategy, approach, URL, request and selectors]()
## Transform
The **Transform** section lists the fields the transformation reads and the fields it adds or modifies, each with a summary of how it is computed when available. Click a field to open it in the [Schema](/docs/ui/workflows/schemas) tab; untouched fields pass through as-is. Workflows without a transformation show a note saying so.
![Pipeline panel on the Transform tab listing two AI-transformed fields with one expanded]()
## Deliver
The **Deliver** section answers one question: where does this workflow's data go?
* **Download** the latest published run as CSV or JSON.
* **API endpoint** shows this workflow's data path. Expand it for the full URL, a copy button, and links to the [REST API](/docs/destinations/rest-api), [SDK](/docs/destinations/sdk), and [MCP server](/docs/sdk/mcp).
* **Destinations** lists the [data destinations](/docs/destinations/data-destinations) that deliver this workflow after each run. Destinations belong to your workspace, not to one workflow: set one up once and any workflow can deliver to it. Expand a destination for its share or bucket, file formats, timing, and a **Change** button. Click **Add** to open the catalog: Amazon S3, Google Cloud Storage, SFTP, Snowflake, Databricks, and Azure Blob Storage. **Set up** opens a [Support Center](/docs/support-center) request prefilled with the workflow, the destination, and the details we need.
* **Notifications** links to the [Notifications](/docs/ui/notifications) panel for Slack, email, and webhooks.
![Pipeline panel on the Deliver tab with download buttons, API endpoint, an expanded Snowflake destination, and notifications]()
## API
* [Get workflow by ID](/api-reference/workflows/get-workflow-by-id) returns the `transformation` object.
* [List workflow destinations](/api-reference/workflows/list-workflow-connectors) returns the workflow's delivery destinations.
# Scheduling
Source: https://docs.kadoa.com/docs/ui/workflows/scheduling
Kadoa allows you to monitor websites for data changes and receive notifications when updates occur.
![Screenshot]()
* When creating or editing workflow, click "Repeat" to set up regular updates.
* Choose an update frequency.
* Real-time monitoring is available on all plans.
To set up notifications:
![Screenshot]()
* click "Get notified when data changes".
* In the monitoring settings, select the fields you want to track for changes.
* Select about what type of changes you want to be notified. We support 3 types of changes:
* **All changes**: Receive notifications for all changes (additions, modifications, and removals). Modifications require at least one [schema key field](/docs/workflows/schemas); without a key, a modified record is reported as removed and added.
* **Only added data**: Receive notifications only for new data that has no exact match in the previous run.
* **Only removed data**: Receive notifications only for removed data.
* Optional: You can add conditions to the monitoring. For example, you can only receive notifications if the data has changed in a specific field and contains a specific value.
* Choose your preferred notification channel (email, Slack, webhooks, and more).
### How Changes Are Detected
For detailed information about the change detection algorithm, matching methods, and examples, see [Change Detection](/docs/change-detection).
When Kadoa detects a change, you'll receive a notification on your configured channel and the change appears in the workflow's run history. See [Change Detection](/docs/change-detection) for how changes are detected, delivered, and retrieved.
To start another workflow on data change instead of notifying, see [Triggers](/docs/ui/workflows/triggers).
# Define Schema
Source: https://docs.kadoa.com/docs/ui/workflows/schemas
Set up data schemas when creating workflows in the UI
Every Kadoa workflow has a **Data Schema** that defines what data to extract from a source.
When you create a workflow with the [Kadoa Assistant](/docs/ui/getting-started), it builds the schema for you, and you can review and customize it in the **Schema** panel. For an overview of schemas and available data types, see [Schemas](/docs/workflows/schemas).
![Schema panel showing the field list, inline field details, and JSON editor control]()
## How the schema is created
* **If you name the fields in your prompt** (for example, "extract title, price, and rating"), the Assistant uses exactly those.
* **If you don't**, the Assistant recommends fields based on the source.
Either way, you can adjust everything before you approve.
The schema lives in its own **Schema** tab; the [Pipeline](/docs/ui/workflows/pipeline) tab shows how fields are transformed after extraction.
## Edit the fields
Click a field for read-only details; the pencil opens an inline edit form. **Add field** opens a dialog, **JSON** enables bulk editing, and **Reorder** rearranges fields. Mark fields as **key fields** to improve record matching and data quality.
Tip: mark scalar `STRING`, `NUMBER`, or `LINK` fields expected to be unique and non-empty as key fields. In JSON, set `isKey: true`. Learn more in [Data Quality](/docs/ui/data-quality).
When you add a field, you can pick a **data type** or a **[function](/docs/workflows/schemas#function-fields)** — a field Kadoa computes for you, such as **Unique ID** or **Custom categories**.
## Use an existing schema
Instead of starting fresh, you can base the schema on:
* **Your schemas**: schemas you have created before.
* **Kadoa schemas**: pre-built schemas for common use cases.
Customize any selected schema by adding, removing, or renaming fields.
If your workflow uses a [template](/docs/ui/templates), its schema is managed by the template. Edit it through the template instead of here.
## Reuse a schema across workflows
To reuse the same schema across multiple workflows, save it in a [template](/docs/ui/templates). A template packages a prompt, schema, data quality rules, and notification settings into a reusable, versioned unit.
1. Open [Templates](https://www.kadoa.com/templates) and click **Create template**
2. Define your fields in the **Schema** tab, or save an existing workflow's schema as a template
3. When you create a workflow, choose the template to pre-fill its schema
Updating the template's schema and publishing a new version rolls the change out to every linked workflow.
# Sources
Source: https://docs.kadoa.com/docs/ui/workflows/sources
Choose the URLs a workflow extracts from
A workflow's **source** is the set of URLs it extracts from. There are two ways to set it:
* **Let the Assistant find it.** Describe what you want and the [Kadoa Assistant](/docs/ui/getting-started) detects the source from your prompt. For example, *"Get me all job postings from sandbox.kadoa.com/careers"* resolves to the careers page automatically.
* **Paste your own URLs.** Provide one or more specific URLs to extract from exactly those pages.
You can review and change the source any time in the **Sources** panel.
![The Kadoa Assistant prompt box with the source detected from the description]()
## Paste one or more URLs
When you already know the exact pages, paste them in the Sources panel:
* **A single URL** for one page.
* **A list of URLs** when you have a specific set of pages from the same site. Ideal when you don't need to extract an entire section or category.
For many similar pages that follow a pattern, describe it in your prompt (for example, "go through pages 1 to 50") and the Assistant generates the URLs for you.
## Supported data types
Kadoa supports following content types by design:
* HTML
* XML
* RSS ...
* XML
* RSS
* JSON
* PDF
# Triggers
Source: https://docs.kadoa.com/docs/ui/workflows/triggers
Start one workflow automatically when another workflow's data changes
Triggers start one workflow automatically when something happens to another's data, with no manual run or external scheduler. Chain workflows, like a downstream extraction whenever an upstream one finishes or its data changes.
A common pattern is watching for content that publishes at an unpredictable time, such as a report you want to process as soon as it's out. Instead of running the full extraction every hour, point a [real-time monitor](/docs/change-detection) at the page and add a **Data changed** trigger that starts the downstream workflow the moment the report appears.
![Workflow triggers]()
## Create a trigger
1. Open a workflow → **Triggers** tab → **Add trigger**.
2. Choose when it fires:
* **Data changed** — data changed between runs
* **Workflow finished** — a run completed
* **Export completed** — an export finished
3. Select the workflow to run → **Create trigger**.
The target runs the next time the event fires. Toggle a trigger off to pause it without deleting.
## What you'll see
* Workflows with triggers show a lightning icon and count in the dashboard table.
* The **Triggers** tab lists outgoing triggers and, below, incoming triggers (workflows that start this one).
## Limits
* Up to 5 triggers per workflow.
* A workflow can't trigger itself.
* Realtime workflows can't be targets.
* Source and target must share the same team (or the same user for personal workflows).
* A triggered run only starts if the target is active, not already running, and has no open support issue. Events that arrive mid-run are skipped, not queued.
* Chains up to 15 levels deep to prevent loops.
For time-based runs, see [Scheduling](/docs/ui/workflows/scheduling). To get notified of these events instead, see [Notifications](/docs/notifications).
# Usage & Billing
Source: https://docs.kadoa.com/docs/usage
Usage-based pricing that charges for active workflows
Kadoa pricing is usage based: you pay for active workflows, measured in slots. A slot holds one active workflow in production. Free it up and reuse it any time.
## What is a workflow?
One data extraction that targets a single source with one schema. Refining the schema is allowed and does not create a new workflow.
## How slots work
* Building and previewing workflows is free and never uses a slot.
* Scheduled and real-time workflows hold a slot while enabled, with a 30-day minimum.
* One ad-hoc run holds a slot for 30 days. Additional runs during that period are included, for example for experiments or sandboxing.
* Disable a workflow to free its slot for another workflow.
## Workflow classes
Most workflows use exactly one slot. You see the class during preview.
Extracted rows and unblocking are covered by fair use. If a workflow exceeds fair use, a multiplier is applied; simple cases get a divider.
| Class | Slot usage |
| - | - |
| **Standard**: the majority of workflows | 1x |
| **Simple**: lightweight GET requests | 0.1x |
| **Complex**: exceptional edge cases, less than 1% on our platform | 2x or more |
## Plans that count approved workflows
Some plans meter approved workflows instead of slots. On these plans, a workflow counts towards your quota once it has been built, approved, and run successfully at least once. Previews stay free.
## Monitor your usage
Track your balance and usage trends on the [**Usage Dashboard**](https://www.kadoa.com/usage).
# Error Handling
Source: https://docs.kadoa.com/docs/workflows/error-handling
Understand failures, get notified, and track workflow reliability
When data is late or missing, someone downstream feels it. Kadoa prevents that by classifying every failure, telling you whose fault it is, notifying you through the channel you choose, and recovering automatically from most errors. When something breaks and gets fixed, you'll know what happened, why, and how it was resolved.
## Understanding Runs vs Workflows
* **Run**: A single execution of your workflow at a scheduled time
* **Workflow**: The configuration that schedules and manages runs
When errors occur, they affect individual runs.
## Error Handling
Every failed run tells you whether the issue is with the website, with Kadoa, or with the data itself.
Kadoa uses a multi-layer approach to handle issues automatically:
### 1. Self-Healing (Automatic)
Our AI adapts to changes without intervention:
* Detects page structure changes and updates selectors
* Adjusts to new layouts and UI elements
* Retries with different strategies when extraction fails
* Agent investigates any issues automatically
Most issues are resolved at this stage without you knowing.
### 2. Human Investigation (Manual)
For complex issues that can't auto-recover:
* A Support ticket gets created and our ops team investigates
* You're notified about the failure via email and dashboard
* Fixes are applied until workflow runs successfully again
* You're notified about the recovery and support ticket gets resolved
Common issues we handle: authentication problems, site maintenance, anti-bot challenges, and network failures.
## Error Format
Errors are returned per run in the [history endpoint](/api-reference/workflows/get-the-workflow-history):
```json theme={null}
{
"errors": [{
"errorCode": "BLOCKED_ACCESS",
"message": "Access blocked by website security",
"timestamp": "2025-01-15T10:30:00Z",
"screenshot": "https://storage.googleapis.com/...",
"context": {
"url": "https://example.com/page",
"httpStatus": 403,
"retryCount": 3
}
}]
}
```
Each error includes a screenshot of what the scraper saw, so you can verify the cause yourself.
## What Happens When Errors Occur
1. **Run marked as FAILED** - visible in the dashboard and API
2. **You're notified** - if you've subscribed to `workflow_failed` events (see below)
3. **Kadoa retries automatically** - self-healing technology attempts recovery on the next run
4. **Ops team is notified** - critical errors are flagged for investigation
5. **Submit a support request** - use the [Support Center](/docs/support-center) for expedited help
![Screenshot]()
## Get Notified About Failures
Don't poll the dashboard. Set up notifications so failures come to you.
Subscribe to **Workflow health** events (`workflow_failed`) via the [dashboard](/docs/ui/notifications) or [API](/docs/sdk/notifications/overview). This covers both failures and recoveries. Choose your channel:
| Channel | Best for |
| - | - |
| **Email** | Daily review, non-urgent updates |
| **Slack** | Real-time team awareness |
| **Webhook** | Feeding into internal alerting (PagerDuty, Datadog, etc.) |
You can also subscribe to `workflow_finished` and `workflow_data_change` events. See [Notifications](/docs/notifications) for the full setup guide.
## Track Run History
Every workflow keeps a full history of runs. Use it to spot patterns like intermittent blocks, declining record counts, or degraded performance.
```bash theme={null}
GET /v4/workflows/{workflowId}/history
```
Each run includes:
* **Status**: `FINISHED`, `FAILED`, `IN_PROGRESS`, `VALIDATING`
* **Timestamps**: `startedAt`, `finishedAt`
* **Records extracted**: row count
* **Screenshot**: what the page looked like during extraction
* **Errors**: full error details with context
# Lifecycle
Source: https://docs.kadoa.com/docs/workflows/lifecycle
Workflow states in the dashboard and API
A workflow defines what to collect and when. Each execution is a run. Kadoa tracks workflow, run, Assistant, and support states separately.
## States
These diagrams show the common transitions.
### Workflow
```mermaid actions={false} theme={null}
flowchart LR
A[Setup] --> B[Preview]
B -->|Approve| C[Active]
C -->|Pause| D[Paused]
D -->|Resume| C
```
Active workflows run once, on a schedule, or continuously. An active workflow can be waiting between runs.
### Run
```mermaid actions={false} theme={null}
%%{init: {"flowchart": {"nodeSpacing": 20, "rankSpacing": 45}}}%%
flowchart TB
A[Running] -->|Hold| B[Verifying data]
B -->|Release| A
A --> C[Finished]
A --> D[Failed]
A --> E[Stopped]
```
Kadoa combines [data quality checks](/docs/data-quality) with agent review. Runs that meet the approval requirements pass automatically. Unresolved issues or required human checks go to a Kadoa reviewer before delivery. Previous results stay visible.
### Assistant activity
```mermaid actions={false} theme={null}
flowchart LR
A[Assistant working] -->|Finish| B[Idle]
B -->|Request| A
B -->|Repair| C[Repairing]
C -->|Finish| B
```
When the Assistant needs input, it is idle and shows **Reply needed**. Repairs appear as **Updating** in the UI.
### Support
```mermaid actions={false} theme={null}
flowchart LR
A[No open cases] -->|Open case| B[In support]
B -->|Close last case| A
```
Support is independent of the other states. An open case does not mean a repair is running.
## UI states
Badges show each state separately. For example, **Preview + Updating + In support** means a preview is being updated and a support case is open.
| Badge | Meaning |
| - | - |
| Setup | Kadoa is preparing the workflow. |
| Assistant working | The Assistant is handling your request. |
| Preview ready | Review and approve, or request changes. |
| Preview | Preview data exists, but approval is currently unavailable. The banner explains why. |
| Running | Collecting data or monitoring continuously. |
| Verifying data | Delivery is on hold for [Kadoa review](/docs/ui/workflows/human-in-the-loop#data-verification). No review is needed from you. |
| Updating | Kadoa is changing or repairing the workflow. |
| Reply needed | Answer the Assistant's question. |
| Scheduled | Waiting for the next run. |
| Complete | Finished, with no further runs scheduled. |
| Paused | Scheduled runs are paused. |
| Failed / Update failed | The latest attempt failed. Check run details. |
| In support | At least one support case is open. |
Previous data stays visible during updates and after failures. The banner explains the current activity or action needed.
## API states
Use `presentation` to display states and `availableActions` to enable controls. List and detail responses include:
| Field | Meaning |
| - | - |
| `state` | Workflow lifecycle, such as `PREVIEW`, `ACTIVE`, or `PAUSED`. |
| `runState` / `execution` | Latest run state, progress, and preview availability. |
| `workspaceActivity` | `idle`, `assistant`, or `repair`. |
| `assistantState` / `sessionStatus` | Assistant conversation state. |
| `awaitingUserInput` | The Assistant needs your reply. |
| `inSupport` | At least one support case is open. |
| `presentation` | Banner status, badges, and dashboard group. |
| `availableActions` | Each action's `allowed` flag and a `reason` when blocked. |
| `publishedJobId` / `jobId` | Run supplying displayed data / latest run. These can differ. |
| `displayState` | Combined workflow and run state, used for filtering. |
A preview being updated returns:
```json theme={null}
{
"state": "PREVIEW",
"displayState": "PREVIEW",
"workspaceActivity": "assistant",
"presentation": {
"status": "UPDATING",
"badges": [
{ "kind": "lifecycle", "status": "PREVIEW" },
{ "kind": "activity", "status": "UPDATING" }
],
"group": "working"
},
"availableActions": {
"approve": { "allowed": false, "reason": "ASSISTANT_ACTIVE" }
}
}
```
The API rechecks permissions and state when you submit an action.
If a preview no longer reflects the workflow's current settings, approving it does not activate the workflow. `PUT /v4/workflows/{workflowId}/resume` starts a new preview run with the current settings and responds with `202` and the new run's `jobId`. Approve the new preview once it finishes. Bulk approval through `POST /v4/workflows/bulk` reports the same case per workflow with `previewRestarted: true` and the `jobId`.
## Filtering workflows
Filter by dashboard group, `displayState`, or lifecycle `state`:
```bash theme={null}
GET /v4/workflows?statusFilters=group:attention
GET /v4/workflows?statusFilters=group:attention,group:failed
GET /v4/workflows?displayState=RUNNING
```
Groups are `attention`, `working`, `support`, `failed`, `active`, `complete`, and `paused`. Open support cases always place workflows in `support`.
You cannot filter by `presentation.status`.
# Overview
Source: https://docs.kadoa.com/docs/workflows/overview
What a workflow is and how to build one
A **workflow** is the core building block in Kadoa. It defines what data to extract, where to extract it from, and when to run, then delivers the results to your destination.
The fastest way to create one is the [Kadoa Assistant](/docs/ui/getting-started): describe what you want in the setup prompt, optionally choose a workflow type, template, notifications, tags, or variables, and it builds and tests the workflow for you. You can also [build with code](/docs/sdk/workflows/create).
![A completed workflow with its full extracted dataset]()
## Explore workflows
Tell Kadoa what to extract and how to get there
Reusable values for your prompts
Define what data to extract
Understand status, repeating runs, and when action is needed
Reuse prompt, schema, and notification settings
Start one workflow from another automatically
Handle and debug workflow failures
# Prompts
Source: https://docs.kadoa.com/docs/workflows/prompts
Tell Kadoa what to extract and how to get there
Every Kadoa workflow is driven by a **prompt**: a plain-language description of what data you need and how to reach it. When you build with the [Kadoa Assistant](/docs/ui/getting-started), your description *is* the prompt, and you can refine it any time by chatting or editing it directly.
## How It Works
1. **Describe what you want** in plain language, with or without a starting URL
2. **The agent reads your prompt**, breaks it into steps, and navigates the site
3. **Data is extracted** into your schema and delivered to your destination
No selectors or click sequences are needed. The sections below help you write prompts that get the best results.
![The Kadoa workflow setup prompt with a prompt editor and configuration controls]()
## Writing good prompts
Good prompts are specific and sequential: tell the agent where to go, what to interact with, what to extract, and how far to continue. The examples below show the pattern.
## Use the setup prompt
When creating a workflow in the dashboard, the setup prompt combines your instructions with optional workflow configuration. Use the controls below the editor to choose a template, notification channels, tags, or the workflow type.
You can also insert reusable items while typing:
| Shortcut | Inserts | Example |
| - | - | - |
| `@` | Notification channel | `Email:Operations` |
| `#` | Workflow tag | `Tag:Sales` |
| `$` | Variable | `Variable:keywords` |
Selecting an item adds a chip to the prompt and updates its related workflow setting. Remove the chip to remove the channel or tag selection. You can create a variable, tag, or notification channel from its selector when needed.
Choose **Workflow** for a dataset that runs once or on a schedule. Choose **Real-time** to monitor a site continuously; real-time workflows require a notification channel unless a selected template provides notifications.
Real-time monitoring is available on request. [Contact support](https://www.kadoa.com/contact?type=support) to enable it for your workspace.
## Example Prompts
```
- Go to the "All Products" page
- Extract: product_name, price, brand, availability, rating
- Click "Next" to continue through all pages
- Stop when there are no more results
```
```
- Log in with $username and $password
- Navigate to "Purchase Orders"
- Filter by status "Open"
- Extract: po_number, vendor, amount, date, status
- Continue through pagination
```
```
- Navigate to the investor relations section
- Find all quarterly earnings reports from 2024-2025
- Download each PDF and extract: revenue, net_income, eps, guidance
- Also collect: report_date, fiscal_quarter, filing_type
```
```
- Go to the careers page
- For each job posting in the list, click into the detail page
- Extract: title, department, location, salary_range, description
- Return to the list and continue with the next posting
- Continue through all pages
```
## Variables
Use reusable variables in your prompt. Variables are replaced at runtime.
* Type `$` in the setup prompt to see available variables
* Select a variable to insert a chip that keeps its stored key and ID
* Secret variables are encrypted and never sent to LLMs
* Store entire prompts as variables to reuse them across workflows
[Learn more about Variables →](/docs/workflows/variables) · [Secret Variables →](/docs/workflows/secrets)
***
## Using in Code
Pass your prompt via `userPrompt`:
```json theme={null}
{
"urls": ["https://example.com/products"],
"userPrompt": "Extract all products. Continue through pagination. Open each item for full details."
}
```
[Create workflows in code →](/docs/sdk/workflows/create)
# Schemas
Source: https://docs.kadoa.com/docs/workflows/schemas
Create and manage reusable schemas that define what data to extract
Schemas define the structure of data you want to extract from websites. They specify field names, data types, and descriptions. Schemas can be reused across multiple workflows.
For example, if you need to extract store locations from 5 different websites, you can create one schema and use it for all of them.
## Managing Schemas
Schemas can be created and managed in two ways:
Create schemas visually using the schema builder, or select from existing templates.
Create and manage schemas programmatically with full control over field definitions.
## Data Types
When defining schemas, you specify the data type for each field to ensure accurate extraction and validation. Kadoa supports the following data types:
| Data Type | Description | Example Use Cases |
| - | - | - |
| **STRING** | String/text content | Product names, descriptions, article headlines |
| **NUMBER** | Numeric values (integers, decimals) | Quantities, ratings, scores, counts |
| **BOOLEAN** | True/false values | Availability status, feature flags, yes/no indicators |
| **DATE** | Date values | Publication dates, deadlines, event dates |
| **DATETIME** | Date and time values | Timestamps, scheduled times, last updated |
| **MONEY** | Currency and monetary values | Prices, costs, revenue, discounts |
| **IMAGE** | Image URLs and references | Product photos, thumbnails, profile pictures |
| **LINK** | URLs and hyperlinks | Product pages, external links, social media |
| **OBJECT** | Nested/complex JSON structures | Structured metadata, complex configurations |
| **ARRAY** | Lists/arrays of values | Tags, categories, multiple images, feature lists |
Choose the appropriate data type to ensure your data is extracted and validated correctly.
### Data Type Formats
Some data types return structured values:
| Data Type | Format | Example |
| - | - | - |
| **MONEY** | `{"amount": number, "currencyCode": string}` | `$124.50` → `{"amount": 12450, "currencyCode": "USD"}` |
The `amount` field is always in the smallest currency unit (e.g., cents for USD, pence for GBP).
## Special Field Types
Beyond regular data fields, Kadoa supports special field types for advanced use cases:
### Function Fields
Functions add fields whose values Kadoa computes for you instead of extracting them from the page. Each function has an output data type, and functions marked with an **AI** badge use generative AI on every record.
* **Custom categories** — assigns each record to a category from a list you define. Useful for sentiment analysis, content categorization, or priority classification.
* **Unique ID** — a stable identifier for each record, well suited as a key field.
Functions vary by workspace: the editor lists only yours, the API rejects the rest, and Custom categories is always available.
[Learn more about function fields in the SDK →](/docs/sdk/schemas/overview#function-fields)
### Raw Content Fields
To keep raw page content, add a regular field that pulls a source value Kadoa already holds — the page URL, the full HTML, or the full Markdown of each page.
[Learn more about source fields in the SDK →](/docs/sdk/schemas/overview#source-fields)
## Transformations
Some workflows transform data after extraction — categorizing records or computing derived fields, for example. A transformation reads input fields as context and produces output fields, each with a summary of how it is computed when available.
To review a workflow's transformation:
* **In the UI**: the [Pipeline](/docs/ui/workflows/pipeline) panel.
* **Via the API**: the `transformation` object in [Get workflow by ID](/api-reference/workflows/get-workflow-by-id).
## Templates
Schemas can be bundled into [templates](/docs/templates) along with a prompt and notification settings. Templates let you apply the same configuration to multiple workflows and manage it in one place.
Need help creating a custom schema? Contact our support team for assistance.
# Secret Variables
Source: https://docs.kadoa.com/docs/workflows/secrets
Store write-only credentials and make them available only to selected workflows
Secret variables are team-scoped, write-only values for passwords, API keys, and other credentials.
Kadoa keeps only non-sensitive metadata in its database. The value stays in the secret provider
you choose.
## How secrets work
* Owners and Admins can create, rotate, rename, and permanently delete secrets.
* Team members can see a secret's key, provider, status, and timestamps, but never its value.
* A workflow receives only the secrets explicitly linked to it.
* Secret values are not sent to an LLM or returned by the API.
* User scripts receive no Google Secret Manager credential.
Kadoa resolves all linked secrets before starting the script. If any value is missing,
disabled, inaccessible, or temporarily unavailable, the script does not start with an empty,
cached, or partial secret set.
![Secret variables display only their storage status and version]()
## Choose where the value is stored
| Provider | Availability | Where the value is stored |
| - | - | - |
| Kadoa-managed | Available | Kadoa's Google Cloud project |
| Google Secret Manager | Available | Your Google Cloud project |
| AWS Secrets Manager | Coming soon | Your AWS account |
| Azure Key Vault | Coming soon | Your Azure subscription |
| HashiCorp Vault | Coming soon | Your Vault deployment |
### Kadoa-managed
Kadoa creates and versions the value in its Google Secret Manager project. Rotating the secret
creates and verifies a new version before disabling the previous version.
![Creating a Kadoa-managed secret variable]()
### Google Secret Manager
Connect a Google Cloud project, then reference an existing secret. The recommended connection
uses a customer grant: grant the displayed Kadoa broker identity **Secret Accessor** on the exact
secret you want to use. Kadoa resolves that secret's latest enabled version at execution time.
You can revoke the grant in Google Cloud at any time. Future executions then fail before the
script starts.
![Referencing a secret from your Google Secret Manager project]()
## Create a secret
1. Open **Team settings** and select **Variables**.
2. Select **Create variable**.
3. Enter a unique key and choose **Secret**.
4. Choose **Managed by Kadoa** or a connected Google Secret Manager project.
5. Enter the write-only value or exact Google secret resource name.
6. Select **Save**.
The submitted value is never shown again. To replace a Kadoa-managed value, rotate it.
The Kadoa-managed value field is for single-line values. For multiline content such as
an SSH private key, store the value in your Google Secret Manager project and create a
customer-managed Secret Variable that references it.
## Use a secret in a script
Link the secret to the workflow, then read it separately from ordinary variables:
```ts theme={null}
import { loadSecrets } from "@kadoa-backend/script-lib";
const secrets = loadSecrets();
const apiKey = secrets.get("STABLE_VARIABLE_ID");
```
Stable variable IDs keep the reference valid when a secret key is renamed. Key access is also
available for existing scripts.
## Log protection and limits
Kadoa removes exact injected plaintext values from managed script stdout, stderr, and uploaded
execution logs before those outputs are stored. This protection includes values split across
output chunks and multiline values.
This is an accidental-disclosure safeguard, not data-loss prevention. It does not protect
encoded, transformed, partial, or deliberately transmitted values, and it does not scan
extracted business data. Review scripts that use credentials and avoid writing secrets to
outputs.
Deleting a Kadoa-managed secret is immediate and permanent. Deleting a customer-managed
reference never deletes the value from your Google Cloud project.
## Secrets used by a destination
A destination, such as [SFTP](/docs/destinations/sftp), can hold a secret variable as
its login credential. The destination references the variable by its stable ID, so renaming
the key keeps the destination working and rotating the value takes effect on the next
delivery.
While a destination references a secret, deleting that secret fails with a conflict. Remove
the destination, or point it at a different variable, before you delete the secret.
## Learn more
* [Variables →](/docs/workflows/variables)
* [SFTP delivery →](/docs/destinations/sftp)
* [Security overview →](/docs/security/introduction)
# Variables
Source: https://docs.kadoa.com/docs/workflows/variables
Create reusable values for your workflow prompts
Variables allow you to define reusable values that can be referenced in your prompts. This is useful when you need to run workflows with different input parameters. In the dashboard setup prompt, type `$` and select a variable to insert it as a chip.
## How Variables Work
When your workflow runs, Kadoa automatically replaces variable references with the actual values you've defined.
**Example:**
If you create a variable named `keywords` with the value `"ESG reports"`, this prompt:
```
- Search for $keywords
- Filter by "2025"
- Extract: data, title, pdfLink
```
Becomes:
```
- Search for "ESG reports"
- Filter by "2025"
- Extract: data, title, pdfLink
```
## Variable Data Types
| Type | Description | Example |
| - | - | - |
| **Text** | Plain text values | `"ESG reports"` |
| **Secret** | Encrypted sensitive values (passwords, API keys) | `"••••••••"` |
| **JSON Object** | Structured data | `{"key": "value"}` |
| **Array** | Lists of values | `["10001", "10002", "10003"]` |
## Creating Variables
1. Navigate to team settings in your dashboard sidebar
2. Click **Create variable**
3. Enter a unique **Key** (no spaces allowed)
4. Select the **Data Type** (Text, Secret, JSON Object, Array)
5. Enter the **Value**
6. Click **Save**
## Using Variables in Prompts
When writing your prompt:
* Type `$` to see available variables
* Select a variable from the suggestion list
* The variable appears as a chip in the prompt editor and remains linked to the stored variable
* To create a variable without leaving the prompt, choose **Create variable** from the suggestions
### Example: Location-Based Search
```
- Loop through ZIP codes $zipCodes in the location field
- Search for nearby stores within $radius miles
- Extract: store_name, address, phone, hours
```
Variables: `zipCodes: ["10001","10002"]`, `radius: 25`
### Example: Authenticated Portal Access
```
- Navigate to the login page
- Enter $username in the email field
- Enter $password in the password field
- Click the login button
- Navigate to the reports section
- Extract: report_name, date, download_link
```
Variables: `username: "user@example.com"`, `password: (Secret)`
## Variable Scope
* Variables are scoped to your **team workspace**
* All team members can use variables in their workflows
* Only Admins and Owners can create, edit, or delete variables
## Learn More
* [Variables SDK Reference →](/docs/sdk/variables/overview) - Manage variables programmatically
* [Variables API Reference →](/api-reference/variables/create-a-variable)
* [Writing Prompts →](/docs/workflows/prompts#writing-good-prompts)
* [Secret Variables →](/docs/workflows/secrets)
# Workspaces
Source: https://docs.kadoa.com/docs/workspaces
Manage your workflows and collaborate using workspaces.
Every Kadoa user gets a workspace automatically when they sign up. There are two types of workspaces:
* **Personal Workspace** - For individual users
* **Team Workspace** - For organizations and teams
## Personal Workspaces
This is your individual space to build and manage workflows.
### What You Can Do
* **Create Workflows** - Build data extraction workflows
* **Manage Data** - View, export, and deliver extracted data
* **Tag Workflows** - Categorize workflows using custom tags
* **Generate API Keys** - Access your data programmatically
* **Set Up Notifications** - Get alerts when workflows complete or encounter issues
***
## Team Workspaces
Team Workspaces enable organizations to collaborate on data extraction workflows with shared resources and centralized management.
### Key Benefits
* **Shared Resources** - Centralized usage tracking, workflows, and API keys across your organization
* **Flexible Permissions** - Role-based access control with Member, Admin, Viewer, and Compliance Officer roles
* **User Management** - Invite and manage team members from a single dashboard
* **Enterprise Security** - Compliance features, audit logging, and access controls
* **Organized Workflows** - Use tags and variables to categorize your team's workflows
### Getting Started
When you upgrade to an Enterprise plan, Kadoa automatically creates your team workspace:
* You become the initial team owner
* You can immediately invite team members
* Your usage limits are shared across all team members
### What's Included
Your team workspace includes:
* **Shared Workflows** - All team members can view and collaborate on workflows
* **Team Settings** - Manage members, permissions, and team configuration
* **Workspace API Key** - Generate a dedicated API key from **Settings > General**
### Manage Team Members
For step-by-step instructions on adding members, managing invitations, and changing roles, see [Team Management](/docs/ui/team-management).
### Roles and Permissions
Each role has specific permissions designed to balance collaboration with security.
#### Member
**Best for:** Team members who create and manage their own workflows
**Permissions:**
* Create new workflows for the team
* Read and execute all team workflows
* Edit their own workflows
* Download data from team workflows
* Use team tags and variables
**Limitations:**
* Cannot manage team settings or members
* Cannot access compliance or audit features
#### Admin & Owner
**Best for:** Team leads, project managers, and administrators
**Permissions (includes all Member permissions):**
* Manage team members (add, remove, change roles)
* Delete or modify any team workflow
* Access team settings and configuration
* Manage team tags and variables
* View usage and billing information
**Limitations:**
* Limited compliance access (depends on organization setup)
#### Viewer
**Best for:** Team members who need to consume data and monitor workflows without making changes
**Permissions:**
* Read-only access to all team workflows and extracted data
* View workflow execution results and logs
* Access tags and variables (read-only)
* View team activity and audit logs
**Cannot:** Create, edit, delete, or run workflows. Cannot manage team settings, members, or notifications.
**Use cases:**
* Data analysts who consume extracted data
* Stakeholders monitoring workflow performance
* MCP users who only need to query and view data
#### Compliance Officer
**Best for:** Compliance teams and auditors who need read-only access with compliance review powers
**Permissions:**
* Read-only access to all workflows and execution logs
* Access compliance reports and audit trails
* View team activity
* Approve or reject workflows pending compliance review (exclusive)
**Cannot:** Modify workflows, settings, or team configuration
**Use cases:**
* Regulatory compliance monitoring
* Security audits
### Advanced Features
Your team settings page provides additional collaboration features:
**Tags** help organize and categorize workflows:
* Create custom tags for projects, departments, or campaigns
* Filter and search workflows by tags
* Use tags for reporting and analytics
**[AI personalization](/docs/ui/ai-personalization)** gives your agents standing instructions:
* Write workspace-wide conventions once instead of repeating them in every prompt
* Applies to every workflow created in the workspace after you save it
* Any member can edit it
**[Variables](/docs/workflows/prompts#variables)** enable template-based workflows:
* Define reusable input variables for workflows
* Share variables across team workflows
**[Workspace API Keys](/api-reference/introduction)** enable programmatic access:
* Generate your API key from **Settings > General** (Admins and Owners only)
* API keys start with `tk-` and are tied to your workspace
* Actions via API key are attributed to your workspace's service account, not individual members
* Your destinations keep working even when team members leave or change roles
View the service account in your team members list. [Audit logs](https://www.kadoa.com/activity-log) show which actions were automated vs. manual.
***
## Switch Between Workspaces
If you're a member of multiple workspaces (e.g., a Personal Workspace and one or more Team Workspaces), use the workspace switcher in the top navigation:
* Your active workspace determines which workflows and settings you can access
* Personal Workspace workflows remain private to you
* Usage is tracked separately for each workspace