Skip to main content

​
Introduction

Data validation lets you define rules that detect anomalies or unexpected results after a workflow run.

​
SDK Methods

​
Generate Validation Rules

Auto-generate rules from your schema or use natural language to create targeted rules. 
// Analyzes your schema and recent data to suggest validation rules
// Rules are created in 'preview' status for review before enabling
// Note: Requires a workflow with completed extraction data
await client.validation.rules.generateRules({
  workflowId,
});

// Generate rule with natural language
// Uses columns from the seeded workflow schema
await client.validation.rules.generateRule({
  workflowId,
  selectedColumns: columns.slice(0, 2),
  userPrompt: "Not null values",
});

​
Manage Validation Rules

List, approve, and update rule status. Rules start in preview status and must be approved before they run automatically. 
// List rules
const rules = await client.validation.rules.listRules({
  workflowId,
  status: "preview",
});

console.log("Rules:", rules);

​
Run Validation

// Schedule validation for a workflow job
const response = await client.validation.schedule(workflowId, jobId);

await client.validation.waitUntilCompleted(response.validationId);

const validation = await client.validation.getLatest(workflowId);

const anomalies = await client.validation.getAnomalies(
  response.validationId,
);
console.log("Validation:", validation);
console.log("Anomalies:", anomalies);

​
API Endpoints

​
List Validation Rules

Retrieve all validation rules for a workflow. View full API reference → 
GET /v5/workflows/{workflowId}/validation/rules

​
Generate Validation Rules

Auto-generate validation rules based on your schema and sample data. View full API reference → 
POST /v5/workflows/{workflowId}/validation/rules/generate

​
Create Targeted Validation Rules

Generate rules for specific columns with custom hints. View full API reference → 
POST /v5/workflows/{workflowId}/validation/rules/generate-targeted
{
  "columns": ["price", "email"],
  "hints": "Check that prices are positive and emails are valid"
}

​
Configure Validation

Update validation configuration for a workflow. View full API reference → 
PUT /v5/workflows/{workflowId}/validation/config
{
  "enabled": true,
  "keyFields": ["product_id"]
}

​
Toggle Validation

Enable or disable validation for a workflow. View full API reference → 
PATCH /v5/workflows/{workflowId}/validation/toggle
{
  "enabled": true
}

​
Get Latest Validation Results

Retrieve the most recent validation results. View full API reference → 
GET /v5/workflows/{workflowId}/validation/latest

​
Rule Structure

Validation rules are expressed as SQL WHERE clauses that identify problematic rows: 
-- Check email formats
WHERE email NOT REGEXP '^[A-Za-z0-9+_.-]+@[A-Za-z0-9.-]+\\.[A-Za-z]{2,}$'

-- Ensure positive prices
WHERE price <= 0 OR price IS NULL

-- Validate URLs contain domain
WHERE url NOT LIKE '%example.com%'

​
Key Fields

Define key fields to track rows across runs for richer insights: 
PUT /v5/workflows/{workflowId}/validation/config
{
  "keyFields": ["product_id", "sku"]
}
Requirements:
  • Values should be present for most rows
  • Values should be unique per row (no duplicates)
  • Prefer stable identifiers (IDs, URLs, SKUs)

​
Validation Results

The validation results endpoint returns: 
{
  "workflowId": "workflow-123",
  "runId": "run-456",
  "issues": [
    {
      "ruleId": "rule-789",
      "ruleName": "Valid email format",
      "status": "NEW",
      "affectedRows": 5,
      "rows": [...]
    }
  ],
  "summary": {
    "totalIssues": 12,
    "newIssues": 5,
    "resolvedIssues": 2
  }
}

​
Notifications

Enable validation notifications to get alerted when issues are detected: 
POST /v5/notifications/settings
{
  "workflowId": "workflow-123",
  "eventType": "validation_issues_detected",
  "enabled": true,
  "channelIds": ["channel-456"]
}
Want to configure validation through the UI?See the UI Data Validation guide to set up validation visually in the dashboard.