Build a dataset

​

1. Research sources

1. Select Agent in the bottom-right corner of any page, then select Workspace.
2. Describe the data, for example “Find career pages of European fintech companies that list open roles.”

​

2. Create the template

• From scratch: select Templates in the sidebar, then Create template. Turn on the parts the template should manage and fill them in. See Create a template.
• From one source: build a workflow for one career page with Add workflow, and fix the sample with the agent until it’s right. Then open the workflow menu (…), select Save as template, choose the parts every source should share, and select Create template. Use this when you want to see real data first.

• The setup prompt says what a record is, such as “Each row is one job posting.” It has no URLs.
• Each field has a short description and the narrowest type, such as DATE for the posted date.
• The key field is stable, such as the posting URL.
• Data quality rules are ones every source can pass.

​

3. Test on a few sources

​

4. Scale to all sources

const careerPages = [
  "https://acme.com/careers",
  "https://globex.com/jobs",
  // one URL per company
];
const failed: string[] = [];

for (const url of careerPages) {
  try {
    await client.workflow.create({ urls: [url], templateId: "TEMPLATE_ID" });
  } catch {
    failed.push(url);
  }
  // Space out the requests
  await new Promise((resolve) => setTimeout(resolve, 2000));
}

console.log("Retry these:", failed);

> Create a workflow from the Job Postings template for each of these career pages: acme.com/careers, globex.com/jobs, initech.com/open-roles

​

5. Deliver to your tools

​

Keep the dataset healthy

• Data quality rules in the template guard every linked workflow at once. See Data quality.
• Check each new source’s first full run against the live site.
• Subscribe a channel to Workflow health to hear about failed runs. Set it as a team default in Settings > Notifications, or add it to the template. See Notifications.

​

Let Kadoa build it