Skip to main content

​
Fetch Data

After running an extraction, retrieve your data using the SDK’s built-in methods.

​
Basic Data Retrieval

const result = await client.extraction.run({
  urls: ['https://sandbox.kadoa.com/ecommerce'],
  name: 'Product Extraction'
});

// Get the extracted data
const response = await result.fetchData({});
console.log(response.data); // Array of extracted items

​
Paginated Data Access

Handle large datasets with pagination: 
// Fetch specific page
const pageResponse = await result.fetchData({
  page: 1,
  limit: 50
});

console.log(`Total items: ${pageResponse.pagination.total}`);
console.log(`Page ${pageResponse.pagination.page} of ${pageResponse.pagination.totalPages}`);

​
Iterate Through All Pages

// Option 1: Iterate page by page
for await (const page of result.fetchDataPages()) {
  console.log('Page data:', page.data);
  console.log('Page number:', page.pagination.page);
}

// Option 2: Get everything at once
const allData = await result.fetchAllData();
console.log('All data:', allData);

​
Real-time Updates

Get instant notifications when data changes using WebSocket connections: 
const client = new KadoaClient({
  apiKey: 'your-api-key',
  enableRealtime: true
});

// Listen to all events
client.realtime?.onEvent((event) => {
  console.log('Event:', event);
  // Handle: EXTRACTION_STARTED, EXTRACTION_COMPLETED, DATA_CHANGED, etc.
});

// Check connection
if (client.isRealtimeConnected()) {
  console.log('Connected to real-time updates');
}

​
Response Format

Data returned by the SDK follows this structure: 
{
  data: [
    {
      // Your extracted fields
      title: "Product Name",
      price: "$99.99",
      inStock: true
    }
  ],
  pagination: {
    page: 1,
    limit: 50,
    total: 150,
    totalPages: 3
  }
}