POST
/
v4
/
crawl
curl --request POST \
  --url https://api.kadoa.com/v4/crawl \
  --header 'Content-Type: application/json' \
  --header 'x-api-key: <api-key>' \
  --data '{
  "url": "<string>",
  "pathsFilterIn": [
    "<string>"
  ],
  "pathsFilterOut": [
    "<string>"
  ],
  "timeout": "1800000 - default 30 minutes",
  "maxDepth": 10,
  "maxPages": 1000,
  "strictDomain": true,
  "proxyCountry": "DE"
}'
{
  "message": "Session started",
  "sessionId": "<string>",
  "error": "<string>"
}

Authorizations

​
x-api-key
string
header
required

Headers

​
x-api-key
string
required

API key for authentication

Body

application/json
​
url
string
required

URL to start crawling

​
pathsFilterIn
string[]

List of path patterns to include in the crawl

​
pathsFilterOut
string[]

List of path patterns to exclude from the crawl

​
timeout
integer

Maximum time for crawling in milliseconds

Example:

"1800000 - default 30 minutes"

​
maxDepth
integer

Maximum depth to traverse during the crawl

Example:

10

​
maxPages
integer

Maximum number of pages to crawl

Example:

1000

​
strictDomain
boolean

Set this flag to true when you want the crawler to stay on the same exact domain, not going to subdomains

Example:

true

​
proxyCountry
string

country flag code to assign to proxy configuration to navigate behind the proxy

Example:

"DE"

Response

200
application/json
Crawling session started successfully
​
message
string
Example:

"Session started"

​
sessionId
string

Unique ID for the started crawl session

​
error
string | null

Null if no error occurred

IntroductionRetrieve the status of a crawling session