POST
/
v4
/
crawl

Authorizations

x-api-key
string
headerrequired

Headers

x-api-key
string
required

API key for authentication

Body

application/json
url
string
required

URL to start crawling

pathsFilterIn
string[]

List of path patterns to include in the crawl

pathsFilterOut
string[]

List of path patterns to exclude from the crawl

timeout
integer

Maximum time for crawling in milliseconds

maxDepth
integer

Maximum depth to traverse during the crawl

maxPages
integer

Maximum number of pages to crawl

strictDomain
boolean

Set this flag to true when you want the crawler to stay on the same exact domain, not going to subdomains

proxyCountry
string

country flag code to assign to proxy configuration to navigate behind the proxy

Response

200 - application/json
message
string
sessionId
string

Unique ID for the started crawl session

error
string | null

Null if no error occurred

IntroductionRetrieve the status of a crawling session