Crawling
Start a new crawling session
Initiates a crawling session with the specified URL and optional filters for paths.
POST
/
v4
/
crawl
Authorizations
Headers
API key for authentication
Body
application/json
URL to start crawling
List of path patterns to include in the crawl
List of path patterns to exclude from the crawl
Maximum time for crawling in milliseconds
Maximum depth to traverse during the crawl
Maximum number of pages to crawl
Set this flag to true when you want the crawler to stay on the same exact domain, not going to subdomains
country flag code to assign to proxy configuration to navigate behind the proxy