/crawl
endpoint with the desired configuration.
View full API reference →
pathsFilterIn
and pathsFilterOut
to include or exclude specific paths.
Adjust timeout
, maxDepth
, and maxPages
to refine the crawling process./crawl/status/<sessionId>
endpoint.
View full API reference →
/crawl/<sessionId>/pages
endpoint with pagination.
View full API reference →
currentPage
: Positive integer, starting from 0.pageSize
: Positive integer, starting from 1.html
: Full HTML structuremd
: Markdown format