GET /crawl/{id}
Poll a crawl job’s status and get its result download links.
GET
Get crawl status
Poll the status of a crawl job started with
POST /crawl. Once completed,
s3PresignedUrls provides download links for the result artifacts — see
Delivery for the full set of ways to receive crawl results.
Path parameters
id(string, required): Crawl job ID returned byPOST /crawl.
Responses
- 200:
id,status(scraping,completed,failed,cancelled),limit,total,completed,failed,pending,inFlight,createdAt,completedAt,duration,s3PresignedUrls(download links, once available),error(failure message whenstatusisfailed). - 404: Crawl job not found.
- 503: Crawl result store unavailable.
Authorizations
Bearer authentication header of the form Bearer <token>, where <token> is your auth token.
Path Parameters
Crawl job ID returned by POST /crawl.
Response
Crawl status and download links
Current state of a crawl job.
Available options:
scraping, completed, failed, cancelled Total pages attempted so far.
ISO 8601 timestamp the crawl started.
Elapsed seconds since the crawl started.
Presigned download URLs for the crawl's result artifacts, once completed.
Failure message when status is failed.
