Scrape

Overview & Purpose

POST /scrape is pline’s synchronous, single-URL endpoint: send one URL, get content back in the same HTTP response. No job to create, no polling — the response is the result. Use it when your workflow needs an answer right now for one URL — a page load in a user-facing app, a single product lookup, an interactive script, or any place where waiting on a background job isn’t an option. Prerequisites: a valid API key on every request (see Getting started). No special scope or permission beyond a standard key is required.

Note: If you’re about to reach for /scrape in a loop over many known URLs, or to discover a site’s structure first, stop — see Batch scrape and Map instead. /scrape is built for one URL at a time.

Best practices

pline is adaptive by default. Every request starts with the cheapest fetch method and opens up from there — escalating to a browser, and to different proxies, only as far as needed to get the data back. Only set js_render or proxy_strategy explicitly once you already know a target site needs it — guessing wrong just adds latency and cost.
Reuse session_id across requests that need to share cookies, instead of re-authenticating every call.
Prefer wait_for_selector-style actions over fixed delays when you can — waiting on a specific element is more reliable than guessing a wait_ms value.
Concurrency: requests sharing the same session_id are serialized; a second in-flight request on that session receives a 429.
Set a tag on requests you’ll want to find again later — it lets you filter and trace requests in Request history for debugging, without having to remember URLs or timestamps. It isn’t echoed back in the response body itself.

Actions (actions) simulate interactions in a real browser before content is captured — implies js_render: true. Use them whenever the data you need doesn’t exist until something happens on the page:

Action	Use it for
`click`, `click_and_hold`	Cookie banners, menus, press-and-hold sliders
`type`, `press`, `select_option`	Filling and submitting forms
`scroll`, `scroll_into_view`	Triggering lazy-loaded content
`wait`, `wait_for_selector`	Letting content settle — prefer `wait_for_selector` over a fixed `wait` when you can
`drag_to`, `hover`, `reload`	Drag interactions, hover menus, restarting after a redirect

For the complete, authoritative list of request/response fields, see the API reference — this page intentionally doesn’t repeat it.

Practical Implementation Example

Scenario: fetch a product page, wait for its JS-rendered price to load, and extract structured fields — a common “one URL, structured answer” workflow.

curl -X POST "https://api.scrapeswitch.com/scrape" \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://shop.example.com/product/42",
    "js_render": true,
    "wait_selector": "[data-testid=\"price\"]",
    "output": ["json"],
    "prompt": "Extract the product name, price, and availability.",
    "schema": { "name": "string", "price": "number", "in_stock": "boolean" }
  }'

import os
import httpx

response = httpx.post(
    "https://api.scrapeswitch.com/scrape",
    headers={"Authorization": f"Bearer {os.environ['API_KEY']}"},
    json={
        "url": "https://shop.example.com/product/42",
        "js_render": True,
        "wait_selector": "[data-testid='price']",
        "output": ["json"],
        "prompt": "Extract the product name, price, and availability.",
        "schema": {"name": "string", "price": "number", "in_stock": "boolean"},
    },
    timeout=60.0,
)
response.raise_for_status()
product = response.json()["data"]["json"]
print(product["name"], product["price"], product["in_stock"])

const response = await fetch("https://api.scrapeswitch.com/scrape", {
  method: "POST",
  headers: {
    Authorization: `Bearer ${process.env.API_KEY}`,
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    url: "https://shop.example.com/product/42",
    js_render: true,
    wait_selector: "[data-testid='price']",
    output: ["json"],
    prompt: "Extract the product name, price, and availability.",
    schema: { name: "string", price: "number", in_stock: "boolean" },
  }),
});

const { data } = await response.json();
console.log(data.json.name, data.json.price, data.json.in_stock);

Error codes

Code	When you receive it
`400`	The request itself is malformed — an invalid URL, an unsupported method/output combination, or a session already bound to a different domain.
`422`	Validation failed — e.g. requesting `output: ["json"]` without `prompt` or `schema`, or another field outside its allowed range.
`429`	The `session_id` on this request is already in use by another in-flight request. Retry after a short delay, or avoid concurrent requests on the same session.
`500`	An unexpected internal error. Safe to retry.
`502`	Every strategy attempt failed — the target blocked the request, timed out, or was unreachable.
`503`	The proxy/unblocker provider this request needs isn’t configured on this orchestrator instance.

Note: Always inspect status in the response before assuming failure from a non-200 body — see Request history to trace a specific request end-to-end with request_id.

Get started

Endpoints

Teams

Billing

Recipes

Monitoring

Overview & Purpose

Best practices

Practical Implementation Example

Error codes

Reference Links

​Overview & Purpose

​Best practices

​Practical Implementation Example

​Error codes

​Reference Links

Overview & Purpose

Best practices

Practical Implementation Example

Error codes

Reference Links