Web change monitoring

POST

scrape

curl --request POST \
  --url https://api.scrapeswitch.com/scrape \
  --header 'Authorization: Bearer <token>'

const options = {method: 'POST', headers: {Authorization: 'Bearer <token>'}};

fetch('https://api.scrapeswitch.com/scrape', options)
  .then(res => res.json())
  .then(res => console.log(res))
  .catch(err => console.error(err));

package main

import (
	"fmt"
	"net/http"
	"io"
)

func main() {

	url := "https://api.scrapeswitch.com/scrape"

	req, _ := http.NewRequest("POST", url, nil)

	req.Header.Add("Authorization", "Bearer <token>")

	res, _ := http.DefaultClient.Do(req)

	defer res.Body.Close()
	body, _ := io.ReadAll(res.Body)

	fmt.Println(string(body))

}

Watch a page for meaningful changes — a pricing page, a terms-of-service update, a competitor’s landing page — without storing and diffing raw HTML by hand.

Use case

Alert when a competitor changes pricing, copy, or product availability
Track terms-of-service, policy, or changelog pages for updates
Detect when a page’s content stops matching what you last indexed

Step 1: Take a clean snapshot

Request clean_html or markdown instead of raw html — boilerplate like ads, session tokens, or rotating banners get stripped by default (only_main_content: true), so the diff only reflects real content changes.

curl -X POST "https://api.scrapeswitch.com/scrape" \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://competitor.example.com/pricing",
    "output": ["markdown"],
    "tag": "monitor.pricing-page"
  }'

import hashlib
import os
import httpx

response = httpx.post(
    "https://api.scrapeswitch.com/scrape",
    headers={"Authorization": f"Bearer {os.environ['API_KEY']}"},
    json={
        "url": "https://competitor.example.com/pricing",
        "output": ["markdown"],
        "tag": "monitor.pricing-page",
    },
    timeout=60.0,
)
response.raise_for_status()
markdown = response.json()["data"]["markdown"]
content_hash = hashlib.sha256(markdown.encode()).hexdigest()

const crypto = require("crypto");

const response = await fetch("https://api.scrapeswitch.com/scrape", {
  method: "POST",
  headers: {
    Authorization: `Bearer ${process.env.API_KEY}`,
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    url: "https://competitor.example.com/pricing",
    output: ["markdown"],
    tag: "monitor.pricing-page",
  }),
});

const { data } = await response.json();
const contentHash = crypto.createHash("sha256").update(data.markdown).digest("hex");

Step 2: Compare against the last hash

Store the hash (and optionally the Markdown itself) alongside the url in your own database. On each run, fetch again and compare:

Python

previous_hash = load_last_hash(url)  # your own storage

if content_hash != previous_hash:
    notify_change(url, markdown)      # your own alerting
    save_hash(url, content_hash)

Step 3: Run it on a schedule

Trigger the scrape from a cron job, a queue worker, or a scheduled function — hourly for fast-moving pages, daily for slower ones. Use a distinct tag per monitored page so you can audit run history and cost per target in Request history.

Request highlights

Setting	Value	Why
`output`	`["markdown"]`	Converts the page to text, filtering out most incidental markup noise
`only_main_content`	`true` (default)	Strips navigation/footers so they don’t trigger false-positive diffs
`tag`	per monitored URL	Lets you trace and cost-attribute each page’s monitoring history

Tips

Diff at the paragraph or section level instead of full-text equality if the page has any dynamic content (timestamps, view counts) you want to ignore.
For JS-heavy pages, set js_render: true and a wait_selector so you snapshot the rendered state, not a loading skeleton — see Scrape.
If you need structured fields (e.g. just the price, not the whole page), add prompt/schema with output: ["json"] instead of diffing raw text — see Output.

Web change monitoring

curl --request POST \
  --url https://api.scrapeswitch.com/scrape \
  --header 'Authorization: Bearer <token>'

const options = {method: 'POST', headers: {Authorization: 'Bearer <token>'}};

fetch('https://api.scrapeswitch.com/scrape', options)
  .then(res => res.json())
  .then(res => console.log(res))
  .catch(err => console.error(err));

package main

import (
	"fmt"
	"net/http"
	"io"
)

func main() {

	url := "https://api.scrapeswitch.com/scrape"

	req, _ := http.NewRequest("POST", url, nil)

	req.Header.Add("Authorization", "Bearer <token>")

	res, _ := http.DefaultClient.Do(req)

	defer res.Body.Close()
	body, _ := io.ReadAll(res.Body)

	fmt.Println(string(body))

}

Get started

Endpoints

Teams

Billing

Recipes

Monitoring

Web change monitoring

Use case

Step 1: Take a clean snapshot

Step 2: Compare against the last hash

Step 3: Run it on a schedule

Request highlights

Tips

​Use case

​Step 1: Take a clean snapshot

​Step 2: Compare against the last hash

​Step 3: Run it on a schedule

​Request highlights

​Tips

​Related

Use case

Step 1: Take a clean snapshot

Step 2: Compare against the last hash

Step 3: Run it on a schedule

Request highlights

Tips

Related