Guides

Example 1: Scrape a website into a table

Describe the data you want and where to find it. The agent figures out the page structure — no selectors to write or maintain:

start = requests.post(f"{BASE_URL}/api/scrapes", headers=headers, json={
    "instruction": "Extract store name, address, phone, and hours",
    "target_urls": ["https://example.com/stores"],
    "fields": ["name", "address", "phone", "hours"],
})
start.raise_for_status()
session_id = start.json()["session_id"]
print("session id:", session_id)

Check for results

Poll until the run is done, then download the delivered files (CSV/JSON):

while True:
    result = requests.get(f"{BASE_URL}/api/scrapes/{session_id}", headers=headers).json()
    if result["status"] == "done":
        break
    time.sleep(5)

print("summary:", result["summary"])
for f in result["files"]:  # each: { name, size, ext, url }
    data = requests.get(BASE_URL + f["url"], headers=headers)
    with open(f["name"], "wb") as out:
        out.write(data.content)
    print("saved", f["name"])

Example 2: Extract data from a document

Upload a PDF (or point at one with target_urls) and describe the data points to pull out. Use multipart form-data for the upload — don't set a JSON Content-Type header here:

upload_headers = {"Authorization": f"Bearer {API_KEY}"}

with open("monthly-coffee-report.pdf", "rb") as f:
    start = requests.post(
        f"{BASE_URL}/api/scrapes",
        headers=upload_headers,
        data={"instruction": "Extract coffee export volume by month as a table"},
        files={"file": f},
    )
start.raise_for_status()
doc_session_id = start.json()["session_id"]
print("session id:", doc_session_id)
# Poll GET /api/scrapes/{doc_session_id} exactly as in Example 1.

Example 3: Refine a run with a follow-up

Sessions are conversational. If the first pass isn't quite right, send another instruction to the same session instead of starting over:

followup = requests.post(
    f"{BASE_URL}/api/scrapes/{session_id}/messages",
    headers=headers,
    json={"instruction": "Also capture each store's email, and drop closed locations"},
)
followup.raise_for_status()
# Poll GET /api/scrapes/{session_id} again; new_files holds the latest turn's output.

Example 4: Watch a page for changes

Turn a finished scrape into a recurring monitor. sieve re-runs it on your schedule and alerts you by email, Slack, or webhook when the data changes:

monitor = requests.post(
    f"{BASE_URL}/api/scrapes/{session_id}/monitor",
    headers=headers,
    json={
        "instruction": "Re-check every day at 09:00 and notify me only when something changes",
        "email_recipients": ["analyst@example.com"],
        "webhook_url": "https://example.com/hooks/sieve",  # optional push notifications
    },
)
monitor.raise_for_status()

# List your monitors any time:
monitors = requests.get(f"{BASE_URL}/api/monitors?mine=1", headers=headers).json()
print(monitors)

Receive webhook events

When a monitored page changes, your webhook_url receives a monitor.changed event with the diff inline:

{
  "event": "monitor.changed",
  "monitor_id": "mon_8f3a2c",
  "run_id": "run_20260709-1600",
  "monitor_name": "Competitor pricing page",
  "summary": {"added": 1, "removed": 0, "changed": 2},
  "data": {
    "columns": ["plan", "price"],
    "changed": [{"changes": {"price": ["$25 / seat", "$19 / seat"]}, "row": ["Team plan", "$19 / seat"]}]
  }
}

Failed runs send monitor.failed; if you turn off "only when changed", successful runs with no diff send monitor.no_change. The full payload reference lives in the in-app docs at scrape.usesieve.com/api-docs.