Skip to main content
ScrapingBuddy
Skip to content
ScrapingBuddyDevelopers
Dashboard
On this page Save the task identity first

Scraping API/Guides

Safe retries

Recover a lost response without accidentally creating another task.

Save the task identity first

Generate a UUID v4 once for each new task. Persist it with the exact URL, format, waitFor and analysis values before sending POST /scraping/tasks.

The Idempotency-Key becomes the task ID. Do not generate a new key inside a retry loop.

Recover an uncertain response

If the connection drops or times out, GET /scraping/tasks/{original-key} first. If it exists, continue with that task. If it returns 404, you can replay the original POST with the same saved key and request.

An identical replay returns the saved task, including a failed, review or expired task. It does not restart processing. Reusing the key with different request data returns 409. Do not fix that conflict by blindly generating a new key.

Respect limits and review states

For HTTP 429, wait at least the Retry-After value, currently 60 seconds. Active-task or rolling daily limits may require a longer wait. Space polling across all your integrations because the account rate limit is shared.

For review or SUBMISSION_UNCONFIRMED, contact support and preserve the task ID. For a confirmed failure, first address the reason, then explicitly decide whether a new task is needed.

Manage API keysAPI referenceContact support
Scraping API v1
Getting startedIntroductionYour first scrapeAuthentication & access
GuidesBulk classificationAI analysis & company contextFormats & page readinessTask lifecycle & retentionSafe retriesErrors, limits & billing
API keys Contact support

Explore the documentation

Scraping APIIntroduction

Public pages, HTML, Markdown, deep research, bulk classification, wallet billing and first integration.

GuidesBulk classification

One question, your own answer labels, and a separate machine-readable result for each website.

GuidesYour first scrape

Go from a public URL to saved page content in four steps.

GuidesAuthentication & access

Connect your server with your customer API key and understand public-page access.

GuidesAI analysis & company context

Extract information, summarize a page or assess relevance with optional OpenAI analysis.

GuidesFormats & page readiness

Choose the content format and when the page is ready to collect.

GuidesTask lifecycle & retention

Know when to poll, when to save content and when to stop.

GuidesSafe retries

Recover a lost response without accidentally creating another task.

GuidesErrors, limits & billing

Understand request failures, task failures and usage-based billing.

POST /scraping/tasksCreate a task

Collect a public page, with optional single-page or deep analysis. During the current testing stage, a valid customer API key is enough to create a public-page task; separate accou

GET /scraping/tasks/{id}Get a task & result

Read task progress and retrieve content when it is ready. Fetch the saved task ID with the same account that created it. Wait at least 6 seconds between progress checks. This reque

GET /scraping/tasksList task history

Find your recent tasks, newest first, without downloading page content. Returns up to 10 tasks per page, ordered by creation time and ID, plus hasMore. Request offset=0, then 10, t

POST /scraping/classificationsCreate a classification batch

Ask one question across up to 100 URLs and constrain answers to your own labels. Send a saved UUID v4 Idempotency-Key and JSON body of at most 256 KiB. The key becomes batch.id. An

GET /scraping/classifications/{id}Get classification results

Retrieve batch progress and all per-URL decisions. Use the batch ID and a key belonging to the same account. Poll at least 10 seconds apart. Returns 202 while queued or running, an

GET /scraping/classificationsList classification batches

Read paginated batch summaries and current classification availability. Returns up to 10 batches, newest first, without items. Use offset=0, then 10, then 20 while hasMore is true.

POST /scraping/classifications/{id}/cancelStop queued classification URLs

Stop batch items that have not started; let in-flight work finish. Send Authorization only; no request body or new Idempotency-Key is required. Repeating cancellation is safe. Only