Errors, limits & billing

Request errors

An unsuccessful API request returns a top-level error string with a non-2xx HTTP status. 400: check your JSON, supported fields, URL and UUID v4 key. 401: the API key is missing, invalid, expired or revoked. 402: starting available balance is below $1. 403: the target is not public, or manual approval is required if restricted access has been enabled.

404: the task or batch ID is invalid, unknown or belongs to another account. 409: the idempotency key belongs to a different request, or a classification batch is already active. 413: the body exceeds 8,192 bytes for an ordinary task or 256 KiB for classification. 422 on ordinary task submission: the target domain could not be resolved; classification records DNS failures per item after admission.

429: honor Retry-After and check the rate, active-task and daily limits. 503: scraping or the requested AI service is unavailable; preserve the original key when recovering the request.

{ "error": "The target must resolve only to public internet addresses." }

Task errors inside a successful response

A GET can return HTTP 200 while task.status is failed or review. Inspect task.error.code and task.error.message, not just the HTTP status.

ACCESS_NOT_APPROVED: pilot access or the target could not be verified. ACCESS_RESTRICTED, LOGIN_REQUIRED or CHALLENGE_DETECTED: the source requires access the integration does not provide. TARGET_HTTP_ERROR or TARGET_UNAVAILABLE: the source could not be retrieved successfully.

EMPTY_CONTENT: no usable content. RESULT_TOO_LARGE: result exceeds the pilot size limit. RESULT_TIMEOUT: the processing window elapsed. PROVIDER_REJECTED, PROVIDER_TASK_FAILED, PROVIDER_RESULT_EXPIRED or PROVIDER_NOT_CONFIGURED: contact support with the task ID.

SUBMISSION_UNCONFIRMED puts the task in review. Do not automatically create another task. Support needs the saved ID to investigate.

Provider execution failures

PROVIDER_ATTEMPTS_EXHAUSTED means the provider returned max_restarts_exceeded: its allowed execution attempts ended without page content. It does not identify whether the cause was a browser connection, page loading, a target restriction or another provider fault. A missing key is not established by this error.

This is not an instruction to keep polling a failed task or automatically create new paid tasks. Preserve the ID and ask support to investigate before deliberately retrying. The same Idempotency-Key returns the existing outcome.

Our API and upstream provider endpoints have different IDs and response formats. Use https://www.scrapingbuddy.com/api/v1 with your ScrapingBuddy customer key, including for result retrieval. Do not put your customer key into an upstream provider URL.

What to include in a support request

Include the ScrapingBuddy task or batch ID, approximate submission time with timezone, API method and path, outer HTTP status, and complete response JSON. Include the submitted URL, format and waitFor when relevant. For classification, include the affected item ID too.

Redact Authorization, API keys, cookies, signed URLs, personal data, private companyContext and confidential instructions or collected content before sharing logs. A task ID by itself cannot explain the failure; error.code and error.message provide the useful starting point. Do not resubmit merely to obtain a new ID.

Current usage limits

All customer API keys share 60 requests per minute per account, including task creation, history and polling across products. Ordinary scraping allows 3 active tasks per account, including review tasks. Classification has its own queue and history.

Ordinary public testing currently has no per-account daily task cap; active-task, request-rate and shared page-budget limits still apply. Classification allows a separate 100 URLs. A shared server page budget also applies. A normal task reserves one page; deep analysis reserves its full maxPages budget, including failed or unused pages. A failed task still counts.

Send one starting URL per ordinary task, at most 2,048 characters, with a JSON body no larger than 8,192 bytes. Classification batch bodies allow 256 KiB. The per-page result size limit is 4 MB. Retrieve ready content within 24 hours of starting-page collection.

Per-page billing and wallet balance

A minimum available wallet balance of $1.00 is required to start a task or classification batch. HTTP 402 means the starting balance is insufficient. Existing idempotency keys can still retrieve their original task below this threshold.

Customer charges are 1.5 times provider costs. Page collection costs $0.00045 per provider-completed page ($0.0003 upstream). A 30-page run therefore costs $0.0135 for collection, plus any AI and web-search usage. Unvisited pages are not charged. A provider-completed page can still have unusable content, such as a login page; collection charges still apply. Confirmed provider execution failures release the page hold.

Each page, analysis, classification or search reserves credit before the paid call. Actual usage replaces the hold when confirmed, with unused credit released. Web search currently needs up to $2.58 of available credit for a conservative temporary hold; this is not its final price. Collected pages are not re-billed for analysis.

INSUFFICIENT_BALANCE stops new work, including queued URLs in the same batch. Successfully collected starting-page content remains available; analysis may be skipped. After adding funds, explicitly create a new run. Already completed steps remain charged, even if later work fails. No automatic resubmission occurs.

BILLING_REVIEW means usage was unconfirmed, such as a lost provider response. Its hold stays pending support review; it is not converted into an estimated charge. Preserve the task ID. billing.customerChargeUsd and billing.heldUsd are decimal USD strings. Each operation rounds up to the nearest $0.000000001. Historical tasks with billing.mode: "pilot" remain free.