Create a classification batch

Ask one question across up to 100 URLs and constrain answers to your own labels.

01 / Request

Make a request

POST/api/v1/scraping/classifications

Send a saved UUID v4 Idempotency-Key and JSON body of at most 256 KiB. The key becomes batch.id. An identical replay returns the saved batch; a changed request with that key returns 409.

A new run requires at least $1 available balance. Each URL collects Markdown before classification. Deep research and web search are optional and billed separately. No file upload or waitFor parameter is accepted by this endpoint.

Returns 202 while batch.status is queued or running, otherwise 200. Location points to the batch and pollAfterSeconds is 10. A completed batch may contain unsuccessful items.

Required headers

AuthorizationstringRequired: Bearer followed by your ScrapingBuddy customer API key.
Content-TypestringRequired: application/json.
Idempotency-KeyUUID v4Required: Generate once per new batch. Save with the request and reuse on uncertain retries. This becomes its ID.

JSON body

urlsstring[]Required: 1-100 public HTTP(S) URLs, each at most 2,048 characters. The array limit is checked before deduplication. Normalized duplicates are removed in first-seen order. DNS validation happens per item during processing.
questionstringRequired: 1-2,000 characters; the same question is applied separately to every URL.
choicesstring[]Required: 2-20 non-empty answer labels, each 1-80 characters. Trimmed labels must be unique ignoring case. Results preserve the supplied label spelling.
companyContextstringOptional: Optional business context, 1-3,000 characters. Applies to this batch only.
depthdeepOptional: Omit for starting-page-only classification. Set deep for bounded multi-page research per URL.
maxPagesintegerOptional: Requires depth: deep. 2-30 per starting URL, default 5; includes the starting page.
webSearchbooleanOptional: Requires depth: deep even when false. Default false. When true, discover public sources beyond the starting website within the same page budget.

POST /scraping/classifications

curl --fail-with-body -X POST "https://www.scrapingbuddy.com/api/v1/scraping/classifications" \
  -H "Authorization: Bearer $SCRAPINGBUDDY_API_KEY" \
  -H "Idempotency-Key: $SCRAPINGBUDDY_BATCH_KEY" \
  -H "Content-Type: application/json" \
  --data-binary @request.json

POST /scraping/classifications

const apiKey = process.env.SCRAPINGBUDDY_API_KEY;
const batchKey = process.env.SCRAPINGBUDDY_BATCH_KEY;
const targetUrl = process.env.SCRAPINGBUDDY_TARGET_URL;
if (!apiKey || !batchKey || !targetUrl) {
  throw new Error("Set the required environment variables before running.");
}

const response = await fetch(`https://www.scrapingbuddy.com/api/v1/scraping/classifications`, {
  method: "POST",
  headers: {
    Authorization: `Bearer ${apiKey}`,
    "Idempotency-Key": batchKey,
    "Content-Type": "application/json"
  },
  body: JSON.stringify({ urls: [targetUrl], question: "Does this company offer managed web scraping services?", choices: ["Yes","No","Maybe"] })
});
const data = await response.json();
if (!response.ok) {
  throw new Error(`${response.status}: ${data.error}`);
}
console.log(data);

POST /scraping/classifications

import os
import requests

api_key = os.environ["SCRAPINGBUDDY_API_KEY"]
batch_key = os.environ["SCRAPINGBUDDY_BATCH_KEY"]
target_url = os.environ["SCRAPINGBUDDY_TARGET_URL"]
response = requests.post(
    "https://www.scrapingbuddy.com/api/v1/scraping/classifications",
    headers={
        "Authorization": f"Bearer {api_key}",
        "Idempotency-Key": batch_key,
        "Content-Type": "application/json"
    },
    json={"urls": [target_url], "question": "Does this company offer managed web scraping services?", "choices": ["Yes","No","Maybe"]},
    timeout=30,
)
data = response.json()
if not response.ok:
    raise RuntimeError(f"{response.status_code}: {data.get('error')}")
print(data)

Base URL https://www.scrapingbuddy.com/api/v1

Set SCRAPINGBUDDY_API_KEY, SCRAPINGBUDDY_BATCH_KEY and SCRAPINGBUDDY_TARGET_URL in your server environment. JavaScript uses Node.js 20+; Python requires requests. These examples call the live, billed API.

request.json / replace the URL

{
  "urls": [
    "https://example.com/about"
  ],
  "question": "Does this company offer managed web scraping services?",
  "choices": [
    "Yes",
    "No",
    "Maybe"
  ]
}

02 / Response

Understand the response

Success returns HTTP 202 while work is active, otherwise 200 with JSON. Inspect item outcomes even when the batch is completed.

batchobject: The batch object described below. Inspect each item.status before using its answer.
pollAfterSecondsinteger: 10. Wait at least this long before checking progress.

Batch object

idUUID v4: The Idempotency-Key supplied when creating the batch. Item task IDs are different from the batch ID.
statusstring: queued or running while work remains; completed or review when terminal. completed does not guarantee every item was classified.
requestobject: Normalized urls, question, choices and any optional companyContext, depth, maxPages or webSearch. Keep private context out of shared logs.
createdAtISO timestamp: When the batch was created.
totalinteger: Number of unique normalized URLs in the batch.
completedinteger: Number of terminal items, including failed, refused, review, cancelled and expired items. Not a success count.
classifiedinteger: Number of items with a usable classified answer. Expired results are no longer included.
itemsarray: All result items in original URL order, after deduplication. Omitted from batch history summaries.

Result item

idUUID v4: Generated task ID for this URL; not the batch ID. Get /scraping/tasks/{id} for collected content, analysis usage, research details and contentExpiresAt.
positioninteger: One-based position after URL normalization and deduplication.
urlstring: Normalized starting page URL.
statusstring: queued, processing, classified, insufficient_evidence, refused, failed, review, expired or cancelled.
labelstring | null: Exactly one supplied choice when status is classified; null otherwise. Never interpret null as No.
confidencenumber | null: Model confidence in its selected outcome, including an insufficient-evidence outcome. Null for refusal or when no decision is available. Not a guarantee of correctness; check status and label first.
probabilitiesarray: Entries {label, probability} for your choices when a choice response is available; otherwise []. Probabilities are model estimates.
insufficientEvidenceProbabilitynumber | null: Estimated probability for the system insufficient-evidence option. Separate from a user-provided Maybe label.
modelstring | null: Server-managed model used for the decision, when available.
sourceUrlsstring[]: Collected source URLs used by the analysis. Not independently verified citations for the decision. Empty when unavailable or expired.
warningstring | null: Partial collection, source-readiness or input-truncation warnings.
errorobject | null: code and message for a collection or analysis failure. Cancelled entries use BATCH_CANCELLED.
billingobject: customerChargeUsd and heldUsd are decimal USD strings for this item. Holds are not extra charges; completed paid steps remain charged after cancellation or expiry.

Example response / illustrative values

{
  "batch": {
    "id": "f824e030-0e09-4d70-a372-fdfd22e84721",
    "status": "queued",
    "createdAt": "2026-10-09T10:00:00.000Z",
    "request": {
      "urls": [
        "https://example.com/about"
      ],
      "question": "Does this company offer managed web scraping services?",
      "choices": [
        "Yes",
        "No",
        "Maybe"
      ]
    },
    "total": 1,
    "completed": 0,
    "classified": 0,
    "items": [
      {
        "id": "8da18a77-6fe2-4e31-a4de-b77f7a7b62d0",
        "position": 1,
        "url": "https://example.com/about",
        "status": "queued",
        "label": null,
        "confidence": null,
        "probabilities": [],
        "insufficientEvidenceProbability": null,
        "model": null,
        "sourceUrls": [],
        "warning": null,
        "error": null,
        "billing": {
          "customerChargeUsd": "0.000000000",
          "heldUsd": "0.000000000"
        }
      }
    ]
  },
  "pollAfterSeconds": 10
}

URLs, IDs, costs, content and timestamps are illustrative. Use your own public hostname.

Errors & next steps

Request errors return an HTTP error status and a top-level error string. Processing failures appear in batch.items[].error even when fetching the result succeeds.

New tasks and batches require $1 available balance; insufficient starting credit returns 402. All customer keys share 60 requests per minute per account. For 429, honor Retry-After; active-task and daily limits may require a longer wait.

Errors & limits Safe retries Task lifecycle