Skip to main content
ScrapingBuddy
Skip to content
ScrapingBuddyDevelopers
Dashboard
On this page Introduction

Developers/Scraping API

From a URL to usable content

Collect a public page. Get HTML or Markdown. Bring the content into your own workflow.

Submit one public URL, check your task and save the result. These guides walk you through the current Scraping API, from your first request to handling errors.

Make your first request
  1. 01
    Submit

    Choose a URL and content format.

  2. 02
    Check

    Wait, then read the saved task.

  3. 03
    Collect

    Save your HTML or Markdown.

Start building

New to scraping APIs? Start with the first-request guide. Use the reference when you need exact fields and response examples.

Your first scrapeA complete walkthrough from access to saved content.Authentication & accessAPI keys, public targets and required headers.Formats & page readinessHTML, Markdown and optional wait conditions.AI analysis & company contextExtract facts, summarize pages and assess relevance.Bulk classificationClassify up to 100 URLs using your own answer labels.Errors, limits & billingPlan credit usage, handle failures and understand holds.

How it works

  1. Prepare your requestUse a public URL. Choose HTML or Markdown and save a UUID v4 task key.
  2. Create a taskSend your API key, task key and JSON body. Save the returned task ID.
  3. Check progressWait at least 6 seconds between checks. A successful HTTP response does not always mean your content is ready.
  4. Save the contentWhen the task is ready, check any warning and save the result within 24 hours of page collection.
Task lifecycle & retentionErrors, limits & billing

Available today

Collect one public starting URL per task as HTML or Markdown. Add optional AI analysis and company context, or research up to 30 pages with optional web search. Bulk classification accepts up to 100 URLs and returns one structured outcome per URL using your chosen answer labels.

Use your customer API key without separate account or hostname approval during the current testing stage. New tasks are billed; only historical pilot tasks remain free. Ordinary tasks and classification batches have separate history endpoints.

Saved company profiles, login or CAPTCHA bypass, custom browser sessions, screenshots, arbitrary extraction schemas and collection-only URL batches are not supported. Classification is the supported batch workflow. See classification requirements.

Explore the product

API endpoints

Base URL: https://www.scrapingbuddy.com/api/v1. Every request requires your customer API key.

Scraping API endpoints
MethodEndpointPurpose
POST/scraping/tasksCollect a public page, with optional single-page or deep analysis.
GET/scraping/tasks/{id}Read task progress and retrieve content when it is ready.
GET/scraping/tasksFind your recent tasks, newest first, without downloading page content.
POST/scraping/classificationsAsk one question across up to 100 URLs and constrain answers to your own labels.
GET/scraping/classifications/{id}Retrieve batch progress and all per-URL decisions.
GET/scraping/classificationsRead paginated batch summaries and current classification availability.
POST/scraping/classifications/{id}/cancelStop batch items that have not started; let in-flight work finish.
Explore API referenceManage API keys
Scraping API v1
Getting startedIntroductionYour first scrapeAuthentication & access
GuidesBulk classificationAI analysis & company contextFormats & page readinessTask lifecycle & retentionSafe retriesErrors, limits & billing
API keys Contact support

Explore the documentation

Scraping APIIntroduction

Public pages, HTML, Markdown, deep research, bulk classification, wallet billing and first integration.

GuidesBulk classification

One question, your own answer labels, and a separate machine-readable result for each website.

GuidesYour first scrape

Go from a public URL to saved page content in four steps.

GuidesAuthentication & access

Connect your server with your customer API key and understand public-page access.

GuidesAI analysis & company context

Extract information, summarize a page or assess relevance with optional OpenAI analysis.

GuidesFormats & page readiness

Choose the content format and when the page is ready to collect.

GuidesTask lifecycle & retention

Know when to poll, when to save content and when to stop.

GuidesSafe retries

Recover a lost response without accidentally creating another task.

GuidesErrors, limits & billing

Understand request failures, task failures and usage-based billing.

POST /scraping/tasksCreate a task

Collect a public page, with optional single-page or deep analysis. During the current testing stage, a valid customer API key is enough to create a public-page task; separate accou

GET /scraping/tasks/{id}Get a task & result

Read task progress and retrieve content when it is ready. Fetch the saved task ID with the same account that created it. Wait at least 6 seconds between progress checks. This reque

GET /scraping/tasksList task history

Find your recent tasks, newest first, without downloading page content. Returns up to 10 tasks per page, ordered by creation time and ID, plus hasMore. Request offset=0, then 10, t

POST /scraping/classificationsCreate a classification batch

Ask one question across up to 100 URLs and constrain answers to your own labels. Send a saved UUID v4 Idempotency-Key and JSON body of at most 256 KiB. The key becomes batch.id. An

GET /scraping/classifications/{id}Get classification results

Retrieve batch progress and all per-URL decisions. Use the batch ID and a key belonging to the same account. Poll at least 10 seconds apart. Returns 202 while queued or running, an

GET /scraping/classificationsList classification batches

Read paginated batch summaries and current classification availability. Returns up to 10 batches, newest first, without items. Use offset=0, then 10, then 20 while hasMore is true.

POST /scraping/classifications/{id}/cancelStop queued classification URLs

Stop batch items that have not started; let in-flight work finish. Send Authorization only; no request body or new Idempotency-Key is required. Repeating cancellation is safe. Only