Public web content, your workflow
Web content. On your terms.
Collect public content across websites. Add AI to answer a research question, extract facts or classify companies. Or keep the HTML or Markdown for your own tools.
Public pages only. AI is optional.
API or a managed solution?Find the right fit
Research across many websites, or collect deeply and repeatedly from specific sites. Choose for the work you need done.
Collect selected public pages from different websites. Apply your question with AI, without building a separate scraper for every site or reading every page yourself.
Research 1,000 company websites in batches. Extract their services or classify which match your criteria, even when source pages use different languages.
For large, recurring collections from specific sites, our team builds and maintains the workflow, including site-specific access and anti-bot handling.
Collect ticket listings and prices every day from a target platform, with agreed coverage, freshness, quality checks and scheduled delivery.
From a page to something useful
Keep the content, ask a question, or get a specific answer your next step can use.
northstar.example/about
Recyclable packaging for retailers. Bristol, UK. hello@northstar.example
Collect the page content.
HTML or Markdown for your own tools. No AI processing.
# Northstar Packaging
Recyclable packaging for independent retailers.
Based in Bristol, United Kingdom.
Contact: hello@northstar.exampleOriginal content, ready for your own workflow.
Illustrative examples, not live requests. See the API reference for exact response fields.
Add your company background when a question needs it. Leave it out for general research or extraction. Context is optional, per request.
Know before you run
Pay for collection. Add analysis only when you need it.
$0.00045 / page
Per provider-completed page. Content-only requests do not use AI.
This self-service API does not solve or bypass CAPTCHA challenges. Protected pages may not be accessible.
The API does not access private accounts or login-only content. Only collect data you are authorized to access.
These are API limits, not the scope of our custom engineering. For anti-bot challenges or authorized login workflows, talk to our team.
Explore custom access handlingA completed page may still have unusable content and incur a collection charge. A URL is not a promise of access. Review results and warnings before using the data.
A few things to know
How collection, research and billing work together.
Full API referenceHTML can contain the original source rather than the final rendered DOM. For text research, try Markdown and wait for relevant text or an element instead of relying only on page load. The wait condition has a 30-second cap and can return a warning. Inspect the collected content and warnings before sending it downstream; neither format guarantees every dynamic section was loaded.
The maximum is a research budget, not a guaranteed page count or a complete-site crawl. The starting page is included, and optional web-search sources use the same budget. Relevant links, accessible content, available credit and service limits affect how far the run proceeds. Review the visited sources and limitations before concluding that a missing fact does not exist elsewhere.
No. Ready means the collected starting-page content is available; analysis.status separately reports the AI outcome. Retrieve and save the content, then inspect the analysis error. Completed collection remains charged even if a later model call fails. An unconfirmed model response needs review, not an automatic retry, because the original call may already have incurred cost.
The minimum admits a run; it is not a price quote or a guarantee that every later step can be funded. Each paid step reserves credit before it starts. Web search currently needs up to $2.58 in available credit for its temporary hold. Confirmed usage replaces the hold and unused credit is released. Read the billing guide before choosing a budget for a multi-page run.
New paid steps stop when available credit cannot cover the next reservation. Completed steps remain charged, and successfully collected starting-page content can still be retrieved; analysis may be skipped. Adding funds does not automatically restart that run. Review the partial outcome before deliberately creating a new run, which can incur new collection and analysis charges.
Batch completion does not mean every website was classified. Check each row's status: insufficient_evidence, refused, failed, review, expired and cancelled have no answer label. Do not convert a missing label into No or into your own Maybe option. Only classified rows carry a chosen label. Route other outcomes separately, and treat confidence values as model estimates rather than guarantees.
No. Page content and analysis are retained for 24 hours after collection; history can retain metadata after the content expires. Retrieve individual task results and save what you need before contentExpiresAt. Export classification results within their retention window too. Reading a record or replaying its original key does not extend retention or run the task again.
Keep the UUID v4 Idempotency-Key and exact request before submission. Check the saved task using its ID or original key. If no task is found, replay the same body with the same key. An identical replay returns the existing outcome, including failed or expired results; it does not restart processing. For review or SUBMISSION_UNCONFIRMED, contact support before creating replacement work.
Custom scraping solutions
Large, recurring datasets need more than individual page requests. Our team specialises in site-specific collection, CAPTCHA and anti-bot handling, and ongoing maintenance.
A few successful requests do not prove a site will work at sustained volume. IP blocks, rate limits, session checks and changing protections can interrupt collection. We assess those constraints and engineer for the agreed workload.
Discuss a custom solutionSpecialist work for CAPTCHA and anti-bot challenges, including DataDome and Akamai, plus authorized login and session workflows. We review permitted access and feasibility first; no universal bypass promise.
Scope pagination, record coverage, request pacing and refresh frequency for your target sites. Validate achievable volume before committing to a delivery schedule.
Extract the agreed fields, check missing values, handle duplicates and deliver a consistent dataset, not just raw page content.
Monitor failed runs, recover where feasible and adapt to source-site changes. Your team uses the data; ours maintains the agreed collection process.
Engineering, repeat runs, quality checks and maintenance all matter. At sustained volume, a purpose-built workflow may reduce the total cost of usable data. Custom is not automatically cheaper; we assess your workload before recommending it.
Start with a target-site review
Share a few URLs, the fields you need and how often the data should arrive. We will review the scope and recommend a route before quoting.
Projects are accepted individually, subject to feasibility and available capacity. An enquiry does not reserve a solution or confirm delivery. Pricing and terms are agreed directly with our team.
Your enquiry goes to our team. No scraping job or payment is started.