Monitor relevant coverage
Follow selected topics and organisations through source-linked records, with a refresh schedule defined for your project.
All use casesBBC news
NEWS RESEARCH / SOURCE-AWARE DATA
BBC news
data.
News. Sport.
Source by source.
Connect headlines, sections and source dates in a dataset built around the coverage your research needs.
FROM PUBLISHER OUTPUT TO YOUR RESEARCH
Follow a topic.
Keep its context.
Bring an agreed set of BBC sources into your workflow. Keep the original service, section and observation alongside each record.
Compare news and sport
Work across agreed sections without losing the distinction between BBC News, BBC Sport and their different field structures.
Build an observation history
Track newly discovered items and metadata changes. Keep source updates separate from collection gaps and items leaving a feed.
THREE VIEWS OF THE SAME RESEARCH WORKFLOW
Feed. Section.
What changed.
Start with the fields your question needs. Keep publisher facts, collection metadata and optional analysis in separate fields.
A feed item, with provenance.
Keep each item connected to its publisher feed, identifier and source date. A short description is not the full article.
- url / headline / feed_guid
- The item identity, source link and publisher feed identifier.
- source_service / source_feed
- Keep BBC News, BBC Sport and individual feeds distinct.
- published_at / observed_at
- The item date and collection time have different meanings.
- summary / image_reference / text_scope
- Available feed fields, not an implied full-text or media licence.
Illustrative schema / not a collected BBC result
{
"example_only": true,
"record_type": "feed_item",
"source": "BBC",
"source_service": "BBC News",
"source_feed": null,
"feed_guid": null,
"url": null,
"headline": "City announces a new public research centre",
"summary": "An illustrative short description, not an actual BBC report.",
"published_at": null,
"updated_at": null,
"observed_at": null,
"text_scope": "feed_metadata",
"full_text_included": false,
"image_reference": null
}Fictional records and values. No live data request.
Fictional example. An RSS item date is not a feed build date. Field availability and business-use permissions must be confirmed for the chosen source.
Illustrative delivery schemas, not collected BBC records or a built-in API response. Fields, coverage and permissions are confirmed for the selected sources.
GOOD DATA KEEPS THE DISTINCTIONS
Same publisher.
Different evidence.
A reliable record says where it came from, what the date means and how much content was actually collected.
A feed is not an archive
Retain the feed or section URL. A changing set of visible items cannot establish complete historical coverage.
A label is not a timestamp
Keep relative labels as supplied. Item publication, feed build and observation times remain separate.
A summary is not a full story
A feed description or listing headline does not include the complete article, author details, transcript or media rights.
THE SIMPLEST SUITABLE COLLECTION ROUTE
Feed first.
Enrich with purpose.
Where authorised, structured feeds can reduce page requests and parsing work. Use page collection only where it adds the fields or coverage you need.
- 01 / DEFINE
Choose sources and rights
Agree services, sections, geography, refresh needs and intended use. Confirm the appropriate permissions before production collection.
- 02 / COLLECT
Read the available structure
Use a suitable feed or initial HTML. Match records and keep each image reference with its own item. Enrich approved missing fields separately.
- 03 / MAINTAIN
Watch freshness and coverage
Validate item dates, monitor failures and preserve successful observations. Report a source gap instead of silently returning an empty dataset.
RSS for available item fields
A suitable feed can expose links, short descriptions and item dates without a browser session or AI call. Feed access is not a business-use licence.
BBC News RSS feedHTML for selected sections
Page markup can supply section attribution, visible time labels and image references. News and Sport need source-specific validation, not one assumed parser.
Raw HTML collection referenceBBC's terms specify a licence for business use of metadata and RSS feeds. Collection does not include that licence, login or paywall bypass. Review BBC Global Terms of Use.
YOUR REQUESTS OR A MANAGED DATASET?
Choose who maintains
the workflow.
Use the API for your own page requests. Choose managed delivery when you need source mapping, matching, scheduled collection and monitored output.
Scraping API
Collect accessible public pages across permitted sources, with optional analysis or classification.
- Content-only mode without AI processing
- Your URLs, parsing and result handling
- No CAPTCHA, login or paywall bypass
Managed BBC data
A source-aware news workflow shaped around your field requirements, refresh needs and approved use.
- Feed and page coverage validation
- Matching, provenance and observed changes
- Scheduled JSON, CSV or agreed API delivery
BEFORE YOU BUILD ON THE DATA
Coverage, timestamps
and permissions.
Does an RSS feed include full articles or the complete archive?
No. The tested BBC News feed contained item titles, links, short descriptions, publication dates and thumbnail references. It was a finite selection, not a complete archive or a full-text dataset. Historical coverage, article text, bylines and transcripts need separately validated sources and permission.
Can BBC News and BBC Sport use the same extraction rules?
Not reliably. Their listing pages have different markup and metadata. We validate each agreed source separately, preserve the originating service and section, and avoid treating a sport attribution as an author or a general news topic. Coverage is defined by the selected feeds and pages, not the BBC name alone.
How do you handle relative times and feed timestamps?
Store a visible label such as "2 hours ago" as a label, not an exact publication time. Use an explicit item timestamp where present. Keep item publication time, publisher update time, feed build time and collection time separate; an absent date remains unknown.
How do you prevent duplicate records across feeds and pages?
Use validated publisher identifiers or normalized URLs, retaining the source context for each observation. A repeated placement does not automatically become a new article. We do not merge similar headlines on title alone, and check redirects and alternate URLs before treating them as the same record.
Does an item disappearing mean it was deleted?
No. RSS feeds and section pages expose changing selections. An item can leave that selection while its article remains available. Keep feed membership changes, headline edits, collection failures and confirmed publisher corrections as separate events.
Is a browser or an AI call needed for every request?
Not when an approved feed or initial HTML already contains the required fields. Read that source first and enrich only missing fields. Browser rendering and optional analysis are separate costs. One successful test does not prove every page or collection schedule will work.
What permissions are needed for commercial use?
BBC Global Terms of Use specify a BBC Public Service licence for business use of metadata or RSS feeds, and note that a fee may apply. Technical access does not include that licence or grant republication, media download or AI-processing rights. We review the intended use and access route before confirming a project.
START WITH YOUR NEWS RESEARCH BRIEF
Which sources
matter to you?
Share the topics, BBC sections and fields your workflow needs. We will review access, permissions and a sample scope before proposing delivery.
- News or Sport sections and example URLs
- Fields, date range and refresh schedule
- Delivery format and intended use
Projects are accepted individually, subject to feasibility and available capacity. An enquiry does not reserve a solution or confirm delivery. Pricing and terms are agreed directly with our team.
We review your brief first. This form does not run a scrape or create a paid task.
ScrapingBuddy is not affiliated with the BBC. The radio-newsroom image and dataset examples are illustrative, not BBC premises, footage or collected news records. Collection, analysis and delivery depend on appropriate access and agreed usage rights.