HTML, Markdown or JSON: choose the output your next step needs

01Three different jobs often get called scraping

Capturing a page means obtaining its content. Rendering means running the page's JavaScript. Extraction means selecting and shaping particular information. An HTML response does not, by itself, tell you whether a browser rendered it. A JSON response does not tell you whether its values were observed, inferred or simply missing.

Start with your downstream task. A developer debugging selectors needs the document structure. A research assistant needs readable passages. A pricing database needs typed fields, stable identifiers and observation context. Choosing the same output for all three makes one of those jobs unnecessarily difficult.

02Match the representation to the consumer

Keep an original capture where your retention policy permits it, even when you deliver a smaller representation. That gives you something to inspect when a transformation drops a table, joins unrelated paragraphs or misreads a price.

03Define missing values before extraction

A missing price must not become zero. An unavailable date must not become today's date. Decide whether null means not published, not found or not collected, and preserve a separate reason when the distinction affects a decision. This matters particularly when AI produces the structured result: a plausible guess is not a substitute for a field on the source page.

For nested data, preserve its relationships. A product can have multiple offers, a menu item multiple sizes, and a property several area measurements. Flattening these into one unnamed number discards the context that made the value meaningful.

{
  "source_url": "https://example.com/product/42",
  "observed_at": "2026-10-11T09:00:00Z",
  "price": null,
  "currency": "USD",
  "price_status": "not_published"
}

04Test the transformation, not just the download

Choose a small fixture set containing a paragraph, a table, a list, an accented name and a missing field. Compare the transformed output with the visible source. For JSON, validate required keys, types and allowed values, then check the meaning of the values separately. A schema can reject a string where a number belongs; it cannot prove that the number is the correct offer price.

Keep collection errors outside the business record. A blocked request should produce a collection failure, not a product whose fields are all blank. Otherwise a downstream system can confuse access failure with a genuine change in inventory.

05Keep the API contract as the source of truth

Use the live documentation to confirm which formats and rendering options a service currently supports. Do not infer capabilities from a file extension or a mode name. When you build an integration, store the request configuration with the result so that two apparently identical exports can be compared fairly later.