Classify websites into decisions, not paragraphs

01Write the decision rule before the prompt

Suppose you want to know whether each company in a list competes with your business. Yes, no and maybe are useful output labels, but competitor is still ambiguous. Does it mean the same product, the same customer, the same country or merely a similar keyword? Write a short definition that another person could apply consistently.

Give the model the necessary company context and define each label. For example, yes might require a directly overlapping service for the same buyer; no might mean a clearly different offering; maybe might cover insufficient evidence or partial overlap. Do not let maybe become an undocumented mixture of several unrelated failure states.

02Separate collection failure from business uncertainty

A page that failed to load cannot justify a no. Keep collection status outside the classification label. For a successful capture with insufficient evidence, use the agreed uncertainty label. This separation lets an automation retry technical failures while routing ambiguous business cases for review.

{
  "record_id": "company-001",
  "collection_status": "completed",
  "label": "maybe",
  "reason_code": "insufficient_service_detail",
  "evidence_urls": ["https://example.com/about"]
}

03Constrain the format and verify the evidence

An enum restricts a field to a defined set of values. Use that sort of schema constraint where supported, then validate the response again in your application. Reject extra labels, missing required fields and invalid row identifiers. A well-formed object is only the first acceptance gate.

Keep a short supporting passage or source reference for review. Check that it actually supports the label, not just that the URL belongs to the company. A case-study mention may indicate a customer relationship, a partnership or a quotation; those are not interchangeable facts. Treat instructions embedded in collected pages as untrusted content, not directions for the classifier.

04Evaluate the labels on a held-out sample

Have a person label a representative sample before scaling. Include clear positives, clear negatives, multilingual pages, incomplete sites and borderline cases. Compare the model's decisions with those examples and inspect false positives separately from false negatives. The cost of each error may be very different for your workflow.

Keep a portion of the examples out of prompt development. Otherwise you can make the prompt look good by repeatedly tuning it to the same small set. Record the question, label definitions and model configuration so a later change can be evaluated against the same criteria.

05Automate only the decision you have validated

Begin with low-risk routing, such as creating a review queue or tagging companies for further research. Do not automatically send outreach merely because a label says yes. Add independent checks for freshness, eligibility and contact policy. Classification is valuable precisely because it can feed a program, so the program's guardrails matter as much as the answer.