How to Research Product Data Without Publishing Unverified Claims

Define the decision before starting the search

A research task should begin with a question that has a clear business use. Find the weight is too vague. A better request is: find the manufacturer's published operating weight for model X, state the configuration and document date, and provide evidence suitable for technical review. The improved request defines the entity, attribute, authority and acceptance test.

Record where an approved answer will be used. A value for an internal comparison may need different evidence from a safety-related compatibility claim or a specification shown on a product page. Higher-risk destinations deserve stronger source requirements and, often, specialist approval.

Set a stop condition as well. Research is complete when the required evidence is found, when approved sources have been exhausted, or when the result is explicitly recorded as unresolved. Without a stop condition, teams repeat searches and quietly substitute weaker sources to finish the task.

• The exact product, variant or organisation being researched
• The field, relationship or claim that is missing
• The destination and consequence of using the answer
• The minimum acceptable source type and freshness
• The reviewer or team authorised to approve it
• The outcome when evidence is absent or conflicting

Create an approved-source policy

Approved-source discovery is not the same as searching the whole web and keeping the first answer. Maintain a source register that identifies permitted manufacturers, regulators, standards bodies, supplier portals, licensed datasets and internal systems. For each source, record what it is authoritative for, any usage restrictions and who owns the relationship.

A manufacturer manual may be authoritative for a technical specification, while the ERP remains authoritative for the sellable SKU and a commerce platform owns its destination product ID. Authority is field-specific. A source that is excellent for dimensions may not be authorised for price, availability or fitment.

Use tiers to guide research, but do not turn them into blind precedence. Primary manufacturer or regulatory evidence normally deserves preference. An authorised distributor may provide useful context. Search snippets, forums and copied catalogues can help locate a lead, but should not become customer-facing proof unless the business has explicitly approved that use.

The W3C Data on the Web Best Practices recommends providing metadata, licensing and provenance information and using persistent identifiers. Those principles are practical controls for commercial research: people should know what the source is, whether it can be reused and how to identify the captured record later.

Capture evidence at the moment of discovery

A URL alone is fragile. Pages change, documents are replaced and portals may render different content after login. Capture enough context for another person to reconstruct the finding: the source title, publisher, stable URL or file identifier, document version, publication date, page or section, access time and the exact value or passage used.

Where permissions allow, preserve a source snapshot or checksum. Do not overwrite an earlier capture when a document changes. Link the new version to the old one and record whether the proposed claim is unchanged, revised or withdrawn.

Separate the captured text from the structured candidate value. If a manual says maximum operating weight with optional counterweight, retain that wording and extract the number, unit and configuration into separate fields. Normalising the number without its qualifier would create a more confident claim than the source supports.

• Publisher, source type and authority scope
• Document or page title and stable identifier
• Version, publication date and capture time
• Page, row, section or selector locating the evidence
• Quoted or extracted evidence with units and qualifiers
• Permitted use, licence or contractual restriction
• Researcher, extraction method and transformation history

Resolve product identity before accepting a value

Most damaging catalogue errors are identity errors disguised as data errors. Two products can share a marketing name while differing by region, size, voltage, model year or pack quantity. Before a candidate value enters review, establish which product or variant the evidence describes.

Match on governed identifiers where available: manufacturer part number, GTIN, supplier item number, platform product and variant IDs, or an approved composite key. GS1 describes the GTIN as an identifier for trade items. It can be a strong identity signal, but a GTIN must still be captured exactly and associated with the correct packaging level and variant.

Do not merge on title similarity alone. Preserve every source identifier and the match decision. If the evidence could refer to several variants, create an ambiguity case rather than distributing the value across all of them.

Identity resolution should produce one of three outcomes: confirmed match, confirmed separate entity, or review required. That status travels with the evidence so later automation cannot mistake a possible match for an approved one.

Extract candidates without turning them into facts

Structured extraction makes evidence review faster. Convert the source into candidate fields such as attribute name, proposed value, unit, language, applicable market, validity period and confidence or rule result. Keep this candidate layer separate from approved catalogue data.

M.I.A.I Data Discovery is designed for approved-source discovery, evidence capture, structured extraction and review queues. Its purpose is to turn a defined research scope into structured evidence ready for human review, including manufacturer research, product evidence and catalogue-gap investigation.

Automated extraction can identify tables, labels and repeated patterns, but it should not invent a missing qualifier or silently convert incompatible units. Record the original representation, the normalised value and the transformation used. A reviewer should be able to see both.

Validation at this stage should flag impossible values, unsupported units, missing identifiers and stale documents. A failed check does not prove the source is wrong; it means the candidate cannot proceed without investigation.

Measure data quality as fitness for purpose

Data quality is contextual. The UK Government Data Quality Framework defines it in terms of fitness for purpose and recommends considering quality throughout the data lifecycle. That prevents teams from declaring a value good merely because a field is populated.

Assess candidates across the dimensions that matter to the decision: accuracy, completeness, consistency, timeliness, validity and uniqueness. A dimension can pass while another fails. A dimension may be accurate but stale; a compatibility record may be current but incomplete because its serial range is missing.

Write checks as specific review prompts. Does the unit match the destination? Is the source date recent enough for this field? Does the value conflict with an already approved record? Is the required variant identifier present? Does another candidate duplicate this research task?

Quality scores can help prioritise work, but they must not conceal critical failures. A high aggregate score should never override a missing identity or mandatory evidence requirement.

Keep disagreement visible

Different sources often disagree for legitimate reasons. A manufacturer may revise a specification, a distributor may use a shipping weight rather than an operating weight, or regional versions may have different components. Store each candidate with its own evidence before selecting a preferred result.

Apply precedence rules only within their documented scope. A current manufacturer bulletin may outrank an old reseller page for a technical value, but it does not automatically own the business's internal product status. Recency alone is not authority.

Send unresolved conflicts to a queue that shows the product identity, candidate values, units, qualifiers, source dates and downstream destinations side by side. The reviewer should approve one value, reject candidates, request more research or record that no safe answer exists.

Absence of evidence is not evidence of absence. If no approved source lists a product as compatible, the result is unknown unless an authoritative source explicitly says it is incompatible. This distinction prevents incomplete research from becoming a negative claim.

A concrete example: investigating an excavator part

A distributor wants to fill a missing fitment field for a replacement idler. An employee finds a reseller page that says the part fits three excavator models. The wording looks plausible, but the page does not show a manufacturer part number, model years or a source document.

The research task records the reseller page as a lead, not as proof. The researcher searches the approved manufacturer library using the part number already stored in the ERP. A current bulletin identifies the same part number and confirms two models, with a serial-number break for one of them. A supplier spreadsheet lists the third model but has no date.

The workflow links all three source records to the internal product and preserves their original identifiers. It extracts two manufacturer-supported compatibility candidates with the serial qualifier. The third remains a conflicting candidate because its source identity and freshness are insufficient.

A product specialist reviews the bulletin, accepts the two qualified relationships and rejects publication of the third. The decision records who approved it, when, and why. The product page can now show two supported applications without presenting the unverified model as fact.

If a later manufacturer bulletin confirms the third model, the research team creates a new evidence version and a fresh review decision. It does not rewrite the history of the earlier rejection.

Design a review queue that supports real decisions

A queue should be more than a list of extracted values. Group work by product and business question so the reviewer can compare evidence, see existing approved data and understand the consequence of the decision. Show conflicts and failed checks first.

Every action needs a defined outcome: approve, reject, request more evidence, merge with an existing case, or mark unresolved. Require a reason for consequential decisions and retain the prior state. Approval should write a new governed fact through the normal change process rather than editing the research record in place.

Route specialist claims to specialist reviewers. A catalogue manager may approve a marketing attribute, while an engineer or manufacturer representative may be required for fitment or safety-relevant data. Role-based queues prevent speed from becoming accidental authority.

Use service levels based on impact. A missing colour can wait; a disputed compatibility claim affecting live customer selection may need an immediate publication block.

Publish approved facts through controlled change

Discovery and publication should be separate permissions and separate events. Once a candidate is approved, create a change record containing the destination entity and field, previous value, new value, evidence reference, reviewer, effective date and rollback path.

Preserve destination identifiers. For a commerce catalogue, the approved value must be applied to the intended product and variant rather than whichever record has a similar title. Validate that the destination ID still exists and that its identity matches the reviewed evidence before writing.

Preview the change and its affected channels. A single approved attribute may flow to a storefront, feed, search index and support application. Each destination can have format or policy requirements. A claim that is suitable internally may still need customer-safe wording.

After publication, verify the resulting value, retain the response or audit reference and monitor for source changes. If evidence is withdrawn or superseded, locate every derived use and decide whether it should be revised, hidden or returned to review.

Measure the research system, not just output volume

Completed searches and extracted fields do not show whether the process is trustworthy. Measure time to usable evidence, review acceptance rate, rejection reasons, unresolved conflicts, stale-source cases and duplicate research avoided.

Track how often identity checks change the result. If many candidates are rejected because they belong to another variant, improve search constraints and source matching before increasing volume. If reviewers repeatedly request the same missing context, add it to the evidence schema.

Measure catalogue-gap closure only after approved publication. Keep discovered, extracted, reviewed and published counts separate. That funnel reveals whether the bottleneck is source access, extraction quality, review capacity or downstream change control.

Audit a sample of approved claims back to their evidence. A healthy process lets a reviewer reconstruct the source, identity match, transformations and decision without relying on the memory of the original researcher.

Evidence-first data discovery checklist

• State the exact product question, destination and risk.
• Search only sources approved for the field being researched.
• Record licence, authority scope and source ownership.
• Capture the precise passage, version, location and date.
• Preserve source snapshots or checksums where permitted.
• Resolve the product and variant identity before using the value.
• Keep raw evidence, normalised candidates and approved facts separate.
• Flag missing qualifiers, invalid units, stale evidence and conflicts.
• Send every consequential candidate through the right review queue.
• Treat unknown as unknown rather than converting it into no.
• Publish through controlled changes using stable destination IDs.
• Monitor source revisions and trace affected downstream claims.