How to Let Teams Ask Business Questions Without Losing Source Control

Start with the questions and decisions people actually face

Do not begin by connecting every document and database. Begin with a short list of questions that cost teams time or cause avoidable mistakes. Examples include why a product cannot be ordered, which supplier record supports a specification, whether a catalogue value has been approved, and which system owns the current stock or price field.

For each question, record who may ask it, which sources may answer it, what a useful answer must contain and what the system must never decide. This creates a testable service boundary. It also exposes questions that sound similar but need different evidence—for example current storefront availability versus expected replenishment.

Ask M.I.A.I is designed for natural-language questions across approved company knowledge and connected systems, with source and confidence context plus human feedback and review. Its approved purpose is to make permitted business information easier to find and use, not to replace the systems that own that information.

Define a permitted source boundary

Create a source register before indexing content. Each entry should identify the system, data owner, permitted collections or fields, supported use cases, refresh method, retention rule and people allowed to query it. A connection being technically possible does not make every record appropriate for every answer.

Keep operational facts close to their authoritative systems. A commerce platform may own product publication and variant availability; an ERP may own inventory commitments, purchasing or fulfilment records; a governed policy repository may own an approved procedure. When two systems legitimately show different views, preserve both views and explain the difference instead of silently choosing one.

Exclude personal, contractual, security-sensitive and draft information unless the use case, access control and retention arrangements explicitly permit it. Review the boundary whenever a source, field or user group changes.

• Source owner and business purpose
• Permitted entities, collections and fields
• User groups and geographic or organisational scope
• Refresh frequency and last successful capture
• Retention, deletion and incident contacts
• Questions the source may and may not answer

Enforce access at retrieval time

The user's permissions must constrain retrieval before evidence reaches the answer model. Instructions such as do not reveal confidential data are not a substitute for access control. Resolve the signed-in identity, roles, tenant and relevant row- or field-level restrictions, then retrieve only within that authorised scope.

Carry the security context through every connector and cache. Search indexes, vector stores, extracted text and saved conversations can all become accidental side doors if they do not inherit the source permissions. A person who cannot open a supplier agreement in the source system should not receive its clauses through a generated answer.

OWASP lists prompt injection, sensitive information disclosure, improper output handling, excessive agency and vector or embedding weaknesses among its 2025 risks for LLM and generative-AI applications. Treat retrieved content as untrusted input, even when it came from an internal repository, and test attempts to override instructions or extract hidden material.

Map authority at field level, not just by system

A system is rarely authoritative for every field it contains. Define ownership for the facts needed by each question: product identity, storefront publication, price, sellable quantity, supplier specification, purchasing status and customer-facing copy may all have different owners.

Use stable identifiers to join records. Preserve the destination product and variant IDs, ERP item ID, supplier reference and any governed cross-reference rather than matching on title alone. Titles change, duplicate SKUs exist and supplier codes can be reused; a convenient text match must not quietly become proof of identity.

When authority conflicts, return the conflict with timestamps and owners. Do not blend two values into a third value that exists nowhere. The next useful step may be data correction rather than a more confident answer.

Retrieve evidence before composing the answer

Build the response from a compact evidence set selected for the question. Each evidence item should contain the source record identifier, relevant field or excerpt, capture time, source version where available, and the rule that made it eligible. Keep enough context to avoid turning a qualified statement into an absolute one.

A response should be reproducible: another authorised reviewer should be able to open the cited record and understand how it supports the claim. W3C PROV-O provides a general vocabulary for describing entities, activities, agents and derivation. A business implementation can use simpler labels, but it should preserve the same practical chain from source record through retrieval to answer.

If the evidence set is empty, stale, contradictory or below the approved threshold for that question, abstain or narrow the answer. Searching harder is not the same as finding better evidence.

Separate facts, calculations, summaries and suggestions

Label what the answer is doing. A retrieved fact repeats a governed field. A calculation combines values according to a named rule. A summary compresses source material. A suggestion proposes a possible next step. These have different review needs and should not be presented with the same certainty.

Show important calculations, including units and inputs. If an availability answer subtracts committed quantity from quantity on hand, identify those fields, their timestamps and the approved formula. If the systems do not define that calculation, do not invent it.

Use careful language for summaries: the records indicate is different from the business has decided. Suggestions should remain clearly optional and must not masquerade as policy, legal advice or an authorised operational instruction.

Show sources, freshness, confidence and limitations

Place source references beside the claims they support, not in a generic list that leaves the user to guess. Display recognisable system and record names, capture times and direct links where the user's permissions allow. For combined answers, show which statement came from which source.

Confidence should describe evidence quality rather than how certain the prose sounds. Define observable statuses such as confirmed by one current authoritative record, corroborated by two approved sources, conflicting, stale or insufficient. Explain the status in plain language.

NIST identifies confabulation, privacy, information integrity, information security, component integration and human overreliance among the risks addressed by its Generative AI Profile. It also emphasises governance, content provenance, pre-deployment testing and incident disclosure. These are practical reasons to show limits and retain a review route instead of optimising only for smooth conversation.

Treat retrieved instructions and output as untrusted

A document can contain text that looks like an instruction to the assistant. A product description, uploaded file or web page might tell the system to ignore its rules, reveal secrets or call a tool. Keep a strict separation between system policy, the user's permitted request and retrieved evidence. Retrieved text can inform an answer; it must not rewrite the workflow.

Validate output before it is rendered, exported or passed to another system. Encode content for its destination, restrict links and file types, and do not let generated text become executable queries or commands. Logging should avoid copying sensitive evidence into places with broader access.

Keep question answering read-only by default. If a future workflow can change a product, place an order or send a message, make that a separate governed action with validation, a preview, explicit confirmation, least-privilege credentials and an auditable result. An answer alone should never imply that an action occurred.

A concrete example: why customers cannot order one product

Suppose a support colleague asks, Why can customers not order product 1042? The workflow first resolves product 1042 to the exact internal item and Shopify product and variant IDs. It retrieves only the permitted Shopify fields needed to show publication, sales-channel and variant-availability state, together with their capture time.

It then retrieves the permitted NetSuite item and inventory records linked by the governed item mapping. Those records might show quantity committed or an operational status, but the answer should not translate that into out of stock unless the business has defined that rule and the required fields are present.

A useful answer separates observations: the Shopify variant was unavailable at a stated time; NetSuite showed specified inventory fields at another stated time; the mapping between the two records was the approved identifier pair. It then explains whether the evidence confirms a cause, exposes a conflict or remains insufficient, and links the colleague to the records they are allowed to inspect.

The workflow does not change inventory, publish the product or promise a cause that the evidence cannot prove. It can suggest the next approved check—such as reviewing the variant policy or item mapping—while leaving the correction with the responsible person.

Design human review around consequence

Not every answer needs the same review. A low-risk pointer to an approved procedure may be delivered immediately. A supplier claim, pricing exception, safety statement, customer commitment or proposed data correction may require a named reviewer before it is acted upon.

Give reviewers the question, answer, evidence, permissions context, confidence status and model or workflow version. Let them approve, correct, reject or mark the issue as missing data. Store feedback as a governed evaluation record; do not automatically turn every comment into source truth.

Make escalation useful. The reviewer should see exactly what is missing or conflicting rather than receiving a vague low-confidence warning.

Log, test and monitor the complete answer path

Record enough to investigate an answer without storing unnecessary sensitive content: user or role reference, question category, sources queried, record identifiers, access decision, evidence timestamps, workflow version, response status and feedback outcome. Apply retention and access controls to the logs themselves.

Create evaluation cases from real approved questions. Include correct answers, insufficient-evidence cases, permission boundaries, conflicting systems, stale records, ambiguous identifiers, malicious retrieved instructions and attempts to request excluded data. Test both the answer and the absence of information the user was not entitled to receive.

Monitor connector failures, empty retrievals, permission denials, stale indexes, citation failures, conflict rates, abstentions and reviewer corrections. A drop in response time is not a success if evidence quality or access control has weakened.

Measure usefulness, not answer volume

Useful measures include time to a verified answer, proportion of answers with current authoritative evidence, successful source opening, unresolved conflict rate, reviewer correction rate, abstention quality and reduction in repeated searches. Track these by question type because a policy lookup and an inventory diagnosis have different expectations.

Do not reward the system simply for answering more questions. A correct abstention can be more valuable than a plausible response. Monitor whether teams can act with less rechecking while still finding and challenging the underlying evidence.

Ask M.I.A.I can support this workflow by giving teams a plain-English route into approved knowledge and connected systems while keeping source context and human review close to important answers. The governance around sources, permissions and decisions remains a business responsibility.

A practical launch checklist

Launch one bounded question set with named owners before expanding. Review the first real answers closely, correct identity mappings and permissions, and document where the workflow must abstain.

Expansion should be earned by evidence: add another source or user group only when its authority, permissions, refresh, tests and incident route are ready. This prevents a useful pilot from becoming an ungoverned index of everything the organisation happens to store.

• Name the questions, users, decisions and prohibited outcomes
• Register approved sources, fields, owners and refresh expectations
• Enforce identity and permissions before retrieval and in every cache
• Join records with stable governed identifiers
• Preserve claim-level evidence, timestamps and derivation
• Label facts, calculations, summaries and suggestions
• Show conflicts, stale data and insufficient evidence honestly
• Test prompt injection, disclosure, ambiguity and connector failure
• Keep actions separate, confirmed and auditable
• Measure verified usefulness and reviewer corrections