Is your business knowledge ready for an AI assistant?

Understand what retrieval contributes

Retrieval finds material relevant to a question and supplies it to a model as context. OpenAI’s retrieval documentation describes semantic search and file-based indexing capabilities. These can help locate useful passages without requiring an exact keyword match. They do not make an obsolete policy current or establish which of two contradictory instructions is authoritative.

Treat the knowledge collection as a maintained business resource. For every important question category, identify the source that should govern the answer. If staff currently need to ask a particular colleague because the written material is incomplete, record that gap rather than assuming the assistant will infer the correct answer.

Create a source register before uploading

List document families such as product specifications, delivery policies, installation guidance and internal procedures. Assign an owner, a review date and an intended audience to each family. Decide how superseded material will be removed or clearly marked. A folder full of dated PDFs is not a version policy.

Check whether permissions can be enforced throughout the proposed system. Search results should not expose material a user could not otherwise access. Ask the implementation team to demonstrate this with users from different roles. A disclaimer in the interface is not a substitute for an access boundary.

Write for retrieval and for humans

Improve headings, terminology and context where the source material is difficult to interpret. A table labelled only “Standard” may make sense to its author but not to a colleague retrieving it months later. Include product scope, units, applicable region and conditions close to the information they qualify.

Do not rewrite every document merely to satisfy a technical preference. Start with the material needed for the first question set. Splitting content into smaller passages can improve relevance, but important qualifications must remain connected to the answer. This is a tradeoff to test with actual questions rather than a universal chunk-size rule.

A hypothetical readiness exercise

Imagine an interiors retailer building an assistant for its sales team. Product sheets describe moisture resistance, while an older sales handbook uses broader language. A customer question about bathrooms retrieves both. The project team discovers that neither document clearly distinguishes splash exposure from a specific installation condition.

The right response is to ask the product owner to clarify the source, not to tune the model until it sounds decisive. The assistant can temporarily explain that the available documents do not settle the question and direct the salesperson to the owner. That behaviour is more useful than confidently combining incompatible statements.

Test answers and missing answers

Create questions that have clear answers, questions requiring several sources and questions outside the collection. Include synonyms customers use, incomplete product names and requests for information users should not receive. OpenAI’s evaluation guidance supports task-specific testing; here, the task includes retrieving the correct evidence and knowing when the evidence is insufficient.

Review the source passages as well as the final answer. A fluent response with irrelevant citations should fail. So should an answer that cites the right document but ignores its conditions. Keep retrieval failures separate from writing failures because they need different remedies.

• Name the authoritative source for each core question category.
• Remove or label duplicate and superseded material.
• Check role-based access with realistic user accounts.
• Test unsupported questions and contradictory sources.
• Assign an owner for new documents, corrections and retirement.

Choose a sensible first boundary

Begin with one collection whose owner can maintain it, such as approved internal product guidance. Expand only when the update process works. A broad launch across every shared drive may create a larger maintenance problem before the team has learned how to detect errors.

Proceed when the source of truth is identifiable and permission handling is demonstrable. Pause when ownership is unclear or essential answers exist only in informal conversations. Knowledge preparation remains valuable even if you later choose ordinary search instead of a conversational assistant: people still benefit from information they can trust and find.