How should an enterprise build its first AI assistant?
Choose one role with frequent questions, reliable source material, and manageable consequences when an answer is wrong. Build a read-only assistant that cites evidence, respects permissions, and escalates when it cannot answer. Connect ERP, CRM, or other operational tools only after that evaluation is stable. A company-wide autonomous agent is not a sensible first release.
Enterprise assistants tend to fail because documents are obsolete, access rights are unclear, scope expands without limit, or incorrect answers and user feedback never become managed improvements. Begin by defining who will use the assistant, where they will use it, which approved sources support the task, and what outcome the task should produce. Model and platform choices follow from that definition.
Choosing the initial scope
| Scope | Typical capability | Delivery difficulty | Main risk | Recommendation |
|---|---|---|---|---|
| Enterprise search or curated FAQ | Finds policies, manuals, and fixed answers | Low | Synonyms may fail and people still maintain the answers | Enough when material is small and questions are stable |
| RAG knowledge assistant | Understands a question, retrieves sources, and answers with citations | Medium | Obsolete documents, retrieval errors, and unsupported generation | Recommended first release for most enterprises |
| Action-taking agent | Looks up orders, creates tickets, sends notices, or changes records | High | Excess privilege, incorrect actions, duplicates, and hostile external input | Enable tools one validated task at a time |
Select a use case worth building
Assess candidate workflows with four questions: How often does the question or task occur? Is there an authoritative source? Can citations, refusal, and review control the harm from an error? Can the time saved be measured? Internal IT guidance, product-manual lookup, and service knowledge are usually safer starting points than automated contract approval or financial decisions.
Write success as a job outcome. For example: “A service engineer can find the valid troubleshooting procedure for the correct model within 30 seconds, with document version and page shown.” “Build an intelligent knowledge base” and “reach 90% accuracy” do not define a usable acceptance test. Explicit exclusions matter too: the assistant does not replace the safety owner, change orders automatically, or disclose project material outside the user's role.
Knowledge, permissions, and answer flow
Before indexing, record each document's owner, version, effective and expiry dates, product line, and permitted roles. Preserve section and table context during parsing. Filter retrieval by identity before content reaches the model; permissions cannot live only in a prompt. Answers should cite specific passages. Missing evidence, conflicting sources, or out-of-scope requests should produce a refusal or escalation.
The interface may live in a web workspace or an enterprise messaging platform, but the user identity should map to the corporate account. Administrators need document-sync status, retrieved sources, feedback, model and instruction versions, cost, and error logs. A correction should pass through the content owner and create a new version rather than overwrite history silently.
Moving from answers to actions
After the read-only assistant passes evaluation, add read tools one at a time: inventory, order status, or device alarms. Review write tools separately. Use field allowlists, least-privilege service identities, idempotency keys, financial or volume limits, confirmation, and reversal paths. Model-produced parameters must pass programmatic validation, and text in external pages or documents must never change authorization.
The test set should include frequent questions, paraphrases, missing and obsolete sources, conflicting material, unauthorized requests, and prompt injection. Measure task correctness, valid citations, citation location, correct refusal, authorization failures, P95 latency, human escalation, and cost per successful task. After launch, measure adoption and time saved by role; demo-question pass rates are not production evidence.
Wavesteam's public BMS battery-management and AI recruiting work illustrates the connection between knowledge, device data, and business workflows. An implementation normally starts with one department and one document domain, then delivers content administration, Q&A, permissions, citations, feedback, and monitoring. Pilot evidence determines whether scope expands.
References
- The original Retrieval-Augmented Generation paper explains the approach; research results do not establish performance on a company's material.
- The NIST Generative AI Profile covers reliability, privacy, information integrity, and human oversight.
- The OWASP GenAI LLM Top 10 2026 addresses prompt injection, sensitive-data disclosure, excessive agency, and unsafe output handling.
The organization must validate conclusions against its approved sources and a fixed evaluation set. External frameworks identify risks but do not replace acceptance testing.