AI projects rarely fail on the model. They fail on the data.
Fragmented, ownerless and legally unclear data turns every AI project into a slower version of the old problem. Five decisions belong with management before anyone chooses a model, plus five acceptance criteria if you plan an internal knowledge assistant.
- 63 %
of organisations do not have, or are not sure they have, the right data management practices for AI.
- 60 %
of AI projects without AI-ready data will be abandoned through 2026, Gartner forecast in February 2025.
Source: Gartner, press release of 26 February 2025 (survey of data management leaders, 2024).
What management has to decide before choosing a model
Start with a workflow, not a pilot
Name the process, the measure and the owner: document review, case handling, catalogue search. A chatbot experiment without a margin, speed or risk target proves nothing.
Relevant ifyou are testing models without knowing where hours are saved or revenue protected.
Give master data an owner
Duplicate product, customer and process data is merged into one binding source before anything is indexed. A model cannot repair contradictions; it repeats them with confidence.
Relevant ifdepartments or subsidiaries keep different versions of the same facts.
Make access rights a condition, not a feature
An AI assistant may only see what the person asking is allowed to see. That has to be shown in the acceptance test, not promised in the sales deck.
Relevant ifconfidential board, HR or legal files sit next to general documents.
Keep your data independent of the model provider
Your data and index stay exportable and the model stays replaceable. Prices and performance of models change faster than your contracts.
Relevant ifa vendor offers an all-in-one solution without an exit path.
Put accountability in the business, not in a lab
The business unit owns the outcome. Results are measured in real workflows, and a person reviews the cases where the system is unsure.
Relevant ifpast initiatives worked technically and were ignored in daily use.
Five acceptance criteria before an internal knowledge assistant goes live
Many companies want one assistant that answers questions from SharePoint, Confluence, ERP and CRM. These are the points I would ask the vendor to demonstrate in the acceptance test. I write these criteria into the contract, run the acceptance test with your business owners and decide with you whether it passes. I do not build or review the architecture myself; independent specialists do the technical testing.
- Permissions are inherited
A user who may not open a document cannot get it summarised either. Tested with real roles, including board and HR material.
- Every statement has a source
Answers point to the document and the passage. When the documents do not contain the answer, the assistant says so instead of improvising.
- Answers are tested against confirmed facts
Before go-live, a fixed set of real questions is checked against facts your experts have confirmed. The same set runs again after every model or data change, so you see whether quality held.
- Exact terms are found exactly
Product numbers, article references and internal abbreviations are matched exactly, not only by meaning.
- Everything is logged
Who asked what, which sources were used and what was answered is recorded and can be reviewed.
Public AI assistants repeat your data, contradictions included
ChatGPT, Claude, Gemini and Perplexity answer questions about your products from whatever they find online: your product pages, old PDFs, distributor listings and foreign versions. If decision 02 is open inside, it shows up outside, in answers you never see.
Internal knowledge assistant
You choose the model, the sources and the permissions. Quality is set in the acceptance test above.
Public AI assistants
You control neither model nor sources. You can measure what they say, trace each error to its likely source and fix your own published data.
Relevant ifyou sell OTC medicines or medical devices and patients ask AI about dosing, age limits or use. How the AI Answer Audit works →
Facing a data or AI decision?
Bring the decision and the current state of your data. I will tell you directly whether it calls for a Digital Baseline Assessment, a fractional mandate or just a second opinion.
Schedule an initial call