BlogData engineering

AI-ready data: what to fix before you build anything

Before you build, audit the data the system will actually use: who owns it, who may see it, how fresh it is, whether copies disagree, whether a machine can read it, how it is described, what personal data it holds and whether you have real examples to test against. The audit takes an afternoon. In the builds we run, a gap in one of those eight areas causes more delay than any choice of model.

What AI-ready data means

AI-ready data is not a state a whole company reaches. It is a property of one data set judged against one use: the returns policies behind a support assistant, the order system behind an agent that issues refunds, three years of service tickets behind a churn model. The same finance warehouse can be ready for a monthly dashboard and useless for all three.

The test we use is simple. Could a competent new employee, given only this data and nobody to ask, do the job correctly? A model is in that position on every request. It cannot know that the PDF in the shared drive was superseded in March, that the file called “Final_v3_USE THIS” is the real price list, or that the HR folder was never meant to be open to everyone. People carry that context in their heads. Software needs it written down, enforced or removed.

The eight things to audit

Run these against the data for one use case, not the whole estate.

1. Where the data lives and who owns it

List every source the system would need. Beside each, write the name of a person who can answer questions about it and approve changes. “IT” is not an owner. If nobody will put their name against the warranty documents, nobody will tell you when they change, and the assistant will quote last year’s terms with complete confidence.

Note the route in as well: an API, a database you can query, or an export someone runs by hand. A source that exists only as a monthly spreadsheet emailed by one manager is a risk you want to see on day one.

2. Access and permissions

This is the one that turns into an incident. An assistant makes it easy to find whatever a user technically has access to, including the things nobody realised were open. Microsoft’s deployment guidance for Copilot lists remediating oversharing as the first of its three pillars. The assistant grounds its answers in the data a user already has permission to open, so every loose permission is an answer waiting to be given.

We hold two rules. First, the AI system respects the source system’s permissions at query time, per user. It does not read everything through one powerful service account into one shared index. Second, fix the permissions at the source before you index. Filtering in the retrieval layer is necessary, but it should not be the only thing between a junior employee and the salary review folder.

For an agent, the question extends from reading to acting: which records it may change, up to what value, and with whose approval.

3. Freshness

Ask how old the data may be before an answer becomes wrong, then check how old it actually is. Stock levels need minutes. Policies need the same day. Product manuals can tolerate a week.

Then check how updates reach the AI system. Deletions matter as much as additions: a withdrawn document that stays in the index is worse than a missing one, because it produces a wrong answer where a gap would have produced none.

4. Duplicates and conflicting versions

Every shared drive has four versions of the important document. A retrieval system will fetch two of them, and the model will blend them into an answer that matches neither.

Pick a source of truth per document type and exclude the rest. Archive folders, personal copies and email attachments should be out of scope by default. Where two live documents contradict each other, which happens with regional policies, that is a business decision to escalate. No prompt wording will settle it.

5. Scans and formats a machine cannot read

Open twenty files at random and try to select the text. Scanned contracts, photographed delivery notes, tables pasted in as images and spreadsheets with merged cells and colour-coded meaning all need conversion before they are usable. Vision-capable models have made that far more tractable than it was, but it is still a pipeline with its own checks, which we cover in Document AI: turning invoices, forms and contracts into data.

6. Structure and metadata

A passage of text with no context is hard to use well. At a minimum each document should carry a title, type, owner, effective date and status (draft, live or withdrawn). With those, the system can filter to live documents, prefer the newest and show a citation the reader trusts. Without them, every query searches everything.

For structured data the equivalent is a plain-language description of each table and column, including units and the meaning of codes. If status = 4 means “cancelled after dispatch”, write that down. A model asked to query the database will otherwise guess, and guess plausibly.

7. Personal data and what may leave your systems

Find out what personal or confidential data is in scope and decide what may be sent to a third-party model. Read the provider’s terms, not a summary of them. OpenAI’s API documentation, for example, says that data sent to the API is not used for training unless you opt in, that abuse-monitoring logs are kept for up to 30 days by default, and that zero data retention needs prior approval. Other providers publish their own terms, and they differ.

Then minimise. Most use cases do not need a customer’s date of birth in the prompt. Drop or redact fields before indexing, and treat automated detection as a help, not a guarantee: Microsoft’s open-source Presidio is a good tool, and its README says plainly that it cannot promise to find all sensitive information. In the UK the reference is the ICO’s guidance on AI and data protection, which is under review following the Data (Use and Access) Act, so check the current version.

8. Labelled examples for evaluation

This is the item teams skip most often. You need real inputs with known correct outputs: genuine questions with the right answer and its source document for an assistant, completed cases with the correct outcome for an agent, historical records with the label you want to predict for a model.

Without them you cannot tell whether a change made things better, and every conversation about quality becomes a matter of opinion. Collecting them also exposes the other seven problems early, because the people writing the expected answers know which document is out of date. We go into building that set in how to evaluate AI agents.

A readiness checklist you can score in an afternoon

Score each area 0, 1 or 2 for the one use case in front of you. Do it with the data owner and an engineer in the same room, looking at real files and not at a diagram.

Area 0 1 2
1. Ownership No named owner Owner known informally Named owner who approves changes
2. Permissions Unknown or wide open Reviewed, gaps remain Correct at source, enforced per user
3. Freshness No update route Manual or occasional refresh Automatic sync, deletions included
4. Versions Many copies, no source of truth Source agreed, clutter remains One live copy in scope
5. Readability Mostly scans or images Mixed, conversion planned Text extractable and spot-checked
6. Metadata File names only Some fields, inconsistent Type, owner, date and status on every item
7. Personal data Not assessed Assessed, handling undecided Classified, minimised, provider terms checked
8. Test examples None A handful, informal Fifty or more real cases with expected results

Add it up out of 16.

  • 13 to 16: build.
  • 8 to 12: build a proof of concept on the cleanest slice while you fix the rest.
  • Below 8: fix first. A build started now will mostly produce a list of data problems, at engineering rates.

A zero on permissions or personal data stops the build whatever the total. The number is not science. Its value is that the owner, the engineer and the sponsor agree on the gaps before anyone writes code.

What to fix first for RAG, an agent or a predictive model?

The eight areas apply to all three, but the order changes.

RAG assistant Agent that takes actions Predictive model
Fix first Versions and permissions Scoped access to act, and test cases Labels and history
Then Readability and metadata Freshness of the records it reads Stable definitions over time
Typical failure A confident answer from a withdrawn document A sound decision made on a stale record A model that learns how data was recorded, not what happened

A RAG assistant can only be as good as the documents it retrieves. A small curated set beats a large messy one, so cut scope hard: live documents, one copy each, permissions correct. It is the usual first step in any RAG or enterprise search build.

An agent needs dependable APIs to the systems it acts on, permissions scoped to the task, and records that are current at the moment it decides. An agent that reasons perfectly over an order status two hours old will still refund the wrong order. If the system of record has no API, that is the first project, ahead of any agent build.

A predictive model needs history: enough past cases, labelled with the outcome, recorded the same way throughout. Check whether a definition changed halfway (what counted as “churned” in 2023 against now) and whether any field was filled in after the outcome was known. That second problem, leakage, produces models that look excellent in testing and fail on their first live day.

A worked example

Take a distributor of commercial kitchen equipment with 60 staff. It wants an internal assistant that answers questions on product specifications, warranty terms and returns.

First pass. Ownership scores 1: the service manager looks after the documents, informally. Permissions 0: one shared drive, open to all staff, with supplier contracts and cost prices in the next folder. Freshness 1. Versions 0: specification sheets from 2019 sit beside the current ones. Readability 1: older manuals are scans. Metadata 0. Personal data 1: returns forms with customer details are mixed in. Test examples 0. Total: 4 out of 16, with a zero on permissions.

Nobody should build on that. But the fix is two or three weeks of unexciting work, not a data programme. Limit scope to the current product range. Move live specification sheets and policies to one location with access set properly, and leave supplier contracts and returns forms out entirely. Tag each file with product family, document type and effective date. Have two people from the service desk write down sixty real questions from last month’s inbox, with the correct answers.

Second pass: 13 out of 16, no zeros. Build the proof of concept on that slice. The scanned manuals become phase two.

What goes wrong when you skip the audit

There are two failures, and they are opposites.

The first is the demo that worked. Someone hand-picks twelve clean documents, the assistant answers well, a budget is approved, and then the system meets the real shared drive. The team spends the build phase doing the audit anyway, under deadline.

The second is the data programme. A company decides to get all its data in order before touching AI, with no use case to say what “in order” means. A year later there is a catalogue and no product.

We would avoid both. Scope to one use case, fix what it needs, and let the next use case pay for the next clean-up.

How to start this week

  1. Write the use case as one sentence, naming who asks what and what the system does with it.
  2. List the sources it needs and get a named owner for each.
  3. Pull a sample of 20 to 50 real items per source and run the eight checks on them with the owner. Score the table.
  4. Deal with any zero on permissions or personal data before anything else.
  5. Start collecting test examples from real tickets, emails or cases now. They take the longest, and nothing else can be measured without them.

The decision at the end of the week is simple: build on a slice, or fix first. If you want a second pair of eyes on it, this audit is the first step of the data engineering work we do before any build.

Questions

Asked often.

What does AI-ready data mean?

It means the particular data a particular AI system needs is findable, permitted, current, machine-readable and described well enough for software to use without a person interpreting it. It is judged per use case, not for the whole company. Data that serves a monthly dashboard perfectly well can still be unusable for a retrieval assistant or an agent.

How long does it take to get data ready for an AI project?

The audit takes an afternoon to a few days. The fixes vary. Tightening permissions and removing duplicates for one document set is usually a matter of weeks. Giving an agent dependable API access to a system of record, or assembling labelled history for a predictive model, can take a quarter. Scope the work to one use case.

Do we need a data warehouse or data lake before using AI?

Not for most first projects. A RAG assistant needs a curated, permissioned document set and an index, not a warehouse. An agent needs dependable APIs to the systems it acts on. A predictive model does need history in one place, but one well-built table often serves better than a platform programme that takes a year to deliver.

Can an AI model clean up messy data for us?

Partly. Models are good at reading scans, tagging documents, spotting near-duplicates and proposing metadata, and we use them for that. They cannot decide which of two conflicting policies is the true one, who should be allowed to see a file, or what you may send to a third party. Those are ownership decisions and they stay with people.

Not on the list? Ask us directly

Keep reading

More notes from the build.

All blogs

Let’s build your stack.

Send a brief and we’ll reply with next steps.