How to run an AI proof of concept that survives real data
An AI proof of concept survives real data when it is designed to test the riskiest assumption, not to produce a demo. Use real data from the first day, write the pass and fail thresholds before anyone builds, make the evaluation set first, keep the work to a few weeks, and agree in advance what a pass and a fail will each lead to.
Why most AI POCs prove the wrong thing
A typical POC goes like this. A team picks a use case, gathers a dozen tidy examples, connects a model and shows a demo in which it works. Everyone is impressed. The project is funded, meets the real data and stalls.
The demo proved something nobody doubted: that a current model can do the task on clean examples. That stopped being the risk some time ago. The risks are elsewhere, and they are specific to you:
- Your data is messier or harder to reach than assumed.
- The model is right on typical cases and wrong on the rare ones that cost the most.
- The systems it must work with will not give it access, or not quickly enough.
- The cost or the response time does not fit the task.
- The people meant to use it do not trust it, or do not need it.
A POC is an experiment, and an experiment that cannot fail tells you nothing. If the only possible result is a successful demo, you are running a presentation.
Pick the riskiest assumption
List what has to be true for the project to work. For each item, ask how likely it is to be false and how bad that would be. Test the one that scores worst on both. One assumption, perhaps two.
Take an equipment rental company that wants an assistant to answer field technicians’ questions from its library of 4,000 service manuals. Its assumptions look like this:
| Assumption | If it is false | Cheapest test |
|---|---|---|
| A model can explain a repair procedure clearly | The project fails | None needed. This is not in doubt. |
| We can find the right passage, for the right machine variant, in scanned manuals of mixed quality | A technician is given the wrong procedure | 150 real questions against the real manuals |
| Technicians will use it on a phone on site | Nobody adopts it | Five technicians, one week, a rough prototype |
| Answers arrive quickly enough on a weak connection | It is abandoned on site | Timed runs over a throttled connection |
The second is the riskiest. It is the most likely to be false, and a confident procedure for the wrong variant is the worst thing the system could do. So that is the POC: retrieval over the real manuals, measured on real questions. Not a chat interface and not a login screen. Adoption is the next question, and it belongs in a pilot.
One check before any of this: make sure a product does not already do the job. Our build vs buy framework covers that decision.
Use real data from day one
Sample, do not curate. Take every question logged by the technical helpline in a recent month, not the twenty the project sponsor remembers. Take the manuals as they are stored, including the photocopies with handwriting in the margin. A curated sample removes exactly the cases that will break you. OpenAI’s evaluation guidance lists test sets that do not reproduce production traffic among its anti-patterns, and we agree.
Real data brings real obstacles, and those are findings too. If it takes three weeks to get an export approved, the production system will meet the same wall. Start the access requests before week one.
Sort out privacy at the same time: what personal data is in the sample, whether it can be redacted, and what the model provider does with what you send. Read the provider’s terms. OpenAI’s data controls page, for instance, says that data sent to its API is not used to train its models unless you opt in. Check your own provider’s terms, and do not assume.
Synthetic data has one legitimate use in a POC: adding rare cases you know exist but could not find in the sample. It is never the base. And if the sample shows that the underlying records are the real problem, that is data engineering work to do before any model is involved.
Define pass and fail before you start
Write the thresholds down before anyone has seen a result. Afterwards, every number looks acceptable to somebody.
Good criteria are specific and measurable, as Anthropic’s guide to success criteria puts it, and they are set against a baseline, meaning what the current process achieves. If the helpline gives the right answer nine times in ten, perfection is not the bar, and neither is “better than nothing”.
For the rental company the thresholds might read:
- Pass: on 150 real questions, the assistant cites the correct manual section for the correct variant in at least 85 per cent of cases.
- Unacceptable error: it gives a procedure for the wrong variant, stated with confidence, in more than 2 per cent of cases. This fails the POC whatever the first number says.
- Cost and speed: a median answer within eight seconds, and a token cost per answer inside the figure the business case assumes.
- Fail: below 70 per cent on the first measure.
- In between: inconclusive, which triggers one extension agreed in advance, not an open-ended one.
Those numbers are examples for an invented company. Yours come from your baseline and from what an error costs. The structure is the part to copy: a main measure, a separate limit on the error you cannot tolerate, a cost and speed check, and a named grey zone.
Build the eval set first
Before the prompts and before choosing a framework. Google’s long-standing Rules of Machine Learning make the same point as rule two: “First, design and implement metrics.”
For a POC we aim for 100 to 200 real cases, each with the expected outcome written by someone who knows the domain. That takes an expert’s time, and it is the commitment most often missing. If the business cannot spare a senior technician for a few hours a week for a month, treat that as information about the project’s priority.
Three rules for the set:
- Mix it as the real work is mixed. Mostly typical cases in their true proportions, then the hard ones, then a few adversarial ones, such as a question about a machine that is not in the library.
- Hold out a third. Develop against two thirds and run the final measurement once against the rest. Otherwise you tune to the test and the number means nothing.
- Grade by code where you can. Did it cite section 7.3 of the right manual? Use an expert on a sample, and a model as grader only after checking that it agrees with the expert.
The eval set outlives the POC. It becomes the regression suite for whatever is built next, which is why our guide on how to evaluate AI agents starts from the same place.
A week-by-week plan
| Week | Work | What you have at the end |
|---|---|---|
| 0 (preparation) | Agree the brief. Request data access. Name the owner and the domain expert. | A signed brief, and data on its way |
| 1 | Data in hand. Draft and label the eval set. Measure the baseline. Run the simplest possible version end to end. | Eval set, version one. A first score, probably poor |
| 2 | Work on the riskiest part only. Read the failures every day and group them by cause. | Error categories, and a score for each |
| 3 | Fix the top two causes. Try one real integration. Measure cost and speed. Go looking for the unacceptable error. | A second score, with cost and latency figures |
| 4 | One run on the held-out cases. Write up. Hold the decision meeting. | A result against the thresholds, and a decision |
Three notes on running it.
Get to an end-to-end run in week one, even if it scores badly. A bad number on real data is worth more than a good architecture diagram.
Spend week two reading failures, not adding features. In the rental example, the reading might show that most wrong answers come from manuals scanned as images with no text layer. That is one fix, and no amount of prompt work would have found it.
Build nothing the question does not need. No interface polish, no authentication, no deployment pipeline.
If the data has not arrived by the end of week one, stop the clock and say so. “We could not get the data” is a result, and a useful one.
The one-page POC brief
The whole brief should fit on one page. Fill it in before the work starts and get it signed by the person who will make the decision.
- Decision. What will be decided when this ends, and by whom.
- Riskiest assumption. One sentence.
- Out of scope. What this POC does not test.
- Data. Source, sampling method, size, date range, who grants access, how personal data is handled.
- Eval set. Number of cases, who labels them, share held out.
- Baseline. How the current process performs on the same measure.
- Thresholds. Pass, fail and the unacceptable error.
- Cost and speed limits per task.
- Time box and budget. Weeks, people, a cap on model spend.
- People. Owner, engineer, domain expert, with hours per week for each.
- On pass. What is funded next, and who signs.
- On fail. Stop, narrow or fix the data, and who decides which.
- Kept and discarded. Kept: the eval set, the findings, the cost data. Discarded: the code.
If you cannot fill in line 2 or line 7, you are not ready to start.
What a pass or a fail should lead to
Lines 11 to 13 deserve more than a line each.
A pass should release something specific: a funded pilot with a named scope and team. If a pass leads only to another discussion, the organisation was not ready for the answer and the POC was premature.
A fail has three respectable outcomes. Stop, and spend the money elsewhere. Narrow the scope to the subset where the numbers were good, such as the newer manuals that were digital from the start. Or fix one named data problem and retest that one thing. What you must not do is quietly move the threshold.
Throw the code away in both cases. POC code is written to answer a question quickly, and it makes a poor foundation. The eval set, the error analysis and the measured cost per task are what carry forward.
How to start
This week, write line 2 of the brief and get the person who will make the decision to agree line 7. Then ask for the data. Those three steps cost almost nothing, and within days they tell you whether the project is real.
None of this needs special tooling. A spreadsheet for the eval set and a script that runs it are enough for the first month. A fixed-scope POC run this way is one of the things we offer under product engineering.