Five questions to answer before you build an AI product

I get a lot of messages that start with "we want to add AI to our product". The idea is usually good. The problem is that the hard part of an AI product is not the model. It is everything around it. Here are the five questions I ask on the first call, and why each one matters.

1. What does a good answer look like?

Write down twenty real examples. Real inputs your users would type, and the output you would be happy to ship. Not perfect, just happy. If you cannot write twenty, the product is not clear yet, and no model will fix that.

Those twenty examples become the test set. Every time a prompt changes or a model is swapped, I run them and look at what got better and what got worse. Without them, every change is a guess and every demo is a lucky one.

2. What happens when it is wrong?

It will be wrong sometimes. The question is what the user does next. Can they edit the answer? Retry? See where it came from? Tell you it was wrong? A product that handles a wrong answer gracefully feels trustworthy. A product that presents every answer as final feels dangerous the first time it slips.

Design the wrong-answer path first. It is the part most teams forget, and it is the part users remember.

3. How much can each answer cost?

Model calls cost money per use, which is different from most software, where the marginal cost is close to zero. Work out what one answer is allowed to cost at your price point. Then design to that number: shorter prompts, smaller models for simple steps, caching where answers repeat, limits where users could run up a bill.

This is also why I pick the model last. Prototype on whatever is fastest to wire up, then let the test set and the cost ceiling choose the model that earns its place.

4. What data can the model see, and what must it never see?

Be precise. Customer records, internal documents, other users' content, payment data. Decide what goes into a prompt and what never does. Decide whether your provider trains on your data, and pick one that does not if that matters, which it usually does.

Write this down before you build, because it shapes the architecture. Adding a boundary after launch is far more expensive than designing with one.

5. Would people pay for this without the word AI?

Strip the label and describe what the product does for someone. Drafts replies to customer emails in your tone. Turns a product photo into a listing. Finds the clause in a contract that matters. If that sentence is valuable, the product is valuable. If the only interesting word is AI, it will be interesting for a month.

The feature is not the model. The feature is what happens around it.

What this looks like in practice

On my projects the first week is spent on these five questions and on the twenty examples. No interface yet. By the end of the week we both know what we are building, what it may cost to run, where the risks are, and how we will know it works. The build that follows is faster and calmer because of it.

If you are at the idea stage, the Brief builder in my Lab walks you through most of this. Describe the product and it returns the open questions, including the ones above.