How to evaluate an AI vendor: a practical checklist for SMEs

Every week another AI startup emails your business with a sales pitch. Most are fine but unnecessary. Some are genuinely useful. A few are very expensive ways to do something you can already do for less. Telling them apart in 15 minutes is a useful skill.

These five questions, asked in this order, will separate the worth-trying from the politely-decline.

1. What does it do that you can't already do?

Most "AI tools" are wrapping a general-purpose model (ChatGPT, Claude or similar) and selling it as a specialised feature. That is not bad - the wrapping can be valuable - but it changes how you should value it.

Ask: "Could I do this myself in ChatGPT Plus with a saved prompt?". If the answer is yes, the value the vendor adds has to be in the workflow, the integrations, or the time-saving. Not the AI itself.

If the answer is no - they are doing something genuinely different (proprietary data, fine-tuned model, deep integration with a system you use) - that is more interesting.

2. Where does your data go?

Read the privacy notice. Specifically:

• Where are the servers (UK, EU, US, elsewhere)?
• Will your data be used to train the vendor's model? (This must be no, in writing, on a paid tier.)
• Will your data be passed to a sub-processor (the underlying AI provider, e.g. OpenAI, Anthropic)? Almost certainly yes if they wrap a third-party model. The question is what that sub-processor does with it.
• How long is data retained? Can you delete it on demand?

If the vendor cannot answer these clearly, that itself is the answer. Our data privacy guide covers what good looks like for AI providers.

3. What does the contract say?

Specifically: the cancellation terms, the price-rise clauses, the SLA, and the data-portability provisions.

AI vendors have been known to raise prices sharply once a customer is dependent on the tool. A 90-day notice period and a "no price rise greater than X% in the first 12 months" clause is reasonable to ask for.

If the contract is take-it-or-leave-it boilerplate from a small startup, you may need to accept that. Just go in with eyes open about the leverage you do not have.

4. What is the cost over 12 months?

Not the headline monthly price. The total. Including extra-seat costs, usage overages, premium features that are billed separately, and the inevitable "Pro" tier that everyone gets pushed to within 90 days.

Then compare it to the most boring alternative: ChatGPT Plus or Claude Pro at £18-20 a month per seat. If the vendor's tool costs five times more, it had better do something five times more useful for your specific work.

Many AI tools fail this test. They are technically impressive and provide real value, but the value-to-cost ratio is worse than a £20 generalist tool plus a saved prompt.

5. How do you exit?

Often the most important question and the one most rarely asked.

If you commit to this tool and decide in 18 months that it is not working, what does leaving look like? Can you export your data, your custom prompts, your fine-tuning, your usage history? In what format? Or are you trapped in the vendor's walled garden?

AI tools that lock in your data tightly should be avoided. The space moves fast. The market leader today may not be the right tool in 24 months. Optionality is valuable.

The trap to avoid

The biggest trap with AI vendors is "feature shopping". You see a demo, the demo looks impressive, you sign up, and three months later the tool is barely used. Sound familiar?

The way out is to start from a real problem in your business that costs you real time, then look for a tool that solves it. Not from "what can AI do?" then look for a problem to apply it to. The 30-day plan sets out the structure.

If a vendor cannot demo their tool solving a problem you actually have, in your actual workflow, you do not need their tool yet.

When to say yes

Three signals worth taking seriously.

Specialised data or fine-tuning - they have built something on top of a base model that you could not realistically reproduce. (e.g., a tool trained on UK construction quotes; a tool trained on accountancy practice management.)

Deep integration - the tool sits inside a system you already use (Xero, HubSpot, Microsoft 365) and does work that requires that integration. The integration itself is the value.

Time saving on a real workflow - you can articulate, before signing, exactly which 30 minutes a week this tool gives you back. And it is plausible.

The honest summary

Most SMEs in North Wales should be very conservative about AI vendor adoption. The base AI tools (ChatGPT, Claude, Copilot, Gemini) cover 80-90% of practical use cases. Specialist tools are worth it occasionally and very rarely worth it before you have got the basics working.

If you would like a second opinion on a specific tool you are considering, that is a 15-minute conversation in a discovery call. The AI Breakfast Club also covers vendor evaluation in its third session.