Modules/Evaluating AI Tools and Vendors/12 Questions to Ask Before You Buy
3/4

12 Questions to Ask Before You Buy

Questions before buying

The best time to ask hard questions about an AI tool is before you've signed up, integrated your data, and built workflows around it. After that, switching costs make you a captive customer.

Here's a checklist of 12 questions that actually matter. Run every serious purchase through it.

Why Most Evaluations Fail

Most people evaluate AI tools on one dimension: can it do the thing in the demo?

That's necessary but not sufficient. You also need to know:

  • What happens when it fails?
  • What does it cost at scale?
  • How hard is it to leave?
  • Who owns the outputs it generates?

These questions don't get asked because the vendor doesn't volunteer them — and buyers are caught up in the excitement of what the tool can do.

The 12 Questions

The Pre-Buy Checklist (Print This)

Before signing any AI tool contract, confirm you've answered:

  • I can state the specific problem this solves in one sentence
  • I know the full cost at 3x my current usage
  • I've seen it run on my actual messy data, not just demo data
  • I know exactly how to cancel and export my data
  • I've checked: is my data used for model training?
  • I've spoken with at least one reference customer in a similar situation
  • I know what "wrong" looks like and how errors surface
  • I've reviewed the ToS for output ownership
  • I know the onboarding timeline and who does the work
  • Any roadmap features I'm depending on are contractually committed

If you can't check all 10 boxes, you're not done evaluating yet.

Quick Check

Quick Check

5 questions · Earn points for speed!

🔀 Random selection — different questions each play!

Key Takeaway

The point of these questions isn't to find a reason to say no. It's to make sure that when you say yes, you've said yes to the tool you'll actually be using — not the one that looked good in the demo.

🎓

Ready to complete this lesson?

You've reached the end! Hit the button below to earn your XP.