Score any AI vendor in 3 minutes.

Without a test set of your real cases, "it works" is a vibe. The 12% that ship define correct first.

Clause to demand: Vendor will deliver an evaluation set of no fewer than N real historical cases, with an agreed accuracy bar, before build begins.

When regulators or plaintiffs ask "who reviewed this output?", an org chart answer is not enough. Override authority must be named.

Clause to demand: A named human role holds override authority for every agent action class; escalation paths are documented before go-live.

You are legally responsible for your vendor. "HIPAA-friendly" marketing is not a signature on your paper.

Clause to demand: Vendor executes client BAA/DPA before any data access; subprocessor list with certifications attached as a schedule.

A demo date is not a milestone. The agent working beside your people on real cases, by a contract date, is.

Clause to demand: Agent runs alongside client staff on live cases by day 21; production cutover criteria defined at kickoff.

A vendor with no skin in the game has priced in your failure. Outcome-aligned terms are the tell.

Clause to demand: If the agreed metric is missed, vendor continues work at no additional fee for up to 60 days against the kickoff baseline.

Auditors and accreditors ask why a decision was made. If the system cannot answer, you inherit the liability.

Clause to demand: Complete per-action audit log (input, logic, output, reviewer) retained and exportable in a standard format.

Models drift and vendors swap them silently. Regulators now expect documented change plans for AI systems.

Clause to demand: Model or prompt changes are tested against the evaluation set and disclosed before deployment to production.

The cheapest vendor is expensive if leaving them means starting over. Infrastructure in your name is the exit plan.

Clause to demand: All work product — code, evals, runbooks, data — resides in client-owned infrastructure from day one.

Case-study PDFs are marketing. A named client and a product you can touch are proof.

Clause to demand: Vendor provides at least one named, contactable reference running the same class of system in production.

Open-ended T&M on an AI project is a blank check. Fixed diagnosis and build fees force scope honesty.

Clause to demand: Fixed fees for diagnosis and build stated in the order form; any T&M work requires prior written approval.