Skip to content
AltonHenley

Ready or Not – Vendor Checklist

Ready or Not

Vendor Evaluation Checklist

Score vendors on eight dimensions, check for red flags, and negotiate key contract provisions.

back to Ready or Not

Before the Evaluation

Complete before talking to any vendor:

We have documented the specific problem we’re trying to solve
We have defined what success looks like (measurable outcomes)
We have identified constraints (budget, timeline, data sensitivity, integration needs)
We have identified who will use the tool daily and included them in the evaluation

The Eight Questions

Score each 1–5 (1 = poor, 5 = excellent). Leave blank if not applicable.

Vendor: ___________________________________

1. Government Experience

Has the vendor worked with local governments? Do they understand public sector requirements? Ask for specific municipal or county references, not vague “public sector experience.”

2. Data Handling

Clear answers on: where data is stored, whether it trains models, who has access, whether data is extractable, and whether protections are in the contract?

3. Transparency

Can the vendor explain how the system reaches its conclusions in terms you can relay to residents? “Proprietary algorithm” is a red flag.

4. Bias Testing

For resident-facing tools: has bias testing been documented? Error rates known? Demographic breakdowns available?

5. Total Cost

All costs identified: licensing, implementation, training, integration, ongoing support, staff time, and future pricing? Ask for Year 1, 2, and 3 pricing in writing.

6. Support

Response time guarantees? After-hours availability? Incident resolution process? What happens at 4 p.m. on a Friday when the system gives residents wrong information?

7. Exit Strategy

Data exportable in standard formats? Reasonable termination terms? Transition support? What happens to your data after the contract ends?

8. References

Government references provided? Did we call them? Were they positive?

Total Vendor Score — / 40

Reference Check Questions

When calling vendor references, ask:

  1. Did the product deliver what was promised?
  2. What surprised you—positive or negative?
  3. What problems have you encountered?
  4. How responsive is the vendor when things go wrong?
  5. Would you buy this product again?
  6. What would you do differently?

Red Flags

Stop and reconsider if a vendor:

Cannot explain what happens to your data
Claims “proprietary algorithm” when asked how the system works
Has no government references—or won’t let you call them
Cannot provide Year 2 and Year 3 pricing
Has no incident response or after-hours support plan
Makes it difficult to export your data or terminate the contract
Cannot document bias testing for resident-facing applications
Offers only “don’t worry” when asked about security or privacy

Contract Provisions Checklist

Negotiate these before signing:

Data ownership: We own our data. The contract says so.
Data portability: We can export data in standard formats at any time.
No model training: Our data is not used to train the vendor’s AI models (or, if it is, this is disclosed and consented to).
Performance guarantees: Uptime, accuracy, and response time commitments are in writing.
Liability: Contract addresses who is liable when the AI produces errors.
Right to audit: We can audit the system’s performance and data handling.
Termination: Reasonable termination terms with adequate transition support.
Pricing protection: Caps or limits on price increases at renewal.
Subcontractor disclosure: If the vendor uses subcontractors, we know who they are and what data they access.

For Embedded AI (Software You Already Own)

When your existing vendor adds AI features:

We’ve identified which current software products now include AI features
We’ve reviewed updated terms of service for data handling changes
We’ve confirmed whether our data is being used for model training
We’ve applied the same governance standards to embedded AI as to a new purchase
We’ve assessed whether the AI features are appropriate for our needs—not just available

© 2026 Alton Henley. Ready or Not.
altonhenley.com · LinkedIn