The 2025 Enterprise AI Readiness Checklist

DAIS 2025 is six weeks out. I've been collecting notes on what separates the enterprise AI teams that are actually shipping production systems from the ones that are still stuck in POC purgatory. Here's the checklist I use before any AI initiative gets to production — not as a bureaucratic gate, but as an honest inventory of what's missing.

Data Readiness

These have to be done before you build anything:

Data quality baseline — run the Data Interrogator against every source table feeding the AI system. Know your null rates, your cardinality, your outlier distributions. If you don't know what the data looks like, you don't know what the model is learning from.

Lineage map — trace the data path from source system to training features. If any step in that path is undocumented, instrument it before you train. You will need this map for regulatory documentation.

Access audit — verify that training data access is appropriately restricted. Models trained on data that shouldn't be accessible to the model owners create compliance problems retroactively.

Model Readiness

Evaluation set — a labeled dataset of at least 200 representative examples, with expected outputs, that has not been used in training. This is your production quality gate. If you don't have this, you don't know if the model is ready.

Baseline comparison — does the AI system outperform the baseline (rule-based system, previous model, human judgment) on the evaluation set? If not, it's not ready.

Failure mode documentation — what inputs cause the model to fail? What does failure look like? What happens in the system when failure occurs? These questions must have written answers before production.

Operational Readiness

Monitoring dashboard — output quality metrics tracked at least weekly. Who owns the dashboard? Who is alerted when metrics degrade?

Cost estimate — annualized inference cost at expected production query volume. If this number hasn't been calculated, calculate it before launch. Include embedding costs, reranking costs, and evaluation sampling costs.

Rollback plan — if the production model needs to be taken offline, what's the plan? Is there a fallback system? Who has the authority to execute the rollback? How fast can it happen?

Human escalation path — for decisions above a confidence threshold, or for defined edge cases, what's the path to human review? This isn't optional for high-risk AI decisions; it's a regulatory requirement for many system types.

Governance Readiness

Model documentation — training data description, intended use case, known limitations, evaluation methodology, approval record. One document, one place, findable by anyone who needs it.

Access policy — who is authorized to call the serving endpoint? A positive list, not "whoever can reach the endpoint internally."

Review cadence — when does this model get reviewed for drift, retraining, or decommission? Who owns that review?

If every item on this list has a clear answer with an owner and a date, the system is ready for production. If any item is "we'll figure that out after launch," figure it out before launch instead. The production failure rate for AI systems that launch without these foundations is not low. As always, I'm here to help.