One Year Later: Which Predictions Came True?
A year ago at DAIS 2024, I made some predictions. It's May 2025, DAIS 2025 is a month out, and it's time for an honest accounting. What did I get right, what surprised me, and what changed my mind.
What I Got Right
The compound AI architecture would become standard. Called it. By the end of 2024, the "single model call" approach to GenAI applications was essentially retired in serious production deployments. Every production RAG system I've seen since then is a compound system: retriever, reranker, generator, validator. The pattern is now the default, not the advanced option.
Cost would be the #1 blocker. Also called it. More than a few GenAI initiatives that looked promising in 2024 stalled out in 2025 when the annualized inference cost landed on the finance team's desk. The teams that built cost-aware architectures from the start are running. The teams that didn't are retrofitting or scaling back scope.
Classical ML + GenAI hybrid systems dominate production. This one turned out to be exactly right. The most impactful production AI systems I've seen in the last year combine structured prediction (ML) with language understanding and generation (LLM). Fraud detection with LLM-generated explanation. Risk scoring with natural language risk summary. Demand forecasting with LLM-based anomaly narrative. The hybrid is the mature pattern.
What Surprised Me
How fast the retrieval engineering discipline matured. A year ago, RAG was still being done with naive top-k vector similarity search and fixed chunk sizes. By mid-2025, production RAG systems routinely use hybrid search (dense + sparse), multi-stage retrieval with reranking, query expansion, and context-aware chunking. The quality gap between a 2024 RAG system and a 2025 RAG system is significant, and the tooling that made 2025 RAG possible developed faster than I expected.
How slowly agent governance tooling matured. I expected Databricks to ship UC-native agent governance by Q1 2025. It hasn't landed yet. The agents I'm working with are still governed through custom instrumentation on top of the framework rather than through UC-native policies. This gap is real and it's causing friction in enterprise deployments where compliance teams want standardized audit trails.
What Changed My Mind
I was more skeptical about natural language data interfaces than I should have been. My critique last year — that the bottleneck was question formulation, not query authorship — was right but incomplete. The better NL interfaces in 2025 have gotten meaningfully better at disambiguation: they ask clarifying questions when a natural language query is ambiguous, they explain what data they're searching, and they acknowledge uncertainty in the answer. That's not the "citizen developer will replace the analyst" claim. But it does make data exploration meaningfully faster for people who understand the data domain. I underestimated the value of the iterative interface. As always, I'm here to help.