LLMs as a Tool, Not a Solution

Two and a half years in, with a production knowledge system, a working orchestration layer, local inference running alongside cloud providers, and prompt hygiene enforced at the infrastructure level — I want to be direct about something the AI industry is not particularly incentivized to say clearly: building this has been hard, it has taken a lot of time, and it is still not something I would recommend to most engineers as a place to invest significant effort without understanding exactly what they're signing up for.

That's not pessimism. It's the honest accounting that I would want from anyone describing a technology investment of this magnitude.

What LLMs Actually Are

Language models are pattern-completion engines trained on text at massive scale. They are remarkably good at generating text that follows patterns similar to their training data. They are not reasoning systems in the way a human expert reasons — they do not maintain internal models of the world that they update as new information arrives, they do not have persistent memory across sessions by default, and they do not know when they are wrong.

Everything in my stack that makes LLMs useful for professional work is infrastructure that works around these properties: the knowledge system compensates for the lack of persistent memory; the orchestration layer compensates for the lack of reliable multi-step reasoning; the output verification compensates for the fact that models don't know when they're wrong; the provider routing compensates for the fact that no single model is best for all tasks.

The tools are powerful. The infrastructure required to make them reliable for professional use is significant. That's the honest framing.

The Pattern-Driven Development Connection

The part of this story that I find most interesting — and that doesn't get enough attention in the AI tooling conversation — is how much the LLM value proposition depends on having well-established patterns in the first place.

I've been writing about pattern-driven development for over a decade: metadata-driven pipelines, configuration-driven frameworks, template-based code generation. The core idea is that consistent patterns enable automation. LLMs extend that idea in a specific direction: if your work follows consistent, recognizable patterns, language models can assist with the pattern-following parts — generating the boilerplate, applying the conventions, producing the structural scaffolding — freeing human attention for the parts that require genuine judgment.

The implication: engineers who already work in highly pattern-consistent ways get more value from LLMs than engineers whose work is more ad hoc. If you've invested in framework design, in consistent naming conventions, in well-documented architectural decisions — all of that investment pays forward into LLM assistance quality. The knowledge base I built is essentially an explicit representation of patterns I had already developed implicitly. Making those patterns explicit also made them accessible to a model.

Who Is Ready for This

The audience for serious AI tooling investment, as of right now, is narrow. You need enough engineering depth to evaluate model output critically — to catch the confident wrong answers and the subtle logic errors that look correct but aren't. You need enough infrastructure comfort to build and maintain the surrounding systems without being blocked by the operational complexity. You need enough patience to invest in tooling that pays back over months rather than days.

You also need either deep pockets or the hardware you bought before GPU scarcity made local inference inaccessible. The engineers running capable local models today are largely those who acquired the hardware before the market moved against them. That window may or may not reopen.

The honest assessment: AI-assisted development, done at the level of investment I've described, is currently viable for a specific population of technically senior, infrastructure-comfortable, high-pain-tolerance engineers. The consumer-ready version — the one that works well without significant setup, without ongoing maintenance, without deep expertise in the tools being used — does not yet exist. It is probably coming. The trajectory of improvement is real. It is not here yet.

What Comes Next

The work continues. The knowledge system needs better automatic ingestion. The orchestration layer needs better error recovery. The provider routing needs more sophisticated cost awareness. The de-identification pipeline needs to handle more edge cases. None of these are done; all of them are improving.

The question I keep returning to is not "when will LLMs be good enough?" — they're already good enough for a significant fraction of the work I do. The question is "when will the infrastructure required to use them reliably be accessible enough that the investment makes sense for a broader population?" The answer to that question depends on how the tooling ecosystem matures, and the answer is not yet obvious.

I'll keep building, keep documenting, and keep being honest about what's working and what isn't. If you're on a similar path — building the infrastructure, not just using the models — I'd genuinely like to compare notes. As always, I'm here to help.