The DIY Temptation
The economics seem obvious at first. OpenAI’s API is cheap. Python wrappers for LLMs are free. Your engineering team is already on payroll. Why pay someone else to build something you could build yourself in a few sprints?
We hear this reasoning constantly, and we understand it. We’ve also seen the aftermath when it goes wrong — stalled roadmaps, significant sunk costs, and engineering teams stuck maintaining brittle pipelines instead of shipping product. The issue isn’t capability; it’s scope. What looks like a three-sprint project routinely becomes an eighteen-month commitment.
Engineering Time You Underestimated
The first prototype is fast. The production system is not. Common sources of hidden engineering time include:
- Prompt engineering and evaluation loops — getting reliable outputs requires iterative testing across hundreds of edge cases. This is a specialised skill that takes time to develop.
- RAG pipeline tuning — chunking strategy, embedding model selection, re-ranking, hybrid search. Each decision affects output quality and requires measurement infrastructure to evaluate.
- Guardrails and safety layers — hallucination detection, output validation, PII scrubbing. Easy to skip in development; expensive to add later when they fail in production.
- Latency and cost optimisation — the system that works in testing often doesn’t work at production load. Caching, batching, and model selection all need deliberate engineering.
The Ongoing Maintenance Tax
Once deployed, AI pipelines don’t sit still. Foundation model behaviour drifts after updates. User inputs evolve in ways your prompt didn’t anticipate. New failure modes emerge under production traffic patterns you never saw in testing.
In a survey of our client engagements, teams that built AI systems in-house spent an average of 30–40% of ongoing engineering capacity maintaining and firefighting their AI infrastructure — time that wasn’t in any original estimate.
Maintenance isn’t optional for AI systems. Unlike traditional software, language model outputs are non-deterministic. A pipeline that passes all your tests today can start producing degraded outputs tomorrow without any code change on your end.
Model Versioning Chaos
Foundation models deprecate. When OpenAI retired GPT-3.5-turbo-0301, teams that had hardcoded that model version faced urgent migration work. When Claude’s behaviour changed across versions, prompt strings that worked reliably started producing inconsistent outputs.
Handling model versioning properly requires abstraction layers, regression test suites, and deployment pipelines that most in-house AI projects simply don’t have. Building and maintaining these adds significant overhead that compounds with every new model you adopt.
When DIY Makes Sense
DIY isn’t always wrong. Building in-house is the right call when:
- Your use case involves genuinely proprietary data handling requirements that prevent using external services
- You have a strong existing AI/ML team with production LLM experience specifically
- The system is a core product differentiator, not internal tooling
- You’re willing to treat AI infrastructure as a product with a dedicated team, roadmap, and engineering standards
A Smarter Approach
The false choice is between “build everything yourself” and “buy a black box SaaS tool.” The third path — the one we recommend — is to partner on architecture and build where it matters.
In practice this means: bring in expertise to design the system correctly, establish the infrastructure and evaluation harness, and transfer knowledge to your team. You own the system; you’re not renting a service. But you’re not reinventing the wheel, either.
The teams that get the best outcomes are the ones who are honest about the difference between “we could build this” and “this is the best use of our engineering time.”