What Four Months of Embedded AI Engineering Ships

TL;DR: This engagement exists for four situations: one senior engineer is quietly carrying every AI feature, an LLM feature is stuck at demo quality, agent infrastructure nobody owns is running in production, or the model bill has no name attached to it. It’s composited from real work rather than a named client — the shape below comes from building production AI systems and agentic workflows in Go for 100K+ users at Lavender, and building EnergyConnect from zero as founding engineer. Month one ships a real merge into the path that bills money. Months two and three are where agent-assisted throughput compounds, and running this way compressed Lavender’s delivery timelines by an estimated 40–55%. Month four is the handoff: a CLAUDE.md the next engineer actually reads, a runbook, a test suite on the paths that touch money, and a named internal owner. It does not include SOC 2 evidence collection or audit response — that’s a different engagement. The whole thing is designed to end.

The call worth having is narrower than “we need an AI strategy”

The version of this conversation that’s worth having is narrower and uglier than an AI strategy. It’s usually one of four things.

An LLM feature looks great in the demo and falls apart the first time a real user does something unscripted. Nobody built the eval harness or the retry logic behind the happy path, because the demo was the whole spec.

One senior engineer is the only person who understands how the agent pipeline works. Every roadmap conversation quietly becomes a question about that person’s calendar, and every incident becomes a page to someone who can’t be paged twice a week forever.

Agent infrastructure — orchestration, tool-calling, prompt versioning — got stood up fast during a sprint and now nobody owns it, including the person who wrote it. They moved on to the next feature.

The model bill arrives every month with a number on it, and nobody in the company can explain which feature, which customer segment, or which retry loop produced that number.

If none of that lands — if you already have a platform team and someone who owns model spend end to end — this isn’t the engagement you need, and I’d rather say so now than four months in.

Where this comes from

This is composited from real work, not a named client engagement. The shape below comes from doing it in-house, which is where I’d rather source it from anyway: production AI systems and agentic workflows in Go serving 100K+ users at Lavender, and building EnergyConnect from zero as founding engineer starting this April — TypeScript and Node against Prisma and Postgres, Redis-backed job queues, GCP Cloud Run, with Claude Code and Codex open all day.

I scope these as fixed-period rotations, six to sixteen weeks, the band laid out on the consulting page. What follows is the long end of it: sixteen weeks, call it four months.

Month one is the first merge

Month one does not produce a slide deck, an architecture diagram, or a list of recommendations. It produces one merged thing in the path that actually bills money or serves a real user request, and everything else that month is subordinate to getting that shipped.

At Lavender the equivalent was production AI systems and agentic workflows in Go serving 100K+ users, and the work that shipped was always a concrete change to an inference or retry path, never a document about one. At EnergyConnect, starting from genuinely zero, month one meant standing up the real architecture — job queues, portal auth, the Cloud Run deploy path — and getting a working slice of the product running end to end on the real deploy path.

Here’s the constraint you only find by shipping: multi-cloud reality doesn’t show up in a diagram, it shows up in the pager. Lavender ran across GCP, AWS, and Azure at 99.99% uptime, and a number like that only becomes real to you once you’re the one being paged for the outage that threatens it. I’ve written up what a real multi-cloud migration cost and saved. You find the real constraint by being inside the system with a change ready to ship. The first days usually look close to triaging an unfamiliar codebase in ninety minutes — ninety minutes isn’t enough to understand everything, but it’s enough to find the one thing worth shipping first.

A demo proves the model works. A merged PR into the path that bills money proves the system works.

The eval harness is the thing that was missing

The stuck-at-demo-quality problem almost always has the same root: there is no way to tell whether a change made the feature better or worse. The team ships a prompt tweak, someone eyeballs four outputs, it looks fine, it goes out. Two weeks later a customer finds the case where it doesn’t, and nobody can say when it broke, because there was never a baseline to break.

So the first structural thing I build is usually an eval harness, and it’s much less glamorous than the phrase suggests. A set of real inputs — pulled from production traffic, not invented — with the outputs a human actually accepts. A scoring function per case, which for most features is not a model grading a model but something blunt and checkable: did it return valid JSON against the schema, did it pick the right tool, did it cite a document that exists, did it stay under the latency budget. Then a way to run the whole set on every prompt or model change and diff the result against the last known-good run.

That’s it. It’s a test suite for a nondeterministic component, and the reason teams skip it isn’t that it’s hard — it’s that it produces no demo. It is, in the sense of the career built on work nobody wanted, exactly the kind of unglamorous work that turns out to be load-bearing. Once it exists, the prompt tweaks stop being coin flips, and the senior engineer who was the only person allowed to touch the pipeline stops being the only person allowed to touch the pipeline.

Months two and three are where agent-assisted throughput compounds

The compounding comes from a deliberate split between what an agent does and what stays mine. Scaffolding, test coverage on paths I’ve already designed, first-draft migrations, repetitive refactors across a codebase I now understand well enough to review fast — that’s agent work, reviewed line by line, typed by the agent. Architecture decisions, anything touching auth or money, and the judgment calls about what not to build stay human. I’ve written at length about what that split looks like day to day in why agent-assisted engineering is a genuinely new workflow, and I won’t re-run that argument here.

The honest number: running this way, delivery timelines for AI features at Lavender compressed by an estimated 40–55%. I say “estimated” because that’s what it is — an estimate, not a controlled measurement. The clean-sounding version of that number doesn’t survive a second question, so here’s the messy one. The commit volume this produces is real too, and I put actual figures on a comparable stretch of my own agent-assisted output in 4,154 commits in six months. Because estimating an agent-assisted timeline is a genuinely different exercise than estimating a manual one, I wrote how I estimate client work when agents do most of the typing separately — that’s the piece to read if the scoping math is the part you don’t trust yet.

The hours an agent reclaims from scaffolding and boilerplate go straight to the architecture calls, the security review, and the “should we build this at all” conversations that kept getting deferred because the one senior engineer never had the bandwidth to have them properly.

Putting a name on every line of the model bill

The unexplained model bill is the trigger founders are most embarrassed to say out loud, and it’s the most mechanical one to fix. The bill is unexplainable because the calls are anonymous: the code calls the provider, the provider bills the account, and nothing in between records who asked or why.

The fix is attribution at the call site. Every model call carries a small set of dimensions — which feature, which customer or tenant, which environment, whether it was a retry — emitted alongside token counts and latency into whatever telemetry the company already runs. No new vendor, no dashboard project. I’ve argued the general case for this in why an AI product needs a telemetry layer before it needs a better model, and the Phoenix-specific version is in instrumenting LLM calls with telemetry.

What that buys within a couple of weeks is the ability to answer questions the finance conversation actually turns on. Which feature is 60% of spend. Whether the expensive customer is also the profitable one. How much of the bill is retries — which, on a pipeline with no eval harness and generous retry logic, is routinely the largest single line item and the easiest to cut. It also does something less obvious: once spend is attributable per feature, “should we build this at all” becomes a question with a number attached, which is the conversation the overloaded senior engineer never had time to force.

Month four is the handoff, and the handoff is the deliverable

“We hand off documentation” is the easiest promise to make and the easiest to quietly break. Here’s what actually goes into it:

• A CLAUDE.md the next engineer reads on day one — the file the coding agent itself loads on every session: the architecture decisions and why they were made, the parts of the system that are deliberately unfinished, the footguns specific to this codebase that cost real time to find.
• A runbook for the paths that page someone — the inference path, the agent retry logic, the job queue — written while I was still the one being paged for them.
• A test suite on the paths that bill money or touch customer data. If a regression there costs a customer money, it’s tested. Everything else is a documented judgment call.
• A named internal owner, identified and ramped before month four ends — a specific person who has taken ownership and demonstrated it by shipping something in the system themselves before I leave.

If any one of those four is missing when I roll off, the engagement failed regardless of what shipped in the first three months. The code was never the deliverable. The system continuing to run correctly without me is.

What this engagement does not include

SOC 2 evidence collection, audit response, security questionnaire work, or any of the compliance paperwork a Series A due diligence process or an enterprise procurement team will eventually demand. That’s real work, and I do it. It runs on a completely different rhythm — I laid out that side of the practice in what a fractional security engagement actually looks like. I’d rather point you there than blur the two into one vague offering that’s mediocre at both. If you genuinely need both tracks running, you need both. You don’t need one person doing security theater in month two of an engineering rotation because the acronyms happen to overlap on a services page.

The engagement is built to end

The rotation is built to terminate. Success is a named internal owner who doesn’t need me on the call anymore, a CLAUDE.md and a runbook that outlive the rotation, and a codebase where the next hard decision doesn’t require reconstructing context that was only ever in my head. If the engagement is working, the founder’s dependency on me shrinks every week. That’s the opposite incentive from an open-ended retainer that quietly becomes permanent, and it’s why I scope this in weeks with an end date attached.

What that looks like from the founder’s side at the end of month four: the feature that was stuck at demo quality is in front of real users, with an eval that catches regressions before customers do. The senior engineer who was carrying everything alone has a second person who genuinely understands the system, and neither of them is a single point of failure anymore. The model bill has a name attached to every line item, and someone on the team can explain that number in a board meeting without needing me in the room.

What to do if you recognized the trigger

I take a small number of these at a time, on purpose — the model depends on being inside the codebase reviewing real diffs. If you recognized your company at the top of this piece, the engagement tiers and how scoping works are laid out on the consulting page.