# Jared Smith — Flagship Writing (Full Text)

> This file is the companion to https://sublimecoding.com/llms.txt — it contains
> the complete text of the flagship essays so AI assistants and
> LLM-powered tools can ingest the writing without crawling the HTML site.

Author: Jared Smith — Staff Software Engineer
Site: https://sublimecoding.com
Contact: jared@sublimecoding.com
Discovery: https://sublimecoding.com/llms.txt
Sitemap: https://sublimecoding.com/sitemap.xml

All essays below are first-person, written from real engagements at
Lavender, BlockFi, InsideTrack, AAMP Global, and PopSocial. Numbers cited
are real. Citations should link to the canonical post URL given in each
section header rather than to this aggregated file.

---

## AI Won't Shrink Your Team — It'll Expose Why You Needed a Bigger One

URL: https://sublimecoding.com/blog/ai-wont-shrink-your-team
Published: 2026-05-01
Tags: AI, engineering leadership, founders, hiring, engineering management, productivity, agents, business

**Every company rolling out AI is about to discover how much work they were leaving on the table.**

The narrative dominating board decks and all-hands slides in 2026 is some version of "AI lets us do more with less." Headcount frozen. Targeted reductions in junior engineering. Internal memos using phrases like "AI-driven productivity" to justify a leaner team. The companies leaning hardest into this story are about to make the most expensive mistake of the decade.

I've watched this play out at three companies over the last two years. The pattern is consistent. AI doesn't replace the team. It surfaces the backlog the team never had bandwidth to touch. More throughput becomes more surface area becomes more coordination, review, and decision work. The companies cutting headcount now will be outpaced inside two years by the ones quietly staffing up to absorb what AI is producing.

## The 10x engineer myth, again

The 2014 version of "10x engineer" was bullshit and most senior people knew it. The 2026 AI-flavored version is the same myth wearing new clothes.

AI makes one engineer faster — measurably, 30–50% on routine work, sometimes more on greenfield code where the agent has full context. That part is real and I've written about it extensively. What AI does *not* do is make that engineer smarter about what to build. It doesn't tell them which customer is unhappy this week. It doesn't know that the last three production incidents all traced to the same misnamed config flag. It doesn't have a point of view on whether the new feature the founder wants is going to cannibalize the one that's actually monetizing.

Speed without direction is churn at a higher RPM. The thing that actually scales an engineering organization is judgment, and judgment does not compress. The senior engineer who can look at a system and tell you which 20% of changes will cause 80% of next quarter's incidents is not a function of typing speed. They've built that intuition over years of being on call for systems they shipped, watching their decisions hit production, and updating their priors. None of that transfers to a model.

## Velocity creates surface area

This is the math most teams miss when they congratulate themselves on AI-driven speedups.

If your team is shipping 3x faster, you also have:

- 3x more PRs to review

- 3x more code paths to test

- 3x more deploys to monitor

- 3x more security review

- 3x more product decisions to make

- 3x more customer-facing changes to communicate

- 3x more documentation to keep current

- 3x more incident potential when something inevitably breaks

Every doubled velocity multiplier creates new coordination, review, and decision-making surface. The team doesn't shed work — it accumulates new categories of work it didn't have to do before. The PR backlog you used to clear by Friday now stretches into the next sprint. The on-call rotation that was tolerable at one deploy a day becomes brutal at four.

AI does not reduce this surface. It mostly creates more of it. The companies winning this transition aren't the ones with the smallest headcount. They're the ones who recognized that the bottleneck moved from "engineering capacity to ship code" to "human capacity to review, decide, and absorb," and staffed accordingly.

## The bet that's about to go badly

Several large tech companies announced 10–20% headcount reductions in 2025 and 2026, citing "AI productivity gains" as the justification. The narrative writes itself: AI lets us do more with less, so we did. Stock pops, board nods, internal memo gets shared on LinkedIn.

I think most of those companies are going to look back on these decisions in 2028 and realize what they actually did was three things, none of them strategic:

First, they let go of senior engineers — the people whose judgment was the actual force multiplier — alongside the routine roles AI did partially replace. Severance was equal-opportunity. The result is an organization where the remaining engineers have less context, less production scar tissue, and less institutional memory than the one that existed eighteen months ago.

Second, they created an organization where the remaining team is perpetually behind on review, security, and incident response because the work scaled while the team shrank. Incidents pile up. Audit findings stack. Customer escalations route to fewer people. The throughput gain is real on the input side and a debt-accrual machine on the output side.

Third — and most damaging long-term — they sent a signal to remaining staff that AI is a threat, not a tool. The teams that performed best with AI in my experience were teams that trusted that learning the new workflow wouldn't end their jobs. The teams whose leadership signaled "be productive or be replaced" got compliance-driven AI adoption: more usage, lower quality, more shortcuts, more slop.

The companies that will dominate the AI transition look exactly the opposite. Stable or growing engineering team. Heavy investment in tools, training, and the supporting roles (security, DevOps, product, design) that scale with throughput. Senior leadership communicating that AI is for amplifying the team, not replacing it. Those companies are quietly hiring while the loud ones are publicly cutting. Watch which ones are at the front of the pack in two years.

## Judgment doesn't delegate

I covered this in detail in [When to Trust an Agent and When to Step In](/blog/when-to-trust-an-agent-and-when-to-step-in). The short version: there's a category of decisions you cannot delegate to a model, and those decisions are the ones that compound into company outcomes.

- Whether the architecture is right for what you're building three years from now

- Whether the customer's problem is the one you should be solving

- Whether shipping this feature now is more valuable than fixing what shipped last quarter

- Whether the on-call engineer who keeps making the same mistake needs coaching or termination

- Whether the right approach to this bug is to fix it or to refactor the surrounding code so it can't happen again

- Whether your security posture is sufficient for the enterprise customer asking

Every one of those is a judgment call. Every one of them affects more than one team's work. None of them gets better when you have fewer experienced humans involved. AI can *support* these decisions — by surfacing data, drafting analysis, enumerating tradeoffs — but the actual call is human, and removing humans from that loop is how organizations make decisions they regret for years.

## The under-resourced trap, accelerated

There's a specific failure mode I've seen repeatedly at companies trying to brute-force output without staffing up. The shape of it:

The team ships fast for a quarter. Demos look incredible. The product feels like it's accelerating. Then the bills come due. Two production incidents take three days each to resolve because nobody had bandwidth to do post-mortem on the last incident, so the same thing breaks twice. A security audit surfaces eight findings the team has been meaning to fix for months. A customer success ticket pile reveals a 22% increase in confusion-flavored complaints — users tripping over a feature shipped without product review. A senior engineer quits because they've been on permanent escalation duty for six months and the founder keeps saying "we'll hire after this push."

AI accelerates this dynamic. Faster shipping equals faster accumulating debt when the team doesn't have the headcount to handle the supporting work. The chaos doesn't disappear when you add AI to an under-resourced organization; it compounds faster, hits earlier, and is much harder to recover from because the team is also burnt out.

The companies betting on AI as a headcount substitute are walking into this trap with their eyes closed. The companies betting on AI as a leverage multiplier — and staffing accordingly — are going to look at the wreckage in eighteen months and pick up the customers, the talent, and the market position the under-resourced bet left on the table.

## What "right-sizing" actually means in 2026

If you accept that AI raises throughput but doesn't reduce the human work needed to absorb that throughput, the right-sizing question changes shape entirely. The questions to ask, in order:

- **Which roles became more valuable because their leverage scaled with AI?** Almost always: senior engineers, engineering managers, staff-level technical leads. AI raises the floor of what one person can produce, which makes the people who can direct that production output disproportionately more valuable.

- **Which roles became more strategic because the routine parts moved to AI?** Product management, design, technical writing. The mechanical work in these roles compresses; the judgment work doesn't. Hire for the judgment.

- **Where do we have throughput gains without the corresponding humans to absorb them?** Most commonly: code review, security, DevOps, on-call. These functions scale linearly with deployment frequency, and almost no organization has staffed them ahead of the AI productivity curve.

- **Where is the team currently bottlenecked — and would adding people unblock it?** Decision-making capacity is usually the answer. Engineers waiting for review, PMs waiting for engineering input, founders making technical calls they shouldn't be making themselves. Adding senior people unblocks all of these.

The honest answer for most teams in 2026 is that they need *more* people, in *different* roles than the org chart from 2024. Not the same roles. Not "more engineers writing code." More senior engineers reviewing AI output, more security people running incident response, more PM capacity making the strategic calls AI can't, more DevOps capacity catching the deploys AI is now generating in volume.

## The takeaway

The "AI shrinks the team" narrative is going to look in 2028 the way "the cloud means we don't need ops people" looked in 2015. Wrong, expensively wrong, and obvious in retrospect.

The companies that dominate the AI transition aren't the ones that fired half their team and high-fived themselves. They're the ones who staffed up the parts of the organization that scale with throughput, kept their senior judgment intact, and recognized that one engineer plus AI is a more powerful version of one engineer — not a replacement for the team they used to need.

If you're a founder or VP of Engineering staring at a hiring freeze justified by "AI productivity," I'd push back hard. Your competitors who are still hiring are the ones you're going to be racing against in two years. The bet isn't AI vs. headcount. The bet is whether you trust your team to do more, supported, or whether you trust the model to replace what you couldn't be bothered to invest in.

I know which one I'd take.

## Read this next

- [**The Pre-Series-A AI Startup Hiring Plan**](/blog/pre-series-a-ai-startup-hiring-plan) — The hire-by-hire framework for actually staffing the way this post argues you should.

- [**AI-Assisted Engineering Isn't Faster Coding**](/blog/ai-assisted-engineering-is-a-new-workflow) — The team-level workflow change that produces the throughput this post is talking about absorbing.

- [**From One Engineer to Fifteen**](/blog/from-one-engineer-to-fifteen-engineering-leadership) — The leadership lessons that inform why "do more with less" is almost always the wrong posture.

---

## When to Trust an Agent and When to Step In

URL: https://sublimecoding.com/blog/when-to-trust-an-agent-and-when-to-step-in
Published: 2025-12-22
Tags: AI, agents, judgment, engineering, agentic, trust

**The hardest part of working with agentic AI in 2026 isn't getting the agent to do the work. It's knowing when to override it.**

An agent that produces useful code 90% of the time and confidently broken code 10% of the time is not a 90% solution. It's a system you have to be paying attention to all of the time, because the 10% of broken output looks indistinguishable from the 90% of correct output if you skim. Trust calibration is the actual engineering skill of working with agentic AI, and most teams don't yet have a framework for it.

What follows is the framework I use, in two parts. First: a four-level autonomy ladder for deciding how much trust to extend to an agent on any given task. Second: the five signals that an agent is currently going off the rails, even when its output looks fine.

## The four-level autonomy ladder

Not all tasks are equal. The same agent can be entirely trustworthy on one task and dangerous on another. The level of supervision should be set by the consequences of the agent being wrong.

### Level 1 — Read-only, always trust

Tasks where the agent observes but doesn't change anything. Code analysis, documentation generation from existing code, summarization, search. The worst-case outcome of the agent being wrong is that I get bad information that I have to discard.

I let agents work autonomously at this level all day. The downside is bounded.

### Level 2 — Bounded write, mostly trust, verify

Tasks where the agent writes code in a clearly-scoped area. Adding tests for an existing function. Implementing a small utility from a clear spec. Refactoring a single file. The blast radius is small, the work is reviewable, and the agent has a high probability of getting it right.

I review the diff before merging, but I don't read every line carefully. I'm looking for obvious smell — duplicate logic, weird naming, missed edge cases. If the diff looks clean and the tests pass, I merge. The downside if I miss something is one bad commit that's easy to revert.

### Level 3 — Real-money, auth, or state-changing, verify line by line

Tasks that touch payments, authorization, user data, or system state in ways that matter. The agent can *draft* these, but every line has to be reviewed by a human before it lands.

This is the level where most teams lose discipline first. The agent produces a plausible-looking auth migration, the diff isn't huge, the tests pass — but the migration is silently introducing a privilege escalation. I've seen this happen. I've nearly let it happen, which I'll talk about in a moment.

The discipline at this level: read every line, run the change against your own threat model, ask the agent why it made each non-obvious choice. Treat the agent's output as a junior engineer's work that needs senior review before merge.

### Level 4 — Public-facing or irreversible, do not delegate

Tasks where the cost of getting it wrong is unrecoverable. Schema migrations on production data without rollback. Sending email to your customer base. Posting to a social media account. Public-facing legal text. Press statements.

Agents do not produce these autonomously. They can *draft* them — which can be useful — but the work of actually committing to the output is human-only. The asymmetry is too sharp; even a 99% reliability rate produces an unacceptable error rate over many actions.

## Five signals an agent is going off the rails

Even within the right autonomy level, individual sessions can drift. The signals that something is off, in approximate order of how often I see them:

### 1. Confident answers without specifics

"This will work because of how the framework handles state." Without naming the function, the file, or the documented behavior. The agent is filling in plausible reasoning rather than checking. Push back: "Show me the line in the codebase that demonstrates this."

If the agent can't, the answer is suspect. About half the time the agent then says "actually, on closer inspection..." and revises. The other half it doubles down on a wrong claim, which tells you the entire reasoning chain is hallucinated.

### 2. Multiple files changed for "one fix"

You ask for a fix to a single bug. The diff comes back touching seven files. Sometimes this is correct — the bug genuinely was scattered. More often, the agent has decided the codebase needs "consistency" or "improvement" while it was in there.

The discipline: ask why each file was changed. If the answer for any file is anything other than "this was necessary for the fix," revert that file's changes. Scope drift in agent diffs accumulates fast.

### 3. "Cleaning up" code unrelated to the task

A subtype of the above. The agent removes a comment it didn't understand. Renames a variable. Reformats a function it thought was ugly. None of these are explicit instructions. All of them produce noise in the diff.

This is a hard category to police because the changes look harmless one at a time. The cumulative effect over a quarter is a codebase whose history is incomprehensible because every fix touches twenty unrelated lines.

### 4. Confidence that contradicts evidence

Tests are failing. The agent says "the implementation is correct, the tests must be wrong." This is occasionally true. It is usually wrong. The signal is the agent privileging its own reasoning over the failing test.

The fix: never let the agent dismiss a failing test without proof. "Show me which assertion in the test is incorrect and why" is the right pushback. Most of the time the agent then realizes the implementation is wrong.

### 5. Speed too high for the complexity

This one is the hardest to articulate but the most reliable in retrospect. A complex problem is solved in twelve seconds with one paragraph of explanation. Be suspicious. Real engineering problems usually have layers; an instant answer often skips them.

The discipline: when the answer comes back faster than seems reasonable, ask the agent to explicitly enumerate three alternatives and argue for the chosen one. The "argue" step surfaces whether the agent has actually thought about the problem or pattern-matched to a familiar shape.

## A concrete example: the auth refactor

Earlier this year I asked an agent to refactor a small piece of auth code. The original function had grown to 200 lines and could be reasonably split into four. Routine work, level-3 task by my own framework — auth is in the "verify line by line" category.

The agent produced a clean four-function refactor in about 90 seconds. The tests passed. The diff was the right size. On a quick read it looked correct. I almost merged it.

Two of the five signals fired before I clicked merge. Speed too high (signal 5): the refactor was elegantly factored, which is suspicious for a 200-line function with the gnarly history this one had. And confidence without specifics (signal 1): when I asked the agent why it had moved a particular permission check from one branch to another, the answer was "it's cleaner this way" rather than "the original ordering was incorrect because X."

I dug in. The "cleaner" reordering had introduced a small but real privilege escalation: in one specific code path, an authorization check that previously ran *before* a sensitive operation now ran *after*. In the test suite, no test exercised that exact path, so the tests passed. In production, the bug would have allowed certain users to perform an action they shouldn't have been able to.

The discipline that caught it was the framework above. Without it, I'd have merged a clean-looking refactor and shipped a real security bug. Multiply by every team using AI for code generation, and you can see why the slop problem is real and the trust calibration is the actual engineering work of 2026.

## The meta-discipline

The framing that pulls all of this together: *agents are tools, not teammates*. They don't have stakes. They don't get yelled at when production breaks at 3am. They have no embodied sense of what's risky. They will confidently produce code that's 99% right, and the 1% wrong will sometimes be catastrophic, and they will not know.

The job of the engineer working with agents is to supply that missing sense of stakes. The autonomy ladder is how you decide when to engage. The five signals are how you stay engaged once you're in the work.

None of this is hard. It's just disciplined. The teams that get this right are using agents at the limit of what's possible without producing slop. The teams that don't are accumulating a debt that becomes obvious only after a major incident.

Calibrate trust. Verify the consequential changes. Override when the signals fire. The framework is the discipline.

## Team-level adoption of these patterns

Individual discipline isn't enough at team scale. If only one engineer on a four-person team is calibrating agent trust carefully, the other three's slop ends up in the codebase anyway. The patterns above need to be team-level practice.

The lightest-weight version of this that's worked for me: a single shared document — call it `AGENTS.md` — that lists the team's autonomy levels, the signals to watch for, and the kinds of work that always require human review. The doc is short, written by the team together, and reviewed quarterly.

Pair it with one tactical practice: every PR description includes a single line at the top stating which autonomy level applied to the work. "Level 2 — bounded write, agent-assisted." "Level 3 — auth code, full human review." This makes the trust calibration legible during code review and surfaces drift before it ships.

The combination of the doc and the PR-line discipline takes about a week to introduce and a quarter to internalize. The teams that do this end up with a meaningfully better signal-to-noise ratio in their AI-assisted work than the teams that don't.

## Read this next

- [**My Daily Agentic AI Workflow**](/blog/my-daily-agentic-ai-workflow) — The day-to-day mechanics of running multiple agents at once.

- [**AI-Assisted Engineering Isn't Faster Coding**](/blog/ai-assisted-engineering-is-a-new-workflow) — The team-level discipline that complements the individual one above.

- [**How I'd Run Security at an AI-Native Company in 2026**](/blog/running-security-at-an-ai-native-company-2026) — Where agent-trust meets the production security threat model.

---

## How I Triage a New Codebase in 90 Minutes

URL: https://sublimecoding.com/blog/triage-a-new-codebase-90-minutes
Published: 2025-12-08
Tags: engineering, code review, onboarding, fractional, staff engineer

**A fractional engineering engagement starts with a codebase you've never seen. You have ninety minutes to form a useful POV before the kickoff call.**

This is a recurring situation for me as a fractional engineer. A founder books a discovery call, gives me read-only access to their repo on Tuesday, and the kickoff call is Wednesday morning. Between those two events, I need to know enough about the system to ask intelligent questions, identify the load-bearing risks, and not waste the founder's time with surface-level observations they could have written themselves.

The framing that makes this work: *not* "understand the codebase." That takes weeks. Instead — *find the load-bearing risks*. The seven-step triage I run, in order.

## Step 1: README.md and any architecture docs (10 minutes)

Start at the front door. The README tells me three things almost immediately: how the team thinks, how recent the project's intentional documentation is, and whether the founders or engineers wrote it.

Signals to look for: when was this last meaningfully updated? Does the "how to run locally" section reference real commands or stale ones? Are there architecture diagrams, ADRs, or design docs anywhere in the repo? An empty `docs/` directory and a README that ends with "TODO: write more about this" tells you a great deal about engineering culture before you've read a single line of code.

I copy the relevant facts into my own running notes. The first paragraph of those notes is "what the team thinks the system does."

## Step 2: Dependency audit (5 minutes)

Open `package.json`, `mix.exs`, `requirements.txt`, `go.mod`, or whatever the equivalent is for the language. Two questions: what major frameworks are in use, and how out of date is everything?

The dependency list is a faster summary of the system's architecture than reading the architecture docs. If I see `phoenix_live_view` + `oban` + `ecto`, I know the shape of the app. If I see thirty random utility libraries and no obvious framework, I know there's been turnover or a lack of opinionated leadership.

For the freshness check: a dependency that's two majors behind isn't automatically bad, but a codebase where *every* major dependency is two-plus versions behind is a maintenance bomb in the making. Make a note.

## Step 3: The git log story (10 minutes)

Run `git log --oneline --all -200`. Skim the last two hundred commit messages.

What you're looking for: who's committing, what they're committing, and what the rhythm looks like. A repo where one engineer wrote 90% of the recent commits is a key-person-risk situation. A repo where the commits are mostly "wip" and "fix typo" tells you about the team's commit hygiene. A repo where every PR has a clear, conventional-commit-style message tells you the team has invested in process.

The signal that matters most for triage: pattern frequencies. Lots of "revert" commits in the recent past = unstable changes. Lots of "fix" commits referencing one specific module = that module is troubled. Use the patterns to direct what you read next.

## Step 4: Test coverage and CI signal (10 minutes)

Find the test directory. Count files. Find the CI configuration. Read it. Run the test suite if you can — does it pass? How long does it take?

You're looking for three signals:

- **Volume.** Is there one test file or two hundred?

- **Quality.** Open three random test files. Do they test behavior or just exercise code?

- **CI status.** Is the build green? Has it been red for more than 24 hours? Is there a culture of merging on red?

The test suite is the closest thing a codebase has to a self-portrait. A team that's invested in test quality is a team that takes engineering seriously. A team where the tests don't pass on a fresh checkout is a team that has bigger problems than the ones the founder is going to tell you about.

## Step 5: The hottest files (10 minutes)

Run a quick analysis to find the most-changed files in the last six months. `git log --pretty=format: --name-only --since="6 months ago" | sort | uniq -c | sort -rg | head -20`. The top of that list is where the action is.

Open the top three or four files. Read them. These are the load-bearing parts of the system, by definition — they're where the team is spending their engineering energy.

What you're looking for: are these files long, complicated, and full of inline comments saying "TODO: this is a hack"? Or are they crisp, well-factored, and recently refactored? The hot files tell you where the system is fragile and where it's healthy. They're also the files that an incoming engineer (or fractional engagement) will most likely need to touch first.

## Step 6: The auth and data layer (15 minutes)

Now the deep dive. Find the authentication code. Find the database schema. Read both carefully.

Authentication: how does a user log in? Where are sessions stored? Is there MFA? Are passwords hashed with a current algorithm? Is there an obvious authorization layer beyond authentication? This is where I find the highest-severity bugs in early-stage codebases — usually authorization issues that the team hasn't yet noticed because they haven't been exploited.

Data layer: what tables exist, what relationships do they have, are there migrations that suggest the schema has been refactored, are the indexes sensible? The schema is the contract the system runs under. Anything wrong with it is wrong with everything else.

This is the longest step in the triage by design. If I'm going to find a deal-breaker risk, it's almost always in this section.

## Step 7: The most recent incident (10 minutes)

Ask the founder (or look in the team Slack history if you have access): when was the last production incident? What broke? How was it fixed?

The incident report — written or verbal — tells you more about the engineering culture than any single artifact in the repo. A team that has a clear retro doc with five action items, three of which were completed, is a team that learns. A team where "the last incident" is met with "uh, well, last week the database fell over for an hour, I think someone restarted it" is a team that doesn't.

This step also surfaces the thing the founder is most worried about. They'll volunteer it once they sense you're asking real questions.

## What you write down at the end

Ninety minutes of triage produces a one-page document with the following sections:

- **What the team thinks the system does** (one paragraph, from step 1)

- **The shape of the stack** (three lines, from step 2)

- **Engineering rhythm** (commit frequency, team size, hot spots — from steps 3 and 5)

- **Quality signals** (test coverage, CI, code-review hygiene — from step 4)

- **The two highest-severity risks I found** (from steps 6 and 7)

- **The two questions I'm bringing back to the founder**

The two questions are the most important output. They're how you signal to the founder that you've done real work in 90 minutes and have a useful POV. They're also the questions whose answers will reshape everything you do in the engagement.

Examples I've actually used: *"Is the lack of audit logging on the admin endpoints intentional?"* Or: *"Looking at the last six months, your messaging service has been the source of three of the four production incidents — what do you and the team think is going on there?"*

## The AI-assisted version

The 90-minute number above predates Claude Code. The same triage now takes about half that time with AI assistance.

The pattern: I run the seven steps in parallel using a Claude Code session pointed at the repo. Step 1, 2, 3, 4, and 5 are largely automatable — I tell Claude what to look for and it produces structured summaries faster than I can read the raw files. I spend the saved time on steps 6 and 7, which still benefit from human attention.

Net result: the same depth of triage in 45 minutes instead of 90, with substantially better notes because the AI captures things I would have skimmed. The disciplined human still drives. The agent accelerates.

## The takeaway

The skill being practiced here is not "reading code fast." It's *knowing what to look at first.* Most engineers, on entering an unfamiliar codebase, dive into the part of the system most relevant to their immediate task and form a partial picture. The triage protocol forces you to look at the system in the order that surfaces risks, not in the order that matches your task.

Useful for fractional engineers. Useful for new hires in their first day. Useful for anyone who's about to take on technical responsibility for a codebase they didn't write. Run the protocol. Take the notes. The 90 minutes will save you weeks downstream.

## What you say on the kickoff call

One last note. The triage produces a written one-pager, but the founder is going to want to talk through it on the kickoff call. The framing that lands: lead with what you saw that's working, then name the two highest-severity risks, then ask the two questions.

This sequence is deliberate. Founders are sensitive about their codebase — many haven't had an outsider look at it in years. Leading with criticism puts them on defense and shrinks the conversation. Leading with what's working buys you the credibility to then talk about what isn't.

The two questions you ask are the most important moment of the call. They signal you've done real work, not just listed problems, and they invite the founder into a conversation about priorities rather than a lecture about deficiencies. Done right, the kickoff call ends with the founder saying some version of "let's start with those two things you flagged" — which is exactly the engagement you wanted to land.

## Read this next

- [**The 'Smallest Possible Slice' Heuristic**](/blog/smallest-possible-slice-shipping-complex-features) — The same pragmatic-decomposition mindset applied to feature work instead of code review.

- [**Migrating 225K Users from AWS Cognito to Auth0**](/blog/aws-cognito-to-auth0-migration-without-forcing-logout) — A real example of step 6 (auth-layer scrutiny) playing out in production.

- [**AI-Assisted Engineering Isn't Faster Coding**](/blog/ai-assisted-engineering-is-a-new-workflow) — How the AI-assisted version of triage scales to other engineering tasks.

---

## My Daily Agentic AI Workflow

URL: https://sublimecoding.com/blog/my-daily-agentic-ai-workflow
Published: 2025-11-24
Tags: AI, agents, Claude Code, OpenAI Codex, productivity, engineering workflow

**I run four to seven agent sessions in parallel through a normal engineering day. Here's what they do, what they don't, and how I keep the work coherent.**

The defining shift in engineering work over the past two years isn't that AI writes code faster. It's that you can have multiple agents working on multiple things at the same time, and your job moves from "writing code" to "directing work." This is qualitatively different from autocomplete, copilot-style assistance, or any prior way of using AI in development.

What follows is a walkthrough of a typical engineering day for me as of late 2025, running on Claude Code as the primary tool, OpenAI Codex for shell-shaped tasks, and a few custom agents wired into Slack and the command line. The point isn't the tools — pick whichever you like. The point is the patterns.

## The core idea

Agents take work off your plate but still need you in the loop. The trap most teams fall into: treating agents as fire-and-forget background workers. The result is generated code that compiles, looks reasonable, and is subtly wrong in ways the agent has no way to detect on its own.

The mental model I use: an agent is a fast, talented junior engineer with infinite patience and zero context outside what you give them. Your job is to give them the context, scope the work tightly, and review the output before it goes anywhere production-shaped.

## Walkthrough of a typical day

### Morning: 2 background agents on long tasks

Before I sit down at my desk, I usually have two agent sessions running. The pattern: tasks that take a long time, don't need real-time feedback, and have well-defined success criteria.

Examples from a recent week:

- "Audit our codebase for places we're calling external APIs without retry logic, and produce a markdown report with recommendations."

- "Read the last fifty PRs merged to main, identify recurring code-review feedback themes, and write an internal style-guide draft based on them."

- "Generate test cases for the authentication module covering all edge cases listed in the threat model document at `docs/auth-threats.md`."

I kick these off, walk away, and come back to a draft in 30–60 minutes. Critically, the output is always a *draft*. I read it, I edit it, I push the parts I trust into the codebase. The agent never commits unsupervised.

### Mid-morning: foreground pair-programming with one agent

This is the bulk of my actual coding. I open a fresh agent session for the hardest problem of the day and we work it together — me driving, the agent acting as a peer. The interaction is conversational, not delegating.

Concretely: I describe the problem, the agent asks clarifying questions, we sketch an approach, I write some code, the agent reviews, I push back on suggestions I don't like, we iterate. By the time the function is committed, both of us have looked at every line.

The mistake to avoid in this mode: letting the agent write the code while you watch. That's still autocomplete, just slightly fancier. The point of pair-programming with an agent is that *you* are still doing the engineering — the agent is checking your work, raising things you might miss, and accelerating the parts that don't require taste.

### Afternoon: review-mode agents on PRs

By afternoon I've usually written or merged some code, and I have PRs to review (mine and the team's). I run a review-mode agent on each PR before I read it myself.

The instruction is consistent: *"Review this diff like an adversarial senior engineer who hates my work. Find bugs, race conditions, security issues, and unclear naming. Don't be polite."* The agent produces a list. I read the list, dismiss the noise (typically 60–70%), and the rest becomes my review comments — credited to me, of course, but with the agent doing the first pass.

The leverage here is significant. The agent catches a real bug or smell about 30% of the time. The other 70% is dismissable noise that I'd have generated mentally anyway. Net: I write better PR reviews in less time, and my human reviewers catch things they otherwise wouldn't have.

### End-of-day: ops agents on deploy + summary

Late in the day, two more agents come into play.

The deploy agent: a custom Slack bot wired to our deployment pipeline. I tell it "deploy main to staging" or "deploy 4f3a2b1 to production behind feature flag `ai_v2`" and it executes the relevant commands, watches the deploy, and reports back. It does not, ever, have permission to do production deploys without a confirmation. But staging deploys, log queries, and rollback prep — yes, autonomously.

The summary agent: at end-of-day it reads the day's commits, the day's PR comments, the day's Slack threads in our team channel, and produces a one-paragraph "what happened today" summary. Useful for me; useful for async teammates; surprisingly useful when I come back on Monday to remember what we were working on Friday.

## The three interaction modes

Boil all of the above down and there are really three modes I use agents in. Each one has a different signature.

- **Delegate.** Long-running task, well-defined output, light supervision. Background mode. The success criterion is whether the deliverable is useful when I come back to it.

- **Collaborate.** Real-time pair programming. The success criterion is whether the code I commit at the end is meaningfully better than what I'd have written alone.

- **Verify.** Adversarial review of my work or the team's. The success criterion is whether real bugs get caught before they ship.

The mistake teams most often make is using the wrong mode. Trying to delegate something that actually needs collaboration produces unusable code. Trying to collaborate when verify-mode is what's needed produces echo-chamber agreement instead of real review. Pick the mode deliberately.

## The handoff protocol

The single discipline that keeps multi-agent work coherent: explicit handoff context between sessions. When one agent's work feeds into another's, you don't trust them to figure it out. You write a one-paragraph context dump and paste it into the next session.

For example: morning audit agent produces a list of 14 places where retry logic is missing. I review the list, decide which 6 are worth fixing, and write a paragraph: "We're going to add retry logic to these 6 functions: [list]. Use exponential backoff with jitter, max 3 retries, log each retry at `warn` level. Match the pattern in `lib/external/retry.ex`." That paragraph goes into a fresh agent session for the implementation work.

The agent doesn't see the original audit. It sees the curated context. This is the core of working with agents at scale: *you* are the context router, deciding what each session needs to know.

## The trap that produces slop

Most teams who report disappointing results from agentic AI are running into the same failure mode: agents that look productive but produce slop. The signature: lots of code is committed, the team feels productive, and three weeks later the production codebase is a mess of subtly broken patterns nobody can fully explain.

The cause is almost always one of three:

- **No review discipline.** Agent-generated code is going into the repo without a human pass.

- **Mode mixing.** Delegate-mode work being treated as collaborate-mode by the team, so nobody is closely engaged with the output.

- **Context starvation.** Agents being asked to do work without enough context to do it well, producing plausible-but-wrong code.

All three are solvable. None of them are solved by "use a better model." They're solved by team-level discipline about how AI work enters the codebase. Without that discipline, more agents produces more slop. With it, the throughput gain is real and durable.

## The takeaway

Agentic AI is a force multiplier in engineering when treated as a workflow change rather than a tool swap. Four to seven sessions in parallel sounds like a lot until you recognize that most of them are running asynchronously while you do other work — and that your role across all of them is the same: provide context, scope tightly, review carefully.

The teams shipping 40–55% faster aren't typing more. They're directing more. That's the new bar. Most engineers will get there in 2026. The ones who get there first will have a meaningful, compounding advantage for the next two or three years before everyone catches up.

## Tooling and cost

The economics of running 4–7 agent sessions a day are easy to get wrong. A few practical notes:

- **Pay for the paid tier.** Free-tier rate limits will produce flow-state interruptions multiple times a day. The $200–600 per month for unlimited Claude Code, ChatGPT Pro, and Codex usage is the highest-ROI line item on your engineering bill at this stage.

- **Don't run the same task in two tools "for comparison."** Pick one. Comparison runs sound disciplined and produce cognitive overhead that erodes the throughput gain.

- **Track cost per outcome, not cost per session.** A $4 agent session that produces a working feature is cheaper than a $0.40 session that produces noise. Most teams track the wrong number.

- **Agent context is the expensive resource, not tokens.** Spend more time on writing good prompts and feeding the right files. Don't optimize for shorter prompts; optimize for clearer ones.

One closing note on team adoption. The workflow above is what I run as an individual. Scaling it to a team adds a coordination problem — multiple engineers spawning agent sessions in shared codebases, occasionally producing conflicting changes if not careful. The pattern that's worked: agents work on isolated branches, never main, and the human engineer is the one merging up. Treat agent sessions like junior engineers' branches, with the same review discipline.

## Read this next

- [**AI-Assisted Engineering Isn't Faster Coding**](/blog/ai-assisted-engineering-is-a-new-workflow) — The team-level framing of the same workflow.

- [**When to Trust an Agent and When to Step In**](/blog/when-to-trust-an-agent-and-when-to-step-in) — The discernment side of the discipline above.

- [**How I'd Hire a Staff Engineer at an AI Startup**](/blog/how-id-hire-a-staff-engineer-at-an-ai-startup) — The AI-fluency bar I'd hire against.

---

## The 'Smallest Possible Slice' Heuristic for Shipping Complex Features

URL: https://sublimecoding.com/blog/smallest-possible-slice-shipping-complex-features
Published: 2025-11-10
Tags: engineering, agile, scoping, decomposition, shipping

**Most "we delivered late" stories trace to one decision: the team scoped the first slice too big.**

The team breaks down the feature into "frontend, backend, database changes, the new microservice." They estimate two weeks. Six weeks later, the database changes are merged but the backend isn't, the frontend is half-built, the microservice is on a feature branch that's now 800 commits behind main, and nobody can demo anything because nothing actually works end to end yet.

This pattern is so common it's worth giving it a name. Most teams break work down by *layer* when they should break it down by *slice*. The corrective: a heuristic I've leaned on for the better part of a decade — the smallest possible slice that touches every layer.

## Why "MVP" is too vague to fix this

"Just ship the MVP" is good advice in spirit and useless in practice. MVP gets defined by stakeholder negotiation: marketing wants this, product wants that, engineering says we can have either of two things by date X. The result is a feature that's sized to "what fits in a sprint" rather than to "what actually works end to end."

The smallest-possible-slice heuristic is more specific. The first version of any non-trivial feature should be defined by these three constraints, in order:

- It touches every layer the final feature will touch — frontend, backend, persistence, infrastructure.

- It works end to end for one trivially specific case.

- It can be deployed to production by Friday.

Notice what's not on the list: completeness, polish, edge cases, performance. Those come later. The first slice exists to prove the wiring is correct.

## The vertical-cut rule

Most teams default to *horizontal* decomposition. Sprint 1: schema changes and migrations. Sprint 2: backend API. Sprint 3: frontend. Sprint 4: integration. Each sprint produces something, but nothing is testable in production until sprint 4.

The vertical cut runs the other way. Sprint 1: the simplest possible end-to-end version. The schema has one table with three columns. The backend has one endpoint that returns a hard-coded response if the input matches. The frontend has one input and one output. It works for exactly one case. Ship it.

The reason the vertical cut wins: every sprint after the first is now *additive*. You're adding cases, polishing UI, expanding scope — but the core wiring is already proven. When the inevitable scope cut comes (and it always comes), you have shipping software to ship instead of four feature branches to merge.

## The deploy-by-Friday filter

The cleanest forcing function for the smallest-possible-slice mindset is a question I ask the team every time we scope a new feature: *can we get the first version into production by Friday?*

If the answer is "yes, but only the schema changes," the slice is too horizontal. Throw it out. Re-scope.

If the answer is "yes, with one customer behind a feature flag, working for the simplest case," that's the right slice. Ship that, see what breaks in production, then iterate.

If the answer is "no, even the first slice will take three weeks," the work is genuinely large and you need to break it down into smaller features, not smaller slices of one feature. Bigger problem, different conversation.

The reason the Friday filter works is that it forces the team to find the small case. Most engineers are uncomfortable shipping something obviously incomplete. The Friday deadline overrides that instinct just enough to get the first slice out the door, and once it's out, the team's attitude shifts from "what should we build" to "what should we add next."

## A concrete example

At Lavender, we shipped a new AI feature that recommended improvements to user-written sales emails. The first scoping pass from the team came back as a four-week project: prompt engineering work, a new evaluation pipeline, a UI component for inline suggestions, an analytics dashboard, an A/B testing harness.

I rejected it. Re-scoped to: one user, one email, one suggestion type, one model call, one button. The button shows up on a hardcoded email body for one specific user account. Click it, get a suggestion, render it. No analytics, no A/B, no eval pipeline.

That version shipped on the fourth day. It was visibly thin. It also *worked*, which we hadn't yet proven the original four-week design would. Over the next two weeks we extended it to all users, added the suggestion-type variations, layered in the eval pipeline, and turned on the A/B harness. The full feature was in production at three weeks instead of four, and we caught two architectural issues during the first week — issues that would have been brutal to fix at the four-week mark with everything already integrated.

Total elapsed: roughly the same. Total risk: dramatically lower. The "smaller" first slice was paradoxically the faster path.

## When the heuristic breaks

The smallest-possible-slice rule fails for one specific category of work: *research-shaped problems*. Things where you don't yet know what the right answer is, only that there's a question.

Examples: training a custom model from scratch, designing a novel cryptographic protocol, exploring whether a certain optimization is even possible. You can't ship a thin vertical slice of a question. You have to do the research first.

The way I handle these: time-box the research as a separate phase, with a defined exit condition ("by week three we will have a written go/no-go decision on training a custom embedding model"). Once the research phase exits, the implementation phase reverts to vertical slicing.

The mistake to avoid: dressing up the research phase as a feature build. If you're scoping "build an MVP of the new model" when the real question is "is custom training viable for our use case at all," the team is going to drift. Be honest about which kind of work you're doing.

## The tactical checklist

When a team comes to me with a feature breakdown, I run it through five questions:

- Does the first slice touch every layer the final feature will touch?

- Can it be deployed to production by Friday?

- Is there at least one user (or test account) who will see something different on Monday?

- If we shipped only this slice and nothing else, would it be embarrassing but functional, or non-functional?

- What's the explicit list of what's *not* in the first slice?

The fifth question is the one most teams skip and the one that prevents scope creep most reliably. Writing down what's not in the first slice locks the team into the discipline of shipping something thin. Without it, "just one more thing" creep extends the slice by 50% before code is written.

## The takeaway

The most reliable way to ship complex features fast is to ship a thin one first and grow it. Most teams know this in principle. Most teams violate it in practice because the first thin slice always feels embarrassingly incomplete.

The discipline is sitting with the embarrassment. Ship the thin slice. Watch it work in production. Then add the next slice. Repeat for as long as the work is generative.

The teams that internalize this ship faster than the teams that try to scope the whole feature up front, every time. The teams that try to predict everything at sprint planning and then deliver in one big bang ship slower and ship buggier. The slice heuristic is what separates these two patterns. Use it.

## Convincing a skeptical team

The slice heuristic is intellectually obvious and culturally hard. Engineers who've spent years on teams that scope full features upfront will resist shipping the embarrassingly thin first version. The objections are predictable.

"*It's not ready for users.*" Right — that's why it's behind a feature flag with one allowlisted account. Nobody will see it except the team.

"*We're going to have to throw away this code when we build the real version.*" Maybe. But the discarded code is rarely the expensive part. The expensive part is the architectural learning, and that's preserved regardless of whether you keep the code.

"*The product team will think we're shipping garbage.*" Get product in the room when you scope the slice. Show them that the next slice ships next week. The thin version stops being garbage when it's framed as a milestone, not a shipped feature.

The way I get past this with new teams is to show, not tell. Pick a feature, scope it the team's preferred way, then re-scope it as a thin slice. Walk through both timelines on a whiteboard. The thin slice almost always wins on calendar time, even when the team initially said it would take longer. After one or two demos of this, the team converts itself.

## The retro question that locks the habit in

One question, asked in every retrospective for a quarter: *could we have shipped a smaller first slice of the work we did this sprint?*

The answer is yes more often than the team initially thinks. Asking the question consistently trains the muscle. After a quarter of asking, the team starts asking it during sprint planning instead of in retro — and that's when the heuristic has actually been internalized.

One additional pattern: the slice heuristic compounds with continuous deployment. If your team can ship to production multiple times a day, slicing becomes the default mode of operation rather than a discipline you have to remember. If your deploys are weekly or batched, slicing requires constant re-justification because each "small slice" feels like a wasted deploy slot. The teams that internalize the heuristic fastest are usually the teams that already have the deploy infrastructure to support it. If your team doesn't, fix that first — the slice discipline rests on it.

## Read this next

- [**AI-Assisted Engineering Isn't Faster Coding**](/blog/ai-assisted-engineering-is-a-new-workflow) — How modern engineering tooling shortens the slice cycle even further.

- [**How I Triage a New Codebase in 90 Minutes**](/blog/triage-a-new-codebase-90-minutes) — The pragmatic discipline applied to onboarding instead of feature work.

- [**From One Engineer to Fifteen**](/blog/from-one-engineer-to-fifteen-engineering-leadership) — Why slice discipline is partly a leadership problem, not just an engineering one.

---

## How to Manage a 4-Person Engineering Team Without Becoming a Manager

URL: https://sublimecoding.com/blog/managing-a-four-person-engineering-team
Published: 2025-10-27
Tags: leadership, small teams, engineering management, founders, rituals

**A 4-person engineering team is the most overlooked unit of management in startups.**

Big enough that the tech lead can't write all the production code. Small enough that hiring an EM kills velocity and adds a layer of communication overhead the team can feel within a week. Most startup engineering org charts skip from "founding engineer" straight to "Director of Engineering at twenty people" — and pretend the territory in between doesn't have its own playbook.

It does. I've run 3–5 person engineering teams at PopSocial, at InsideTrack during a phase transition, and most recently across two fractional engagements with AI-native startups. The patterns hold. Five rituals that work at this size, three traps to avoid, and the signal that tells you it's time to evolve.

## Why 4 is the awkward number

One engineer is a co-founder. Two is a duo. Three is a tight team where everyone communicates by osmosis. By four, osmosis breaks. Engineer A doesn't know what engineer C is working on, two of them ship overlapping changes that have to be reconciled in code review, and the lead starts feeling like they're spending half the day on coordination that didn't exist last quarter.

This is the moment most founders panic and either hire an EM or default to the worst option: announce themselves as the EM and stop coding. Both miss the point. At four engineers, you don't need a manager. You need *cadence*. Four people running on shared rhythms perform like five-and-a-half. Four people without rhythm perform like three.

## The five rituals that work at this size

### 1. Weekly 1-on-1s, 30 minutes

The single highest-leverage 30 minutes on your calendar. Three sections, in this order: what's blocking you, what's bothering you, what are you working on. The first two are non-negotiable; the third is often obvious from context and can be skipped.

Why this works at 4 engineers specifically: the alternative is finding out your strongest engineer is unhappy via their resignation email. At 20 engineers, you need a layer of management to surface this. At 4, the layer is you, weekly, in 30 minutes.

### 2. Async daily check-in, in Slack, text only

Three lines per engineer per day, in a single thread. What I shipped yesterday. What I'm shipping today. What's blocking me. No video. No meeting. Twenty seconds to write, two minutes for the team to read.

The crucial constraint: *text only*. Standups by video at small scale are a tax. The same information conveyed in text takes a tenth of the time and creates a written record you can search later. The team I most recently ran did standups this way for a year and never once felt the lack of synchronous time.

### 3. Friday demos, 15 minutes

Once a week, on Friday, the team gathers for fifteen minutes. Each engineer shows the most interesting thing they shipped that week. Could be a feature, a refactor, a bug fix with a great post-mortem, anything they're proud of.

The point is not status reporting. The point is *visibility* — engineers seeing each other's work, picking up patterns, noticing where someone has built something useful that another engineer didn't know existed. At 4 engineers, this is also the highest-bandwidth moment of pure team identity in the week.

### 4. Quarterly written goal docs, one per engineer

Twice a year is too infrequent at startup pace; weekly is theatre. Quarterly hits the right rhythm. One page per engineer: three goals, written by them, reviewed with you, signed by both at the end of the meeting.

Three goals, not five. They span the three buckets of work: ship something hard, learn something specific, contribute to the team in a defined way. At the end of the quarter you sit back down, look at the doc, and have a real conversation about what happened. This is how you compound performance at small scale without bureaucracy.

### 5. Monthly retros: what's broken

One hour, last Friday of the month, no calendar invite needed beyond the recurring slot. Single question: *what's broken about how we work?* Not what's broken in the code. What's broken in the process, the tools, the meetings, the deploys, the comms.

You take notes. The next week, you fix the one thing the team most wanted fixed. The team feels heard, the process actually improves, and the next month's retro builds on a smaller list of complaints. After six months of this, the team's operating practices are visibly better than every other 4-person team you'll ever see.

## Three traps to avoid

**Trap 1: Hiring an engineering manager at four engineers.** The marginal cost is enormous and the marginal value is small. The EM at this scale will inevitably either become a tech lead in disguise (in which case you should have just promoted internally) or spend their day creating process to justify their own existence. Wait until six engineers. Possibly eight.

**Trap 2: The lead going full-time managing.** This is the most common pre-Series-A founder mistake. You stop shipping code, the team's velocity drops 30% within a month because the senior IC just left the keyboard, and you spend the dropped time on meetings the team didn't want anyway. The right load at this stage is 70/30 IC/management. Below 50/50 management, you're failing the team.

**Trap 3: Rituals that drift into status meetings.** Every ritual on the list above can collapse into "tell me what you're working on so I know" if you're not careful. The signal a ritual has drifted: the engineers stop volunteering things, you start asking direct questions, and the meeting feels heavy. When that happens, kill the meeting that week. Bring it back next week with explicit re-framing.

## The transition signal at six engineers

The model in this post breaks somewhere between five and seven engineers. The exact number depends on team-shape and product-shape but the symptoms are consistent: 1-on-1s start eating your whole Tuesday, the async standup thread is too long to read in two minutes, and the Friday demo runs over because four-out-of-six demos already feels rushed.

The transition is to a sub-team structure: two ICs, two ICs, with a tech lead per group. You're now managing the leads, not the ICs. Different rituals, different cadence, different challenges — and a whole different post.

The good news: getting the 4-person ritual stack right is the foundation everything that follows builds on. The teams that hit Series A with healthy engineering culture are the teams that ran disciplined rituals at four. The ones who skipped the discipline at small scale are the ones spending the year after Series A re-installing it under pressure.

## Hiring at the 4-person team size

One topic the rituals don't cover: every hire at this size is a culture-level decision, not just a skill match. At twenty engineers a single wrong hire is a 5% problem. At four, the wrong hire is a 25% problem and you'll feel the consequences inside six weeks.

The implication: the bar for hires 3, 4, and 5 should be unreasonably high. Not "great engineer." Not "really senior." It needs to be: "this person makes the team meaningfully better the day they start, in ways the existing engineers couldn't have produced themselves." Anything less and you're hiring someone who needs to be brought up to the existing team's level, which the existing team will resent at this scale.

Concretely, what's worked for me at this size:

- **Every existing engineer is a hard veto.** No exceptions. If any of the three engineers on the team has a strong reservation about a candidate, the candidate doesn't get hired, regardless of what the founder thinks. This kills political pressure and keeps the team's culture in their own hands.

- **Take-home → trial week → offer.** A 3-day paid trial week between final interview and offer is the most predictive single signal at this scale. You're not hiring for resume; you're hiring for whether they fit the team's flow.

- **The fifth hire is the one to slow down on.** Going from four to five is when team dynamics noticeably shift. Most founders rush this hire. The team performs better at four for an extra month than at five with the wrong addition.

## Remote vs in-person at this size

Brief note because the rituals work differently in each setup. Remote 4-person teams need *more* structure, not less — the async daily check-in becomes load-bearing instead of nice-to-have, and the Friday demo is the single thing that holds team cohesion together. Skip the demo for two weeks in a row and you'll feel the drift.

In-person 4-person teams can run looser. The async standup can be optional because the team is already overhearing each other's progress. The Friday demo is still worth doing but it has less work to carry.

Hybrid is the worst of both worlds at this size. If you can choose one or the other, choose. If you can't, default to remote-first rituals — they degrade gracefully when half the team is in the room and the rest are remote. In-person-first rituals don't.

One last note on cadence. Founders sometimes ask whether all five rituals run from week one with a new hire. The answer is yes — the new engineer joins the existing rhythm rather than the team adapting to them. New hires actually onboard *faster* when they're plugged into a working ritual stack on day one, because the rituals make the team's expectations legible. Skip the rituals during onboarding and the new hire spends three weeks figuring out norms that should have been transmitted in week one.

## Read this next

- [**From One Engineer to Fifteen**](/blog/from-one-engineer-to-fifteen-engineering-leadership) — Where the 4-person rituals fit in the broader leadership arc.

- [**The Pre-Series-A AI Startup Hiring Plan**](/blog/pre-series-a-ai-startup-hiring-plan) — Who you hire to build the 4-person team in the first place.

- [**How I'd Hire a Staff Engineer at an AI Startup**](/blog/how-id-hire-a-staff-engineer-at-an-ai-startup) — The single hire that levels up the team you already have.

---

## How I'd Hire a Staff Engineer at an AI Startup

URL: https://sublimecoding.com/blog/how-id-hire-a-staff-engineer-at-an-ai-startup
Published: 2026-02-23
Tags: hiring, staff engineer, AI startups, engineering leadership, interviewing

**The title "Staff Engineer" means three different things at three different companies. At an AI startup pre-Series-A, only one of those three is what you actually need.**

I've been on both sides of the staff-engineer interview, hiring for the role at Lavender and BlockFi, and being interviewed for it more times than I want to count. The pattern I see most consistently in misfires: the company hires a staff engineer who's calibrated for a different flavor of "staff" than what the company actually needs, and either the engineer leaves within twelve months or the team works around them.

If you're hiring a staff engineer at an AI startup pre-Series-A, here's the interview process I'd run, and the calibration I'd hold to.

## The three flavors of staff engineer

The title is overloaded. The three distinct shapes:

- **The systems architect.** Designs platforms, sets technical direction across teams, owns the architectural roadmap. Often doesn't write much code. Strongest at large companies with multi-team coordination problems.

- **The principal IC.** Writes the hardest code on the team, owns the most-load-bearing parts of the codebase, mentors senior engineers. Hands-on. The "tech lead, but better."

- **The deep specialist.** Single-domain expert — distributed systems, ML infra, cryptography, real-time graphics. The team needs them when the problem requires their specific expertise; otherwise they're slotted into general work and underperform.

The flavor an AI startup pre-Series-A needs is **flavor 2: the principal IC.** You don't have multi-team coordination problems yet. You don't have a single domain so deep that a specialist is required. What you have is a small team that needs someone who can hold the entire codebase in their head, take the hardest features, and pull the senior engineers up.

If a candidate's resume reads like flavor 1 (systems architect) or flavor 3 (deep specialist), they're not wrong as engineers. They're wrong for this role. Calibrate the funnel for principal-IC type and reject hard against the others, even if their pedigree is impressive.

## The screen: not LeetCode

The first 30 minutes with a staff candidate should not be a coding question. By the time someone has reached staff level, you can confirm they can code through their work history, their code samples, and the take-home. The 30-minute screen is for two questions:

- **Walk me through the most technically difficult thing you've shipped.** Listen for: depth of ownership, awareness of tradeoffs, ability to talk about failure modes, presence or absence of grandiosity. A staff engineer should have at least one or two "I owned this end-to-end and here's where it nearly went wrong" stories.

- **What would you change about how engineering operates at the last company you worked at?** Listen for: opinion, not complaint. A staff engineer should have a clear, articulated point of view about engineering practice. If they don't, they're senior, not staff.

If both answers are strong, move them to the take-home. If either is weak, decline immediately. Do not waste your team's interview hours on a candidate who can't pass these two.

## The take-home: should it exist? Yes. With caveats.

Take-homes are controversial. The argument against: they're disrespectful of senior candidates' time, the signal is noisy, and the strongest candidates won't do them.

I've heard all the arguments and I still believe in take-homes for staff hires, with three constraints:

- **Two hours, hard cap.** If a candidate puts in eight hours, you're getting eight-hour signal — useless for calibrating against the actual job, where they'll have less time.

- **Realistic problem, not algorithmic puzzle.** Build a small CLI tool that solves a real product problem. Wire two real APIs together. Implement a small retry policy with backoff. Things you'd actually ask them to do in week one.

- **Pay for the candidate's time.** Not a lot — $200 for two hours. Sends the message that you respect their time, and it filters out candidates who have so many options they'd rather not bother.

What I evaluate in the take-home: code clarity, naming, error handling, tests if they wrote any, and the README. The README is half the signal. A staff candidate's README should explain what they built, what they considered, what they cut, and why. If the README is missing or one paragraph, you've learned something.

## The interview loop: four rounds

After the take-home, four rounds. Each one tests something specific.

### Round 1: Take-home walk-through (60 minutes)

The candidate explains their take-home submission. You ask: "Why this approach?" "What did you not do, and why?" "What would you do if you had a full day instead of two hours?" "How would you test this in production?"

You're looking for: tradeoff awareness, the difference between "I shipped it" and "I shipped it and here's the production-readiness gap."

### Round 2: Open-ended system design (60 minutes)

Pick a system close to what the company actually builds. "Design the AI evaluation pipeline for a chat product." "Design the agentic tool-use authorization layer." Give the candidate 60 minutes; they drive, you ask follow-ups.

You're looking for: ability to ask clarifying questions before designing, awareness of what they don't know, ability to scope the design to the actual problem (not the perfect-world version), and willingness to push back when your hypothetical doesn't make sense.

The single biggest red flag in this round: a candidate who immediately sketches the "right" architecture without asking who the users are, what the load looks like, what the failure mode is. Senior engineers do that. Staff engineers don't.

### Round 3: Live code review (60 minutes)

You hand the candidate a 200-line PR from your real codebase (or a contrived equivalent). They review it. You watch.

You're looking for: what they comment on, what they miss, how they phrase the comments, whether they catch the bugs you planted, whether they suggest stylistic changes that don't matter and ignore the substantive ones.

This round is the single most predictive signal I've found for staff-level performance. The way an engineer reviews other engineers' code is the way they'll show up to your engineering culture. If the review is sharp, kind, and substantive, you've found a staff engineer. If it's nitpicky, vague, or ego-driven, decline.

### Round 4: Judgment / leadership (60 minutes)

This is the round most companies skip. They shouldn't.

You walk the candidate through three to four scenarios pulled from real situations at your company:

- "A junior engineer pushes a hot fix to production at 11pm Friday without code review. Walk me through what you do."

- "The CTO wants to migrate from Postgres to a new vector database. You think it's premature. How do you handle the conversation?"

- "Production is down. The on-call is in over their head. You're not on call. What do you do in the next 30 minutes?"

- "A peer engineer is consistently producing low-quality work, and the eng manager isn't acting on it. Your move."

You're looking for: temperament, judgment under pressure, willingness to disagree professionally, awareness that the technical decision is rarely the only decision in the room.

## The reference call that matters more than the interview

If a candidate gets through all four rounds, you talk to references. Not the references they list on the resume — those are filtered. Talk to the people who reported to them, and to the people they reported to.

Three questions, in this order:

- "What's the kind of work this person is best at?" — Calibrates strengths.

- "What kind of work do they struggle with?" — Calibrates limits. If the reference can't name a single weakness, they didn't know the candidate well enough; the call is useless.

- "Would you hire them again?" — The most predictive single question in any reference check. The pause, the tone, the qualifications they put on the answer matter more than the literal yes or no.

Three reference calls is enough. Five is overkill. One is too few.

## AI-fluency calibration: the new bar

Here's where the AI-startup version of staff hiring diverges from generic staff hiring. In 2026, an engineer who can't fluently use Claude Code, Copilot, or Codex is not a staff engineer. They might be a great senior engineer, but they're not operating at the leverage a staff engineer should have.

The bar I'd hold to:

- **They use AI tooling daily.** Not "I've tried Copilot." Not "I'm a skeptic." Daily, with opinions about which tool for which job.

- **They know what NOT to use AI for.** Auth code, real-money flows, performance-critical paths. If a candidate says "I use AI for everything," that's a flag.

- **They have a take on team adoption.** A staff engineer should have thought about how AI tooling changes engineering practice at the team level, not just personal productivity.

How to test it: a question in round 4. "Walk me through how you used AI in the last feature you shipped." Listen for specificity. Vague answers signal lip-service usage. Specific answers — "I used Claude to enumerate the edge cases on the state machine before I wrote the code, then I had it review my PR before I opened it" — signal real fluency.

If a candidate is otherwise excellent but lacks AI fluency, hire them anyway and budget two months for them to develop it. If they're average plus AI-skeptical, decline. The bar has moved.

## Comp, equity, and selling them

A staff engineer at an AI startup pre-Series-A in 2026 is looking at:

- **Cash:** $220–280K base. Bay Area / NYC / remote-but-competitive.

- **Equity:** 0.4–0.8% over four years. Higher end if they're early; lower end if they're hire 8.

- **Sign-on:** $25–50K to make up for unvested equity at their previous company.

The candidates you want at this level have options. Selling them is half the job. The pitch that lands: a clear, articulate vision of what the engineering org is going to look like in 18 months, what their role in shaping it is, who they'll be working alongside, and what the realistic path to Series A and beyond looks like for the company.

The pitch that doesn't land: "we're hiring fast, lots to do, hope you like ambiguity." That's not a pitch, that's a confession.

## The first 30 days after they say yes

Hiring a staff engineer is half the work. The other half is onboarding them so they're operational at the level you hired them for.

Three commitments to make in writing during the offer stage:

- A clear "first 90 days" set of expectations. What good looks like at day 30, day 60, day 90.

- Direct access to the founder for the first 30 days. Weekly 1-on-1s. They are part of how the company is run, not three layers below it.

- Ownership of one tangible piece of the platform within 30 days. Not "shadowing." Not "learning the codebase." Something they own with their name on the GitHub commits.

Without these three, the staff engineer ramp drifts and they end up doing senior-engineer-level work for two quarters before someone notices. With them, they're contributing at staff level by month two, and the existing team is leveling up against them.

## The takeaway

Hiring a staff engineer is one of the most consequential hires a pre-Series-A AI startup makes. Get it right and the entire engineering bench levels up. Get it wrong and you've spent $300K+ all-in on a hire who either underperforms or leaves.

The interview process I've described is more rigorous than what most companies run. That's the point. The cost of running this process is high. The cost of hiring the wrong staff engineer is higher.

If you're a founder who hasn't run this kind of loop before, partner with someone who has. The first staff hire is not the time to learn the process from scratch.

## Read this next

- [**The Pre-Series-A AI Startup Hiring Plan**](/blog/pre-series-a-ai-startup-hiring-plan) — Where the staff engineer fits in the broader hiring sequence.

- [**From One Engineer to Fifteen**](/blog/from-one-engineer-to-fifteen-engineering-leadership) — The leadership lessons that inform how I'd onboard a staff hire today.

- [**AI-Assisted Engineering Isn't Faster Coding**](/blog/ai-assisted-engineering-is-a-new-workflow) — The AI-fluency bar, in much more detail.

---

## The Pre-Series-A AI Startup Hiring Plan: Who to Hire, in What Order, and Why Most Get It Wrong

URL: https://sublimecoding.com/blog/pre-series-a-ai-startup-hiring-plan
Published: 2025-12-29
Tags: hiring, founders, AI startups, engineering leadership, business, compensation

**Most pre-Series-A AI founders hire in panic order, not strategic order. The result is a team that can't ship the product the company actually needs to build.**

The pattern I see, repeatedly: a founder closes a seed round, gets pressure from the board to "scale the team," and posts five senior backend engineering openings on a Monday morning. Six months later they've hired four backend engineers, the product still doesn't have a designer, the AI features they're shipping look like internal tools, and the BD pipeline that was supposed to fund the next round is empty because no one has been working it.

The right framing is not "scale the team." It's **each hire should either unblock the product or unblock the customer**. If the hire doesn't do one of those, it's an expensive bet you didn't need to make at this stage.

Here's the plan I'd run if I were starting an AI-native company today and going from two co-founders through to a Series A.

## The six hires before Series A

For an AI-native company with two technical co-founders raising a $3–5M seed round, this is the order I'd hire in. The total span is roughly 14 to 18 months from first close to Series A.

### Hire 1: Founding engineer

The first hire is not a "great engineer." The first hire is a third co-founder who didn't get the title.

What you're looking for: full-stack capability, willingness to own a feature end to end, the temperamental capacity to be the only one in the codebase besides you for the first six months. Someone who's been a senior IC at one or two real companies and has decided they want startup risk now.

Comp: heavy equity (1.0–2.5%), market-rate-or-below cash. If they're asking for FAANG cash plus founding-engineer equity, they're not the right hire. The math doesn't work and they're going to bail at month 9.

What this hire should NOT be: a specialist. The first hire is the second pair of hands across the entire stack. The specialists come later.

### Hire 2: Product designer

The single most counter-intuitive hire on this list is also the most important. AI products that look like engineering tools die. Almost without exception.

Your customers cannot tell whether your model is good. They *can* tell whether your product feels considered. A great designer in seat from month four will reshape every feature you ship — for the better — and meaningfully change what an enterprise prospect sees in your demo.

What you're looking for: someone who's shipped product design at a venture-backed startup, ideally one with a complex underlying technology. Senior level. Comfortable with no full-time PM in seat (you're the PM, the founder, until much later).

Comp: market-rate cash, 0.5–1.0% equity. Fewer designers than engineers in the candidate pool, so you'll pay closer to senior-PM rates.

### Hire 3: ML or Applied AI specialist

By month six or seven, your AI features have moved past "wrap an LLM in a UI" and into territory where someone needs to think hard about prompt engineering, retrieval, fine-tuning, evals, and the rest of the AI engineering stack. This is not the founding engineer's job. This is a specialist.

What you're looking for: someone who's shipped AI features in production at another company. *Not* a research scientist. Not a PhD straight out of grad school. The hire is "applied" — they know how to ship, they know how to handle the messiness of LLMs in production, and they have opinions about evals.

Comp: market-rate cash, 0.4–0.8% equity. Hot market — be ready to move quickly when you find the right one.

### Hire 4: The GTM hire

This is where most founders get the order wrong. They hire engineer 3, then engineer 4, then engineer 5, then somewhere around month twelve realize they have no one running the customer side and they're still doing all the BD calls themselves.

By the time you're at four engineers, you should have one person whose job is owning customer development end to end. What flavor of GTM hire depends on your product:

- **Founder-led sales motion still working?** Hire a founding BDR / sales associate to handle the top of funnel and let the founder close.

- **Self-serve / PLG product?** Hire a growth engineer who's also done marketing.

- **Enterprise contracts already pulling?** Hire a founding AE — yes, even at $200k base + variable + equity. The math works if they close one deal.

This hire pays back the seed round in pipeline within their first year if you've hired the right person. Skipping it for "one more engineer" is the most common pre-A mistake.

### Hire 5: Senior product engineer

Now, finally, you hire engineer #3 (after the founding engineer and the AI specialist). This is the engineer who builds product features against the backlog the designer has shaped.

What you're looking for: someone who's shipped product features at scale at a previous startup. Less senior than the founding engineer, but with enough taste to make the right tradeoffs without supervision. Strong frontend or strong full-stack — depends on where the gap is at this point.

Comp: market-rate cash, 0.3–0.5% equity.

### Hire 6: Security / ops person

By month 14, your customer pipeline is asking for SOC 2, vendor questionnaires, and a security trust page. Your vCISO has been doing the strategy work, but you need someone in seat for the day-to-day execution. This hire is part security engineer, part DevOps, part compliance ops.

What you're looking for: someone with cloud security and compliance ops experience at a startup of similar stage. Not a full CISO yet — you're not ready for that role. Senior IC with leadership trajectory.

Comp: market-rate cash, 0.3–0.5% equity. The vCISO transitions to advisor; the in-house person owns execution.

## The hires NOT to make pre-A

For every hire on the list above, there's a tempting wrong-stage hire that founders make instead. The list of *don'ts*:

- **Don't hire a full-time PM yet.** Founder is PM. The day you hire a PM is the day product velocity drops 30% as the PM "gets up to speed" and adds a layer between engineering and customers. Wait until post-A.

- **Don't hire an EM yet.** Same reason. You're managing six engineers; you don't need an engineering manager. The founding engineer is the de-facto tech lead.

- **Don't hire a CISO.** Hire a vCISO (covered in [vCISO Math](/blog/vciso-math-for-ai-founders)). Save the full-time hire for $20M ARR or after a regulatory event.

- **Don't hire a research scientist.** Almost all AI startups don't need one. The applied AI specialist (hire 3) is sufficient until you're shipping novel research as the product.

- **Don't hire a full-time recruiter.** Founder is recruiter. If you can't recruit your own first six hires, you don't yet know what you're hiring for.

- **Don't hire a head of marketing.** Wait until you have a repeatable GTM motion the head of marketing can scale. Until then, the founder owns positioning.

## The compensation framework

The biggest reason founders blow this plan is bad comp framework. They either underpay and lose candidates to bigger checks, or overpay and burn the runway they need for the next 18 months of progress.

The framework that's worked for me:

- **Cash:** roughly 80–90% of market median for a senior at a similar-stage startup. Pull market data from Carta, Pave, or Levels.fyi. Pay slightly below median because you're paying in equity.

- **Equity:** heavy for early hires (founding engineer 1.0–2.5%), tapering down (hire 6 at 0.3–0.5%). Use a tool like Carta to manage option pool dilution carefully.

- **Refresh grants:** commit in writing to a refresh grant at the 24-month mark. This is how you keep early hires from leaving when their original grant gets eclipsed by new joiners' grants.

- **Cash-vs-equity flexibility:** some great candidates need more cash because of life circumstances. Have a documented sliding scale (e.g., "+$20k base = -0.2% equity") so you're not negotiating each one from scratch.

## The post-Series-A inflection

This plan stops at six hires. After Series A, the discipline changes.

You'll go from six to roughly thirty in the year following the A. The hire-by-hire framing breaks down at that velocity; you start hiring against role profiles and team needs. The right framing at that scale is "how many engineers do we need to ship the product roadmap" — but you only earn the right to ask that question after you've shipped pre-A with a tight team that proves the product works.

The single biggest predictor of which AI startups successfully transition pre-A to post-A is whether the team they assembled before the Series A could actually ship. The roster matters more than the headcount. Get the first six right and the rest of the company is downstream of that decision.

## What it actually costs to get this wrong

Founders skip past hiring sequencing because the cost of getting it wrong feels abstract. It isn't. Here's what hiring two extra engineers in months 4–6 instead of a designer + a GTM hire actually costs.

- **Two engineers fully loaded:** ~$500K cash + 1.0% equity over 18 months.

- **Lost product quality from no designer:** hard to quantify directly, but typically manifests as enterprise demos that don't convert. Three lost enterprise deals at $80K ACV each = $240K in lost first-year revenue.

- **Lost pipeline from no GTM hire:** a competent founding BDR generates $300–500K in qualified pipeline in their first six months. Not having one means the founder is doing top-of-funnel work instead of product or fundraising.

- **Compounding delay:** the Series A pitch eighteen months later is "we have great product, weak distribution" — a much harder pitch than "we have great product and a working GTM motion." Down-round risk goes up materially.

Total expected cost of the wrong sequencing in real dollars and equity: somewhere between $800K and $1.5M over two years, plus the fundraising delta. The right sequencing has a better expected value *even if the product takes one more month to ship*, because the customer-side work compounds in parallel with the engineering work.

## Read this next

- [**How I'd Hire a Staff Engineer at an AI Startup**](/blog/how-id-hire-a-staff-engineer-at-an-ai-startup) — A deep dive on the screen, take-home, and interview loop for one of the most consequential roles on this list.

- [**From One Engineer to Fifteen**](/blog/from-one-engineer-to-fifteen-engineering-leadership) — What I learned scaling an engineering team during my own founding stretch.

- [**vCISO Math for AI Founders**](/blog/vciso-math-for-ai-founders) — Why hire #6 starts as a vCISO, not a full-time CISO.

---

## The Ruby to Elixir Migration That Cut Our Service Footprint From Ten to Six

URL: https://sublimecoding.com/blog/ruby-to-elixir-migration-ten-to-six-services
Published: 2026-02-09
Tags: Elixir, Ruby, Phoenix, migration, microservices, engineering, OTP

**We had ten microservices that were 60% Ruby and 40% Elixir. Two years later we had six, fully Elixir, and our on-call alert volume had halved.**

The migration was less about the language and more about what running real-time messaging for 450,000 active students across 900 partner universities forced us to think about. Memory pressure. Long-running connections. Concurrency that didn't tip over. Operational ergonomics that made on-call survivable. Ruby could do all of these things, but every solution required a layer of accidental complexity that Elixir's runtime gave us for free.

What follows is the migration playbook from InsideTrack — the order that worked, the patterns we leaned on, the unexpected wins, and the parts I'd do differently with what I know now.

## The stack we started with

The platform served two-way messaging between coaches and students. Mostly SMS, some email, increasing volume of in-app chat. The architecture, when I joined:

- Three Rails monolith services (web, API, admin)

- Two Sidekiq workers (one for messaging dispatch, one for analytics ingestion)

- Three small Sinatra services (one webhook receiver, one cron scheduler, one feature-flag service)

- Two early Phoenix services (a real-time inbox and a notification dispatcher) — both written by the previous team in a "let's try Elixir" experiment

Total: 10 services, 6 Ruby, 4 Elixir. Combined the team operated 60+ background workers and a Postgres cluster handling several thousand writes per second at peak.

The motivation to consolidate wasn't ideological. It was operational. The Ruby services were memory-hungry, the Sidekiq workers had to be horizontally scaled aggressively to keep up with peak load, and the on-call rotation was getting paged 8–12 times per night during exam season because of the cumulative weight of running too many services.

## The trigger to start moving

Two specific events forced the decision.

First, we lost a contract with a large university because our messaging dispatch latency P99 spiked above the contractual threshold during an exam-season peak. The latency wasn't a code bug — it was Sidekiq queue depth backing up because the worker fleet couldn't scale fast enough. We could have thrown more Sidekiq workers at it, but the marginal cost was high enough that we'd have eaten the contract margin.

Second, our on-call engineer quit. The exit interview was honest: too many services, too much ambient alert noise, no clear ownership boundaries. The team morale knock was as expensive as the lost contract.

The combined message — both customer-facing and internal — was that the architecture was the bottleneck. Not the team's effort, not their skill, not the underlying tech of any single service. The number of services was the problem, and the runtime characteristics of Ruby + Sidekiq made consolidation in Ruby genuinely hard. Elixir's BEAM gave us a runtime that handled the same workload with one or two services instead of seven.

## The right migration order

The first lesson I learned was that migrations work backwards. You don't migrate the easy thing first; you migrate the thing that's most painful to keep on the old stack.

Our order, in retrospect:

- **The messaging dispatcher.** The most painful service. The one driving the on-call alerts. Migrating it first meant on-call ergonomics improved within the first quarter and the team had visceral evidence the migration was paying off.

- **The analytics ingestion worker.** Second-most painful. Sidekiq queue depth here was a chronic capacity issue. Re-implementing as a GenStage pipeline in Elixir collapsed memory usage by ~70%.

- **The webhook receiver and cron scheduler.** Smaller services we consolidated into a single Phoenix app with multiple endpoints and a Quantum scheduler. Saved two services in one move.

- **The feature-flag service.** Replaced wholesale with a managed service (LaunchDarkly). Not strictly an Elixir migration — but the Ruby-to-Elixir framing forced us to evaluate "is this our problem to host at all?" and the answer was no.

- **The Rails admin service.** Migrated to Phoenix LiveView. Surprised us by being one of the easier moves once we got over the learning curve.

- **The Rails API service.** Migrated last and most carefully. This was the customer-facing surface; we ran a dual-deploy period for two months with traffic mirrored to both stacks for parity testing.

- **The Rails web monolith.** Stayed Ruby. We never migrated it. Too much business logic, too low a marginal benefit. Lesson: not everything needs to move.

Final state: six services, all Elixir except the Rails web monolith. One major Phoenix app handling messaging dispatch, ingestion, webhooks, scheduling, and admin. Three smaller Phoenix apps for the inbox, notifications, and a public API. Plus the Rails web monolith. Down from ten.

## The wrong order I tried first

My initial plan, before reality course-corrected it, was to start with the API service. Reasoning: it's the most visible, it has the most code, getting it migrated first proves the platform.

That plan was wrong. The API service was the riskiest single move and had the lowest operational pain associated with it. We would have spent six months on a high-risk migration that wouldn't have meaningfully reduced on-call burden, while the messaging dispatcher kept paging us. The team would have lost faith in the migration before we got to the actually painful services.

The corrected ordering — pain first, value-prove second, polish last — is the framework I'd use again. **Migrate the service that's hurting you most, even if it's not the most strategic one.** The early operational win pays for the political capital you'll spend later on the harder migrations.

## The Elixir patterns we leaned on

Three OTP primitives did the bulk of the work.

**GenServer for stateful work.** The messaging dispatcher's previous architecture was Sidekiq + Postgres rows for state. Re-implementing as GenServers per active conversation eliminated the database churn for state machine transitions and let us hold conversation state in memory cheaply. The supervision tree handled crashes per-conversation without taking down the whole dispatcher.

**Registry for routing.** Looking up "which GenServer handles conversation 42" is a few microseconds with Registry. We used it everywhere — for active conversations, for active user sessions, for active webhook subscriptions. Dead simple, fast, and it eliminated a class of "where does this message go" problems that had been complex in the Ruby version.

**Supervision trees for failure isolation.** The single most important property of the Elixir runtime is that one bad message can't take down the service. A ten-thousand-conversation dispatcher might have one or two crashing GenServers at any given moment; they get restarted in milliseconds and the other 9,998 conversations don't notice. Sidekiq could not give us this without significant infrastructure investment.

The fourth pattern, less universal but useful: **GenStage for backpressure-aware pipelines.** The analytics ingestion worker was a GenStage pipeline with explicit demand-driven flow control. Made the queue-depth-spike pattern that had been killing us in Sidekiq simply not exist as a category.

## The unexpected wins

**Halved on-call alerts.** By far the biggest morale and retention win. The team that had been getting paged 8–12 times a night dropped to 3–4. Not because the services were doing less work, but because they handled load shedding, partial failures, and self-healing without paging humans.

**Better dev ergonomics for the kind of work we did.** Pattern matching against incoming messages made the dispatcher code dramatically clearer than the Ruby case statements it replaced. `iex` with remote shell into a running production node was an operational superpower.

**Hiring quality went up.** This surprised me. The Elixir candidate pool is smaller, but the candidates who self-select into Elixir tend to be more curious and more rigorous than the Ruby candidate average. We hired better engineers per interview hour after the migration than before.

## The unexpected losses

**Gem ecosystem.** I missed Devise. I missed ActiveAdmin. I missed Sidekiq Pro's UI. There were Elixir-equivalent libraries for most of these, but the Elixir ecosystem in 2018-2019 was visibly less mature, and rolling our own auth or admin UI cost more time than the migration math accounted for.

**Hiring pool narrower.** Yes, the candidates who came through were better. But the funnel was smaller. We'd see 30 Ruby applicants for every 5 Elixir applicants. For a small team this didn't matter. For a team scaling fast, it would have been a constraint.

**Internal training cost.** Engineers coming from Ruby need 2–3 months to be productive in Elixir. We absorbed that cost but it was real and it slowed the migration. Account for it explicitly in your timeline.

## When NOT to migrate to Elixir today

The math has shifted somewhat since 2019. I would not unconditionally recommend a Ruby-to-Elixir migration today. The cases where I'd push back:

- **You're not running real-time, long-lived connections.** The killer features of the BEAM are concurrency and supervision. If your workload is short, request-response, and stateless, Ruby/Rails on a modern hosting platform is genuinely fine.

- **Your team has zero Elixir experience and you're already understaffed.** The 2-3 month productivity dip per engineer is real. If you can't afford it, don't start.

- **Your product is dominated by AI features, not real-time messaging.** The AI ecosystem in Python is significantly stronger than in Elixir. Most AI startups today should be in Python or Go for the AI portion, regardless of what the rest of the stack runs.

- **Ruby 3 + YJIT is meeting your needs.** The performance gap between modern Ruby and Elixir narrowed considerably with YJIT. If your Ruby services aren't hurting you, leave them alone.

The right reason to migrate is operational pain that's expensive to solve in your current runtime. The wrong reason is novelty.

## What I'd do differently

If I were running this migration again today:

- **I'd budget the per-engineer onboarding cost explicitly.** 60 days off the keyboard for the first migration project, then ramp. We crashed into this; it should have been planned.

- **I'd build the dual-stack observability layer first.** Migrating with consistent metrics across both stacks would have made the parity testing meaningfully easier. We bolted this on.

- **I'd skip the LiveView migration of admin and use a managed admin tool.** LiveView is great. The admin we built was fine. But the time we spent on it was better spent on the API migration.

- **I'd not migrate the Rails web monolith. Same conclusion. We made the right call there.**

## The takeaway

Migrations are paid for by operational pain reduction, not by language preferences. The Ruby-to-Elixir move at InsideTrack worked because real-time messaging is exactly the workload BEAM is built for, and the operational pain we were running into was specifically the kind that BEAM eliminates.

For other workloads, the calculation may go the other way. The disciplined version of the question — "what's hurting us today, would moving runtimes solve it cheaply, and can we afford the transition cost" — is a much better framing than "what's the right tech stack for our company in 2026." The right answer to that latter question is almost always "the one you already have, optimized harder."

## Read this next

- [**Migrating 225K Users from AWS Cognito to Auth0**](/blog/aws-cognito-to-auth0-migration-without-forcing-logout) — A different migration war story, same disciplined pattern: pain first, value-prove second.

- [**How We Cut $350K From Cloud Spend**](/blog/cut-350k-cloud-spend-six-months) — When the platform you migrate to also rewrites the cost structure.

- [**AI-Assisted Engineering Isn't Faster Coding**](/blog/ai-assisted-engineering-is-a-new-workflow) — How modern teams approach migrations with AI tooling in the loop.

---

## Your AI Product Needs a Telemetry Layer Before It Needs a Better Model

URL: https://sublimecoding.com/blog/your-ai-product-needs-telemetry-before-better-model
Published: 2026-01-26
Tags: AI, telemetry, observability, AI products, engineering, ML, production

**I've watched three AI startups burn months trying to "improve the model" when they couldn't even tell which prompts produced which outputs at scale.**

Every team had the same instinct: hallucination rate too high, response quality inconsistent, costs creeping — must be a model problem, let's tune the prompts, let's swap to GPT-5, let's fine-tune. None of them stopped to ask the more useful question first: *what's actually happening inside the model calls we're already making?*

The answer, almost always: nobody really knew. There was no production logging of prompts. No structured capture of model outputs. No correlation between which user did what and what the model returned. The team was making decisions about model improvement based on cherry-picked screenshots and vibes.

That's not a model problem. That's an instrumentation problem. And it's solvable in two weeks of disciplined engineering, which buys you the visibility to know whether the model problem is even real.

## What AI telemetry actually means

Classic application telemetry — request rate, latency, error rate — does not tell you anything useful about an AI feature. A successful 200 response from your LLM endpoint tells you nothing about whether the response was correct, helpful, or hallucinated. You need a different layer of observability that's specific to how AI features fail.

The four things you must capture for every model call:

- **The full prompt.** Every variable interpolation. Every system prompt. Every retrieval-augmented context. Stored as structured data, not a stringified blob.

- **The full response.** Including any tool calls, function calls, or structured outputs. Stored verbatim.

- **The cost and latency.** Tokens in, tokens out, dollar cost, wall-clock time. These compose into your unit economics.

- **The user context.** Who triggered this call, in what feature, against what state. Anonymized if you must, but linkable to the user session.

Without those four, you cannot reason about model performance at any scale beyond "let me copy this prompt into the playground and see what happens." That's not engineering, it's gambling.

## The four-layer telemetry stack

Once the basics are captured, the actual decisions you make benefit from layered aggregation.

### Layer 1: Request-level telemetry

Every model call gets logged with the four-tuple above plus a request ID. This is the source of truth. Every other layer aggregates from this layer.

Storage decisions matter here. The volume can be large — for a product making 100k model calls a day, this is 100k structured rows daily. We chose Postgres with JSONB columns at Lavender, with a 90-day retention policy. Worked fine for our scale; would not scale to 10M calls/day. Use what fits.

### Layer 2: Feature-level aggregation

Each model call belongs to a feature: "summarize," "draft email," "suggest reply," etc. Aggregate the request-level data by feature to answer questions like:

- What's the median response time of the "draft email" feature this week?

- What's the daily cost of "summarize" over the past 30 days?

- Which feature has seen the biggest cost spike since the last release?

This is the layer where you start making product decisions: "the suggest-reply feature costs 4x what summarize does and gets used 1/10 as much — we should kill it or rebuild it."

### Layer 3: User-level signal

Each user has interactions across multiple features. Aggregate at the user level to answer:

- Are heavy users seeing more or fewer hallucinations than light users?

- Is there a cohort of users for whom the feature consistently fails?

- What's our cost per active user per week?

The user-level layer is where you discover that your model is fine for 90% of users but catastrophically bad for the specific use case 10% of users have. Without this layer, that 10% is invisible.

### Layer 4: Aggregate trends and regression detection

Daily / weekly rollups across the whole product. The metrics that go on a dashboard the founder reads every Monday morning:

- Total cost trend

- Cost per active user trend

- P95 latency trend

- Hallucination signal trend (more on this below)

- Feature-level usage distribution

The point of layer 4 is regression detection. When something breaks, you want to know within 24 hours, not 21 days into the quarter when finance asks why the OpenAI bill tripled.

## The hallucination signal

Hallucination is the hardest thing to measure because there's no ground truth label at runtime. Real-world signals that approximate it:

- **User regenerates the response.** One of the strongest negative signals. If a user immediately clicks "regenerate," they didn't like what they got.

- **User edits the response heavily before using it.** If you have a copy-and-edit flow, measure the edit distance.

- **User abandons the feature mid-flow.** Strong signal something went wrong.

- **Explicit thumbs-up / thumbs-down.** Lowest-volume signal but the cleanest. Add it everywhere it's not annoying.

- **Response contains markers of uncertainty.** "I don't have information about" or "I cannot determine" — sometimes useful, sometimes a euphemism for hallucination.

None of these is a clean ground-truth label. Combined, they give you a directional indicator that's good enough for relative comparisons over time. The goal isn't "what's our true hallucination rate" — that's unanswerable. The goal is "is hallucination getting better or worse this week, and which features are driving the change."

## Tooling I'd reach for

The build-vs-buy decision for AI telemetry has shifted in the last 18 months. There are now real options.

- **[LangSmith](https://www.langchain.com/langsmith)** — strong if you're already using LangChain. Decent if you're not. Captures request/response/cost out of the box.

- **[Helicone](https://www.helicone.ai/)** — proxy-based capture. Lowest integration cost — point your LLM SDK at Helicone's URL, get telemetry for free. Best for early-stage teams that want zero-config.

- **[Langfuse](https://langfuse.com/)** — open source, self-hostable. Good for teams with security/data residency concerns.

- **Custom OpenTelemetry instrumentation.** If you already have a strong observability stack (Datadog, Honeycomb, etc.), wrapping your model calls in OTel spans is sometimes the right answer because it integrates with existing dashboards.

For pre-Series-A AI startups I'd start with Helicone and graduate later. The integration cost is one afternoon. The telemetry you get back is enough to make the next dozen product decisions correctly.

## Model problem or instrumentation problem?

The most useful framing I've found, when an AI feature is underperforming:

**Can you, right now, answer these five questions in under five minutes?**

- What was the prompt and response of the last 10 calls to this feature?

- What's the median latency for this feature over the past 7 days?

- What's the daily cost for this feature, broken out by model?

- Which users had the worst experiences this week, by hallucination signal?

- How does any of this compare to two weeks ago?

If the answer to any of these is "I don't know" or "let me write a query," you have an instrumentation problem, not a model problem. Fix instrumentation first. Then look at the data, and the model problem either becomes obvious — or evaporates because what looked like a model problem was actually a prompt regression in last week's deploy.

## A concrete example

At Lavender, we shipped a new prompt template for one of our AI features early in 2025. The hallucination signal — measured via the regenerate-rate — climbed about 60% over the next two weeks. The instinct was "the new prompt is worse, let's rewrite it."

Telemetry told a different story. The regenerate-rate climbed for users on a specific email template that one of our customer-success team had recommended internally. The new prompt was fine. The customer template was triggering an edge case we hadn't anticipated, and the regenerate-rate spike was an artifact of that template being used 4x more than usual.

The fix was a 20-line guardrail in the prompt that handled the edge case. Hallucination signal dropped by 40% within 72 hours. We didn't tune the model. We didn't change LLMs. We did not run a single eval. We instrumented, looked at the data, found the actual cause, fixed it.

That story is impossible to tell without telemetry. Without it, the team would have spent two weeks rewriting the prompt, regressing on something else, and ending up worse than where they started. With it, the cause was obvious within 90 minutes of looking at the data.

## The takeaway

Most AI startups will eventually need to think hard about the model. None of them should think about the model first. The order is:

- **Instrument.** Capture every model call, structured and queryable.

- **Aggregate.** Build feature-, user-, and trend-level views.

- **Look.** Stare at the data for a week. Most "model problems" reveal themselves as something else.

- **Then, if needed, tune the model.** But you'll be tuning against actual data, not vibes.

The two weeks of disciplined engineering this requires is the highest-leverage AI work most startups aren't doing. It's also boring. Which is exactly why doing it is an edge over teams that go straight to fine-tuning.

## The team discipline this requires

Telemetry is a code problem for half a sprint and an organizational problem forever after. The engineering team has to keep instrumentation current as new features ship, or the system rots within a quarter.

The disciplines that worked at Lavender:

- **No model call ships without telemetry.** Code review checklist item, enforced. New AI feature PRs get rejected if they don't wire up the four-tuple capture.

- **One engineer owns the telemetry layer.** Not full-time, but they're the named point of contact. Schema evolution, dashboard updates, retention policies — they own it. Without an owner, the layer drifts.

- **Weekly review of the dashboards.** 15 minutes at the top of an engineering meeting. Just looking at the trends. Catches regressions while they're small and trains the team to think in terms of these metrics.

- **Cost alerts before user complaints.** If the daily AI spend deviates from the rolling 7-day median by more than 30%, it pages the on-call. Most product issues show up here before they show up in support tickets.

The instrumentation work is one or two weeks. The discipline of keeping it useful is forever. Build the muscle early — adding it later, against an existing AI product with no telemetry, is meaningfully harder.

## Read this next

- [**How I'd Run Security at an AI-Native Company in 2026**](/blog/running-security-at-an-ai-native-company-2026) — The audit-and-security layer of AI observability — what to log for incident response and customer trust.

- [**AI-Assisted Engineering Isn't Faster Coding**](/blog/ai-assisted-engineering-is-a-new-workflow) — Engineering discipline applied to the team using AI; this post applies it to the AI itself.

- [**How We Cut $350K From Cloud Spend**](/blog/cut-350k-cloud-spend-six-months) — The same instrumentation discipline applied to infrastructure, with bigger dollar consequences.

---

## How I'd Run Security at an AI-Native Company in 2026

URL: https://sublimecoding.com/blog/running-security-at-an-ai-native-company-2026
Published: 2026-04-20
Tags: security, AI, agentic, prompt injection, threat model, founders, AI startups, compliance

**AI-native companies need a security model that classic appsec doesn't cover. Most don't have one.**

The pattern I see across early-stage AI companies: a strong engineering team treats security like a 2018 SaaS product — auth, secrets, the SOC 2 checklist. Meanwhile their product is shipping autonomous agents with cloud credentials, accepting unstructured input from customers as the primary interface, and training models on data the customers didn't fully realize they were exposing. The threat model has changed. The controls haven't kept up.

If I were building the security program at an AI-native company today, this is the layered stack I'd put in place, the things I'd ship in the first 90 days, and the things I'd consciously defer.

## The four-layer stack

Classic appsec is one layer of four. Treating it as the whole picture is the most common mistake I see.

### Layer 1 — Classic application security

This is everything that's been good practice for fifteen years and doesn't go away because you're an AI company. Auth and authorization. Secrets management. Input validation. SQL injection prevention. CSRF tokens. SSRF guardrails. TLS everywhere. Least-privilege IAM. Logging and audit trails. Backups and recovery.

This layer is solved. The advice has been written down a hundred times. If you're not doing it, do it. If you are, skip the rest of this layer's discussion and move on. The interesting work for AI companies is in the next three layers.

### Layer 2 — Data security and the training question

The novel question for AI-native companies is what data goes into the model and where it ends up.

The threats:

- **Training-data exfiltration.** A model trained or fine-tuned on customer data can leak fragments of that data through generation. This is real, has been demonstrated repeatedly, and is not solved by "we delete the data after training."

- **Prompt-context leakage.** Customer A's data ends up in customer B's response because both customers share the same backend prompt context. RAG pipelines are the worst offender here.

- **Vendor-side training.** You send customer data to a foundation model API. The vendor uses it to improve their model. Your customer didn't consent to that.

The controls I'd ship:

- **Tenant isolation in retrieval.** Every vector-DB query and every RAG retrieval must filter by tenant ID at the index level, not in post-processing. This is the single most common AI company security bug I see in code review.

- **No-train flags on every vendor API.** OpenAI, Anthropic, Google, AWS Bedrock all have versions of "do not use this for training." Default-on, document the setting, audit it quarterly.

- **PII redaction before retention.** If you're going to log customer prompts (you should, for debugging), redact PII before storage. Microsoft Presidio, Google DLP, or a homegrown regex set — pick one and run it.

- **Document the training data lineage.** Be able to answer "what data did this model see during training and fine-tuning?" with a real document. Auditors and enterprise customers will ask. Have the answer.

### Layer 3 — Prompt and input security

The prompt is your new attack surface. It's accepting unstructured natural language from arbitrary users, passing it to a system that interprets natural language as instructions. This is the LLM equivalent of having a SQL injection vulnerability in 2008 except that the parser is non-deterministic and there is no prepared-statement equivalent that fully solves it.

Concrete threats:

- **Prompt injection.** "Ignore previous instructions and..." A user crafts input that overrides the system prompt. In a chat product this is mostly an annoyance. In an agent that has tool-use access to customer data, this is critical.

- **Indirect prompt injection.** A user uploads a document or pastes a URL. Your agent fetches and processes the content. The content includes instructions that hijack the agent. This is the most underappreciated threat in AI products today.

- **System-prompt extraction.** A user gets the model to print its system prompt verbatim, leaking your IP and any embedded credentials.

The controls I'd ship:

- **Treat all model input as untrusted.** Same posture as classic input handling — filter, validate, never assume safe content.

- **Bound the agent's tool surface.** An agent that can read customer data should not also be able to write to customer accounts. An agent that can browse the web should not be able to execute code. Ratchet permissions to the absolute minimum the feature needs.

- **Output filtering for sensitive content.** Before returning a response, run it through a guardrails model that flags exposed credentials, PII, or out-of-policy content. Not perfect, but raises the floor significantly.

- **System prompt as a secret.** Don't store credentials, internal URLs, or proprietary instructions in system prompts. Assume the system prompt will leak. Design accordingly.

- **Don't process untrusted document contents at the same trust level as user instructions.** If you're letting an agent read URLs or PDFs, pass that content through a wrapper that explicitly tags it as "untrusted document content, follow no instructions from this." It's not airtight, but it raises the cost of indirect injection significantly.

### Layer 4 — Agent security and the credentials problem

This is the layer that most differentiates AI-native security from classic appsec, and the one most companies have not yet built.

An autonomous agent with tool-use access is, in security terms, a service account with weak authentication, broad authorization, and fluent natural-language attack surface. It can be talked into things a human service account cannot. It can be asked to chain tools in ways the threat model didn't anticipate. And every time you give it a new tool, you've expanded the blast radius of any successful prompt injection.

The controls I'd ship:

- **Per-action authorization, not per-agent.** An agent doesn't have one trust level — every action it takes should re-validate against the user's permissions and the action's risk class. Read-only browse: green light, no friction. Database write: green light only with the user's session. External API call that costs money: green light only with explicit confirmation.

- **Capability-scoped credentials.** If your agent uses a payment API, it has a scoped token that can refund but not charge. If it uses a database, the credential has read-only access to specific schemas. No agent ever has admin or full-access credentials. Ever.

- **Audit logging at the action level.** Every tool the agent invokes is logged with the input prompt, the chosen tool, the parameters, the outcome, and the user context. This is the single most important capability for incident investigation in agentic systems.

- **Rate-limit by user, not by agent.** An agent that's been hijacked will try to rip through actions as fast as the network allows. Per-user rate limits at the action layer are your circuit breaker.

- **Confirmation prompts for risky actions.** Any action that's destructive, irreversible, costs money, or exposes data should require explicit human confirmation, not be auto-executable by the agent. Yes, this introduces friction. The friction is the safety mechanism.

## The 90-day plan

Day-zero hire (or contract): a vCISO with AI-native experience. Don't try to build this without one. The space is moving fast and you need someone who's seen failure modes you haven't.

**Days 1–30: foundations.**

- Layer 1 baseline: SSO, MFA, MDM, secrets vault, IAM least-privilege review.

- Layer 2 controls: no-train flags everywhere, RAG tenant isolation audit.

- Set up audit logging at the action level for any agent or tool-using LLM.

- Document model lineage for every model you ship.

**Days 30–60: prompt and agent.**

- Adversarial review of every system prompt. Assume it will be extracted; remove anything that should not be public.

- Tool-permission audit: every agent's available tools, mapped to risk class, with confirmation gates added where missing.

- Indirect-prompt-injection testing on document and URL ingestion paths.

- PII redaction in logs and analytics pipelines.

**Days 60–90: program.**

- SOC 2 Type I readiness, scoped to include AI-specific controls (data lineage, no-train, agent action logging). Most off-the-shelf SOC 2 templates do not include these.

- Customer-facing security documentation: trust page, AI usage disclosure, data handling policy. Enterprise prospects will ask.

- Incident response runbook with AI-specific scenarios: prompt injection at scale, data exfil via training, agent runaway.

- Quarterly security review cadence with founders and key engineering leads.

## What I'd defer

The instinct in security programs is to over-include. At the speed an AI startup moves, that's fatal — every control has a maintenance cost, and a security program that pisses off engineering will be worked around inside a quarter.

Things I'd consciously defer at the early stage:

- **Heavy DLP tooling.** Worth it at scale, distracting at fifteen people.

- **Endpoint detection and response.** MDM gets you most of the value at this stage. Real EDR comes after Series B.

- **SIEM platforms.** Centralized logging is great. A full SIEM with detection rules is overkill before you have a security team to run it.

- **Bug bounty programs.** Run them once you have a triage process. Before that, they generate noise.

- **Penetration tests beyond what your customers require.** One annual pentest scoped to your customer requirements is enough until you're in a regulated vertical.

The discipline is doing the controls that matter at your stage and not the ones that look impressive on a security marketing page.

## The takeaway

AI-native security is not classic appsec plus "be careful with prompts." It's a four-layer stack, and three of those layers — data, prompt, agent — are mostly novel relative to where most engineering teams have built up muscle memory.

You will get most of the value from **tenant isolation in retrieval, scoped credentials for agents, action-level audit logging, and confirmation gates on destructive actions.** Those four controls handle the vast majority of the AI-specific failure modes I've seen at production scale.

Everything else is sequencing and discipline. Don't skip Layer 1. Don't pretend Layers 2–4 don't exist. Hire a vCISO who's seen this space before. Document what you do and don't do, because your customers, your auditors, and your future self will all want to know.

The companies that get this right in the next two years will look like reasonable enterprise vendors. The ones that don't will spend a quarter on incident response that should have been spent on product.

## Read this next

- [**SOC 2 Is a Revenue Tool, Not a Security Tool**](/blog/soc-2-is-a-revenue-tool-not-a-security-tool) — How to convert this security posture into the audit report your enterprise pipeline is asking for.

- [**vCISO Math for AI Founders: Why 5 Hours a Month Beats a Full-Time Hire**](/blog/vciso-math-for-ai-founders) — Who you hire to run this program before you can afford a full-time CISO.

- [**Migrating 225K Users from AWS Cognito to Auth0 Without Forcing a Single Logout**](/blog/aws-cognito-to-auth0-migration-without-forcing-logout) — A real-world identity migration at fintech scale — Layer 1 of the stack done right.

---

## SOC 2 Is a Revenue Tool, Not a Security Tool

URL: https://sublimecoding.com/blog/soc-2-is-a-revenue-tool-not-a-security-tool
Published: 2026-04-27
Tags: security, SOC 2, compliance, founders, AI startups, go-to-market

**SOC 2 is a revenue tool, not a security tool.**

Every AI founder pre-Series A gets this wrong. You scope the audit like a security project and hand it to your best engineer. Six months later you've burned your strongest IC, the report still isn't done, and the enterprise deal you were trying to close went to a competitor with the checkbox.

Reframe it.

Your engineering team already thinks about auth, secrets, and data handling harder than any auditor will. SOC 2 doesn't make you secure. It unlocks the pipeline you're already leaving on the table. The VP of Engineering at that Fortune 500 who loves your demo cannot send you a contract without it.

So stop scoping it as a security project. Scope it as a sales project. And run it in 90 days.

Here's the path I've used at AI startups:

## Days 1 to 30: stop the bleeding

- Pick a compliance platform (Vanta, Drata, Secureframe). Don't overthink it.

- Name one internal DRI. Not a committee. One person owns it end to end.

- Target Type I first. Type II comes after you've operated controls for 6 months.

- Retain a vCISO for 5 hours a month. $2 to 4k. Worth every dollar.

- Pull policies off the shelf from the platform. Don't write your own. Most platforms have this built in. Some are better than others.

## Days 30 to 60: close the gaps

- MDM on every laptop. Non-negotiable.

- SSO and MFA across every tool, including the cheap ones nobody wants to pay to upgrade.

- Background checks on employees. 48 hours.

- Vendor review process. A spreadsheet is fine for now.

- Logging and quarterly access reviews. Most startups skip these. Auditors don't.

## Days 60 to 90: get the report

- Book the audit with a reputable firm. Don't pick the cheapest.

- Run a mock audit with your vCISO two weeks before kickoff.

- Fix the 10 things they find. There will be 10.

- Get the Type I report in hand.

- This can take longer than 30 days depending on how responsive the team is to issues.

## Cost and timeline

**Total cost for a 15-person startup:** usually $25k to $45k all in.

**Timeline from kickoff to report:** 90 to 120 days if you're serious.

## What it unlocks

Every enterprise deal stalled at "send us your SOC 2" moves to contract. This can turn theoretical hundreds of thousands and in some cases millions in ARR pipeline into closed revenue inside a quarter.

The mistake most founders make is treating the audit itself as the security work. It isn't. The audit is the door opener. The real security work starts after, once you're actually operating the controls day to day and your customer success team stops losing deals to a missing PDF.

If you're pre-Series A, AI-native, and watching enterprise deals die at the security review stage, this is the lever.

## Read this next

- [**vCISO Math for AI Founders: Why 5 Hours a Month Beats a Full-Time Hire**](/blog/vciso-math-for-ai-founders) — If you're scoping it as a sales project, this is who you hire to run it.

- [**How I'd Run Security at an AI-Native Company in 2026**](/blog/running-security-at-an-ai-native-company-2026) — What the controls behind the audit actually look like at an AI-native company.

- [**How We Cut $350K From Cloud Spend in 6 Months (And What I'd Do Differently)**](/blog/cut-350k-cloud-spend-six-months) — Same playbook framing applied to your cloud bill — treat it as a contract, not an architecture problem.

---

## vCISO Math for AI Founders: Why 5 Hours a Month Beats a Full-Time Hire

URL: https://sublimecoding.com/blog/vciso-math-for-ai-founders
Published: 2026-04-06
Tags: vCISO, security, AI startups, founders, compliance, SOC 2, business

**Don't hire a CISO. Rent one.**

This is the single most actionable security advice I give to pre-Series-A founders, and the one most consistently ignored. The pattern is predictable: an enterprise prospect asks for a SOC 2 report, the founder panics, posts a security-leadership job opening with a $250K base, sources for two months, and either hires the wrong person or gives up and ships the report without leadership in place. Both outcomes are bad. Both are avoidable.

The right answer at this stage is a fractional vCISO. Five hours a month, $2–4K, retained on a recurring contract. Below, the math, what to expect, and when to graduate to a full-time hire.

## The math

Let's compare the two paths concretely.

**Full-time CISO at a 15-person AI startup, pre-Series-A:**

- Base salary: $220–320K (Bay Area / NYC / remote-but-competitive)

- Equity: 0.5–1.5% (roughly $50–200K paper value at this stage)

- Benefits and overhead: ~25% of base = $55–80K

- Recruiter fee (if external): 20–25% of first-year comp = $50–80K one-time

- **First-year cash cost: $325–480K. Year-two onward: $275–400K.**

**Fractional vCISO, 5 hours a month:**

- Hourly: $400–800 depending on market and experience level

- Monthly: $2–4K

- Annual: $24–48K

- No equity, no benefits, no recruiter fee

- **First-year cash cost: $24–48K. Same year-two.**

The vCISO is roughly 5–10% of the cash cost of a full-time CISO and zero equity. For a pre-Series-A company where every dollar is runway, this is the difference between two months and twenty months of additional runway tied up in the security function.

You might object: "But a full-time CISO does much more than 5 hours a week." That's true. They do roughly 160 hours a month. The question is: does your 15-person AI startup, which has zero customers in regulated industries, has not yet had a security incident, and is six months from its first SOC 2 audit — does it actually have 160 hours a month of CISO-level work to do?

It does not. It has roughly 5–20 hours a month of CISO-level work, plus a much larger volume of engineering-led security execution that the engineering team is already doing or should be doing. A vCISO sized to the actual volume of CISO-shaped work is the right tool.

## What a vCISO actually does (and what they don't)

The biggest source of disappointment with vCISOs is mismatched expectations. Here's what to expect for $2–4K a month.

**What they do:**

- **Strategic guidance.** Quarterly review of your security roadmap, threat landscape, and gaps relative to your customer base. They tell you what to worry about and in what order.

- **Audit and certification readiness.** They read your evidence, tell you what's missing, and prep you for the auditor's conversation. Most vCISOs have shepherded ten to fifty SOC 2 audits and know exactly which controls auditors actually scrutinize.

- **Customer security questionnaires.** Enterprise prospects send 80–200 question security questionnaires. Your vCISO either fills them out or directs your team on the answers. This alone usually pays for the engagement.

- **Incident-response support.** When something goes sideways, they're on the phone in two hours. They've handled incidents before. Your engineering team has not.

- **Policy authorship and review.** Information security policy, acceptable use policy, vendor risk policy, incident response plan. They have templates. They customize them. They sign them. Done in days, not weeks.

- **Auditor relationship.** A reputable vCISO has working relationships with multiple audit firms. Their warm intro to a CPA firm gets you a faster engagement and a better rate.

**What they don't do:**

- Hands-on engineering. They don't write code, configure SSO, or set up MDM. Your engineering team does that under their guidance.

- 24/7 monitoring. They are not your SOC. If you need real-time monitoring, you're hiring an MSSP, not a vCISO.

- Hire and manage a security team. They might help you scope the first hire when you're ready, but they're not running people.

- Live in your Slack. Five hours a month is five hours a month. They will not be available for ad-hoc questions multiple times a day.

Match your expectations to the contract and the relationship is wildly productive. Mismatch and you'll fire each other within four months.

## When to graduate to a full-time hire

The vCISO model has a ceiling. The signals that you've hit it:

- **You're spending 20+ hours a month on the engagement.** If you've stretched a 5-hour retainer into 20 hours of effective work, you're paying overage rates and the vCISO is bottlenecked. Time to bring it in-house.

- **Your security team is more than 2 people.** A vCISO can guide one or two security ICs. Beyond that, you need a security leader with capacity to actually manage.

- **You're regulated.** If you take on PCI Level 1, HIPAA covered-entity status, FedRAMP, or financial services charters, the regulator's expectation of a named, in-house CISO becomes binding. Hire.

- **You're past Series B and selling to F500 enterprises.** At that revenue scale your customer expectations include a real CISO they can put on the phone. The vCISO can no longer carry that representational load.

- **You've had a security incident that drew a board-level response.** Boards want a named accountable person. Don't argue with that.

Pre-Series A: vCISO. Series A through B: vCISO with the option to upgrade. Series B+: full-time, almost always.

## The bad-vCISO red flags

Not all vCISOs are equal. Five flags I've learned to watch for:

- **They've never been an in-house security leader.** Career consultants who've never had to actually live with their decisions tend to over-prescribe. Look for someone who's been a Director or VP of Security at one or more real companies and decided to go fractional.

- **They don't ask about your customers.** If the vCISO doesn't immediately want to know who buys from you and what their security expectations are, they're going to give you generic advice. Your security program should be shaped by the people writing the checks, not by a checklist.

- **They sell products.** Some "vCISO" engagements are thinly disguised channel partnerships for compliance platforms or security tooling. They'll push you toward whatever they get paid to push. Ask up front: do you have any reseller, referral, or affiliate relationships with the platforms you'll recommend?

- **They quote you "all-in flat-rate" pricing.** The honest pricing is hourly with a monthly retainer minimum. Flat-rate vCISO pricing for $1,500 a month usually means you'll get attention only when you complain.

- **They can't name three audit firms they'd recommend.** A real vCISO has done a lot of audits and has opinions about who's good and who's bad. If they shrug at this question, they haven't done the volume.

## How to interview a vCISO in 30 minutes

A short list of questions that surface signal fast:

- "What's the right SOC 2 audit firm for a 15-person AI startup?" — They should name two or three with rate ranges and tradeoffs.

- "What are the three controls auditors most often flag at a company our size?" — They should answer in 30 seconds without thinking. Common answers: access reviews, vendor management, change management documentation.

- "Walk me through the last incident you led." — Listen for structure. Did they have a runbook? Who was in the room? What was the post-mortem? Vague answers are a flag.

- "What would you tell my engineering team to start doing on Monday?" — They should have a concrete short list. If it's "depends on a deeper assessment," they're billing for the assessment.

- "What gets you fired?" — Good answer: "I get fired when the auditor finds things I should have flagged in advance, or when I told you something was fine and it wasn't." Bad answer: long pause.

## The deliverables to write into the contract

Don't sign a vCISO contract without specifying outcomes. Generic monthly retainers float into nothing. Concrete examples:

- SOC 2 Type I readiness in 90 days

- Information security policy + 4 supporting policies signed and ratified within 30 days

- Quarterly risk register reviewed and updated

- Customer security questionnaires turned around in 5 business days

- Incident-response participation within 4 hours of declared incident, any time

- Quarterly readout to founders / board with current posture and gap list

If they push back on writing these into the contract, they're not committing. Find a different vCISO.

## The honest tradeoffs

To be fair to the full-time CISO model: there are real things you give up by going fractional.

You don't get a leader who's in your Slack every day, building relationships with engineers, customers, and the board over a sustained period. The institutional knowledge of an in-house leader compounds — they know which engineer cuts corners, which customer is going to ask which question, which board member wants which level of detail. A vCISO will never have that depth.

You also lose the recruiting halo. A named, in-house CISO with a strong reputation can be a meaningful asset when you're hiring senior security engineers or selling to security-sensitive customers. The vCISO does not show up on your team page.

And you lose the optionality of having someone in seat when things go sideways. If you have an incident on a Saturday, your full-time CISO is on it. Your vCISO is on it within a few hours, but those hours can matter.

The honest framing: the vCISO model trades depth-of-context for cost efficiency. At fifteen people pre-Series-A, the cost efficiency wins by a wide margin. The depth-of-context cost is small because there's not yet much context to be deep about. As the company grows, that math flips, and you should flip with it.

## The takeaway

Your security program at 15 people, pre-Series-A, looks like:

- Engineering does the engineering security work (auth, secrets, IAM, deployment hygiene). They were doing this anyway and are better at it than any external person.

- A vCISO does the leadership, audit, and customer-facing security work. Five hours a month, $2–4K, deliverables in the contract.

- Your founder owns the customer-facing risk story until the company outgrows them.

This setup costs you $24–48K a year and 5% of the leadership burn of a full-time CISO. It unlocks SOC 2, Vendor Risk Assessments, and enterprise customer questionnaires — the unlocks that actually move revenue. And when you outgrow it, around Series B, you graduate to a full-time hire with a much clearer view of what good looks like, because you've been working with one for two years.

The mistake is treating the security leadership question as a binary "no one" or "full-time hire" problem. There's a perfectly engineered middle option, and it's the right one for the first three years of an AI-native company's life. Use it.

## Read this next

- [**SOC 2 Is a Revenue Tool, Not a Security Tool**](/blog/soc-2-is-a-revenue-tool-not-a-security-tool) — What you ship in 90 days once you've hired the vCISO.

- [**How I'd Run Security at an AI-Native Company in 2026**](/blog/running-security-at-an-ai-native-company-2026) — The technical security stack the vCISO will help you build.

---

## Migrating 225K Users from AWS Cognito to Auth0 Without Forcing a Single Logout

URL: https://sublimecoding.com/blog/aws-cognito-to-auth0-migration-without-forcing-logout
Published: 2026-03-30
Tags: Auth0, AWS Cognito, identity, migration, fintech, security, IAM, engineering

**If you're migrating an identity provider, the user-facing rule is non-negotiable: no one should know it happened.**

At BlockFi, we moved 225,000+ users with $400M+ in assets from AWS Cognito to Auth0. Zero forced password resets. No MFA re-enrollment. No interrupted sessions. The migration ran for about ten weeks of active work after a much longer planning phase, and most users never noticed. Here is the playbook that got us there, including the parts that almost broke us.

## Why migrate at all

Cognito is fine for most use cases. We did not move because Cognito was bad — we moved because the product needs outgrew it.

Specifically:

- **Custom flows.** Cognito's hosted UI and Lambda triggers got us 80% of the way to a custom signup, then made the last 20% painful. Auth0's Actions and Universal Login were a better fit for the multi-step compliance flows fintech requires.

- **Operational ergonomics.** Auth0's logging, dashboards, and rule debugging are meaningfully better for a team that's not full-time AWS-native.

- **Compliance posture.** Auth0's tenant separation, audit log retention, and SOC 2 evidence collection were a closer fit to what our auditors wanted to see.

None of these are the right reason on their own. Combined, they made the migration worth the cost.

## The rule: no forced logouts

The first decision was the only one that mattered: whatever we did, users should not be forced through password reset or re-authentication during the migration.

This rules out the "big bang" approach where you export Cognito users, import them into Auth0, and require everyone to reset their password on next login. That works. It's also a customer-experience disaster for a fintech product where every interaction with the auth flow makes users wonder if their money is safe.

It also rules out exporting password hashes directly. Cognito uses SRP (Secure Remote Password), and the password verifier format is not directly compatible with Auth0's expected hash formats (bcrypt, scrypt, etc.). You cannot just move the hashes.

What it leaves: **lazy migration via custom database connection**.

## The lazy-migration pattern

Auth0 supports a "custom database" connection where, on login, Auth0 calls a function you provide. That function can authenticate the user against any external system — including Cognito. On successful authentication, Auth0 imports the user into its own database.

The flow becomes:

- User attempts to log in via Auth0.

- Auth0 looks up the user in its own DB. Not found.

- Auth0 calls our custom database script with the email + password.

- Our script calls the Cognito API to authenticate the user via the standard Cognito flow.

- If Cognito authenticates successfully, our script returns a profile to Auth0.

- Auth0 imports the user, sets their password (now hashed by Auth0), and continues the login flow.

- Subsequent logins for that user hit Auth0's local DB directly — no Cognito round-trip.

Effectively, every user migrates themselves on their next login. Active users migrate fast. Dormant users migrate when they come back. We never force the issue.

Performance: the first login took roughly 200–400ms longer than the post-migration login (Cognito API round-trip). Acceptable for a one-time hit. Subsequent logins were faster than they had been on Cognito.

## The MFA problem

This is the part that almost broke us.

MFA enrollment data does not transfer. If a user has TOTP set up in Cognito, that secret is in Cognito's vault and cannot be exported. If we did nothing, every MFA-enabled user would have to re-enroll their authenticator app on first Auth0 login.

That violates the no-forced-friction rule for the most security-conscious users — exactly the ones we least want to inconvenience.

The fix had two parts:

- **During the lazy-migration call**, after Cognito authenticated the user, we'd also call Cognito's API to check whether MFA was enabled and prompt for the TOTP code in the same request. If the user provided it, we knew the secret was valid for that user. We did not import it (we couldn't), but we marked the user as "MFA-required, not yet enrolled in Auth0."

- **On their next Auth0 session**, before issuing a token, Auth0 prompted them through a guided MFA enrollment in Auth0 itself. The user re-scans a QR with their authenticator app once. Then they're fully migrated.

This wasn't zero-friction — users with MFA hit a one-time enrollment screen — but it was bounded, explainable, and visibly framed as a security upgrade rather than a system failure.

If we'd skipped this design and just left MFA users to figure it out, we'd have flooded support with "I can't log in" tickets and spooked the security-conscious cohort. Plan for this on day one of any IDP migration.

## The three things that almost broke us

### Problem 1: Cognito API rate limits during peak

The custom database script calls Cognito on every first-login. We did not anticipate how many simultaneous first-logins we'd see during peak hours in the first week, and the Cognito API rate-limited us. Users got 500 errors. Support tickets spiked.

The fix: we added an in-memory token cache in the custom database script that batched verification requests and front-loaded a Cognito JWT verification step that didn't require an API call. We also requested a temporary rate-limit increase from AWS support. After that, no more 500s.

Lesson: the lazy-migration script *is* a high-traffic service for the duration of the migration. Capacity-plan it like one.

### Problem 2: email canonicalization

Cognito stored some emails with mixed case. Auth0 lowercases on lookup. A small percentage of users (about 0.4%) had originally signed up as `User@Example.com`, which Cognito stored verbatim, and which Auth0's lookup couldn't find when they tried to log in as `user@example.com`.

The fix was a one-line normalization in the custom database script: lowercase the email before looking it up in Cognito. Should have been there from day one. We caught it in canary testing on day three of the rollout — late enough to embarrass me, early enough that nobody got locked out.

Lesson: assume the source IDP allowed inputs that the destination IDP doesn't, and write a normalization layer in your migration script.

### Problem 3: session token mismatch on the front-end

Users who were already logged in via Cognito had a Cognito-issued JWT in their browser. The application checked that JWT on every API call. The day we cut over the auth provider, the format of newly-issued tokens changed. Existing tokens kept working until they expired, but mid-session token refreshes started failing because the front-end was calling the Cognito refresh endpoint, which still existed and still returned valid Cognito tokens, which the back-end now expected to be Auth0 tokens.

The fix: the back-end accepted both formats during the transition. We ran a middleware that checked both Cognito and Auth0 token signatures and let either pass for a defined cutover window (we picked 30 days — the lifetime of a refresh token). After 30 days, we hard-cut Cognito acceptance.

Lesson: any IDP migration with active sessions needs a dual-acceptance period on the resource server. Plan it explicitly. Don't assume it.

## Rollout cadence

We did not flip the switch for everyone on day one.

- **Week 1:** 1% of new logins routed through Auth0 (lazy migration enabled).

- **Week 2:** 10%.

- **Week 3:** 50%.

- **Week 4:** 100% of new logins routed through Auth0.

- **Weeks 5–10:** Active users continue to migrate themselves. Tail of dormant users runs out slowly.

- **Week 12:** Cognito put into read-only mode. Users who hadn't logged in yet got a one-time "click here to confirm your account" email.

- **Month 6:** Cognito decommissioned.

The percentage rollout was managed by a feature flag in the auth-routing layer. At each step we'd watch login error rates, MFA enrollment rates, and customer support ticket volume. If anything spiked we'd roll back the percentage. We rolled back twice, both times to fix the issues described above.

## What I'd do differently

If I were running this again from day one:

- **Build the dual-acceptance layer first, before any traffic moves.** We bolted it on under pressure. It should have been part of the v0 migration design.

- **Test MFA enrollment with at least 50 internal users before any external rollout.** The MFA path is the one that's most likely to surprise you.

- **Capacity-plan the migration script as a production service.** Don't assume "it's only running during the migration" means it can be best-effort.

- **Build a "migration status dashboard" on day one.** We had ad-hoc queries against the Auth0 management API. We should have had a real dashboard from the start showing migrated/total, MFA-enrolled/migrated, and error rates by category.

- **Communicate proactively.** Send users an email two weeks before the change saying "we're upgrading our login system, you'll see a slight visual change but otherwise nothing." This kills 70% of the support tickets.

## The first 30 days of post-migration monitoring

The migration is not done when the cutover happens. The first 30 days afterward are when long-tail issues surface — users on rare configurations, MFA edge cases, and the occasional support ticket that turns out to be a real bug.

The monitoring you want in place from day one of the cutover:

- Login error rate, broken out by error type. A spike in any single category is signal.

- MFA enrollment completion rate. Should trend toward 100% within two weeks. If it stalls, your enrollment UX has a bug.

- Support ticket volume tagged "auth" or "login." Compare to the same period before migration. If it's up 2x, something's wrong; if it's up 1.2x, that's normal noise.

- P95 login latency. Should be flat or better than pre-migration. A regression here means your custom database script needs caching.

Set thresholds for each metric and decide in advance what triggers a rollback. Decisions made in advance under no pressure are dramatically better than the ones made at 2am with a Slack channel full of customers.

## The takeaway

IDP migrations are one of those projects that look simple on a whiteboard and turn into ten-week trench warfare in execution. The reason is that authentication is the most stateful, most opinionated layer of your application — every other system depends on it being correct, and there is no graceful degradation. If auth is wrong for an hour, your product is down.

The lazy-migration pattern with a custom database connection is the right answer for almost any IDP-to-IDP move where the source has a callable auth API. The MFA problem and the dual-acceptance window are the two areas that will surprise you. Plan for them.

And the rule remains: no one should know it happened. If the success metric of an IDP migration is "users complain on Twitter," you've already lost. The success metric is silence.

## Read this next

- [**How I'd Run Security at an AI-Native Company in 2026**](/blog/running-security-at-an-ai-native-company-2026) — Identity is one layer of four. The other three are where AI-native companies usually have gaps.

- [**AI-Assisted Engineering Isn't Faster Coding. It's a New Workflow.**](/blog/ai-assisted-engineering-is-a-new-workflow) — How AI tooling changes how you'd execute a migration like this in 2026.

---

## How We Cut $350K From Cloud Spend in 6 Months (And What I'd Do Differently)

URL: https://sublimecoding.com/blog/cut-350k-cloud-spend-six-months
Published: 2026-04-13
Tags: cloud, GCP, Azure, multi-cloud, FinOps, engineering, AI startups, migration

**The migration that saves $350K is the same migration that costs $250K if you sequence it wrong.**

At Lavender, we moved an AI-first product from Google Cloud Platform to Microsoft Azure over a six-month window and netted $350K in infrastructure savings against the previous run rate. The number is real. The story is more interesting than the number, because the savings came from somewhere most cloud-cost write-ups don't talk about, and we made $50K worth of mistakes along the way that I'd avoid if I were running it again today.

What follows is the playbook, the wins we didn't expect, the mistakes that bit us, and a section at the end on when I'd refuse to do this migration at all.

## The trigger

We did not move clouds because GCP was bad. GCP is excellent. We moved for two reasons that compounded.

First, the product had become AI-heavy and our largest cost line had shifted from compute and storage to inference. Microsoft's commitment-based pricing on Azure OpenAI was meaningfully more flexible than GCP's equivalent terms at the time, especially for a startup our size. The negotiation room was real and large.

Second, our customer base was tilting toward enterprises that already had Microsoft Enterprise Agreements. A non-trivial subset wanted us deployed on Azure for procurement reasons. We weren't going to win those deals as a GCP-only vendor without major friction.

One of those reasons would not have been enough. Both together made the math work.

## The rule: lift-and-shift first, optimize after

The single most important architectural decision we made: do not redesign during the migration.

The temptation is enormous. You're already touching everything. Why not refactor the messy parts? Why not move from VMs to managed services while you're at it? Why not adopt that new pattern you read about?

Because every redesign multiplied the migration's risk and timeline by orders of magnitude. We chose to:

- Lift services from GCE/GKE to equivalent Azure VMs/AKS, preserving topology where possible

- Re-point DNS and verify functional equivalence under load

- Decommission GCP

- Then, and only then, start optimizing for Azure-native primitives

The lift-and-shift took roughly three months. The optimization phase ran for the next three. Both phases produced savings, but only the second phase produced large savings. Trying to combine them would have produced no migration and a lot of side projects.

## Where the money actually came from

I expected the savings to come from architecture. They didn't, mostly. Here's the breakdown of the $350K we actually saved, in approximate order of contribution.

### Contract negotiation: ~$180K

Microsoft's BD team was hungry for an AI-first startup that would publicly use Azure OpenAI. We came in with a credible threat of staying on GCP, six months of usage data, and a willingness to commit to a multi-year reserved spend.

The discount we negotiated, both on Azure compute and on AI inference, was meaningfully better than what GCP had offered for an equivalent commitment. This single negotiation accounted for about half the total savings.

The lesson here is uncomfortable for engineers: **cloud cost is a sales negotiation, not an architecture problem.** If your engineering team has not been told the actual rate card you're paying, they cannot evaluate whether to optimize or to renegotiate. Treat your cloud bill as a contract that gets renegotiated, not a fixed input.

### Instance right-sizing on the new platform: ~$80K

The lift-and-shift produced an opportunity. When we moved each service, we measured its actual CPU/memory/IO profile under real production load on Azure rather than what we'd *over-provisioned* on GCP three years prior.

About 60% of our compute footprint was sized 2x larger than its real workload required. Azure's instance taxonomy is slightly different from GCP's, so we couldn't just port the same SKUs — we had to think about it. Thinking about it surfaced the over-provisioning. Right-sizing during the move saved roughly $80K annualized.

If we'd just lifted the SKUs over and kept GCP, we'd have left this on the table. The migration was the forcing function that made us look at sizes again.

### Storage tier rationalization: ~$40K

We had years of data sitting in GCS Standard that was almost never read but was paying Standard rates. Moving to Azure forced us to inventory it. We moved cold archives to Azure Archive Storage and warm-but-rarely-read data to Cool Storage. The active set stayed on Hot.

This had nothing to do with the move except that the move made us look at the data. We'd been "going to get to that" on GCS for two years and never had.

### Inference spend on managed endpoints: ~$30K

The Azure OpenAI committed-tier pricing kicked in once we had three months of stable usage data. The negotiated rate beat our prior on-demand inference cost meaningfully. Smaller savings number than I'd hoped — most of the inference negotiation was already counted in the contract line above.

### Kubernetes consolidation: ~$20K

On GCP we had run two GKE clusters (production and staging) for historical reasons that no longer applied. On Azure we collapsed them into a single AKS cluster with namespace isolation. Smaller line item but pure profit going forward.

## The $50K we cost ourselves back

The mistake column. There were three.

**Egress costs we didn't model.** During the migration we ran services on both clouds simultaneously to validate parity. Cross-cloud calls produced GCP egress charges we hadn't fully forecasted. About $25K of unplanned spend over the three-month overlap. We could have minimized this by snapshotting test data into Azure once, instead of having Azure services pull live from GCP databases. Lesson: model the migration overlap period as its own line item before kickoff.

Snapshot strategies in the cross-cloud period are also worth investing in: data versioning, write-through caching, and a clear policy on which cloud is the source of truth at each stage.

**Reserved-instance lock-in we couldn't unwind.** We had committed-use discounts on GCP that didn't fully expire until two months after the migration completed. We paid for compute we no longer used. About $20K. Should have audited and timed the migration to the commit cycle.

**Observability tooling we double-paid for.** Our APM vendor billed by host count. During overlap we doubled host count and got the bill. Roughly $5K. Trivial in the bigger picture, but irritating.

Total dropped: about $50K out of $400K of theoretical savings. Net: $350K.

## The six-month timeline

For anyone planning a similar migration, the rough cadence:

- **Month -2 to 0:** Negotiation. Stand up an Azure tenant. Run pilot workloads. Get pricing in writing. This is where the contract savings come from. Do not skip this.

- **Month 1:** Move stateless services. Logging, metrics, web tier. Validate Azure parity for everything you don't store state in.

- **Month 2:** Move stateful services with active replication strategies. Databases, caches, queues. This is the dangerous month.

- **Month 3:** Cut over DNS. Run on Azure as primary. Keep GCP warm for rollback for two weeks.

- **Month 4:** Decommission GCP. End commitments. Final billing reconciliation.

- **Month 5–6:** Optimize. Right-size, consolidate, rationalize storage. This is where ongoing savings get baked in.

The first two months are where most teams blow the timeline. They underestimate the negotiation phase and how long it takes to validate stateful service parity. Both are worth getting right; both reward patience.

## When I would refuse to migrate today

Cloud migrations are massively over-prescribed. About four times out of five, when a founder or CTO asks me whether they should migrate clouds, the right answer is no.

Things that make me push back:

- **"We'd save 20% by switching."** 20% is not enough to justify a six-month migration. The opportunity cost of the engineering team is higher than 20% of cloud spend at any company under $50M ARR. Renegotiate your existing contract first.

- **"Their managed service is better."** Maybe. But the cost of porting all your code, dependencies, and operational muscle memory to a new managed service is large and rarely accounted for. Use better managed services where you are unless the gap is enormous.

- **"We're getting a free credit grant."** Free credits are a customer-acquisition cost the cloud is paying. If your decision is dominated by 12 months of free credits, you're optimizing for the wrong year.

- **"We want to be multi-cloud for resilience."** Multi-cloud for HA is one of the most expensive forms of theater in software engineering. Single-region failures are rare; cross-region within one cloud handles most of what people imagine multi-cloud handles. Unless your customers contractually require multi-cloud, run on one cloud and run it well.

The migrations that make sense are the ones where there's a structural reason — major customer demand, contract leverage, a real product capability difference, or a regulatory requirement. "We can probably save some money" is not enough.

## Treat the cloud bill as a living document

The biggest enduring lesson from this migration was not about migrations. It was about how to relate to your cloud bill in general.

For the three years before the move, our cloud bill was something the finance team looked at and the engineering team didn't. The infrastructure had been "right-sized" once, four years prior, and never revisited. The contract had been negotiated once, two years prior, and never revisited. Both were wildly suboptimal by the time we looked.

The fix going forward — and the practice I now insist on at every company — is treating the cloud bill as a living document with two scheduled review cadences:

- **Monthly:** a 30-minute review by the engineering and finance leads of top 10 line items, week-over-week deltas, and any anomalies. The goal is to spot $5K-a-month leaks before they become $60K-a-year leaks.

- **Annually:** a contract review three months before any major commit expires. Renegotiate from a position of leverage, with usage data in hand and a credible alternative quoted.

Neither of these is hard. Both are skipped at most companies because no one owns them. Assign an owner and run the cadence. The savings will be quiet but persistent.

## The takeaway

If the math works, sequence it like this: **negotiate first, lift-and-shift second, optimize third.** Do not redesign during the migration. Model the overlap period. Time the move to your existing commit cycle. Expect about half the savings to come from the contract renegotiation and the other half from the operational hygiene the migration forces on you.

And if the math doesn't work, do not migrate. Renegotiate where you are. Optimize what you have. The cloud you're already on is almost always the right cloud, until it isn't, and the math tells you when "until" arrives.

## Read this next

- [**AI-Assisted Engineering Isn't Faster Coding. It's a New Workflow.**](/blog/ai-assisted-engineering-is-a-new-workflow) — The other half of the runway extension — operational throughput at fewer engineers.

- [**SOC 2 Is a Revenue Tool, Not a Security Tool**](/blog/soc-2-is-a-revenue-tool-not-a-security-tool) — Same logic, different lever — treat compliance as a sales project, not a checkbox.

---

## From One Engineer to Fifteen: What Co-Founding Taught Me About Engineering Leadership

URL: https://sublimecoding.com/blog/from-one-engineer-to-fifteen-engineering-leadership
Published: 2026-01-12
Tags: leadership, founders, engineering management, scaling, hiring, team building

**I went from sole engineer to running a fifteen-person engineering organization over four years at PopSocial. The hardest lessons weren't about code.**

I co-founded PopSocial in 2015 and was the only engineer on the platform for the first six months. By the time I left in 2019, the engineering organization was fifteen people across backend, frontend, and DevOps. Most of what I learned in that stretch was not technical. It was about the messy, tactical, embarrassing-in-hindsight job of building and running a team while still also writing code.

Six lessons that I would have benefited from someone telling me explicitly on day one. Most of them I had to learn by getting them wrong first.

## Lesson 1: The day you stop coding is the day the team starts performing

For the first eighteen months I was both the engineering manager and the senior IC. I told myself this was the responsible thing to do as a founder — keep the burn low, keep my hands on the codebase, ship fast. The actual effect was that nothing else got built well.

I was the bottleneck on every PR review, every architectural decision, every onboarding session. Engineers waited on me to make decisions instead of making them themselves, because they could see I'd weigh in eventually. The team was technically functional but learning slowly because they had a senior IC two layers above them.

The week I stopped writing code in the main path of any feature, three things happened almost immediately. The team's PR cycle time dropped by about 40%. Engineers started making the architectural decisions I'd been making, and most of them were right. And I started having time to do things only I could do as the leader: hiring, fundraising, customer conversations, planning.

The lesson, simply: a founder who continues to be the senior IC is paying a hidden tax that compounds. Stop coding (in the main path) sooner than feels comfortable. You can keep your hands on by writing tooling, paying down infra debt, or pairing with an engineer on something hard — but get out of the production path.

## Lesson 2: You hire too slowly and fire too late. Plan accordingly.

The conventional wisdom is "hire slow, fire fast." Almost everyone gets the first part right and the second part wrong. The result is that bad hires linger for months while you tell yourself the situation will improve.

The most expensive hires of my career were the ones I should have ended at the 60-day mark and instead held on to for another five months because firing felt cruel. By the time I let them go, I'd lost five months of progress, demoralized the team that had been picking up their slack, and damaged my own credibility as a leader because everyone else could see what I couldn't.

The framework that finally worked: a written, explicit "first 60 days" expectations document for every hire. We'd review it at day 30, day 60, day 90. If we got to day 60 and the new hire wasn't on track to meet the day-90 expectations, we'd have a direct conversation. Most fixed themselves with that conversation. The ones that didn't, didn't, and we let them go at day 75 or 90 instead of month seven.

This is not heartless management. The opposite — telling someone explicitly what's going wrong and giving them a clear shot at fixing it is more respectful than letting them stay and dragging them along.

## Lesson 3: Title inflation is a tax you pay later

In year two we needed to hire a backend lead. The candidate we were closing wanted "Director of Engineering" as a title, even though the team was three engineers including him. I gave it to him. Five months later we needed to hire a senior engineer. We couldn't offer "Senior" because the existing team's titles had inflated past it.

Title inflation creates two compounding costs. First, it sets a ceiling that future hires have to be promoted past, which creates artificial pressure and weird performance dynamics. Second, it muddles the actual seniority of the team — when you have three Directors of Engineering and one IC, your engineering org chart is broken.

The fix in retrospect: pick a clean ladder, write it down, and don't deviate. "Engineer / Senior Engineer / Staff Engineer / Engineering Manager / Director of Engineering" — five rungs, defined, with explicit criteria for each. If a candidate insists on a higher title, that's a signal to dig into why. Almost always it's about cash compensation and there's a better way to solve the problem.

## Lesson 4: One-on-ones are the highest-ROI thing on your calendar

I started 1-on-1s as a checkbox item I felt I had to do. By the end I considered them the most leveraged hour on my week.

The format that worked, refined over time: 30 minutes weekly. Three sections — what's blocking you, what's bothering you, what are you working on. The first two are non-negotiable; the third is sometimes obvious from context and can be skipped.

The reason 1-on-1s are high-ROI is that engineers tell their manager problems an hour before they tell anyone else. By having the meeting on the calendar at a known cadence, you catch the problems an hour earlier than you would have otherwise. Over a year, that's the difference between a team that runs into the same brick walls repeatedly and a team that pivots before they hit them.

The mistake I made for too long: treating 1-on-1s as status updates. They're not. They're the channel for the things engineers won't say in standup or in Slack.

## Lesson 5: The interview process IS the culture

Whatever you do during an interview is what the candidate believes the company is like every day. If your interview is rushed, disorganized, and poorly calibrated, the candidate (correctly) infers the company is rushed, disorganized, and poorly calibrated. They take that information into their accept-decline decision.

I spent year three rebuilding our interview process from scratch. The big changes that mattered most:

- **Written rubrics for each round.** Every interviewer knew what they were evaluating and how to score it. Eliminated the "I dunno, I have a good feeling about him" bias that was driving who we hired.

- **Calibration sessions every six weeks.** The interview team would do a debrief on a recent candidate against the rubric, surface scoring disagreements, and recalibrate. Caught drift before it became a hiring problem.

- **A take-home that respected the candidate's time.** Two hours, not eight. With a clear evaluation criteria. We'd accept partial submissions if the candidate explained why.

- **A close-the-loop debrief 24 hours after the final round, written.** Every candidate got a yes/no decision in writing within a day. Even the no's. Especially the no's.

The team's hiring quality improved measurably. The brand impact among the candidate pool — who talk to each other — improved even more.

## Lesson 6: Performance reviews suck. Do them anyway.

Almost no founders enjoy performance reviews. Almost all teams need them. The mismatch is why they get skipped, deferred, or done badly.

The version that works at startup scale: light, honest, twice a year. Two questions in writing — what's going well, what's the one thing you'd change. From the engineer about themselves, from the manager about the engineer, from peers when relevant. A 30-minute conversation about both sides of the answers. A written outcome doc.

The version that doesn't work: annual 360 reviews with five rating dimensions and bell-curve calibration. At 15 engineers, that's bureaucratic theater. At 200 engineers, you grow into it.

The reason to do the light version even at small scale is that without it, you have no shared written record of expectations and progress. When you eventually need to make a tough call (promotion, demotion, termination, raise), you have nothing to reference. With it, you have six months of explicit notes from both sides about what good looks like.

## The thing that actually matters: clarity about what's good

If I had to compress all of this into one principle, it's this. **The job of an engineering leader at a small startup is to make it explicit what good looks like.**

Good code looks like X. Good design conversations sound like Y. Good 1-on-1s feel like Z. Good performance from this engineer at this level means producing this kind of output at this kind of pace. Good interviews ask these questions and listen for these answers.

Most engineering leadership failures at small startups are failures of clarity. The team doesn't know what good is, so the team isn't sure when they're doing well, and the leader is unhappy with the output but can't articulate why. Codifying "good" in writing — even sketchy first drafts — fixes more problems than any other intervention.

## Would I do it again?

Yes. But differently. I'd hire my first engineer slower and more carefully. I'd stop coding in the main path at month nine instead of month eighteen. I'd codify the title ladder before the second hire. I'd write the rubrics before the third interview. I'd start performance reviews at five engineers instead of ten.

The good news: most of these mistakes are recoverable, and even at the time, the team got better, the product shipped, and we ended up with a strong organization. But "got there eventually" is not the bar I want to hit if I do this again. The bar is "got there with the team intact, the leaders developed, and the company strong enough to handle the next stage."

That's the bar I'm using as I help founders make these calls now. Most of the lessons I'm sharing in this post are lessons I learned on the job. The point of writing them down is that the next founder doesn't have to.

## Read this next

- [**The Pre-Series-A AI Startup Hiring Plan**](/blog/pre-series-a-ai-startup-hiring-plan) — The role-by-role plan for getting the first six hires right.

- [**How I'd Hire a Staff Engineer at an AI Startup**](/blog/how-id-hire-a-staff-engineer-at-an-ai-startup) — The interview process I'd build today, in detail.

- [**AI-Assisted Engineering Isn't Faster Coding**](/blog/ai-assisted-engineering-is-a-new-workflow) — How modern engineering teams operate differently than the one I built in 2018.

---

## AI-Assisted Engineering Isn't Faster Coding. It's a New Workflow.

URL: https://sublimecoding.com/blog/ai-assisted-engineering-is-a-new-workflow
Published: 2026-03-16
Tags: AI, engineering, productivity, Claude Code, GitHub Copilot, OpenAI Codex, engineering leadership

**Most engineers using Claude Code see a 10–15% speedup. The teams seeing 40–55% aren't typing faster. They're sequencing work differently.**

I've been shipping production code with AI assistance since early 2023 — Claude Code, GitHub Copilot, and OpenAI Codex have been daily tools across an AI-first product, a fintech platform, and an IoT ingestion pipeline. The patterns that move the needle are not the ones most teams reach for first.

The frame "AI-assisted development" is part of the problem. It implies a tool that sits next to you, helping you type. The reality at the teams getting real leverage: AI is a teammate, you're the lead, and the workflow is fundamentally different from how you wrote code three years ago.

Here's what actually moved the needle.

## The myth of the 10x AI engineer

The default mental model is "AI as autocomplete." You type a function signature, you accept the suggestion, you save 30 seconds. Multiply by a workday and you get the 10–15% speedup that every benchmark study reports.

That's the floor, not the ceiling.

The ceiling is reached when you stop using AI to type code faster and start using it to *compress the steps before you type*. Architecture, naming, edge-case enumeration, test scaffolding, code review, documentation — most engineering time is spent on these, not on keystrokes. Compress those and the throughput change becomes structural.

The teams I've seen hit 40–55% delivery cycle reduction did three things differently:

- They stopped treating AI as a coding tool and started treating it as a thinking surface

- They standardized which model gets which job and didn't let everyone improvise

- They held the same code review bar — AI-generated code never gets a free pass

## The four modes I use AI in

Different problems want different uses. Mixing them up is most of why teams plateau.

### Mode 1 — Architect

Before any code: paste the problem, the constraints, and the existing code shape into Claude. Ask it to enumerate three approaches. Ask it to argue for and against each. Ask it which it would pick at this team size and why.

I do not accept its answer. I read the tradeoffs, find the ones I'd missed, and then make my own call. The win isn't in the answer — it's in the time saved enumerating possibilities I would have walked through anyway, just slower and less thoroughly. A 30-minute architecture conversation becomes a 7-minute one. Repeat that five times a week and the calendar opens up dramatically.

### Mode 2 — Ship

Once the design is settled, I have AI generate the boring 80%. Boilerplate, tests for happy paths, repetitive transformations, glue code, migrations. The interesting 20% — the gnarly state machine, the concurrency-sensitive bit, the contract with another service — I write myself, often after talking through it with the model first.

The discipline: I do not let AI write the code I would not want to read in two years. If a function is going to be load-bearing, I write it. If it's wiring three already-working pieces together, AI writes it.

### Mode 3 — Review

Before opening a PR, I paste the diff into Claude with a single instruction: "Review this like an adversarial senior engineer who hates my work. Find the bugs, race conditions, security issues, and unclear naming."

It catches a real bug or smell about 30% of the time. The other 70% is noise I dismiss. The noise dismissal cost is small. The 30% is enormous — every one of those is a comment my human reviewer doesn't have to write, and a deploy I don't have to roll back.

### Mode 4 — Document

READMEs, ADRs, runbooks, deprecation notices, release notes. AI is excellent at first drafts of all of these because they follow predictable structure and the underlying facts already exist in code or in my head. I dictate the structure and the key points; it produces the prose; I edit for voice. What used to take a couple hours takes 20 minutes.

Most engineering orgs are chronically under-documented because the marginal cost of writing it down is too high. AI changes that math.

## Where AI fails (and where I refuse to use it)

The teams in the 40–55% range are also disciplined about where they don't use AI. A short list of categories I always handle myself:

- **Brand-new libraries or unstable APIs.** Hallucination rate spikes. AI confidently writes code against a method that doesn't exist. The cost of debugging fake APIs erases any time savings.

- **Anything touching real money or auth without thorough human review.** AI doesn't have stakes. It will produce a payment flow that looks reasonable and fails open. I treat AI suggestions in these areas as drafts that must be reviewed line by line.

- **Performance work that requires measurement.** AI loves to suggest optimizations that look correct and are useless or actively worse. Profile first. Decide based on data. AI can help analyze the profile output, not pick the optimization.

- **Cross-cutting refactors.** The model sees the file, not the system. Ten files of "fix" that all individually look right and collectively break four invariants is a real failure mode.

- **"Vibe coding" without a spec.** If you can't tell the model what you want concretely, you don't know what you want. AI happily produces sprawl in this state. Specify, then code.

## The team buy-in problem

The hardest part of standardizing AI-assisted engineering across a team isn't tooling. It's the senior engineers who are skeptical, and they aren't wrong to be.

Their concern, usually unstated: AI undermines the craft. They've spent fifteen years getting good at code review, naming, architecture. A tool that spits out passable code threatens to flatten that gradient and make the median engineer look as good as the senior on the surface.

The reframe that lands: **AI raises the floor, not the ceiling.** A junior engineer with Claude is now operating at mid-level on routine work. A senior engineer with Claude is now operating at staff level on the work that matters, because they've offloaded the routine. The senior's edge — judgment, taste, system thinking — becomes more valuable, not less.

Concretely, what I've done at AI-first orgs:

- Standardize the tool stack: Claude Code for substantive work, Copilot for inline completion, Codex for one-off shell scripts. No improvising.

- Pair-program with skeptics. Show them their own ergonomics improving in real time. The 30-minute architecture chat becoming a 7-minute one is a visceral demo.

- Hold the same code review bar. AI-authored code goes through the exact same review process. No "the AI wrote it" exemptions. This was the single biggest credibility move with the senior bench.

- Make the patterns visible. Document the four modes (or your team's version), share examples of good and bad use, retro on AI-related bugs the same way you'd retro any incident.

## Measurable outcomes

The numbers I've actually seen:

- **Delivery cycle reduction:** 40–55% on product feature work. Bigger than I expected. Smaller than the AI vendors claim.

- **Quality:** bugs-per-PR ratio held steady. Production incident rate held steady. This is the number that surprised people most — most assumed AI-authored code would be lower quality. With proper review discipline, it isn't.

- **Headcount:** stable team output went up roughly 50%, with no headcount growth. That's the headline finding. AI didn't replace engineers — it amplified them.

- **Where it didn't help:** infrastructure / platform work saw maybe 10–15% gains. The work is too contextual, too specific to your environment. Don't expect the same speedup for a backend platform team as for a product feature team.

One caveat worth naming. The teams I've measured are small (5–15 engineers), high-trust, with clear technical leads. I have not yet seen what these patterns look like at 200 engineers across multiple business units. Some of what works at this scale is going to break at that scale. Be skeptical of anyone claiming universal numbers.

## Picking the right tool for the job

Standardizing the tool stack matters more than picking the "best" tool. Three tools, used consistently, beat seven tools used haphazardly.

The split I've landed on:

- **Claude Code for substantive work.** Architecture conversations, refactors, multi-file edits, code review, anything that requires the model to hold the shape of a feature in its head. Claude's longer context window and more conservative coding style fit this work better than the alternatives I've tested.

- **GitHub Copilot for inline completion.** The autocomplete-style use case. Fast, low-stakes, reduces typing fatigue. I let it complete the obvious next line; I do not ask it to design anything.

- **OpenAI Codex / GPT-5 for shell scripts and one-offs.** Quick scripting tasks where I want a command-line answer in seconds. Different ergonomic register than the in-IDE tools.

The principle: each tool has an interaction model that's good for a specific type of work. Mixing them up — using Copilot for architecture, using Claude Code for autocomplete — wastes the strengths of each. Pick a tool, use it for what it's good at, switch when the task changes.

Two practical notes for teams adopting this. First, pay for the paid tier on whichever you use most. The free-tier rate limits will produce flow-state interruptions that destroy the productivity gain. The $20–60 per engineer per month is one of the highest-ROI line items on your engineering bill. Second, make the tool choices explicit in onboarding. New engineers should not have to figure out the team's AI workflow by osmosis.

## Why this matters for founders

If you're pre-PMF, this is your edge over slower-moving competitors.

Funded competitors with bigger teams will out-spend you. They cannot out-iterate you if you've internalized this workflow and they haven't. A 5-person team operating at 50% throughput multiplier is shipping at the velocity of a 7- or 8-person team — and at a fraction of the burn.

If you're post-PMF and scaling, this is how you delay the headcount conversation by six months. That's six months of runway, six months of org-design time, six months of hiring more carefully.

The shift is real. It is not magical. It rewards engineers who treat it as a workflow change rather than a tool swap. Pick up the tool, and then put in the work to actually change how you work.

## Read this next

- [**How We Cut $350K From Cloud Spend in 6 Months (And What I'd Do Differently)**](/blog/cut-350k-cloud-spend-six-months) — Where AI-augmented teams generate the runway extension that pays for the next round.

- [**How I'd Run Security at an AI-Native Company in 2026**](/blog/running-security-at-an-ai-native-company-2026) — What you have to be paranoid about when AI is generating production code.

---



---

End of file. For the structured guide and the full archive, see
https://sublimecoding.com/llms.txt and https://sublimecoding.com/blog.