Testing AI Agent Outputs in Elixir with ExUnit
An ExUnit suite for agent loops in Elixir that asserts on tool-call order and hallucinations as well as final strings, plus evals that survive model upgrades.
We can't find the internet
Attempting to reconnect
Something went wrong!
Attempting to reconnect
Writing · Tag
62 posts on AI, page 4 of 6. Or browse the full writing index →
An ExUnit suite for agent loops in Elixir that asserts on tool-call order and hallucinations as well as final strings, plus evals that survive model upgrades.
Token-by-token LLM streaming in Phoenix LiveView without React: the 2026 async APIs, plus the production failure modes most tutorials skip.
One operating system runs a team whether the teammate is a person or an agent. My blueprint: trust, pods, outcomes over hours, managing agents like interns.
Classic SRE postmortems can't explain agent incidents. The sections to add (decision-time context, autonomy rung, permissions delta), with a template.
Your accumulated agent memory is a bet on one vendor's format. Build a portable, git-versioned knowledge layer any agent can read instead.
I shipped more code in 2026 than the previous four years combined. The commits are real. The productivity is real. What I lost is harder to measure.
An homage to Peter Welch's Programming Sucks, updated for agents: a genius intern with amnesia, hallucinated packages, and a closet that eats your auth layer.
Prompt injection defense a five-person team can ship in a week: trust boundaries, least-privilege tools, approval gates, and what not to build yet.
Most AI agent frameworks reinvent a job queue badly. Oban already is one: durable, idempotent, retry-aware. Run an agent loop that survives a deploy mid-run.
Amazon mandated AI coding, let it touch infrastructure unwatched, and lost millions of orders. The fix was more humans per deploy.
Apple raised prices on Mac, iPad, and HomePod, blaming an AI-driven memory shortage. The cost of intelligence is falling; the hardware to run it isn't.
Eight failure patterns I see running AI coding agents daily: the confident wrong answers, the lost context, and the bugs they reliably ship.