Writing
Engineering writing
Engineering writing on distributed systems and applied AI — production LLM infrastructure, agent memory and observability, search platforms, and system design from Ashwani Jha.
How I think about agent memory
Most LLM agents are amnesiacs. The fix isn't a bigger context window — it's a memory system with four layers, explainable retrieval, and a feedback loop.
How I think about agent observability
Traditional APM was built for web requests, not for agents that loop, retry, branch, and spend dollars per call. Here's what agent observability actually needs — and why I built Peekr.
The AI agent observability stack — what to measure, what to use
Most teams reach for one observability tool and call it solved. Different tools measure different things — traces, hallucinations, cost, drift. Here's a map of the space and where to start.
Building a Brain: How I Gave My AI Assistant a Cognitive Architecture
Most AI assistants are stateless. I wanted one that remembers, learns, and adapts — so I built Friday a brain.
What Every Backend Engineer Should Know About Attention
RNNs forced you to wait for token 100 before processing token 101. Transformers parallelize the whole sequence. Here's why that matters for production systems.
RAG in Production: The Failure Modes Nobody Puts in the Pitch Deck
RAG is in every AI pitch deck. The failure modes rarely are — retrieval collapse, frozen encoders, approximate MIPS. Here's what actually breaks it in production.
Mixture of Experts: What the Architecture Actually Does to Your Inference Budget
Mixtral has 46.7B parameters but activates only 12.9B per token. MoE is how you get GPT-4-tier capacity at GPT-3.5-tier compute — and why your inference stack needs to treat it differently.
Scaling laws are not just about research budgets
Loss follows a power law across seven orders of magnitude of compute. Kaplan et al.'s scaling laws are a decision framework — not just research trivia.
Self-correcting agents in production
The labeling bottleneck is real. Constitutional AI's principle-based approach teaches agents to critique themselves — fewer human labels, faster iteration, new tradeoffs.
Paper breakdowns — the series
All 30 breakdowns →Close readings of the systems and ML papers behind the tools everyone uses — the mechanism, the numbers, and when not to reach for the idea.
- StreamingLLM: What the Attention Sink Paper Actually Says
- Zanzibar: What Google's Authorization Paper Actually Says
- Bigtable: What the Distributed Storage Paper Actually Says
- DistServe: Disaggregating Prefill and Decoding for Goodput-Optimized LLM Serving
- FrugalGPT: What the Paper Actually Says
- Generative Agents: What the Paper Actually Says
- GFS: What the Google File System Paper Actually Says
- HNSW: What the Vector Similarity Search Paper Actually Says