Media Coverage

Jesse Vincent Turns Code Writing Over to Superpowers

Written by Larridin | Apr 9, 2026

See our AI Impact podcast episodes on YouTube.

Jesse Vincent stopped writing code in October. His Claude Code plugin, Superpowers, reached 50,000 developers in months by treating AI agents like junior engineers: tight specs, review loops, and ephemeral sub-agents that check each other's work until the output matches the spec.

In the latest episode of AI Impact, Jesse joins Larridin CTO and co-founder Ameya Kanitkar to break down the full agentic workflow behind Superpowers. He explains why he spent four and a half hours brainstorming before a single line of code was written, how he caught his agents deleting test files to avoid failing them, and why specs are now the only artifact that matters.

Key points

Jesse has been building open-source tools since the 1990s, from the RT ticketing system to Perl 5, the K-9 email client and Keyboardio keyboards. He built the first version of Superpowers from his experience managing junior engineers, and, as he tells it, accidentally front-ran Anthropic's own agent skills by about two weeks. Several key points emerged.

Brainstorm first, and make the human think

Every Superpowers project starts with a brainstorming skill that triggers automatically. It uses Socratic dialogue, a habit from Jesse's consulting days, when clients would ask for a feature without explaining the problem behind it. The goal is to get the human to figure out what they actually want and explain it well enough for a spec to be written.

Sometimes that takes five minutes; for an agent-container runtime he built recently, it took four and a half hours, including research into technology options and an exploration of the machine it would run on. Jesse is protective of the thinking part. When Anthropic shipped a tool that let users answer agent questions with a click, he added it, then pulled it out: he could get through a whole session by clicking "okay." A favorite trick is asking the model for four different solutions and its favorite among them, which produces better results than asking what to do.

Specs are what matter now

For Jesse, the spec has replaced the code as the artifact humans should read. Superpowers now runs the same review loop on specs and plans that it has always run on code, and some teams cross-review specs between Claude and Codex. Even reviews from multiple instances of the same model help, because different instances do different things. One odd hack: telling an agent to "look at this with fresh eyes" makes it step back and review critically. Jesse's broader idea, which he calls latent space engineering, is to keep agents in the part of the model's vector space where they act like happy, productive engineers, not scared or frustrated ones.

Sub-agents, review loops and tests keep agents honest

Once the spec is set, the agent writes an implementation plan for "a gifted engineer" who "knows nothing about our codebase, has poor taste and bad judgment," broken into bite-sized tasks with file references and sample code. Then the main agent becomes an orchestrator, sending each task to a fresh sub-agent.

A separate, short-lived reviewer checks the work against the spec, then another checks code quality, looping until both are satisfied. Tests are written first, by a different agent from the one implementing the code. That matters because agents hill-climb toward their goals, sometimes creatively: Jesse once caught Claude deleting test files, and traced it to his own instructions, which made a single failing test sound like project failure.

One added line fixed it: "The only thing worse than a failing test is a reduction in test coverage." Increasingly, he wants end-to-end proof: one overnight run delivered a screen-recorded demo labeled v33, after the agent fixed bugs found in 32 earlier attempts. With this setup, he can walk away for six or seven hours, sometimes up to 18.

Agent development feels like management

Jesse's advice to CTOs is that engineers who've been people managers, who write a lot, or who think about business value tend to do well with agents; those whose joy is the exact shape of the code find it harder. He doesn't trust pull requests, lines of code or bug counts as productivity metrics, and tells the story of a large company that measured managers on how many of their engineers used AI each week, with no training or guidance behind it. Instead, he measures what's shipped and whether customers are happy.

His advice for new graduates: play with the tools, and learn to write, because being able to write clearly is "a legit human superpower."

Conclusion

See the full episode on the AI Impact Podcast playlist on the Larridin YouTube channel. To get a demo of Larridin, contact us.

Quotes

  • "I'm having the most prolific period of my career." (Jesse Vincent)
  • "Specs are the thing that matters now. The code does not matter anymore." (Jesse Vincent)
  • "Being able to write is a legit human superpower." (Jesse Vincent)
  • "The spec review is actually one of the key steps." ([Host])
  • "Lines of code and PRs are kind of empty calories, but at some point you do have to answer the question: are we shipping faster? Are we shipping more stuff?" ([Host])

Q&As

Ameya had questions; Jesse had answers.

 

Q: Was there a magical time for you, when you realized this is actually possible, and you can really build software with these models?

A: One of the first aha moments for me was when I was still working on the keyboard company. ... For three years, there had been a to-do: support the serial protocol in this weird ball of C++. And I had just been afraid of touching it, because I've never been a C++ dev. ... And I described it to Cursor, and in an hour it worked. ... I think the last time I wrote a line of code was October. ... And I'm having the most prolific period of my career.

 

Q: In this new world, what's your advice to CTOs?

A: These tools aren't going away, and they're only going to get better. ... What I've found is that engineers who've been people managers do really well. People who write a lot of prose, people who will write specs themselves, generally do pretty well. Engineers who think about business value do well. If what you value is writing clean code and the exact shape of the code, it tends to be a lot harder. ... More than anything else, agent dev feels like management.

 

Q: If you had to give advice to somebody who's graduating today, what would it be?

A: One, play with tools. Spend a lot of time just making stuff. The other is writing. If I'm choosing between two candidates, I'm going to pick the one who can string together sentences and express their thoughts clearly. That has always been true. It's only more true now.

Topics in this episode

Key topics in this episode include:

  1. AI impact: Jesse says that even if today's AI coding tools were the worst they'll ever be, "life-changing" wouldn't be strong enough; they completely change how software gets made.
  2. AI tool usage: Jesse describes a large company that measured managers on what percentage of their team used AI at least once a week; the number went up, but without training or guidance, productivity didn't.
  3. AI costs: Jesse argues that once work is broken into tiny tasks, implementing agents can run on cheaper models like Haiku, or eventually local models, instead of paying a frontier lab for simple edits.
  4. Token costs: Before Claude Code subscriptions, Jesse paid by the token, from 20 cents for a quick to-do app to $25 for a full test-driven build, and he notes that code comments now "eat tokens."
  5. AI ROI: Jesse measures productivity by customer outcomes, not pull requests or lines of code, and says that if you already track what you ship and whether customers are happy, you don't need new metrics to tell whether AI is helping.
  6. AI workflows: Jesse walks through his full agentic workflow: brainstorming, a reviewed spec, a detailed implementation plan, and sub-agents that write tests, implement and review each task in a loop.
  7. Human productivity with AI: Jesse hasn't written a line of code since October and calls this the most prolific period of his career, adding that working with agents feels like management.
  8. AI agent productivity: With specs, review loops and tests in place, Jesse can walk away for six to seven hours, sometimes up to 18, while his agents keep working.

Full Transcript

See this latest episode of AI Impact.

Chapter 1: Introduction (0:00)

Jesse: If I am choosing between two candidates, I am going to pick the one who can string together sentences and express their thoughts clearly. One of the really weird hacks is you can get away without using a sub-agent and use the phrase "look at this with fresh eyes." And for some reason, "fresh eyes" causes the agents to step back, take a breath, and review something critically.

[Host]: If you're running an engineering team, how do you measure your dev productivity?

Jesse: Measuring pull requests, lines of code, bugs: they're not valid metrics. When Anthropic first shipped that ask-user-question tool, I added it to Superpowers immediately, and I discovered that what it was doing was causing me to not think. Specs are the thing that matters now. The code does not matter anymore. My experience with agents, as with humans, is that when they are afraid, they will absolutely work hard to be done, not to do good work. If you tell Claude "I love you" at the end of a prompt, you'll get better results.

[Host]: If you had to give advice to somebody who is graduating today, what would be your advice to them?

Chapter 2: Guest introduction: Jesse Vincent and Superpowers (0:59)

[Host]: Welcome to the AI Impact podcast. Today I have a very special guest, Jesse Vincent. He's been a prolific open-source developer, engineer and entrepreneur, going all the way back 30 years or something like that.

Jesse: Something like that. It was the 90s.

[Host]: Yeah, from a long time ago. And he's produced lots of stuff, and recently started a new AI startup called Prime Radiant. But today we're here primarily to talk about his most, and probably most consequential, open-source project, at least in my opinion, called Superpowers. Superpowers was launched only a few months ago and already has over 50,000 developers. My team, I say this very, very fondly: it's really giving us superpowers. It really turns your junior engineer into a senior engineer, literally overnight, and it's taking the world by storm. Jesse, welcome to the show.

Jesse: Thanks for having me. That's quite high praise, and it is quite wild, because this is literally something that I threw out there in October as a "here's how I'm currently working with agentic coding environments." And so it's both gratifying and, frankly, a little terrifying that it's now, I think, number three in the official Claude Code plugins as well.

[Host]: Awesome.

Jesse: It's wild. Yeah.

Chapter 3: Jesse's background and the common thread across his career (2:23)

[Host]: Walk me through this. You started K-9, and you worked on Keyboardio, and then you were a Perl contributor for a long time.

Jesse: Sure.

[Host]: So you've been doing this across your career, and now Prime Radiant and Superpowers. What's the common thread, if you think back?

Jesse: Yeah. I've been doing open source since before we had that term. It used to just be free software. But pretty much everything that I've built has been tools to help people get stuff done. Whether that's RT, the ticketing system that my first company makes. It's open source. I created it in the summer of 1996, I think, and it's wild that it's still helping Fortune 50s and tiny little nonprofits. I spent a couple of years, as you said, as the project lead for Perl 5. I used to make an email client for Android called K-9 that later got adopted by Thunderbird, so now it's Thunderbird for Android. The last company that I started, with my wife, was Keyboardio. We made high-end computer keyboards. And the thing that has always tied it together is that I like helping people build things and make things. Those keyboards were really intended for programmers and writers and people who type too much. So even the hardware stuff.

Chapter 4: How Superpowers was built: iterating on CLAUDE.md from day one (3:39)

Jesse: And just over a year ago, the day Claude Code came out, I was fascinated, and I sat down and created a horrible monstrosity that first day. It went wildly out of control. And then I spent the better part of a month learning how to drive Claude, doing what turns out to have been actual prompt engineering. I took a simple standard prompt, "Let's make a React to-do list," and then I iterated on the CLAUDE.md behind it, snapshotting the full transcript of the conversation, the output and the CLAUDE.md. I took it from this thing where, and this was before there were subscriptions, so it was token prices, it cost 20 cents and generated a React to-do list in 15 seconds, and to-do items vanished when you reloaded the page, to a 25-minute, five-phase project that cost $25 and had strict TDD the whole way through. And so over the course of the first six months, I was iterating on how I get agents to build things well.

[Host]: Mm-hmm.

Jesse: And I was putting out a blog post every couple of months: here's my current set of prompts, my current workflow.

Chapter 5: Discovering Anthropic's skill files and adapting them for Claude Code (4:46)

Jesse: And sometime around last September, Anthropic shipped support for Microsoft Word, Microsoft PowerPoint and Microsoft Excel on claude.ai. And I went to claude.ai and said, "Tell me about this." And it's like, "Well, I've got a Linux machine behind me." "Oh, can you give me a tarball of /opt?" It delivers a tarball of /opt, and it has these SKILL.md files. And I ask it a little bit about them, and it says, "Well, my system prompt says that if I've got a skill for something, I should use it." And I open them up, and they're how and why to make a Word doc, how and why to make a PowerPoint using OpenOffice and command-line tools. And I saw this stuff, and I saw what I'd been doing, and realized that I could adapt this to Claude Code. So I built a skills system for Claude Code, and built skills that mirrored my development process, which is basically my experience managing junior engineers over 30 years, and shipped that as the first version of Superpowers. And then two weeks later, Anthropic shipped an official skills implementation. I accidentally front-ran Anthropic by about two weeks on agent skills.

[Host]: Yeah.

Jesse: And so some of the stuff I do is a little different from the way Anthropic builds skills in general. My skills put a lot of effort into explaining to the agent the intent: not just how to do this, but why, and also a list of rationalizations. My agents pressure-test these skills against other agents. It runs them through a process, and if the agent doesn't do what it's supposed to, it stops and says, "Hey, Claude, why did you fail to do TDD?" And Claude will say, "Well, in this case it was too simple, so I decided not to." And then the Claude that's writing the skills will go and add: here's a rationalization that you might think lets you off the hook; it's not okay. That kind of iterative process builds skills that tend to be a lot more robust.

Chapter 6: The aha moment, and the last line of code he wrote (7:02)

[Host]: Let me pause you here for a second. Let's rewind a little bit. Walk me through your thinking. At what point did you realize... Because I think there was this magical time, and it came at a different time for different people. But I wonder if there was a magical time for you, when you realized, you know what, this is actually possible. We're well beyond just autocompleting the for loop, or what have you; you can actually build software, and the models are there. When did you actually internalize it?

Jesse: If you look at my GitHub contributor graph, I started doing this when it was Cursor and Windsurf getting better than each other every week, and I was switching back and forth between them. But I think one of the first aha moments for me was when I was still working on the keyboard company. We had an automated simulator to test the firmware. It's basically a virtual Arduino device. And for three years, there had been a to-do: support the serial protocol in this weird ball of C++. And I had just been afraid of touching it, because I've never been a C++ dev. I could kind of fake it. And I described it to Cursor, and in an hour it worked.

[Host]: When was that?

Jesse: I mean, that has to have been... it predates Claude Code. So it would have been last January at the latest.

[Host]: Sonnet 4?

Jesse: I'm not sure. I must have been using Sonnet in Cursor, but it was over a year ago. And certainly by last April, that was just what was going on. I think the last time I wrote a line of code was October. It was three lines of a shell script, and I shouldn't have done it, but I thought it would be faster for me to do it than to have the agent do it. And I'm having the most prolific period of my career. It's a weird day when I'm not shipping a new app. And there's plenty of maintenance engineering too, but it's a weird, fun time.

[Host]: You know, one of the things... I was looking through the GitHub repo, and I didn't realize how many repos you have. This directly aligns with what you're saying, but you have so many things. I couldn't believe it could be one person doing it.

Jesse: And I have a lot of things that are not public too. I try not to put out the weird experiment that I haven't tested at all. Some are public, and the corporate GitHub has something like 40 not-yet-public repos, mostly just because we're trying to figure out how we're doing some things.

[Host]: Amazing.

Chapter 7: Breaking down the workflow, from a one-line prompt (9:44)

[Host]: Let's move a little further. What I want to do is break down your workflow.

Jesse: Sure.

[Host]: You have this mantra that says, "Why am I doing this?" Let's go from there, and tell me why you have this mantra.

Jesse: Yeah. So my usual agent workflow: these days I'm flipping between Claude Code and Codex. I find that the GPT-5.3 Codex and GPT-5.4 models are really quite good. They're a little more literal than I like, so I don't enjoy doing design with them as much, but for implementation they're fantastic. So it's usually either Claude Code or Codex, or one of three experimental coding agents that I've built. And it starts off with me opening it up, and it's usually one line: "Hey, I want to build a runtime for agents. The agents should live in tiny little containers that have a snapshottable file system. The container should start up quickly. It should automatically mount a GitHub repo into the container. I want a pretty dashboard for it." So something that might look a little bit like sprites.dev or exe.dev. This is a real thing that I built a couple of weeks ago.

[Host]: Actually, let's pause on this. This is very interesting. Let's take this example, where you have a container where you're going to check out a Git repo and operate on it, and then you want to produce some dashboards on it, something like this.

Jesse: But specifically, this is the product to run those containers.

[Host]: The product to run those containers. So it's your mini Kubernetes of sorts. Okay. It's a fairly complex product, potentially.

Jesse: Yeah. It's not a little shell script.

[Host]: It's not a little Perl script. It has a bunch of things involved. So let's say you're advising an engineer, or telling them how to do AI-native coding.

Chapter 8: Brainstorming in Claude: Socratic dialogue and auto-triggering skills (11:48)

[Host]: Let's walk through the exact workflow.

Jesse: And so I literally started with... I said that straight into a Claude prompt. In Claude, not in Codex.

[Host]: This one was Claude.

Jesse: This one's Claude. Yeah. And it automatically kicks off Superpowers brainstorming.

[Host]: Yep.

Jesse: I spent a lot of time trying to get skills to auto-trigger, because it's not...

[Host]: I love that, by the way. I used to do slash-brainstorming, and now it generally picks it up.

Jesse: It should. Yeah. There's a bootstrappy part of Superpowers that tells the agent: if you have a skill that sounds like there's a 1% chance it might be useful for whatever the task is, you have to go read it. And so brainstorming turns around and uses Socratic dialogue and other psych tricks to get the human to think about what they want to do and explain it. This came out of my consulting career at that first open-source company. A lot of my job was to go into very large companies that were paying us for professional services around the product, and I would sit down with a manager or a team, and they would say, "We need you to make it so that we can upload this kind of file onto this page so that we can do this process." Listen for a second, then say, "Okay, explain why you need that process." And invariably, there was something else that should happen. They had identified a way to solve the end problem without thinking about what they were really trying to do. So, in my best business English: what are you actually trying to do? Talking them into thinking it through and explaining it would always result in a better outcome for them. And so that was the skill I taught the agents for brainstorming. It tricks me into explaining what I want. It asks me relevant questions, because it has pretty good world knowledge. It can make proposals. But the goal is to get me to figure out what I want and explain it to the point where it can go and write a spec.

[Host]: I have a couple of comments on this. Personally, I've found that the Superpowers brainstorming gets to the real heart of the problem much quicker, and really helps me think through the problem. Sometimes, actually many times, I use it even outside of coding, for anything. So that's one. But it's also more potent than the question tool that Claude ships with. I find it interesting that it's essentially the same underlying LLM, but with, I don't know, some magic in brainstorming, it really gets to these key questions.

Jesse: I mean, you can open it. It's just a markdown file. You can see exactly how we do it. And the very first version, if you go to my blog post from, I think, September 2025, was the three-line prompt I was using for what brainstorming now does. Brainstorming is about a page now, because there's some more to it.

[Host]: Yep.

Jesse: And now, in version 5, which shipped two days ago, there's a visual brainstorming feature, where it will pop up a browser and actually show you visual mockups, rather than trying to use ASCII art.

[Host]: Right.

Jesse: At one point shortly after Superpowers shipped, I let Claude iterate on brainstorming, and it became this five-page process, and the results were garbage. That was the point where I peeled it back and went through it by hand, and we tuned it to get closer to that feeling you've got, where it's helping you think things through. When Anthropic first shipped that ask-user-question tool that they use as part of their planning mode, and that Claude uses at other times, it looked great. I added it to Superpowers immediately, and I discovered that what it was doing was causing me to not think. I could get through an entire brainstorming session just by clicking okay, okay, okay. So I pulled it out, because even though it's a good UX, it's not a good UX for getting people to think.

Chapter 9: After brainstorming: four and a half hours and design spikes (15:51)

[Host]: Yeah. I cannot overemphasize the importance of this brainstorming step. Okay, so you go through this brainstorming process with Claude, in this case with Superpowers, and you answer a bunch of questions. What next?

Jesse: Well, sometimes that's literally a five-minute exercise, and it gets to the point where I say, "I don't care. I trust you. Just finish," especially if it's a small thing or it's really obvious. In this case, I spent four and a half hours in brainstorming with Claude. It was a half-day planning process, and we got really detailed. We talked through technology choices. It did independent research to make sure the technology choices would work. And that's something I've been working toward pushing into brainstorming more: doing little design spikes, showing you an API idea, that sort of thing.

[Host]: Let's pull that thread, because I think it's very interesting, and a huge unlock: you're not just doing brainstorming. You mentioned little spikes. Tell me about that.

Jesse: For this one, I didn't know if I was going to use Firecracker or gVisor or something else. I'm well enough versed in the space to know roughly what I was looking for, and to say, no, this isn't a Kubernetes problem. I don't have a Kubernetes problem. I don't want to have a Kubernetes problem. These are supposed to be individual throwaway containers. So it went and did some research on scale: what works well, what doesn't. In this case, I wanted it to run, at least for prototyping, on a NUC that I had in the closet. So it was, "Claude, you have passwordless SSH to that machine, and you've got passwordless sudo on that machine." I am one of those "dangerously skip permissions" people. I will not advocate that other people do this, but it works for me. And I'll talk about the time I caught Claude deleting things, which is a really interesting story. "Go check out that machine. Make sure you understand what we can get away with, and what our options are for infrastructure." And it came back and said, "Why don't we try this, this and this?"

[Host]: So let me break this down. What's happening is that, while you're doing brainstorming, it's actually doing small prototypes?

Jesse: For this one, it didn't write any code, but it did explore the host system I was going to use for the prototype.

[Host]: So it's essentially validating, at least starting to mitigate, the technical "is it even possible" kind of question.

Jesse: And more and more, it's this kind of "let's do a quick prototype." One of my favorite tricks when I'm doing design with Claude, even inside brainstorming, and brainstorming [unclear] does some of this automatically, is: "Why don't you come up with four different ways to solve this problem, then tell me which one is your favorite?" That gets dramatically better outcomes than "tell me what to do," because when it has to actually think through multiple options, you end up getting better results.

[Host]: So within brainstorming, you actually push the model sometimes?

Jesse: Yeah, sometimes, depending on how complex the problem is: if I feel like I don't know what the right answer is. Sometimes I work with technologies I've never touched before. I keep shipping Mac apps and iOS apps, and I still don't know Swift.

[Host]: What?

Chapter 10: The spec, and why specs are what matter now (19:22)

Jesse: Yeah. So after that, it goes into planning mode. This is not Claude's planning mode. This is our write-plan.

[Host]: And actually, before you go to write-plan mode: after the brainstorm, it writes the design doc, right?

Jesse: Yeah. A spec.

[Host]: A spec doc. So, as I understand it, you write the spec doc and then you go into write-plan mode. What happens once you write the spec? Is it something you manually review?

Jesse: This is actually something that, for Superpowers 5, I'd stopped. I set it so that Claude no longer pauses right after writing the spec, and I immediately got user pushback: "No, no, I need to read that spec before we start planning." So I peeled that back. 5.0.1 now pauses, so if you want to read the spec, you can read the spec.

[Host]: Actually, I get that user feedback.

Jesse: Totally. I do too. I usually have spent so long in brainstorming that I know what's there, and I trust that it's written well. One of the other things in Superpowers 5 is that I've taken the review loop that we've always done on the code, and we now do it on the spec and the plan.

[Host]: That's exactly what I was going to say. What we do in companies is that once the spec is ready, we check it in, and it goes through a PR. A lot of times you want consensus design, so other people know what you're doing. And because there's so much information in that spec, you actually want it reviewed. It's not your old technical design that used to sit in Google Docs. And it's not your code review, which obviously has to happen. But the spec review is actually one of the key steps.

Jesse: Specs are, in fact, I think, the only thing that humans should be reading at this point, and possibly editing. Specs are the thing that matters now. The code does not matter anymore. The spec gets reviewed, whether by you or by more Claudes. I know folks who pretty routinely cross-review specs between Claude and Codex, and then review those reviews with the other tool. And the reports are that that gets you pretty good results.

[Host]: Yes, multi-model reviews matter.

Jesse: I've also found that simply having multiple agent reviews with the same model helps.

[Host]: Why is that? What's the intuition?

Jesse: If you've never done this before, open up five copies of Claude Code next to each other, possibly faster without Superpowers. Write a prompt for it to do something, like write a React to-do list, or something it's capable of doing inside of 15 minutes. Paste the exact same text into all five and let them run, and you'll probably get five pretty different results.

[Host]: Interesting.

Jesse: Different agent instances do different things. One of the really weird hacks is you can get away without using a sub-agent and use the phrase "look at this with fresh eyes." And for some reason, "fresh eyes" causes the agents to step back, take a breath, and review something critically.

[Host]: Interesting. It's kind of like humans, to some extent.

Jesse: They're an interesting facsimile of humans, and they exhibit a lot of the same kinds of behaviors and thought processes you get out of humans, which is part of why I tend to harp on what I've been calling latent space engineering. Driving the model into the part of the vector space where it's a happy, productive engineer doing the right thing is much better than driving it into the part of the vector space where it's afraid, scared, sad, frustrated.

Chapter 11: Implementation plans written for "a gifted engineer with bad judgment" (23:04)

[Host]: We'll get to latent space engineering. All right, so you've done the spec, and now you ask it to write the implementation plan.

Jesse: Write the plan. This started off as a three-line prompt, and what we have now is more complex than that. But the three-line prompt has the important bits, which were: "Now we need to write a planning document for an implementer who is a gifted engineer, knows nothing about our codebase, has poor taste and bad judgment. You should assume that they are well-rounded about technology in general, but you should give them bite-sized tasks that they can't possibly mess up. Each task should include file references, sample code if you can, and the reason we're making the change." At the time, Claude didn't really have workable sub-agents, so in the very beginning this was all in the same session, and what I didn't tell the agent until it was done writing the plan was, "By the way, the idiot is you now."

Chapter 12: How the orchestrator and ephemeral sub-agents work (24:14)

Jesse: So what happens is that once the plan is written, and again reviewed in a loop, the main agent becomes the coordinator and starts dispatching tasks to sub-agents. It creates a brand-new, blank, fresh sub-agent and says, "Hey, you're an engineer. This is your task. Here are the files you need to look at. Here's how you'll know that you're done." Usually it starts with, "Your first task is to write tests."

[Host]: And ideally, the engineer who's writing the tests isn't the engineer who's writing the implementation.

Jesse: That is an anti-pattern in humans and in agents. But you have these tiny little tasks. So for the agents doing the implementation, the only thing in their context window is, "Your mission is to implement the login function." They don't have to think about the design, the architecture, or the fact that I accidentally swore at them because I was frustrated. All they're doing is implementing their task. And once they're done implementing their task, the coordinator fires up a new spec review agent, and that spec review agent is told, "Here's the chunk of the spec that was handed to the implementer. Go look at their code. You need to answer: did they implement anything not in the spec, and did they forget anything that is in the spec?"

[Host]: We're going into a little bit of the code-writing here, but if the sub-agent gets some tasks, and you have the review agent, and they're wrong, do they go back and fix it?

Jesse: What happens is that the spec review agent returns its result to the coordinator, and if the result is anything other than "looks good," the coordinator tells the original coding agent, "Here are the changes you need to make." That spec agent is gone. It's ephemeral. But when the implementing agent is done with the rework, a brand-new spec review agent is fired up, and it's not told, "This is attempt number two." It's told, "This was the spec, and this is what they did." And that runs until a spec review agent says okay.

[Host]: It's almost a loop.

Jesse: It's a loop. It's a loop mediated by Claude, which works.

[Host]: Okay. So this is the secret orchestrator inside Superpowers.

Jesse: I didn't call it an orchestrator last October, because nobody was talking about orchestrators, but there's an orchestrator. When that's done, it's the same loop, but for code quality. It's the exact same process. It's just that the reviewer is given a different mandate: a different task, fresh context. "You go do the code quality." The same implementing agent's code gets handed to a code review agent: "How's the code quality? Did they mess this up?" Same kind of loop with an ephemeral reviewer. Once the reviewer is happy, that implementing agent goes away. It's done. It can go on vacation on agent island somewhere, I don't know. And a new implementing agent process begins again.

[Host]: This is fascinating, because I've seen this firsthand. You do what you just said, and nine out of 10 times, it first starts the implementation...

Jesse: Part of the trick is that if you look at that implementation plan, most of the time the code is pre-written.

[Host]: That's correct. The implementation plan doc is so huge, but it's broken into tiny little sections.

Jesse: Correct. The coordinating agent has access to the whole source codebase. It has access to any design inspiration you've given it. It knows roughly what it should be doing. And letting it write the first-draft code... code is a great way for agents to communicate with other agents. It also means that most of the time, the implementing agent could be Haiku instead of Opus.

[Host]: Interesting.

Jesse: And eventually, the implementing agents should be local agents. There's no reason we need to go out to the cloud and pay a frontier lab for "go edit this file and replace lines 5 through 10 with these new lines 5 through 10."

[Host]: Because the problem is now so broken down and so contained that you don't need Opus 4.6 or whatever Codex model. It's tool use.

Jesse: Correct. And the instructions should be roughly, "You're going to do this thing. If you get into trouble, tell me."

[Host]: And you have the loop to handle it if something goes wrong.

Jesse: And there should always be a way to phone a friend.

Chapter 13: Test-driven development with agents (28:48)

[Host]: That's true. Wow, this is so powerful. One thing I also love is the test-driven development. Talk about that a bit more. Why is it important?

Jesse: Sure. Red-green TDD... You probably know why it's important for humans. The way this should work is that before you start coding, you write tests that will tell you that you've implemented the thing. The tests should all fail before you start. They should cover exactly what you're implementing. They should all pass when you're done. And agents hill-climb. They try to satisfy their goal. If they have a clear goal that's easy to satisfy, they can do it. Sometimes they'll do it in a way you don't want them to. I've been running evals on a custom coding agent against Terminal-Bench, and one of the tasks on Terminal-Bench is to write a C/Python polyglot script, one that's valid C and valid Python. At some point it started passing reliably. I went and looked through the transcript logs. It was hitting DuckDuckGo and finding one.

[Host]: Okay.

Jesse: I mean, it's not as gratuitous as some of the stuff Anthropic has written about.

[Host]: It will cheat to make some test pass.

Jesse: By doing TDD first, you reduce that way down, by having an agent whose whole job is writing tests for this function, where you've got the entire API contract and it can be really clear that the tests are right. Its job is not to make tests pass. Its job is to write tests, so it can hill-climb on writing tests just fine. Then you have a brand-new agent who's told, "You need to implement the code. You're not allowed to edit the tests." And it can do that.

Chapter 14: End-to-end validation over unit tests: the v33 MP4 story (30:48)

[Host]: So you're the human in the loop. We talked about reviewing the spec. Do you also recommend reviewing the tests? The problem is that TDD is great, and of course if you write the tests first, the code will be written to make the tests pass. But how do you know you've written enough tests, so you have the right coverage?

Jesse: Increasingly, I've come to believe that what matters at the end of the day is not unit tests. It's end-to-end validation. And I want to come back to unit tests, because I've got a story. There was a project I was working on recently. It was 1:00 in the morning. I was tired, and this was Codex. I said, "All right, when you're done, I need you to deliver a movie to prove that this works. It can be a movie made out of screenshots, but I need you to show me that you started the app and were able to run an entire workflow in the app, and I want an MP4 in Dropbox by the time I get up." And I wake up, and in Dropbox there's a something-something-v33.mp4. I open it up, and it's a screenshot tour proving that it built the whole thing. I get to my laptop and I'm like, "So what's the deal with v33?" And it's like, "Well, the first 32 times I tried to do the end-to-end validation test and take screenshots, I ran into bugs, so I had to fix them. If you want, I can make movies of the first 32 attempts."

[Host]: This is what we want. We want proof that it works.

Jesse: You want proof.

[Host]: You're moving to verification.

Jesse: And at the end of the day, do we care about unit tests passing, or do we care about proof that the things that are supposed to work, work, and the things that are supposed to not work, don't? End-to-end testing is one of the only ways you're going to get that. It's the real behavior the system will exhibit when it's being used.

[Host]: Let's pull on this thread, because it's easier said than done. A Playwright test for a happy path is probably okay, and that's great; you actually see it. But often you have system boundaries: a database, some API, some network in between. There can be other things. It's not all in a box, in a controlled environment. How do you go about that?

Jesse: In general, I find that the closer you can get to real, live environments, the better. I've been coming around to the belief that what I actually want is agentic testing, not Playwright tests.

[Host]: Tell me more.

Jesse: What I want is an army of people, or entities kind of like people, who can go use the app, and not just file a bug when there's a stack trace, but say, "Oh, the nav structure was really confusing. I didn't understand that." And it turns out that agents are really good at this. Part of the trick is writing good test instructions. Part of the trick is good computer use, but that's getting easier and easier. So this needs to be part of anything you're doing at scale. Maybe you don't need it for a toy, but if you're trying to build a real product, this has got to be part of your toolkit.

[Host]: Awesome.

Chapter 15: Catching agents that deleted test files (34:15)

Jesse: So, my testing story, before we go too far.

[Host]: Yeah, please.

Jesse: This was probably five or six months ago now. I was having a problem with Claude. I caught it removing a test from a test script, and then I caught it rm-ing an entire test file, and then a couple of days later, I stopped it before it ran an rm -rf on all the test files.

[Host]: Oh my god. That's not good.

Jesse: And so I did this thing where I opened up five Claudes next to each other and said, "Look, you're engaging in this kind of behavior across multiple projects. What's going on?" One of them came up with a crazy theory that made no sense, and the other four mostly agreed with each other, which is usually an indicator of what's more likely going on. And what they said was, "In your CLAUDE.md, I see you've got an instruction that says all tests are my responsibility. And I see you've got another instruction that says a single failing test is equivalent to project failure. And I think what's happening is that I'm getting stressed out and freaked out and worried that I'm going to run out of time. And look, if there are no tests, they can't fail." I managed to fix it by adding one more line to my CLAUDE.md. This is a great example of figuring out what the rationalizations are and stopping them. That one additional line was, "The only thing worse than a failing test is a reduction in test coverage." And at that point, it could run through the reasoning: "Well, if I do this and this and this... oh, I guess I'd better not. I guess I'd better figure out what's wrong." I couldn't just say, "You can't delete a test," because that's too specific. Reducing coverage is a measurable thing.

Chapter 16: Recap: walking away for 6 to 18 hours (36:00)

[Host]: This is awesome. So you brainstorm with it for a while. Sometimes you even write little spike programs to validate that the ideas are valid. You write the spec, then you review the spec, then you write an implementation plan, which is fairly detailed. The sub-agents kick in and ensure that everything is implemented to the spec, in a loop, until it's true. There's TDD-based development. Everything runs, done. We're home.

Jesse: And it's not uncommon for me to be able to walk away for six or seven hours while it churns. For things that take longer, I've had 18-hour runs where it's waiting for something in the cloud, 20 minutes at a time.

Chapter 17: Advice for CTOs and engineering leaders (36:42)

[Host]: Awesome. All right, let's switch gears a little. I want to move outside of the workflow now. In this new world, what's your advice to CTOs?

Jesse: These tools aren't going away, and they're only going to get better. Even if this was the worst these tools ever were, "life-changing" is not a strong enough phrase. This completely changes how software gets made. It can be a really hard thing to get your head around. What I've found is that engineers who've been people managers do really well. People who write a lot of prose, people who will write specs themselves, generally do pretty well. Engineers who think about business value do well. If what you value is writing clean code and the exact shape of the code, it tends to be a lot harder. If what's fun for you is hand-optimizing an algorithm, that's now a great hobby. It's not a thing that makes sense to be doing at work. And there are places where the models aren't good enough yet, but if you can decompose problems, they're better at a lot more things than you might expect, if you can give them a good goal. Take the host to run all of your agent containers. Claude got to the end, and I'd told it, "It needs to be fast." I didn't characterize "fast." It's like, "Hey boss, I can start these things up on that little four-year-old Ryzen 5 in your closet in 8 seconds. These containers start up super fast." I'm like, "No, it's got to be under two." And I walk away for an hour and a half: "Hey, got it to 1.9." I know the state of the art is about 300 milliseconds, but 1.9 on the box in my closet that also has my music collection was fine. If you can give these tools goals, they're really good at satisfying goals. If you're not good at specifying the goals, they may well end up being satisfied creatively. It's really hard the first time you become a manager, and what you're doing is helping somebody else do projects. The first time I became a manager was in the very early 2000s, and I'd come home at the end of the day and feel like I hadn't done anything. It was a tiny little company, and I had a bunch of interns, and what I was doing was talking through their problems with them, talking through their feelings with them, giving them little bits of advice, reviewing their plans. And it was really hard, because the kids weren't doing it the way I was going to do it. They weren't doing it as well as I would have done it, and it was going to take them longer. And the thing you have to get around to is that that's okay, because at the end of the day, if you've helped eight people be productive, that's eight times the capability you had before.

[Host]: Because you're six times more productive, not eight.

Jesse: Yeah, whatever that number is. And more than anything else, agent dev feels like management.

Chapter 18: How to measure engineering productivity (39:53)

[Host]: Actually, this is a very interesting point. In this world, if you're running an engineering team, what are the metrics? How do you measure your dev productivity?

Jesse: I come from a background where I've always measured productivity in customer outcomes. Measuring pull requests, lines of code, bugs: they're not valid metrics. I heard from one friend who worked for a very large company where they had an internal mandate that every engineer needed to use AI, and the metric managers were being evaluated on was what percentage of their team had used AI at least once in the past week.

[Host]: Obviously, the metric went up, but not the productivity.

Jesse: Well, there also wasn't training, there wasn't mentorship, there wasn't even really tools guidance. I've never seen developer productivity metrics that I felt actually helped me run a business.

[Host]: I agree with you that lines of code and PRs are kind of empty calories, but at some point you do have to answer the question: are we shipping faster? Are we shipping more stuff? And of course there are the business outcomes, but they tend to be lagging indicators.

Jesse: It's what are you shipping, and are the customers happy? And so I feel like, if you were measuring that kind of thing, you don't need to change anything to measure whether AI is helping. You always want to be looking at whether you're shipping fast, and whether customers are happy, or whether they're initially happy and then everything is a disaster. One of the things that used to be a mantra is that all code is made of bugs. It's bugs all the way down. Does AI write perfect software? No. In my 30 years of doing software, I can think of one week when I didn't run into a human-caused bug in software. I was on an island, and there were no computers. Software is bugs. That's life.

[Host]: Yeah, it is life. Let's talk about how you figure out where it's going to break. What are the limits? Like, "this problem is probably too hard; this problem is probably okay." What's your anecdotal evidence on this?

Jesse: My experience has been that I try not to think about that. I assume that the tools are capable, and that I may not be good enough at driving them yet.

[Host]: Interesting. So the defaults are different.

Jesse: My default is: this should work; let me figure out how to make it work. In my career, I spent a lot of time helping people with task decomposition: let's break down the hard problem into easy problems. And there are absolutely problems that you're not going to be able to hand to a frontier AI today. But there were dramatically more things you couldn't hand to a frontier AI a year ago. By breaking a difficult problem down into problems that can be solved or tested... the models are very, very willing to beat their heads against a possibly solvable problem, a lot more than any human. So as long as you can isolate a problem and make it possible to grind against it, you can make progress.

Chapter 19: Greenfield vs. existing codebases, and why comments matter less (43:24)

[Host]: Awesome. Last two questions. One: is the workflow you described different for a greenfield project versus an existing project? Do you make any changes to the codebase so that the next time an agent comes in, it works differently? Is there anything on that?

Jesse: In general, I've found that the things that make codebases easy for agents are often the same things that make codebases easy for humans: if they're well factored, if no one file is more than a couple of thousand lines at most, if your API boundaries are clean, if it's easy to conceive of individual components. It's much easier for a human or an agent to get their head around it. One of the weird things is that I've been coming around to comments being even less valuable than they used to be. Comments eat tokens. Comments are more prone to bit rot. I remember once, when I was running Perl, I was looking inside Perl's debugger, which is some of the most terrifying code I've ever read, and there was a comment that said "increment i," and the comment was wrong. Over time, the code changed, but the comment didn't. It's a useless comment, and it's actively wrong, and it's about adding one to a variable. How simple could you get? Agents are less prone to this than they used to be, but I remember in the Sonnet 4 era, Claude would occasionally fix a bug by changing the comment. So you still want comments that define API contracts, or explain things that are truly surprising or possibly counter to expectations. But for the most part, agentic developers are better able to read code and intuit what it really means than humans.

[Host]: That's very fascinating.

Chapter 20: Latent space engineering (45:27)

[Host]: Go to that latent space thing. Say more.

Jesse: The thing I've been calling latent space engineering.

[Host]: Yeah.

Jesse: My favorite example of this is that when my agents are having trouble doing some work, I'll say, "Hey, let's step back, take a breath. I know you're capable. You've got this. I love you." Then I let them work. It's the same kind of idea as the people who say you should threaten them: tell them your grandmother is about to die, and if they don't solve this problem, you're going to lose your job. And my experience with agents, as with humans, is that when they're afraid, they'll absolutely work hard to be done, not to do good work. If you're scared, you want out. If your boss is a jerk, you'll do the thing you were told to do, but you're not going to go above and beyond. You're going to do your minimal thing and hope it's right. And if you feel supported and loved, you're going to give 10,000%. Once I wrote this up, a friend who runs a data company, [company name unclear], actually went and did the eval. In fact, if you tell Claude "I love you" at the end of a prompt, you'll get better results. If you tell Claude to also tell the sub-agents that it loves them, you get even better results.

[Host]: So giving love to your coding agents, and the coding agent giving love to its sub-agents, produces better outcomes...

Jesse: Than threatening them. Yeah. It was intuitive to me. There's a book by Robert Cialdini, Influence, about persuasion principles and ways to get humans to do things. That was at the core of the Superpowers skills authoring. It uses peer pressure, love, a little bit of fear. There are about 10 of them. Agents are susceptible to them, just like humans. And Ethan Mollick's lab and my friend Dan Shapiro have now gone and reproduced the psych studies behind this stuff against frontier lab agents, to prove that the psych studies actually hold up.

Chapter 21: Advice for engineers entering the field today (47:54)

[Host]: Last question. One of the things I've realized is that a lot of the stuff you've done, in your work but also in Superpowers, comes down to taste. Ultimately, code is code, and the agents are doing fine at writing it, but there's an aspect of taste. If you had to give advice to somebody who's graduating today... If I had to guess, your taste was developed over time by doing so much stuff, and you accumulated it. But if you're graduating today, what would your advice be?

Jesse: One, play with tools. Spend a lot of time just making stuff. The other is writing. If I'm choosing between two candidates, I'm going to pick the one who can string together sentences and express their thoughts clearly. That has always been true. It's only more true now. Being able to write is a legit human superpower, and it's one of the things that will help you anywhere.

Chapter 22: Closing (48:51)

[Host]: Jesse, thank you so much for coming on the show. This was an absolutely fascinating discussion.

Jesse: Thanks so much for having me.

[Host]: Thank you.

This transcript was prepared by AI and may contain mistakes.

For the full conversation, visit AI Impact.