Five weeks to production with ralph-loops

§0 TLDR

During five weeks I discovered, designed, vibe-coded, ralph-looped, and deployed a production-ready booking system at an external company. To avoid the pitfalls of using AI two principles have guided me throughout

1. Alignment between consumer needs and product features is the ultimate goal.
2. The fewer ambiguities the AI has to resolve, the better.

I used AI (Claude Code) to help me in every step. From designing discovery interviews to making an integrated eval framework for the LLM part of the chat. The actual code was implemented with up to seven concurrent ralph-loops. 1,545 commits, 79.3% by autonomous agents. From 63 to 2,196 tests and it’s now live. Functionality is verified but I have not read the code. To combat tech debt the project underwent six refactor sessions, one with 54 issues.

I’m also looking for forward-deployed engineer work, so consider this post a work sample, and judge for yourself!

Docker-vm and ralph-loop scripts

This project very much felt like it split cleanly into five phases:

§1 BACKSTORY

My family has a small real estate business, Klara Förvaltning, that rents out properties in a smallish town in Sweden. It has 3 employees in the office. Every month 5–10 apartments come up for re-letting and new tenants have to be found.

They are a non-technical company with lots of manual processes. Nine years ago I built them an application system by hand, which took me about two months. In November 2024 I rebuilt the whole thing in one week with Cursor. So besides the previous application functionality and an initial refactor, it was a greenfield project. And completely within my control.

During my career I’ve built so much stuff that no one uses, so here the focus was truly customer value. As I’ve leaned on AI more and noticed I was the bottleneck. I had discovered ralph-loops playing around and combined with the structure of superpowers/GSD/Matt Pococks skills, I had a feeling it could offer balance between autonomy and alignment while moving me away from implementation and still trust the code.

My background is mostly from the game industry, having spent many years in Unity, and my roles have spanned developer/producer/CEO for the last 10 years. I am by no means a tech wizard and sometimes feel awe at the things I read on HN. I am a curious generalist who can code and loves to learn. As it was a family company, the project was pro bono. Now, onto the good stuff!

§2 INTERVIEWS → LOCKED SPECS

I came in not knowing what problem I’d solve, so the first phase was discovery. I researched some frameworks, talked it through with Claude, and designed a discovery process. Both the CEO and I wanted numbers so we could identify what most valuable problem to tackle was.

So I sat with each employee for the first couple of hours, mapped out their week — what they did, how often, how long it took — and kept coming back with questions to nail down specifics. I used several interview tricks, for example from The Mom Test, where I listened for pain: what did they not enjoy? Then a following round on the tasks that took time, was frequent, or often erroneous. Where does the data come from? How do you make this decision? What happens if X, if Y, if Z?

Out of that I could rank what was actually suitable to automate. Optimally, I knew the task would have:

• testable data
• a natural human-in-the-loop
• somewhere to eval it
• and not much domain experience required

Then I placed each candidate on a chart: easy or hard to build, low or high value. By the end of day two I’d mapped basically all their workflows and identified a top candidate. It took them roughly 40 hours a month to find applicants, invite them, schedule a viewing with the outgoing tenant, get people through the door, and rent it out. Due to communication being through email and text it was a lot of scattered work that interrupted the workday. If I could automate most booking and calendar logistics, we figured we could save around 20 hours of that a month. The goal became to develop a scheduling automation that coordinates outgoing tenants and incoming applicants against the employees’ calendars, books viewings automatically, and notifies staff when something needs a human.

flowchart TD
    classDef human   fill:#fde2b3,stroke:#d98c00,color:#5a3d00
    classDef system  fill:#cfe3ff,stroke:#2f6fd0,color:#0b2c5c
    classDef trigger fill:#eeeeee,stroke:#888888,color:#333333
    classDef done    fill:#d9f2d9,stroke:#3a9c3a,color:#14501a
    classDef header  fill:#eaeaea,stroke:#bbbbbb,color:#111111

    subgraph NEW[" "]
        direction TB
        Nhdr["After — one click, then the system runs it"]:::header
        Nhdr --> N0(["Apartment cancelled"]):::trigger
        N0 --> N1["Employee clicks Start search"]:::human
        N1 --> N2["Contact applicants, register interest"]:::system
        N2 --> N3["Contact outgoing tenant, book slots against the employee calendar"]:::system
        N3 --> N4["Offer times to applicants, book viewings"]:::system
        N4 --> N5["Notify staff of scheduled viewings"]:::system
        N5 --> NF{"Is booking full or applicant pool empty?"}:::trigger
        NF -- No --> NMore["Invite more applicants from pool"]:::system
        NMore --> N4
        NF -- Yes --> N6["Hold the viewing"]:::human
        N6 --> NQ{"Anyone want it?"}:::trigger
        NQ -- No --> N1
        NQ -- Yes --> NEND["Evaluate applicant + draft contract"]:::done
    end

    subgraph OLD[" "]
        direction TB
        Ohdr["Before — manual (almost every step is a person)"]:::header
        Ohdr --> O0(["Apartment cancelled"]):::trigger
        O0 --> O1["Find suitable applicants in the database"]:::human
        O1 --> O2["Email the shortlist"]:::human
        O2 --> O3["Wait for replies"]:::human
        O3 --> O4["Contact outgoing tenant to book a time"]:::human
        O4 --> O5["Wait for the tenant's reply"]:::human
        O5 --> O6{"Time works in the calendar?"}:::trigger
        O6 -- No --> O4
        O6 -- Yes --> O7["Contact applicants"]:::human
        O7 --> O8["Wait for replies"]:::human
        O8 --> O9["Manually handle scheduling"]:::human
        O9 --> O10["Hold the viewing"]:::human
        O10 --> OQ{"Anyone want it?"}:::trigger
        OQ -- No --> O1
        OQ -- Yes --> OEND["Evaluate applicant + draft contract"]:::done
    end

The same job, before and after. Amber = a person, blue = the system. The manual flow is almost all amber; after, one click hands the whole booking dance to the system, and the employee only steps back in for the viewing.

With the target chosen, the next question was how to build it. I’d built a lot with AI, and I’d learned that the moment I started caring about the code, I became the bottleneck. I wanted to avoid that. I was deep into ralph-loops at this point, and since this was a complete, standalone web app, I knew I could spin several up. That was tantalizing — I felt I had a really good grasp of the process already.

I also like Kanban a lot and thought that if I could centralize the workflow around user stories, I could care less about the implementation and more about customer value, which was the point. So I sat with Claude for a few hours, researched how spec-driven automation projects are run, and put it all into the first skill, /skill-workflow-creation (prefixed with “skill” so as not to confuse it with commands and plugins).

Then I started dumping in raw voice transcripts — everything I thought I knew about the process — and had Claude sort them into structured workflow documents: actors, flow chart diagrams, assumptions listed, and with a section of a list of open questions at the end. It became a loop: add new information, invoke the skill, read the workflow — especially the Mermaid diagrams, they surfaced a lot of misalignment — read assumptions, answer the open questions, repeat. The diagrams worked great as it became a visual tool that naturally circumvented Claude’s tendency to over deliver on words, and the open-questions section offered a natural way to guide the AI to identify uncertainties.

This process spat out open questions by the dozen — the booking workflow alone spawned 20. After the first dump it settled into a rhythm of asking the AI “what open questions remain?” And they just unloaded. Do you want a travel buffer between viewings, and how long? Do you book viewings over lunch? What happens if someone who says no, then say yes? What hours should the communication happen during? Etc.

Each one got categorized: can I answer it myself, can I research it, or do only the employees know? The employee ones I batched, and walked over to ask in person — being onsite for the whole project, that direct contact was a huge asset.

This is where the hardest part of the later build first showed up: corner cases. So. Many. Corner. Cases. The process was completely manual and they wanted to keep as much flexibility as possible — a fully automatic search, plus adding applicants anywhere in the process, plus hot-adding non-applicants who just call in. Multiple actors, timing, all of it interacting. There was a lot of conversation with employees to figure out where to cut down on flexibility, and where to keep it. It all got specified over a few days.

I was running Claude with --dangerously-skip-permissions, naturally, so: how could I be sure nothing changed in areas we hadn’t discussed? How could I trust specs I hadn’t carefully read every loop? I decided to run it all through a rough “gate” at the end — where I defined the main happy paths and asked Claude, in small sessions, to verify each one was still represented. It didn’t cover the full mapping, but it confirmed the spec represented the product at certain key areas — enough to derive an architecture from. A bit like a bandaid on a bandaid, but it made me confident the main parts were accounted for in preparation of the next steps.

We iterated until there were no employee or user open questions left — though we did park some engineering details for the build phase. I was part of the loop here, Claude was explicitly told never to commit anything unless there was explicit instructions and I was fully in the loop until every question was answered. The focus was solely on WHAT, never HOW: no autonomy, so that the next stage could be as autonomous as possible.

§3 TWO PROTOTYPES + THE DESIGN REFERENCE

During the spec session it became clear that some open questions needed closing through experimentation. The major questions were:

• Are the employees comfortable with a chatbot communicating with their customers and booking into their schedule?
• What type of data can I access from their software?
• What visual UI should I ground this in?

While the first two questions were implementation, I knew I needed a UI that the ralph-loop could use as reference for new features. I also wanted the employees to be comfortable in the new software, so I made a few visual sketches, and eventually landed on a vertical Kanban board (too many states for horizontal, I felt).

The first questions became: were the employees comfortable with a text bot? Was their calendar open for CRUD operations? Previously all communication with tenants and applicants was through email but text has several advantages, most notably a quicker response time which we assumed would lead to filling up viewings faster.

To test this I built a web app with a chat interface and connected to Twilio where we could simulate the actors — outgoing tenants and applicants — and connected to a Microsoft calendar and an Anthropic LLM. It was a little bit of fiddling, but the basic premise was proven. We had a three-way conversation, and it booked the viewing into the calendar. It gracefully redirected to the office when it was unable to answer. The chat prototype was built May 18–20 and presented on the 20th, together with a very simple UI draft I’d made in Claude (Artifacts).

The chat prototype — a three-way conversation (simulated outgoing tenant + applicant) wired to Twilio, a Microsoft calendar, and an Anthropic LLM.

It was a wireframe, basically showing the flow of apartments, where and how info could be available. The chatbot’s conversations had to be available, so they could inspect them. Exactly what else was on the page mattered less, but there had to be a place for the information to live.

The design reference — a wireframe of the apartment flow, built in Claude (Artifacts).

…and the apartment-detail view, where the chatbot’s conversations stay inspectable.

I also had to investigate what data I could access from the software (Din Hyresvärd) they use to manage tenants, contracts, applicants, etc. While not strictly necessary (the information is easy to enter by hand), it saved time and complexity if apartments showed up automatically when they were marked “terminated” in the software. Pulling from the API was never a deal-breaker, though — if it didn’t work we could fall back to manually entering the details. One day of prototyping — figuring out which values mapped where, and how they changed when apartments were terminated — and it was verified. We also wrote some PII redaction scripts, to make sure as little PII as possible was touched at every stage.

With the chat prototype and visual layout confirmed by employees, and the tenant software able to give data, all open questions were locked on May 22nd.

§4 PREPPING THE REPO FOR AGENTS

May 20–21st.

So now.. It was time to prepare the repo for the onslaught of what was to be the implementation. Several things had to be done before I could confidently let loose a ralph-loop, based on experience:

• Architecture, so that the loops had a clear direction to follow
• A UI front-end reference with close to 100% feature parity and production-ready design
• Skills and scripts ported from previous projects
• Full e2e testing capabilities for each individual loop
• A refactor of the current project, both backend and frontend

Architecture

The architecture came easily, due to all the specifications we had. There were tradeoffs but by taking the final specifications and using a fan-out-and-reconcile pattern and evaluating different choices a top candidate was born. It was documented in one document, architecture-spec.md. Here is a snippet from the file:

author: lead (final convergence over 5 reconciled canonical slices + the Phase-4 grill)
inputs:
  - 0.scan-notes.md
  - 1.domain-state-model.md          # charter 1 — the spine (states/transitions/entities)
  - 2.runtime-events-jobs.md         # charter 2 — events, jobs, webhook, scheduler wiring
  - 3.integration-boundaries.md      # charter 3 — DH / Twilio / MS Graph / LLM ports (A2 resolved)
  - 4.conversation-design.md         # charter 4 — SMS sub-state-machines + Swedish copy
  - 5.control-surface-and-app-fit.md # charter 5 — board/modals/commands + app mount
provenance: 15 worker drafts (3/charter) → 5 verify+reconcile passes → this convergence → a Phase-4 grill that closed D1–D12. Every load-bearing claim is grounded in a repo file; citations live in the slice docs.
---

# 0. KF automation — unified architecture spec

## Why this layer exists / what it unifies

The eight workflow specs (P1–P7 + S1) and the two prototypes already describe *what* the Klara
Förvaltning letting automation must do. What was missing — and what this layer locks — is the
**single source of truth for how the pieces coordinate**, so implementation never re-derives the
spine inconsistently from scattered specs:
…

examples…
3. Consolidated module map (where every piece lives)
Feature code in backend/kf/ (D9); shared external adapters in backend/integrations/; broadcast SMS in
the existing backend/notifications/. Each file ≤300 LOC.
backend/
  kf/
    models/            # objekt, round, visning, efterföljning, escalation + the conversation/interest + message tables (shared Base)
    repositories.py    # ~10-line Repository[Model,Create] subclasses
    routers/
      board.py         # GET /api/kf/board|objekt|conversations  (projections)
      commands.py      # POST /api/kf/* : refresh, start-search(general|named), quick-add, pause/unpause, escalate, restart  (NO mark-done — D5)
      webhook.py       # Twilio inbound + status callback (signature-gated, public mount)
      settings.py      # PUT /api/kf/settings (recipients, område-map, MS-Graph binding)
    services/
      inbound.py       # conversation lookup + intent→handler shell (consumes the LLM port)
      intent_handlers.py # the HANDLERS table — state-machine actions
      booking.py       # atomic FCFS slot claim
      escalation.py    # escalate() + ai_stuck counter (net-new)
    jobs/{run.py, dh.py, viewings.py, rounds.py}   # the ~10 idempotent sweeps + the `python -m` dispatcher
    utils/{send_window.py, objekt_state.py}        # window helper + the 8-enum transition guard
    db_api.py          # the passande contract over `applications`: register_search / next_applicants / mark_contacted / quick_add
  integrations/
    dh/                # DHClient + DHGateway + project()  (lift dhprototype)
    sms/               # conversational Channel + TwilioSmsChannel + deliver_outbound + gsm + twilio_signature
    calendar/          # CalendarService + GraphCalendarService + ms_auth  (Settings via repo)
    llm/               # IntentRouter + AnthropicIntentRouter + ScriptedIntentRouter
  notifications/       # EXISTING — wire TwilioTransport.send() for broadcast SMS (one method)
  config.py            # EXISTING — add typed ms_*/anthropic_*/dh_* settings
  main.py              # EXISTING — include kf_router at /api/kf + the signature-gated webhook
  alembic/             # NEW — one migration for the `applications` column-extension (D10)
frontend/src/
  components/kf/       # SchedulingBoard, BoardBand, ObjektCard, AptModal, ChatModal(read-only), QuickAddDialog, KfSettings (≤300 LOC; layout = user co-design)
  router/index.js      # EXISTING — add the /scheduling route with meta.nav "Schemaläggning"
  api/index.js         # EXISTING — add the frozen api.kf client group (+ a 'conflict'/409 code)

The UI reference

With the architecture set and referenced I needed a UI reference. This was done by taking the previous Claude wireframe design and making it into a high-fidelity prototype and manually prompting to fix it. While it was not 100% feature parity, it was clear where everything was to go, and how it should look. “Every phase-related action goes here in these green buttons”, apartment info goes here, settings looks like this, etc. This was surprisingly easy, we already had the major layout done in the wireframe, which helped. The only thing that changed was the orientation of the board, from vertical to horizontal, as it all fit nicely in the hifi design. Finished, download a zip, put in project, done.

Hi fidelity UI design

AI-Info: what I carry between projects

I have a few things I bring with me from each project to the next. I’ve put these in a template folder I instruct Claude to use to set up a new project. They include:

• A Dockerfile and a script so I can ./run-claude.sh and spin up a new VM
• Peon ping (love this), and a script to enable it inside a Docker VM
• Setup scripts for Claude Code shared credentials between VMs, Playwright, statusline, etc.
• Some aliases (almost only use cdv, claude --dangerously-skip-permissions --verbose, godsend)
• Skills like /how-to-create-a-skill, /how-to-write-ai-instructions (derived from Anthropic’s repo), and /meta-agent, that I use when I work on skills and AI repo infra, not the code itself

My colorful statusline. Explained: <Model first letter>:<version> <folder> | <branch> <colored token context window bar> <%> | <time since context start> <tokens>

Besides these I also have a few development skills I use. They’re originally from Matt Pocock (thanks Matt), and slightly adapted to each project. I’ve used /grill-me, /write-a-prd, /prd-to-issues, /tdd and /improve-codebase-architecture for some time, moving on from Superpowers and then GSD.

I also moved over my older ralph-loop scripts, to be changed later.

Enable full e2e testing

With the above scaffolding in place (template files in, PAT set, main branch protected, etc) wiring everything up is as simple as launching cdv and explaining the end goal of full e2e testing in front end, back end, e2e testing, database seed, start/stop scripts with reset and reseed, and having it analyze and suggest a plan that conforms to the new architecture design we just made, and NOT the current (un-refactored) old project. In situations like this, I talk to claude about alignment. “Analyze thoroughly and mirror back what you propose to do to achieve the goals so that I can verify that we are in alignment. State any assumptions clearly”. E2E testing is crucial to enable the agent to verify it works beyond unit tests.

Refactor

So with the new architecture design, new front end design reference project (just html, css and .jsx, but inspectable by agent), agentic environment set up, skills loaded, testing strategy set, it was time to go to town. The first task was to refactor old into new, and I used first /grill-me > /write-a-prd > /prd-to-issues, and then have a ralph-loop.sh execution script to loop overnight. It calls a ralph-loop-pick.sh to find an issue, and a ralph-loop-done.sh to close and suicide. A very simple PROMPT.md was injected at start of each session with instructions along the lines of “if you have unstashed changes you have been sigkilled. If so, figure out where in the ongoing work you are. Otherwise start the work fresh. Call ralph-loop-pick.sh to get the gh issue you are to work on. Invoke /tdd and when done, invoke /ralph-loop-done.sh”. /tdd told it to update gh issue plan. Since the issues were so thoroughly spec’d and only one agent was running. It could just find the issue and repeat over and over. The instruction was enough, making it spend maybe a 0.5-1 min to resume every time, and then continue.

My adapted skills are extended with explicit context padding, i.e. include decisions in the issues, always list both unit and e2e tests (if applicable) that needs to be fulfilled as acceptance criteria, and a bit more, as I felt that the information sometimes gets lost between steps.

Even so, it was slow going. It took about the whole day to shave away at stuff, from start to finish. At the end of the day it seemed to work, and I tried letting it finish the last issues overnight.

§5 THE LOOP: FIVE DEV PHASES

Ignition & the two-VM tap · May 23

I came back to unfinished work. So work started on making the loop more effective. I started it off it running and watched it closely, having three VMs up at once: one running the loop, two with /meta-architect doing fixes. When the loop did something wrong, I analyzed immediately, fixed, pushed, killed loop, and restarted it. Sometimes a nudge was enough, but most of the time decreasing complexity through determinism is the key unlock. Better bash scripts, deterministic prompt injection, etc. Iterations went fast and the focus was on adapting existing infrastructure to be more efficient. Through this work, I quickly noticed the skills and workflow needed improvement.

The backlog flywheel · May 23

I’ve reflected on this since. I think it’s where my flow breaks from spec-driven development: I’d specced everything up to this point into architecture-spec.md, where Claude defines how to build — and now the user stories became the what. An interesting flip, but it works well.

At this point I discarded the PRD step. I’d hit problems with Matt Pocock’s PRD system before — mainly that it loses knowledge between steps: the issue it eventually creates is self-contained, but questions settled during the grill session sometimes aren’t respected, and the AI guesses on the implementation. I’d tried patching it (stuffing the decision tree and all the context into the PRD and issues) but it never worked great.

I wanted to drive everything off user stories (once again, to raise the odds that everything that got built was actually valuable). I created some skills to mine stories from the spec and suggest them in a back and forth format. So a new document user-stories-atomized.md got populated, picked up by a new /ralph-prio skill, and promoted into user-stories-list.md (“decision: we should build this”), then read by /ralph-create-issues — a mix of /grill-me, /write-a-prd, and /prd-to-issues — with instructions to scan the references, prototypes, and architecture. The goal: issues executable by a /ralph-tdd running ./ralph-loop.sh.

user-stories-atomized.md — the raw stories mined from the spec.

/ralph-prio promotes the ones cleared to build into user-stories-list.md.

So — having already done the heavy lifting on architecture, models, and UI, and with a curated story list — I leaned on one last skill, /ralph-create-issues. It takes stories from user-stories-list.md and does everything needed to turn them into self-contained issues: an orchestrator-fan-out-reconcile-and-combine pattern, subagents checking every relevant module and seam, the story lists, and open and newly-closed GH issues (so no duplicates), then it grills me down every decision tree. Finally it posts a set of standalone, comprehensive stories, each sized to be “reasonable in about 900s” (15m).

What /ralph-create-issues turns a one-line story into: a standalone issue with acceptance criteria, an E2E test plan, and a decision log.

This day was also hygiene around blockers and deterministic guards — how to add and recognize blocked items, for instance. Sometimes GH issues weren’t in the right numeric order, and the picker had to handle that.

The catch showed up fast: that atomized story list mattered a lot, and Claude was happy to invent stories found nowhere else in the repo. Keeping it clean — only the stuff that was good and wanted — became one of the more manual tasks.

Planning got durable · May 24–25

I moved logic to deterministic gates. Instead of tracking state via GH issues I introduced current-plan.md, generated by /ralph-tdd-plan from current-plan-TEMPLATE.md; a deterministic guard in ./ralph-loop.sh decided whether an issue was in the planning state or not. The suicide logic moved into scripts too — ralph-loop-plan-done.sh, ralph-loop-kill-self.sh. Small tweaks paid off: telling it “current-plan.md is always fresh, you don’t need to verify it” saved a full minute of “let me make sure this plan is up to date” every loop.

May 25 — improved PROMPT.md and the picker logic. Played with timeouts; moved more work into subagents to save context, and raised the planning-phase timeout (almost all subagent work) to a multiple of the normal one. Improved /ralph-create-issues to research stories, UI, and backend with subagents — by now we were getting stories that were similar but not identical, so nuances had to be identified correctly.

Concurrency: 1 → 4 → 7 loops · May 26–27

/ralph-tdd-plan was hallucinating with a single planner agent, so I implemented a more durable planning step: three agents each for frontend, backend, and UI, each writing to its own .md, then the main agent reconciles them into one plan and verifies everything the three disagree on — which catches a LOT of mistakes. Out comes a detailed plan for the issue, ticked off locally as it goes, with separate prompts for the planning phase and the normal phase.

And then — the labels. At this point one agent was happily chugging along correctly and it was time to introduce multiple loops. I had two VMs constantly speccing new issues with /ralph-prio → /ralph-create-issues, and the single loop couldn’t keep up. So I started another: a GitHub-label claiming system bolted onto the existing pick-issue logic, with the ralph-loop-done.sh script unclaiming the label. I added a RALPH_LOOP_ID to an .env so each VM could shift its local ports for E2E testing and identify its own claim label. This worked… ridiculously well.

Supervised autonomy — loops chugging across the desk, day and night.

May 27 — it worked so well I’d burned 40% of my weekly plan in about 24 hours. Time for some tokenomics: Opus for planning and orchestrating, Sonnet for subagent work, and I cut the planning fan-out back from 12 agents to 9. Then the model reset on a Friday with the Opus 4.8 drop — a real relief, since I was nearly out of weekly tokens despite pulling back.

The ceiling I actually hit: the Max (20×) plan — weekly all-models at 87%.

Cruise control, then the quality wall · May 29–30 →

By now things were spinning. The pipeline existed, I knew what I wanted to build, and it went fast. My days were /ralph-prio and /ralph-create-issues in two windows, creating as many issues as I could while four loops chugged through the day; at night I opened six and let them rip. Every morning I came back, the issues were done. The bottleneck was clearly me — but it was going so fast, with no major problems, and I couldn’t see how to remove myself from the issue-creation part as the grilling sessions often surfaced things that needed alignment.

“Every morning I came back, it was finished” — the agents chugged.

May 29 — I fixed a batch of small reliability issues over; iteration time dropped once they were gone.
May 30 — rolled back from Opus 4.8 (bugs), introduced communication tests (evals), and started figuring out how to test those. This was SLOW compared to everything else — a lot of it inexperience, maybe. From here the loop itself barely changed; I was experimenting with evals, evaluating options, building my own integration tests, and finding nothing that felt reliable (Swedish? lack of time and experience?).

flowchart TD
    classDef human   fill:#fde2b3,stroke:#d98c00,color:#5a3d00
    classDef system  fill:#cfe3ff,stroke:#2f6fd0,color:#0b2c5c
    classDef trigger fill:#eeeeee,stroke:#888888,color:#333333
    classDef done    fill:#d9f2d9,stroke:#3a9c3a,color:#14501a
    classDef header  fill:#eaeaea,stroke:#bbbbbb,color:#111111

    subgraph AUTH[" "]
        direction TB
        Ahdr["My machine — human in the loop"]:::header
        Ahdr --> A1(["Ask Claude to add a user story to<br>user-stories-atomized.md in the correct format"]):::human
        A1 --> A2[["/ralph-prio<br>promote story: user-stories-atomized.md<br>→ user-stories-list.md"]]:::human
        A2 --> A4[["/ralph-create-issues<br>thorough research and grill session to align"]]:::human
        A4 --> A5["GitHub issue #N created<br>label: agent-ready"]:::trigger
    end

    subgraph VM[" "]
        direction TB
        Vhdr["Loop VM — no human present"]:::header
        Vhdr --> L1[/"./ralph-loop.sh"/]:::system
        L1 --> PICK[/"ralph-loop-pick-gh-user-story.sh<br>find current claim or claim new issue #N"/]:::system
        PICK --> D{"current-plan.md exists?"}:::trigger
        D -- "NO" --> NOPLAN["inject #N into PROMPT-plan.md<br>run claude"]:::system
        NOPLAN --> PS[["/ralph-tdd-plan<br>→ current-plan.md"]]:::system
        PS --> PD[/"ralph-loop-plan-done.sh<br>delete subagent reports and SIGKILL"/]:::system
        PD --> PICK
        D -- "YES" --> HASPLAN["inject #N into PROMPT-tdd.md<br>run claude"]:::system
        HASPLAN --> TS[["/ralph-tdd<br>red-green-refactor + e2e"]]:::system
        TS --> TD[/"ralph-loop-done.sh<br>tests green? close #N · unclaim · SIGKILL"/]:::system
        TS --> TO["TIMEOUT — SIGKILL"]:::trigger
        TO --> PICK
        TD --> PICK
    end

    A5 ~~~ Vhdr

One user story, start to finish. Amber = human in the loop, blue = the loop on its own. Rounded = me asking, double-bordered = a skill invocation, slanted = a bash script. Each iteration runs a fresh claude session under a hard timeout and gets SIGKILLed — the claim label and current-plan.md are the only memory in between.

§6 WHAT BROKE

So, what broke? Several things. Everything easily testable worked great. The corner cases I hadn’t thought of, and the LLM’s replies, are where the time went. Just now in production, an outgoing tenant confirmed the chat bots confirmation, and the LLM happily booked again — a duplicate event that made it think the employee had blocked themselves. Create instead of upsert. Not hard, but I hadn’t thought of it at spec time. Would I have, building it by hand? Maybe. Not every time.

The LLM seam

The biggest problem was the LLM itself. This was my first production chatbot — hell, my first real Python and Vue too — so a few landmines were fair. It was Swedish instructions in a system prompt with no clear structure, a multi-actor flow with time, reminders and notifications. I tried to write evals, but the soft passes never felt right, the loop couldn’t iterate on them, and the API bills climbed. I even thought about moving the orchestration to n8n just to see the flow, but that felt too complex to bolt on. So I went back to testing the flows by hand. After a few failures I decided I wanted more determinism, and moved a lot of the replies to regex matching: “Here are the available times — 1, 2, 3 — answer with the number.” with states that only did number matching.

It’s a little demoralizing when it goes so fast and then stops. With the ralph-loop it just fucking chugs, it just goes, goes, goes, and all of a sudden you’re standing still, writing debug and test tooling, velocity at zero.

Phantom-code

Then there’s lack of knowledge, or the phantom-code feeling. In meetings I couldn’t always answer the employees’ questions about the code which as a developer feels.. weird? In one text the floor number was missing, and I didn’t know whether the SMS is built in the frontend or the backend. I’d been in the design conversation, but I hadn’t written it, so the best I had was “I think it’s X, let me check and get back to you.” I could ask Claude every question that came up, but that took minutes, and the discussion had moved on. It felt like a phantom limb — phantom code, phantom knowledge. I know it’s there, but I can’t touch it.

What I shifted to instead was customer value. “Everything is possible. Tell me what you want from the user experience, and I will make it so.” They’d tell me; I’d note it, have Claude research it after, fire off /ralph-create-issues for anything not built, and bring the result next meeting. My mindset moved from “I have to know the code” to “I’m going to make a valuable product for you, and I know it can be done — it’s just a question of the right instructions.”

Skill drift

Skills drifted, too. Once the specs were built and employees started testing, new wants came in — “here it should say this,” “it should always end with a signature” — plus things I hadn’t planned, like logging and error alerts. I’d fire /ralph-create-issues on the spot, but the new features weren’t anchored in the spec, so the skills still expected the old pipeline and I could see in the thinking text Claude got confused, wasting context. It’s a debt, a smell related to skills: drift that shows up first as repetitive instructions quietly violating DRY. The fix is same as software: refactor. For this, I will admit I let it run and/or sometimes violated DRY by repeatedly adding a line to the prompt: “this feature does not exist in user-stories-list.md”.

Refactoring

Refactoring mid-flight is tricky, because issues are anchored to file and line references and drift often caused confusion. Each refactor meant stopping the loops, waiting for in-flight work to land, then re-planning before the next batch fired. Multiple concurrent ralph-loops feels like a heavy stone wheel on an axle: spinning fast, work piling on. Refactoring means waiting for it to slow, lifting it off, fixing the axle, then getting it spinning again. It’s way more noticeable when you have sequentially dependent issues. Next time I’d put LOC limits in the Claude hooks instead of instructions so the Claude is forced to refactor on the fly; god objects and thousand plus lines of code was a recurring problem.

Home made Eval Harness

Every unit test was green, and booking still had five clear bugs in a single day. They came from two almost-identical functions, used interchangeably, that returned different results — none of my refactors or scans had flagged them. Only a deliberate debug session found it and reconciling the duplicates cleared five bug issues at once. Anything unit or Playwright-testable worked well after each issue; the conversation chatbot didn’t — not enough experience on my side resulted in a home made eval harness that didn’t fulfill its purpose. I believe, hope, offering better testing to the AI would have had a higher chance of catching this.

§7 REPO NUMBERS

I looked at the code maybe a handful of times. Every time I did I got a little frustrated — there’s so much I’d want done differently — but in the end, if it works, it works.

The shape of the whole thing: very few commits at the start, all understanding and aligning; then the middle, pushing like rocket fuel; then the polish, which took forever. And the “autonomous” loop kept office hours — every morning I came in, it had already run dry. The bottleneck was clearly me.

§8 WHAT I’D DO AGAIN + DIFFERENTLY + LESSONS LEARNED

Again

Spec hard, up front. The intensive spec phase that produced architecture-spec.md kept the planning agents in line for five weeks. I can’t remember a single architecture, design, or technical question coming up that it had already covered. It really proved useful, guiding every issue from the starting point.

The UI design reference. Also a godsend, and a clear lesson from previous projects: whenever Claude has to design a UI by itself, it’s never what you want it to be.

Multiple VMs to fix tap-the-loop pattern. The “multiple VMs to fix one ralph-loop” approach worked surprisingly well. I set ralph-loop.sh to reload itself every loop so changes got picked up. Most of the time I nudged it, sometimes I stopped and pulled the latest change (which it did by itself too), but the speed at which the ralph-loop got durable and competent was great. It went a lot faster than when I’d done it the other times.

Differently

Find reference projects first. One thing I’ll take to the grave: when you’re coding with AI and don’t already know how to instruct it, find reference projects. In my case I’d never built a chatbot before, so I should have downloaded some official samples or an open-source project with an architecture I liked. Even Opus 4.8 on xhigh, which is what I always run, went in the wrong direction sometimes; with further research it invalidated its earlier thoughts. I’m somewhat mad at myself I didn’t do this. :P

Decide the testing strategy on day one. If I’d spent a bit of time thinking about the process up front I’d have realised I needed a way to test the “booking dance”: multiple people scheduling, calendar conflicts, rebooking, and so on. I should have spent more time thinking about how to actually test it, since tests are crucial if you want to rely on the ralph-loop. Swedish + many states + multiple actors + time was tricky.

Write the LLM log tool sooner. Adding a simple column the chatbot module could write decisions and flow to, with a Python script that fetches last runs, gave Claude immediate and detailed access to a run, and it was really helpful to debug. It’s been instrumental in figuring out what happens in the situations where it isn’t apparent.

Lessons

Backlog feeding is real work. For ralph-loop issues, I think this is what a lot of the work eventually comes down to. Once everything is set up, what’s required is to feed the pipeline and make decisions. I realise this can be automated from logs and such, and I want to try connecting Papertrail from Heroku here. But it is apparent that when creating something new somewhere you have to align the human and the AI unless you want to backtrack and revert. In my experience, developing new software will uncover situations not thought of, and they have to be handled. The issue creation felt natural for that part.

Context was not correctly wired up I will admit I uncovered this while writing the blog post, which is a bit embarrassing. But here goes: Despite architecture-spec.md there needed to be refactors. This writing made me reflect, “why”? I investigated and architecture-spec.md was referenced in issue creation but NOT for implementation. So implementation skills were provided limited guidance in some areas. An estimated ~65%, 40 refactor issues, could have been avoided if this single line was referenced correctly. (Rest of refactors was due to unforeseen requirements and the LLM-chatbot architecture change).

§9 A SHORT NOTE ON SECURITY

With CS background I knew about the major pitfalls. Brute forcing passwords, open endpoints, prompt-injection etc. I ran several security checks and made sure the basics are locked up. Is it Fort Knox? No. Is it better than I would have done manually? Definitely.

§10 WAS THE RALPH-LOOP WORTH IT?

If I had not used AI as much as I did, I would not have been able to build this in the time allotted. At the same time, I would not even have TRIED to build this product had it not been for AI. I would have found a smaller slice that fulfilled part of the value proposition. Would the employees been as happy? Arguably no.

The system is live and verifiably functioning. It’s secure, at least as secure as I would have done manually. The bills are not skyrocketing. Functionality exists that would not have been built if not for AI, and it has delivered real consumer value. So I would say yes, it was worth it to remove myself as the bottleneck.

Plus, I got to spend some serious time building autonomous AI, which I thoroughly enjoy.

§11 CLOSING + JOB PITCH

I’m looking for work

The system’s been live a few days. It’s already booked viewings and surfaced corner cases we’d missed, fixed in a couple of hotfixes. We estimate it saves ~20 hours a month of a boring, monotonous task, and the non-technical employee who runs it seems genuinely happy there’s no more triaging tenants over SMS and email. The payback is estimated to be about a year, and arguably would drop fast with more apartments or employees.

I found the whole process to be rewarding and I’m looking for a job like Forward Deployed Engineer, Applied AI Engineer, or adjacent roles where I can build applied AI infrastructure and help solve real problems. Small-ish teams, whip-smart colleagues, big AI budgets, customer contact if possible. Reach me at caewerbring@gmail.com.

Docker-vm and ralph-loop scripts