Three days of research before a line of code

§0 TLDR

Over three days in July I pointed ~94 subagents at a product I did not know how to build, a lease analysis SaaS for the Swedish commercial real-estate market, to research the architecture before I wrote a line of it. Same principle as the last project, aimed one step earlier: the fewer ambiguities the AI has to resolve, the better. Only this time the ambiguities were mine.

It ran off a single markdown file that was spec, work queue and journal at once, restarting itself through 10 usage-limit deaths without losing work. Only 30 of the ~94 agents did any actual researching — the rest existed to check them or write them up. Two blind researchers per topic, a reconciler that settled every disagreement with a fresh web search, then seam checks, red-teams and a completeness critic on top. It caught ~21 fabrications, including one paper that said the opposite of what it was cited for.

I asked for an architecture and also got a lot of business numbers I hadn’t asked for, including a break-even at 8–13% of market share, which is a meh number. A fraction of the research has been read by me, which was the plan.

3 days · 3 working sessions · ~94 subagents · 85 files · ~536,000 words · 326 mermaid diagrams · 690 dated citations · ~21 caught fabrications · 38 open questions handed back to me

I’m still looking for forward-deployed engineer work, so same as last time — consider this post a work sample, and judge for yourself!

§1 BACKSTORY

The last project I wrote up here was five weeks to production with ralph-loops — a booking system I built with autonomous agents at a small real estate company. I got some (already known, sigh) lessons from that one, the biggest one I got wrong: find reference projects first. Reference projects are fucking great. When you’re coding with AI and you don’t want it to make bad assumptions, a solid reference project removes so many ambiguities. I didn’t do it that time (for the chatbot bit) and I paid for it with hours and hours of debugging.

So when Fable came out I was thinking of a new idea with a friend, a lease analysis SaaS for the Swedish, and potentially the European, commercial real-estate market. Property managers and landlords sitting on thousands of lease contracts, invoices, maintenance agreements, none of it in a standard format, none of it queryable.

And I realized pretty quickly that this was quite big. It had a lot of things, a lot of moving parts, and I don’t have the experience to have built all of this. Document mining and reviewing, multi-tenancy with cross-org delegation, an invoicing layer that connects to whatever system the customer already runs, knowledge of when to train LoRAs, and how to deal with data sovereignty issues and where data is allowed to live. I knew that at some point I could look for specific reference projects to deal with individual problems - but how would it all fit together? And, most importantly, how could it be developed with autonomous agents?

I have a partner in this, who is on the sales side with industry experience. To derisk early, I had developed two prototypes through vibe code that showed what worked right now to extract (Opus > Pymudpdf > rerank with Haiku, after trying out deterministic solutions such as Docling) as well as a simple UI of what was to be shown. They both worked well enough for us to want to continue on this trajectory.

It’s vacation time in Sweden, and so when Fable came out and there was a few days of limited access I thought it would be fun to create a huge database of information that I could mine instead. I thought I would use the credits and see what it can come up with. That’s basically the basis of this research task.

The goal: a repository of information that can be used to guide the technical implementation and high level architectural choices that will later follow, so that I would not paint myself into a corner.

Here’s how it went down.

§2 WHAT CAME BEFORE: BUSINESS MODEL, THEN A GRILLING

To start things off I did do a little bit of research with Claude on the web in research mode. Through discussions we made some competition analysis, finding suitable references (prophia.com) and an overall plan. While I did not include the artifact in the research prompt, the initial work helped me shape the structure of my ask.

To start things off, I started with an open question, dumping info, and invoking the /grill-me skill. The research spec became locked. Some details:

§1 of the plan file — what we’re building, who buys it, and the domain model in my own words. Locked before the main research run.

In addition, we specified a few other things:

• Data sovereigny was a risk, and US/EU/Swedish deployment options was to be researched
• It needed to investigate fine tuning vs prompting vs regular APIs, the data required for each, and the situations where each made sense
• For specific features and what they entailed, prophia.com was to be used as a reference to infer what type of features were to exist. ESPECIALLY document upload > review > data connected to source.

This is the same move as last project’s locked specs, applied to a research run. Ambiguity is what kills autonomous work, and so I tried to see. How much valid research could I do upfront?

§3 THE REQUIREMENTS I HAD FOR THE DEVELOPMENT

The grilling turned into these requirements that the product we researched had to fulfill, which became the goal of the agents to find a path to.

• I wanted to be able to push my docker vm to a VPS and develop it agentically e2e from there, which meant…
  • The docker VM had to contain all infrastructure, so monolith deployment but split service codebase
  • No Supabase/BaaS
  • Entire stack had to work in Linux VM
• It should support adding features without breaking existing ones
• Able to switch between US Model provider API, <-> EU model provider API <-> self hosted vLLM on S3, EU bucket <-> on-prem servers
• Expose product API such that we could create a potential chat bot overlay with easy access
• The data extraction is separate from the deterministic use of the data

Each thing in and of itself is not a majorly hard thing and would take a few days to pin down the manual way. But all of it, together, with corner cases? I wanted something that I could rely on, and my hope was that Fable could find options that would stand up to all of the requirements, such that I did not have to backtrack on something after some time of development.

§4 THE PLAN FILE IS THE PROGRAM AND THE MEMORY

So, with this in mind a plan started to be born. We did two initial runs. One to analyze features of prophia, such that I could approve/disapprove of them beforehand. And one to evaluate using self hosted open weights model instead of Anthropic API. Besides this, I had the two prototypes.

The plan was designed so that a ralph loop in this sense, (a simple bash script, research-loop.sh) could pickup from anywhere and continue. This was needed as I realised Fable drained the tokens like no other model and I was requiring a lot of subagents. So session limits would be a problem.

The one markdown file became a spec, work queue and journal all at once. Claude was fired with an offset, a 5 hr rolling window, and a prompt:

PROMPT='/goal complete "workflow/research/1.initial-architecture-research/3.12072026 - Architecture Research Plan (execute with goal).md" — resume from the first unchecked checkbox in §7, verifying against files on disk: tick boxes whose output file already exists; redo agents whose output is missing; stop only when all checkboxes are complete'

The structure of the plan file was this

• §0 driver instructions, incl research method, using webfetch, Fable for everything, driver orchestrates, how to manage files and input/output to subagents, file naming, etc.
• §1 product brief
• §2 the 13 locked decisions
• §3 seven working hypotheses the agents must hold or explicitly refute with evidence
• §4 the dual-researcher protocol
• §5 the 15 workstream briefs
• §6 cross-cutting phases
• §7 a 72-checkbox execution tracker
• §8 a fill-in-the-blanks mission brief template for subagents.

Importantly, the orchestrator never reads the files, only executes the research and lets the agents write to disk.

§5 THE RESEARCH MODE: A/B BLIND → RECONCILE → WRITE

We all know agents hallucinate. In order to mitigate this, I went back to the trusty fan-out and-reconcile pattern. This was used for all of the 15 specific areas.

1. Researcher A and Researcher B spawned in parallel with an identical brief and blind to each other. Both do full web research and write a complete report to disk.
2. Reconciler reads both. Agreements means high confidence and Every disagreement between the two gets a fresh, targeted web search. Single-source claims get verified or tagged [UNVERIFIED].
3. Deep-dive writer — turns the reconciled report into the decision-oriented final document, with diagrams and per-profile notes.

In phase 1 we had 15 × 4 = 60 agents. In phase 2, we added an adverserial layer to do seam checks. 11 agents investigated if the documents work together, each taking one spine — the data spine, the consumption spine, the inference spine, the money spine, profile consistency, the handoff ledger. Does the WS2 data model support the WS10 tools?

After which, it aimed to fire off 3 red-teams, 1 completeness critic.

Red-teams went out to attack the results presented in the earlier research. One aimed at business, and two separate ones, in different order, aimed at security and data isolation. (Arguably.. as I am writing this, maybe we should have aimed one at architecture as well. Though it turns out the seam checks already were that.)

In total, of the 94 agent runs, only 30 were researchers and the others existed only to verify the research was valid. Then there were agents that did deep dives, and synthesis.

§6 THE BREADTH: FIFTEEN WORKSTREAMS AND WHAT I POINTED THEM AT

While this is not a particularly engaging list, what is interesting is the breadth of the plan and the way they work together. Each of them covers specific areas, and while not all of them are prioritized right now it is… comforting to know that the direction we choose in the beginning is something that scales to all of it.

As you can see, a new scan of prophia was included. feature catalog, tiers, integrations, ai features. Also at integrations towards popular services in the existing space. This is the reference-project lesson from last time, done sideways.

And there was also deliberately speculative ones I was not sure we were going to use but could come up and I wanted to use the tokens: Invoice connection, billing rails, etc. WS11 is the clearest case: it came back with the actual Swedish landscape and a build-order recommendation that says don’t build your own Peppol access point until the volume justifies €4,150/yr in fees. I asked for architecture and was served business logic around it, which was a nice addition.

§7 RUNNING IT

All of this was done in an hour or two, and so the start of the research began on Sat 12th of July. After about 40 minutes, the session context was done. So.. the script ran once every five hours, and restarted claude 4h 50 later. I ran it on my laptop, so longest run was 22 hrs. It took 10 sessions in total, and since it reclaimed all of the things on file (and apparantly subagents are allowed to finish even though you reach limit, to some extents) nothing was lost. I started with concurrency of five subagents thinking it would help but quickly raised it to 20 just to make it go quicker.

I did have some snags with pinned models from imported scripts, but the instructions always smoke-tested due to the Fable-as-subagent instruction and fixed it on the fly.

I did have one mistake happening, two seam check batches ran in parallel. Thankfully they both turned out almost identical which supports their validity.

All in all, it just chugged. 928m input tokens for the agents. 15 line summaries * 94 to the orchestrator.

§8 THE NUMBERS

Out of all of this. What did I use?

I have pointed references to this for implementation of the next round of prototyping. So far there are references to WS2 (domain model), WS8 (platform/tool registry), WS12 (frontend), WS14 (infra/dev env). Links existing in the current architecture-spec/ D-decisions.

§9 WHAT THE FILTER CAUGHT

The A+B > Reconciler found some interesting hallucinations. In total, 21 concrete facts were disputed, which is not a whole lot tbh.

For example, in one area both agents hallucinated. One suggested, “$27.85M total funding over 8 rounds”, refuted by a fresh fetch ($19.2M over 4). The other researcher stated “no recent funding event” was also wrong — a real June 2026 round exists.

One researcher cited arXiv:2503.18562, which is a paper about LLMs judging self-reported confidence, which the authors actually found opposite results of the ones AI stated.The critic also did its job. As it went through the deep dives looking for what nobody had covered, it found six things that had to be solved. Most of them were holes rather than wrong facts. One was different: the Swedish market size, tagged [UNVERIFIED], and the only one it judged that more web research could actually narrow. So to solve this it took a detour and triangulated info from the Swedish Central Bureau, the branch organisations (Fastighetsägarna and Sveriges Allmännytta) and the numbers from the largest real estate news site, to end up with a (more) reliable estimate. Thats how the market share break even point below came to be. This was very cool to see.

After the final synthesis a final verifier rechecked all 126 internal links. It also checked all 19 diagram renders and re-fetched 10 architecturally important sources that the synthesis leaned heavily on. 8 came back clean, 2 confirmed with nuance.

Arguably, many more could be wrong! It is not unlikely the AI has overlapping hallucinations. The reconcilers and then the final verifier checked against live sources twice over, and in that, I feel personally like it can be somewhat trusted. I am sure there are some mistakes, but the overall picture, which is what it was supposed to deliver? That feels solid.

§10 THE END RESULT

One big fat synthesis, which was derived from the 15 deep dives. The big fat synthesis contains architecture, inference substitution matrix, diagrams of docker containers and microservices, components, sequences, deployment details, and more. All linked to the deep dives for further info. It landed at 2,076 lines / 20,538 words with 19 embedded diagrams, split into three segments so I would not have to open the whole thing every time: 1,649 lines of architecture (where all 19 diagrams live), 220 lines of open questions and the reference index, and 187 lines of business. This was my starting point for starting to digest the information, see section 12.

§11 I ASKED FOR AN ARCHITECTURE AND GOT A BUSINESS MODEL

So I asked it for tech details, including setup time for FDE labour for new documents, etc, and it came back with.. valuable business insights, based on the tech. Of the 15 areas, competition and cost model was purely money related, but every area priced itself as well and used it to evaluate the suitability of different choices.

Am I confident to build a business on these? No certainly not. But they are valuable to start a thought process. For example, break even points, LoRA training, and document review hours.

In general, one thing stood out: “Run-rate costs of the three deployment profiles converge within ±2% at 200 customers, so the choice is about sovereignty and market access, not cost.” Meaning that from a cost perspective, it didn’t make a lot of sense to train our own models if we had an as labour intensive model as prophia.

One important finding regarding pricing that makes immediate sense when I talked about it with my partner: don’t price per seat (as is the standard in the industry), price for onboarding and per document. It made sense to my partner as he has experience of whole organisations sharing licenses in order to save money.

It also found a stark reality, this business is TOUGH. Working with the assumptions the AI did, break even landed at 8-13% of market share. BREAK EVEN. Crazy. So much for going into a low margin business, eh? While this is not necessarily true, or right, or the only way, it is a great starting point for a conversation of 1) what we would need to do differently than prophia, 2) how to expand market size or 3) how to find another market to start in, like asset transactions, instead of asset management. It’s a number I did not ask for, but a number that showed up.

I fired this thing to answer how do I build it. It also answered is this a business, because you it couldn’t evaluate the three-profile architecture without pricing all three, and once everything is priced it started adding it up. It was a cheap side effect of every agent showing its cost, and I believe, having the initial prompt about this being a startup case, the goal was to understand the viability of the endeavour (from the tech side, but I got business as well).

§12 HOW I ACTUALLY INGESTED 536,000 WORDS

So what did I do with all of this data?

Well at first it was just a lot of words. And diagrams. And… well, information overload.

So I went into a new folder, and started asking questions. I asked a bunch of them, and asked claude to draw me a lot of diagrams for specific parts. This, I realized in my previous project, works very well to cut down on information transfer while surfacing what is important. Diagrams of platform architecture overview, the dev-environment container design, the MVP vertical slice and how it grows, and how one feature spreads across the codebase. Questions questions questions, diagrams diagrams diagrams. This dialogue surfaced more requirements that I later took with me into the project.

Here’s one of them — the MVP vertical slice, and what happens to the codebase each time a feature is added:

flowchart TB
    classDef spine fill:#eeeeee,stroke:#888888,color:#333333
    classDef f1 fill:#d9f2d9,stroke:#3a9c3a,color:#14501a
    classDef f2 fill:#cfe3ff,stroke:#2f6fd0,color:#0b2c5c
    classDef f3 fill:#fde2b3,stroke:#d98c00,color:#5a3d00
    classDef f4 fill:#f7d6d6,stroke:#c0392b,color:#4a1010

    subgraph SPINE["Shared spine — built once, unchanged as features are added"]
        direction LR
        USER["User (webapp)"]:::spine
        UP["Upload + mining pipeline<br>classify · extract · ground · entity-link"]:::spine
        TERMS["Effective-terms engine<br>current terms after amendments"]:::spine
        EXEC{{"Tool registry + executor choke point<br>authz + RLS + audit on EVERY call"}}:::spine
        DATA[("PostgreSQL + object store<br>facts + raw PDFs")]:::spine
        USER --> UP --> TERMS --> EXEC
        EXEC --- DATA
        UP --- DATA
    end

    subgraph F1["1 · MVP — report of current leases"]
        D1["doc type: lease + tillägg"]:::f1
        TL1["tool: current_leases / rent_roll<br>(READ)"]:::f1
        R1["report: what each lease says today<br>with click-to-source"]:::f1
        D1 --> TL1 --> R1
    end

    subgraph F2["2 · annual reports, to calculate rent correctly"]
        D2["+ doc type: tenant annual reports"]:::f2
        TL2["tool: rent_calc — joins lease × annual report<br>index / turnover rent (READ)"]:::f2
        RV2["review task: 'need annual report for lease Y'<br>a missing input, not a wrong number"]:::f2
        D2 --> TL2 -.-> RV2
    end

    subgraph F3["3 · maintenance bills vs contract — show the diff"]
        D3["+ doc types: maintenance contract + bills"]:::f3
        TL3["tool: maintenance_audit<br>contracted cost vs actual invoiced (READ)"]:::f3
        D3 --> TL3
    end

    subgraph F4["4 · send correcting invoices to rentees"]
        TL4["tool: outgoing_lines_generate → settle<br>WRITE / EXTERNAL — confirm-gated + audit"]:::f4
        CONN["connector layer (WS11)"]:::f4
        RAILS["Peppol / ERP / bank rails<br>the first external egress"]:::f4
        TL4 --> CONN --> RAILS
    end

    UP -. "+1 extraction schema" .-> D1
    UP -. "+1 extraction schema" .-> D2
    UP -. "+1 extraction schema" .-> D3
    EXEC ==> TL1
    EXEC ==> TL2
    EXEC ==> TL3
    EXEC ==> TL4
    TL2 -. "rent diff" .-> TL4
    TL3 -. "maintenance diff" .-> TL4

    style SPINE fill:#fafafa,stroke:#888888
    style F1 fill:#f1fbf4,stroke:#3a9c3a
    style F2 fill:#f2f7ff,stroke:#2f6fd0
    style F3 fill:#fff8ef,stroke:#d98c00
    style F4 fill:#fdf1f1,stroke:#c0392b

The MVP vertical slice and how it grows. Grey is the spine, built once. Every feature after it is +1 extraction schema into the same pipeline and +1 tool behind the same executor — only feature 4 lights up the connector layer and leaves the building.

I have not read all of the research, not nearly, but the main thing I took away from it was a feeling of confidence. I felt that this can be built. It is not rocket science, after all. And it should be able to be built somewhat quickly, given that a LOT of the thoughts had been thought through already. I was also confident that I could start, and not have to worry about major overhauls later.

All of the Questions I had about “how” to design this, made sense after a few hours of partly diving into the material, and mostly having conversations with the AI.

The way I understood was to make the AI teach me, with pictures, like a five year old. It worked surprisingly well! Last project I said “I have not read the code”. While not entirely true, I can jokingly say, “I have not read the research” this time.

§13 HOW I USE IT IN THE CURRENT PROJECT

The project now runs a three-layer document model, and the research is layer 1:

• A skill, /ralph-arch, sits between /ralph-prio and /ralph-create-issues. It fires off an armade of subagents, some of them reads the research, and then through interview locks a track’s decisions into layer 2 before any issues get written.
• Decisions are namespaced per track (T14-D1) so parallel runs never contend on a counter, and a new track never edits a prior track’s decisions - it can only record an override. A bash script helps the skills to quickly walk through decision chains.
• Issues reference decisions by D-number, never copy them. This is the direct fix for the five-weeks post’s most embarrassing finding — architecture-spec.md was referenced at issue creation but not at implementation, which cost an estimated 40 refactor issues.
• The amount of D-decisions currently trace back to the research: 59 of 181

Currently, the stack lives. It has services of mining, inference, frontend, backend, database, and platform-worker. Is deployable as its own VM, and the ralph-loop works well on it. We’ve taken it to the next step, actually building a foundation for interpreting the leases correctly with measured accuracy, and it’s extendable.

§14 WHAT I’D DO AGAIN, DIFFERENTLY, AND WHAT IT COST

Again

• Grilling session to lock the decisions in a human interview before any agent runs. 13 decisions, marked do-not-reopen, helped shape the flight.
• Two blind researchers plus a reconciler that resolves disagreements with fresh evidence.
• The checkbox tracker. It’s what let the run survive its own execution environment killing it 10 times.
• Making the AI teach me the result with diagrams instead of trying to read 536k words.
• Giving it broad reigns and general thoughts, without which I would not have had business model implications to this degree.

Differently

• Do two waves. A lot of tokens (est… 20-30%? of the 928m) was spent that did not have to be spent, and maybe that would have been found at the start had I done an initial wave of planning it out and having another grilling session, then adapting instructions to focus on what was important and reporting structure before start of the real ones. But the setup took only a few hours, and then it was hands off for a few days, so it all worked out well in the end.
• State the end goal more closely. I knew I wanted to use this mostly for architecture design, and that was not clear in the prompt.

§15 CLOSING

Was three days and 928 million tokens worth it? So far, yes. Over two weeks I’ve built a solid document data extraction tool based on this architecture. Agentically, with amazing (subjective, of course), architecture.

What I can point at right now: 59 of the project’s 181 locked decisions cite the research directly, the stack deploys as its own VM, and I haven’t had to go against the architecture yet.

All in all, it was one run to get insights and decision support that would have taken me a lot longer to gather myself. And (so far) it holds up.

I’m looking for work

This is the second of these I’ve written. The first was a system in production; this one is the thinking that comes before one. It’s exciting when someone hands you a domain you don’t know, and the job is to learn, and figure out a suitable structure.

I’m looking for Forward Deployed Engineer, Applied AI Engineer, or adjacent roles — applied AI infrastructure, real customer problems, small teams, whip-smart colleagues, big AI budgets. Reach me at caewerbring@gmail.com.

If you’d rather look at the machinery than my write-up of it, the plan file and research-loop.sh are both up here as-is.