Subliminal Labs
Subliminal Labs Blogsubliminal.inc

Every Company Is an Agent Swarm

Now we just have to rebuild it for AI.

Complete trajectories of every employee.Every on-screen action and connector event, with provenance.

Subliminal (on screen)SlackGoogle Workspace
arrives from outsidestays insideleaves for outside

Introduction

Consider a company. It's made up of independent actors, each with their own view of the work and their own tools. They message each other, hand work off, and together turn inputs from the outside world (customers, suppliers) into outputs.

Sound familiar? It's an agent swarm. The agents are just powered by human brains instead of LLMs.

To replace that swarm, you first have to understand it. Luckily, a swarm of humans can be observed exactly like a swarm of AI agents: through their trajectories. AI agents log theirs; humans' are a little messier, scattered across screens, apps and chats. Capturing them, and stitching them into one record of the whole company, is what we built Subliminal to do.

By analyzing these trajectories, we can write down the program the human swarm has been running all along. Then we rebuild it for AI agents and let an AI swarm run it instead. From there, we can continuously optimize the program based on how it performs.

There is one more missing piece: testing. How do we know the new AI swarm will actually work in the real company?

After weeks of recording, we derive the program using the earlier weeks only and hold back the last few. Then the AI swarm runs that program through the held-back weeks against a simulated outside world, and we check that it does as well as or better than the human swarm did. With this, the company's history has become its own benchmark.

Recording the swarm

An AI agent's trajectory is written as it works, with the harness logging every observation and action it makes. People leave nothing like that behind. One piece of work is spread across a ticket, a Slack message, a payment tool and an email, mixed in with everything else they're doing. So we build each employee's trajectory ourselves from their screen recordings, extracting everything they saw and did throughout the day (e.g. “read a support ticket”, “issued a refund”) along with the details and a timestamp for each action.

The screen only shows what someone looked at, though. An email nobody has opened yet, or the rest of a file someone only skimmed, never appears on it. So we also connect to the company's tools, like Gmail, Slack, GitHub and Google Drive. These connectors fill in what nobody looked at, and they're how an AI swarm will eventually act on the company's systems. Every event, from the screen or a connector, is tagged with its provenance: the footage and keystrokes it came from, the Slack message, or the repository and commit in GitHub.

Fig. 1 follows one case, ticket #4812. A customer has been charged twice. Maya, the support lead, assigns it to Daniel. Daniel finds two identical $49 charges, asks Grace in finance whether to refund one, waits six minutes for her answer, then refunds the later charge and replies. In their trajectories, #4812 is interleaved with the other cases they had open (the grey stretches), and pulling it out is part of the work. Hover over any step in the figure to see the screen it was written from.

outside the companysupport leadsupportfinancecompany boundaryother casesother casesother cases09:141sendsticket09:152assigns itto Daniel09:163reads theticket09:174finds two$4909:185“ok torefund+6 min09:246“yes, thelater one”09:257refunds$4909:268replies
Fig. 1. Ticket #4812 across four people, numbered in order, with their other work in grey. Hover a step to see the screen it was written from.

Trajectories link wherever one person's action becomes another's observation: a message sent and read, a ticket assigned, a row someone edits that someone else later opens. Daniel's “ok to refund one?” is an action in his trajectory and an observation in Grace's. It doesn't say which ticket it's about, so we match it from the content, the amounts and the timing. Joining every such link, on one clock, gives the joint trajectory: the record of the whole company.

Fig. 1 also marks the company boundary. The ticket crosses it inward, so it's an input. The refund and the reply cross it outward, so they're outputs. The lookup, the question, the six-minute wait and the answer all stay inside.

A trajectory is only a record, one sample of employee 's policy . Run together, the policies of all employees make up the company's joint policy , and the joint trajectory is one sample of it:

holds many cases at once, interleaved. Switch Fig. 1 to Outside and one case shrinks to three things: the input , the company's record up to the moment it arrived, , and the outputs that crossed the boundary. Every case in gives one such triple, and if we treat cases as independent, each is a sample of the company's behaviour seen from outside, . Cutting one long sample into many cases is what makes the company testable.

The customer never sees Maya, Daniel, Grace or the six-minute wait, only the refund and the reply. So many choices of , with very different trajectories inside, give the same . If a different swarm gives this customer an equally good refund and reply, its joint policy is equally good.

That is what lets us change everything inside. Next we write down what happens inside the box, then rebuild it.

Writing down the program

The program isn't written down anywhere in the company. It's implicit in what the human swarm does: who to ask, which charge to refund, what to check first. But every time someone follows it, they leave evidence of it in the recording, and from enough of that evidence we can infer the program itself.

We write it down as a wiki: plain-text pages on the company's people and roles, who hands what to whom, its systems and rules, and how past cases went (Fig. 2). Models read the joint trajectory, group cases by kind, write a page for each kind and keep the pages current as recording continues. Every claim on a page keeps its provenance: the cases and trajectories it was drawn from. None of it comes from interviews.

The pages are specific enough to act on: exact steps, thresholds, which reply template to use. Each rule says how often it was seen and where people did it differently, and the wiki flags what it doesn't know. Whether it's actually useful is measured in the first result.

This wiki is the company's program as it is today, waits and detours included. It is also what we edit from here on. We change the program, and the benchmark scores the joint policy that results when agents run it.

How do I… › Support tickets

Refund a double charge

Who
Support, with Finance’s OK
When
A customer reports being charged twice
Systems
Zendesk, Stripe, Slack
  1. Find both charges in Stripe: same amount, same day.
  2. Ask Finance on Slack before refunding. approved all 31 times asked
  3. Refund the later charge, never the first. seen 31 times
  4. Reply with the “duplicate refunded” template and close the ticket. seen 34 times
Gotcha. Two charges with different amounts are usually an upgrade, not a duplicate. Check the plan history before refunding. seen 3 times
Conflicting practice. Two versions seen for annual plans: refund at once, or ask Finance. This page defaults to asking Finance.

Written from 5 weeks of recording: 12 people, 300 person-days. Describes what was seen being done, not official policy. Updated daily.

Fig. 2. One page of a company's wiki, written from screen recording, with how often each step was seen.

Rebuilding it for AI agents

The obvious move from here is to give every employee an AI replacement and keep the program as it is. But that would ossify the patterns the company has today. Instead, the AI swarm should be free to do the work in whatever way suits it.

The program the people run is shaped by human limits. Each person has one screen, working hours, a narrow role and a set of permissions. Nobody holds the whole picture, so work gets handed off and waits in queues. An AI swarm has different limits, so we ought to rebuild the program for it. The rebuilt program is the wiki rewritten for AI agents. Its pages on the company's systems, rules and past cases stay; its pages on who does what now describe agents instead of people, each with the whole wiki as context and acting on the company's systems through the same connectors we record with. Agents running it give a new joint policy, . The only constraint is the company boundary: the same inputs have to lead to outputs that are at least as good.

Fig. 3 shows a simulated twelve-person company both ways. Today ticket #4812 passes through Maya, Daniel and Grace. Rebuilt, a router agent hands it to a support agent that sees the charges and the plan history at once. It also knows from the wiki that Finance approved all 31 such refunds it was asked about. That is evidence the approval rarely changes anything, not proof, so dropping it is a change the benchmark has to back up on held-out cases. In Fig. 3 the agent refunds the charge itself. Each kind of agent owns one set of systems (tickets and Stripe, the CRM, the ledger, orders), and each runs as many copies as there is work.

InputsMayasupport leadMarcoopsElenasalesJordansalesDanielsupportPriyasupportLucassupportHannahsupportGracefinanceSamfinanceTomopsAishaopsOutputs1 · assigns it2 · checks, asks Finance · 4 · refunds3 · answers after 6 minMon 11:02
12 minTicket #4812, time to resolve
45 minAll work, median time
2.2Hand-offs per case
12People

ticket #4812, the double charge from Fig. 1 all other work · today, dots beside someone are waiting for them; rebuilt, each ticket gets its own agent instance · simulated

Fig. 3. A simulated company run by twelve people, and the same company rebuilt as AI agents, with ticket #4812's path highlighted. The times come from the simulation, not a measurement.

Nobody rewrites the wiki by hand. A model proposes edits to it, the benchmark in the next section scores the result, and an edit is kept only if the score holds or improves.

This puts all the weight on the benchmark. If it scores the wrong thing, the search will optimize toward the wrong thing and report success.

Every company is its own benchmark

By the time we rebuild anything at the company, their joint trajectory already pairs weeks of inputs at the company boundary with the outputs the company produced in response, the standard any replacement has to meet. Three pieces turn that into a benchmark, and Figs. 4–6 follow one ticket through them:

  1. Temporal holdouthold the later weeks back as tests.
  2. Closed-loop simulationrun the AI swarm through them against a simulated outside world.
  3. Outcome equivalencescore each case on whether it ended as well as it did for the company.

Temporal holdout

A temporal holdout cuts the joint trajectory by time: we write the wiki from the earlier weeks only and hold the later ones back. The held-out weeks are split once more. The first block is used to choose between versions of the program, and the last block, which nothing is tuned on, gives every number we report. So nothing from a test case can leak into the program it tests.

Every input after the cut starts one test case, paired with what the company actually did. Cases share queues, accounts and systems, so each test starts from the company's real state at the moment its input arrived.

history: the wiki is written from thesechoosereportwk 1wk 2wk 3wk 4wk 5wk 6wk 7wk 8#5127
Fig. 4. The recorded weeks cut by time. The wiki is written from weeks 1–5, week 6 chooses between versions, and weeks 7–8 are reported. Ticket #5127 is one held-out case.

Closed-loop simulation

The simplest way to run those cases would be open-loop replay, which feeds the AI swarm the recorded messages in order. But this breaks the first time the swarm does something differently to the humans, which is the whole point of the rebuild. Once the agent answers differently from Daniel, the customer's next recorded message no longer fits.

So instead, we simulate the outside world. A simulated persona of each counterparty, whether a customer or a supplier, is built from all our recorded trajectories. It knows what the real one wanted and responds to what the AI swarm sends during the testing.

In Fig. 5 the simulated customer asks whether they can switch back to monthly, which the real customer never had reason to ask. The swarm acts in a sandbox copy of the company's systems, so a refund changes a balance without moving money. This is a closed-loop simulation, and each run of the AI swarm through a held-out case is a rollout, recorded as a joint trajectory of its own.

Ticket #5127, as it really happened from the recording · 45 min

  1. “Charged twice: $49, then $99?”
  2. Daniel asks Finance
  3. waits 38 min
  4. Grace: “the $99 is the upgrade”
  5. Daniel explains the upgrade

The AI swarm, replayed simulated customer · sandbox Stripe · 6 min

  1. “Charged twice: $49, then $99?”
  2. agent checks the plan history
  3. agent explains the upgrade
  4. “Then can I go back to monthly?”the real customer never said this: the simulated one is reacting to the agent
  5. agent explains how
Fig. 5. Ticket #5127 as it really happened, beside the AI swarm's run against a simulated customer.

Outcome equivalence

A rollout will never match the recorded trajectory word for word, and a different path to an equally good outcome shouldn't count against the swarm. So what we score is outcome equivalence.

A judge sees the recorded joint trajectory for the case, the rollout's, and the wiki, and compares where they ended up: what money moved, what the customer was told, which rules were followed. It gives one of four verdicts: better, as good, worse or harmful. Outcome equivalence means better or as good, and a “better” has to cite its provenance in the wiki. Money and records are checked mechanically against the sandbox. The judge also flags any case where the swarm skipped a check the company used to make, even if the outcome came out the same.

Put together, the benchmark estimates how often the rebuilt swarm's outputs are judged at least as good as the company's , over the held-out cases. Cases share people and systems, so we compute error bars by week, not by case.

1 The judge asks how the ticket ended, in each run

The company, as recordedThe AI swarm
Did any money move?NoNo✓ same
What was the customer told?The $99 is the upgrade to annualThe $99 is the upgrade to annual, and how to switch back✓ same answer
Was the plan history checked first, as the wiki requires?Yes, by GraceYes, by the agent✓ same
How long did it take?45 min, two people6 min, one agentnot asked

2 It gives the whole case one verdict

As good The ticket ended the same way both times.

The four possible verdicts:harmfulworseas goodbetterthese two count as outcome-equivalent

Fig. 6. How the judge scores ticket #5127. It asks how the ticket ended in each run, not how it got there. The answers match, so its verdict is “as good”, which counts as outcome-equivalent.

The judge and the simulator are both models, and either could be wrong. So before they score anything, we test them on cases where we already know the answer: the real cases in the joint trajectory.

For the judge that means three kinds of test. Given a recorded case as it happened, it should grade it the way it actually went. With a key action removed, like a refund that was never issued, it should fail it. With a minor perturbation, like a reworded reply, it should still pass it.

In the first run below, on 74 test cases, the judge gave the same verdict on 73 when run twice. When we took cases it had passed and removed their key actions, it failed every one. And the minor perturbation test passed all cases as well.

The simulator gets the same treatment: given the company's real replies, it should act in a way consistent with what the real customer did in the trajectories. And these sims too can be stress tested by presenting it different perturbations of the trajectories to see how they react.

First result

We've run a first version of this at a telehealth company's support team. We wrote the wiki from earlier weeks of recording, then had an agent work later tickets it had never seen, replaying each case against what the customer actually sent, with sandboxed tools so nothing real changed. A judge scored every case for outcome equivalence, and we ran the same model again without the wiki (Fig. 7).

On 44 held-out cases, two runs each, the agent's handling was outcome-equivalent 92% of the time with the wiki and 69% without it (95% intervals 84–99% and 56–82%). Eleven cases came out right only with the wiki and two only without it, and no handling was judged unsafe.

We then compared the agent with the employees directly on 307 cases, checking every claimed mistake by hand against the recordings. The employees mishandled 10.1% of cases. The agent, with the wiki, mishandled 4.9%, mostly minor service slips.

This run worked each case on its own, against the recorded customer. It didn't use the rebuilt program or the simulated customer yet. Those come next.

agent with the wikisame model, withoutthe employeesagent with the wiki
Fig. 7. The real result, from a telehealth company's support team. Top: held-out cases the judge scored outcome-equivalent, with and without the wiki, with 95% intervals. Bottom: cases mishandled, after every claimed mistake was checked by hand.

Bringing a company to the frontier of AI is the hard part. It means reconstructing every employee's trajectory and deriving the program the human swarm runs.

But thanks to the benchmark, keeping it at the frontier is much simpler, as the program can be hill climbed against it. A frontier model proposes changes to the wiki and to the agents' setup, cheaper models run them as the swarm, and the held-out score decides which joint policy ships. Each new model is tried the same way, on the same held-out weeks.

It is in this way that real world companies can actually leverage the exponential increase in LLM capabilities and intelligence, rather than continue to play catch up.

This is what we do at Subliminal Labs: we record how your company runs, rebuild it for AI agents, and keep it at the frontier. If you want yours there, work with us.

Citation

The Subliminal team, “Every Company Is an Agent Swarm”, Subliminal Labs, October 2026.

@article{subliminal2026swarm,
  author = {The Subliminal team and Subliminal Labs},
  title  = {Every Company Is an Agent Swarm},
  year   = {2026},
  url    = {https://subliminal.inc/blog/every-company-is-an-agent-swarm}
}