Specialize the context, not the harness.

This is part two of a three-part series on how we're building Cooper, an AI coworker for insurance. Part one argued that the models are already smart enough; the hard part is getting them the right context. This post explains why we replaced predefined workflows with a general-purpose agent, and the five subsystems that make it work. Part three will cover how Cooper learns from production work and proposes improvements that our engineers review, revise or reject.

The agentic harness at the center of Cooper knows nothing about insurance. It spends its day doing insurance work, but we don't bake insurance expertise or workflows into its core. Everything it needs to know about the job is injected into its context.

The underlying models have encountered insurance, of course. But they don't know how a particular office prepares a proposal, why a specific client needs an extra review or which version of a report a colleague actually meant. That knowledge has to come from somewhere. A procedure, a remembered correction, a template and a client's preferences are all knowledge, whether we happen to store them as skills, memories, notes or reference material.

In the first post, I argued that getting this knowledge in front of the model is the hard part. This post is about why we built a general-purpose agent to use it, rather than an increasingly specialized insurance system. We actually tried the specialized system first.

We were building software again

Our first approach made sense to us at the time. We understood a set of insurance workflows, so we broke them into steps and put AI into each step: an agent extracted information from documents, another matched a submission to markets, and other steps filled forms or generated emails. Some calls returned structured data, others made decisions or called tools, and deterministic code kept everything moving through the process.

It worked, which was part of the problem. There wasn't one obvious failure telling us to throw the approach away, because we could keep improving the individual steps and each improvement would make that particular workflow better, even as the limits of the overall approach were becoming harder to ignore. The workflow still decided what could happen, and every organization had a slightly different idea of what should happen.

We were asking someone to specify the process ahead of time, when the whole difficulty was that real work kept producing situations nobody had specified.

Those differences weren't just preferences about where a button should go. They changed what information was needed, who had to review it, what counted as complete and what you should do when something was missing. We could build a new version for each organization. Or we could pull all those differences into configuration, which eventually meant building a language complicated enough to describe every way an insurance organization might work.

Neither option was appealing. We were building software again, with the same limitation the industry had been hiring people to work around for years. We were asking someone to specify the process ahead of time, when the whole difficulty was that real work kept producing situations nobody had specified.

Adding more intelligence to the individual steps wasn't going to fix that.

The person was the blueprint

So we took a step back. Insurance companies already had plenty of software, and yet a person remained the most flexible "technology" available to them. We wanted to understand what we were missing.

Someone can join an insurance organization without any insurance background and still become productive very quickly. They already know how to read documents, use a computer and communicate with other people, and they can learn a procedure they've never seen before, ask when something doesn't make sense and remember the answer without needing the same explanation every morning. These are general capabilities. The organization supplies the knowledge that makes them productive in this particular job.

Some of that knowledge is formal: regulatory requirements, report formats, the steps for preparing a submission. Some are much less formal, like the fact that a certain client expects a phone call before an email, or that the template in one folder is the one everyone uses even though another file says "latest." A new hire gradually picks up both, and can apply them in combinations nobody wrote a procedure for.

That's the flexibility we wanted. The person doesn't become a different employee when you ask them to do something new. They take what they already know, learn what's missing and use the same tools to work it out.

This became our North Star for the product: interacting with Cooper should feel like working with another person on your team. It also became a practical way to make engineering decisions. You should be able to correct Cooper while it works, rather than wait for a designated step. If it needs more information, it should ask and pick up where it left off. The next report should take less explaining once Cooper has learned how you want it prepared.

Interacting with Cooper should feel like working with another person on your team.

We retired the earlier workflow engines and built a general-purpose agent from scratch. There was a lot to build, but a handful of core subsystems had to be right for this product direction to work. Cooper needed somewhere to work, a way to read and use existing software, and a way to retain and apply knowledge. A weakness in any one of those would limit what Cooper could do.

We're an engineering team of just eight people building across problems that support entire companies elsewhere: agent infrastructure, document understanding, browser use, memory and skills. We use providers whose work we value, and we've built the systems around them that make Cooper useful. The challenge is getting all of these capabilities to work together well enough that someone can hand Cooper a job, rather than operate a collection of AI tools.

A computer of its own

The first thing we had to give Cooper was room to work. If it can only perform operations we've anticipated and exposed as individual tools, we'll eventually find ourselves building another tool for every variation of every task.

A person doesn't need a purpose-built application to reconcile two spreadsheets. They can open them, inspect the data and figure out what to do. An agent should have similar room to maneuver, including the ability to write and run a small program when that's the sensible way to solve the problem. But letting generated code run means accepting that it may do something we didn't intend. We have to design for that before asking customers to trust it with their work.

E2B gives Cooper a complete, isolated computer where the entire agent loop runs, rather than just sandboxing individual tool calls. Its pause-and-resume support preserves files and running processes, so Cooper can wait for someone to answer a question without losing its working environment. A coworker shouldn't have to clear their desk every time they need to ask you something. Cooper's working environment is separate from the trusted services that own identity, permissions and durable records.

Provider credentials stay behind a gateway. The agent can make an authenticated request to an authorized service without having that service's API key sitting in its environment. Its workspace storage uses temporary, narrowly scoped credentials. The agent decides how to do the work. The platform decides what it can access.

The other half of this design is continuity. Someone might need to answer a question, take over a browser or come back later with a missing document, and none of that fits particularly well into a system that treats the end of a model call as the end of the work. Orchestration tracks execution outside the agent process, and files persist in the workspace. The lifetime of the work shouldn't depend on keeping one process alive forever.

The agent decides how to do the work. The platform decides what it can access.

That's the environment. What fills it, most of the time, is a collection of files somebody expects Cooper to understand.

Reading what is actually in the document

This sounds like it ought to be the easy part by now. Models can read PDFs. But a document arriving at an insurance office doesn't promise to be a well-behaved PDF, or even the kind of file its extension says it is.

We built the Insurance Agent Benchmark to test how well agents handle actual insurance work, starting with documents. We ran 17 models across the same 166 cases, both with our document system and as direct model calls. In those runs, every model scored higher with Cooper. The biggest gains came on files the models struggled to ingest on their own: long documents, oversized workbooks and damaged inputs. Choosing a capable model was only part of the job. We also had to get the information in front of it.

Google's Gemini Flash stood out in our benchmark because it combined competitive document accuracy with low cost, rather than forcing us to choose between the two. We use it for the parts of document understanding that need a model, including interpreting visual information that a text-only reader would miss. We still use local readers where a model call isn't needed. A machine-readable page can often be extracted directly. A scan needs a different path. An Outlook email can contain an entire packet of attachments, and a workbook can spread the relevant information across dozens of sheets. The agent needs a useful way to read all of these without having to become an expert in their file formats.

Some dynamic PDF forms show nothing but Adobe's "Please wait" message in ordinary readers. Our pipeline mistook that placeholder for an unreadable document, even though the completed answers were stored inside as structured data. We needed a different reader, not a better vision model looking at the same placeholder.

Long documents present a less exotic version of the problem. Dumping everything into context is expensive and can make the relevant detail harder to find. Our document system breaks large files into manageable pieces and supports targeted reading, while preserving the connection between extracted information and its source. That lets Cooper cite the document and, where available, the page behind an answer, so someone checking its work can go back to the evidence rather than search the whole file again.

Our document system breaks large files into manageable pieces and supports targeted reading, while preserving the connection between extracted information and its source.

There's a trust requirement underneath all of this. If a field is blank, the correct answer may be that the information isn't there. If a page couldn't be read, we need to distinguish that from a page that contained nothing relevant. A plausible answer with a page number attached can still be wrong; the citation gives someone a way to check it.

And after reading the material, Cooper often has to leave the files behind and do something in somebody else's software.

Working through the browser

That is where a lot of insurance work still lives. A carrier portal might be the place to retrieve a document, enter the information you've just assembled or find out why a submission won't go through. Waiting for every one of these systems to offer the right API would rule out too much of the work people already do.

Running remote browsers at scale is an infrastructure problem of its own, and we'd rather build on Kernel's work than operate that infrastructure ourselves. Kernel provides the cloud browsers and live view; we spend our engineering effort on coordinating the work inside them, so Cooper and a person can share a session, hand over control and pick up where the other left off. The interesting problem starts after the browser can click a button. It has to keep working when a page behaves differently, a connection drops or somebody needs to step in.

Some systems allow only one signed-in session per user. If two workers try to help in parallel, the second login can kick the first one out. So adding more workers can actually make the task less reliable. We coordinate access to those shared sessions and account for human takeover as well. A worker waiting for someone to finish signing in can look inactive, but that doesn't mean another worker should take its browser away.

We also don't ask a model to rediscover every familiar interaction. Skills supply deterministic scripts for known paths, which the agent executes. That avoids spending a model call on every field and makes familiar operations faster and more predictable.

The important difference from brittle automation is that the agent can work on the script as well as run it. When a website changes and a supported script breaks, Cooper can inspect what happened, patch the script and re-run the failed step to check the repair. It doesn't have to wait for an engineer to update every changed selector. If the repair fails, Cooper should stop or ask for help rather than keep trying blindly.

When a website changes and a supported script breaks, Cooper can inspect what happened, patch the script and re-run the failed step to check the repair.

That gives us the speed and repeatability of a script without making every website change a permanent break. The agent can repair known operations as it goes, and those operations still don't dictate the shape of the larger task. This is why we can use deterministic code without giving up the flexibility we wanted in the first place.

Learning how you work

Suppose Cooper gets through that task, but you have to correct how it prepares the final report. It applies the correction and gives you the right result. Then, the next time you ask for the same kind of report, it makes the same mistake.

You would find that irritating in a colleague, and reasonably so. The task got done, but the explanation was part of the work too. Having to repeat it means you haven't really finished teaching them.

That is what our memory layer is for. A correction is knowledge Cooper should carry into the next task. Turbopuffer lets us search remembered knowledge by both meaning and specific words, combining vector search with BM25 in the same system. That gives us the retrieval foundation while we concentrate on what makes memory useful: deciding what to retain, reconciling corrections and bringing the right knowledge into the next task. Remembered facts are scoped to a user within their organization, rather than automatically becoming knowledge everyone else can access.

A correction is knowledge Cooper should carry into the next task.

The foreground agent's memory tool is read-only. A separate server-side process handles extraction and reconciliation after completed turns. We don't want every tentative answer or passing detail to become a durable belief just because it appeared in a conversation.

Even then, memory has ordinary database problems that are easy to overlook when you're thinking about intelligence. Two conversations can update the same fact, and an older write can finish after a newer one, which means the last thing the database receives isn't necessarily the latest thing the user told us. We retain source lineage and guard against stale updates, then reconsider a conflicting change against the current state. Otherwise the system can successfully learn a correction and subsequently lose it because another task finished later.

Remembering what happened is one way Cooper acquires knowledge. But you shouldn't have to make it learn everything through corrections. Much of what a person needs on their first day can be taught before they start.

Teaching Cooper the job

A skill is a way to package that knowledge so the agent can use it. It contains procedural instructions and the resources needed to follow them, which might include templates, reference material or scripts. Some describe shared insurance practices. Others describe the way a particular organization wants something done.

For a proposal, that might include the required sections, the agency's preferred layout and the checks to make before sending it. For a portal, it might explain a sequence of operations and the exceptions the on-screen instructions don't tell you about. This is knowledge a capable employee would need too. We load it into the agent's context when the work calls for it.

That brings us back to the opening claim. The general-purpose agent hasn't become an insurance-specific engine. It has been given the knowledge to do an insurance job. A remembered preference and an authored procedure may have different histories, but both inform the work through context. Neither needs to become a branch in the agent's core code.

The distinction changes how we build the product. Insurance experts can contribute procedures and corrections, with validation, previews and review before publication. When a task can use the capabilities already available, teaching it doesn't require a backend release. We still have to build missing capabilities, and we still have to verify that the teaching works. But we no longer need a new software feature just because another organization follows a different procedure.

A remembered preference and an authored procedure may have different histories, but both inform the work through context. Neither needs to become a branch in the agent's core code.

Our integrations are skills too. Cooper works with agency management systems such as Applied Epic and AMS360, HubSpot/Salesforce for CRM, and Microsoft 365 through Outlook, OneDrive, SharePoint and calendars. The platform handles authorized access; knowledge of how to operate each system lives in skills, using APIs where available and a browser where needed.

That matters when the tools insurance organizations use number in the tens or hundreds. We don't have to encode every operation and every customer's variation in a custom backend connector. One agency's way of filing documents or updating an account can live in its own skill, which we can change without altering how another agency uses the same system. Access and authentication still need engineering, but teaching Cooper a business's way of using a tool doesn't mean maintaining a separate fork of the integration.

A request might therefore involve finding a file in SharePoint, checking information from an email, applying the agency's reporting convention and saving the finished document to its system of record after the appropriate confirmation. To the person asking, that's one job. They shouldn't have to assemble a sequence of integrations themselves.

We started out thinking about submissions, proposals and certificates. The skill library now also covers medical timelines, claims correspondence and responses to regulatory complaints. Each requires different knowledge and careful verification, but the core agent doesn't need to be rewritten for each one. That is the part of this architecture I find most exciting: the range of useful work isn't limited to the workflows we knew to build at the beginning.

The same person, through a different channel

Once you take the coworker idea seriously, insisting on a single place to interact with Cooper starts to feel like an unnecessary constraint. You email a colleague when you're forwarding a packet, message them when you have a quick question and call when talking is easier. They're still the same colleague.

You can reach Cooper through the web app, email and Microsoft Teams/Slack. Those surfaces use the same underlying agent, with the presentation and available interactions adapted to the channel. Some work needs a richer interface than a message thread can provide, particularly when a person has to take over. That doesn't require us to build another insurance assistant for each surface.

Phone calls are next. The direction is to make Cooper reachable where the work happens, using the knowledge it's authorized to use, instead of asking people to reorganize their day around our application.

Getting better at the job

The encouraging thing about the adoption we're seeing is that this direction gives us room to keep going. We don't have to know every task an insurance organization might eventually ask Cooper to do before we can build something useful for it. We need general capabilities that hold up, a way to supply the right knowledge and a way to notice when that knowledge needs to change.

That last part is the subject of the next post. Scheduled agents already review production conversations and propose skill improvements as pull requests. Our engineers review them, merge some and reject others. A rejection or a request for revision becomes information for the next proposal. The system proposing improvements has to learn from being corrected too.

Scheduled agents already review production conversations and propose skill improvements as pull requests. Our engineers review them, merge some and reject others.

Part three will cover that feedback loop across skills, the agent and its surrounding system, along with the work of keeping learned knowledge useful. Memory needs reconciliation and cleanup. There is something appealing about the analogy to sleep: doing the work is one part of learning, and organizing what you learned afterward is another. We'll explain what that means in our system, without pretending a growing pile of memories automatically makes an agent better.

These are the problems we're spending our time on. If they sound like the kind of work you'd enjoy, we're hiring.