The situation
The client is a commercial law firm of roughly 60 attorneys whose corporate practice group handles a high volume of routine contract review for its business clients: NDAs, master service agreements, and standard vendor contracts, the kind of document a first- or second-year associate reviews dozens of times a month. The partner running the group estimated that associates were spending 15-20% of their billable time on a first pass through these routine documents: reading the whole thing, checking it against the firm's standard negotiating positions, and marking up deviations before a senior associate or partner ever looked at it.
None of this work was actually hard, in the sense that experienced associates could do it quickly and correctly. It was expensive because it was billed at associate rates for work that, once the firm's own negotiating positions were made explicit, was largely a matter of comparing a document against a known standard, the kind of first-pass triage that didn't need a lawyer's full judgment applied to every clause, only to the clauses that deviated from the standard.
Why off-the-shelf didn't fit
Contract-review AI tools are one of the more crowded categories in legal tech, and the firm had already licensed one before calling us. It worked reasonably well as a generic clause-flagging tool. It would highlight an indemnification clause, a limitation-of-liability clause or an unusual termination provision, but it flagged everything against a generic, vendor-maintained model of "market standard" language rather than this specific firm's own negotiating positions, which the partners had refined over years and which differed in specific, deliberate ways from generic market norms (a more associate-favorable insurance requirement here, a firm-specific position on data breach notification timelines there). The result was a tool that generated a long list of flags, many of which weren't actually deviations from this firm's playbook at all, just from some generic industry baseline, which meant associates still had to read the whole document carefully to figure out which flags actually mattered, largely defeating the purpose.
Scoping the real workflow
The thing that mattered most during scoping wasn't about the technology at all. It was getting two partners in the same room to actually write down the firm's negotiating playbook in specific, comparable terms, which, it turned out, had never been fully documented in one place. Different partners had informally different tolerances on a few clauses, and the scope call surfaced a real, unresolved internal disagreement about the firm's standard position on limitation-of-liability caps that had to be settled by the partners, not by us, before we could build anything. We treated that resolution as a prerequisite deliverable of week one, not a side conversation, because building an agent against an undocumented or internally inconsistent standard would have produced a tool nobody trusted.
Once the playbook was documented, roughly 40 specific clause categories each with the firm's preferred position, an acceptable fallback position and a "must escalate to partner" threshold, the actual technical scope became clear: an agent that reads an incoming document, maps its clauses against the 40 categories, and produces a structured markup showing where the document matches the firm's standard, where it falls within an acceptable fallback, and where it requires partner attention.
What we built
The agent ingests an incoming contract (via the firm's existing document management system, triggered when a document is tagged for review), identifies and extracts the relevant clauses against the 40-category playbook, and produces a structured review memo: for each category, whether the document's language matches the firm's standard position, falls within an acceptable fallback, or requires escalation, with the specific problematic language quoted directly and a suggested redline drafted in the firm's own preferred phrasing pulled from the playbook itself, rather than generic alternative language. The memo is attached to the document in the DMS before an associate opens it, so the associate's actual review time is spent confirming the agent's categorization and drafting negotiation strategy on the flagged items, not reading the entire document cold.
Every categorization is logged with the specific clause language and playbook rule that produced it, both so an associate can quickly sanity-check the agent's read and so the firm has a clean audit trail if a client or opposing counsel later disputes how a document was characterized internally.
Where it got hard
Clause identification across genuinely varied document formatting was harder than the scope call suggested it would be. Law firms don't just review their own template. They review whatever the counterparty sends, and counterparty documents varied enormously in structure, numbering conventions, and where in the document a given topic (say, indemnification) actually appeared. Our first version, tuned against the firm's own template, performed noticeably worse against counterparty-drafted documents with unconventional structures. We rebuilt clause identification to work off semantic content rather than positional or formatting cues, and validated it against a deliberately messy sample set of 30 real counterparty documents the firm had on file from the past year, several of which were poorly OCR'd scans of documents that had clearly been through several rounds of fax and photocopying before reaching a digital form.
The escalation threshold for limitation-of-liability caps, the exact clause the partners had disagreed about during scoping, needed a second round of partner review after the agent had been running in shadow mode for a week, because the initial written threshold turned out to be ambiguous in a way that only became apparent once real documents were run against it. That's a good example of a scope document doing its job: the ambiguity surfaced early, in week three, rather than as a disputed markup six months into use.
Rollout & results
The firm's own billing data over the two months following launch showed a 68% reduction in associate hours billed to first-pass contract review, with associate time redirected toward negotiation strategy and client counseling on the items the agent actually flagged for escalation, work the partners consider a better use of a junior associate's development time than mechanical first-pass reading. Partner review load on routine documents dropped as well, since the memo format meant partners were reviewing a structured summary rather than a full markup on documents that mostly matched the firm's standard positions anyway.
"The real deliverable in week one wasn't code. It was finally getting two partners to agree, in writing, on something they'd been quietly disagreeing about for years. Everything downstream of that was comparatively easy." Managing Partner, Corporate Practice Group, client engagement
What we'd do differently
We'd budget explicit time in the scope phase for "playbook consolidation" as its own line item on any legal engagement, rather than treating it as part of general scoping. It's a distinct kind of work (getting stakeholders to agree on and document a standard that previously existed only informally) with its own timeline risk, separate from the engineering timeline, and conflating the two nearly led us to underestimate the calendar time this specific engagement needed.