legal-research

How it is built

Architecture overview

A legal research platform lets a lawyer find the decisions and statutes that govern a question, check that they are still good law, and increasingly draft text that cites them. Its value sits in a layer a search engine lacks: a citator graph. Every decision is parsed for the citations it makes, each citation is labelled with how the citing court treated the cited one (followed, distinguished, criticized, overruled), and the labels are combined with a court hierarchy that says which court binds which. A hit is therefore never just "relevant"; it is relevant, binding or only persuasive for this court, and still good law or not. Around it sit ingest, an editorial layer of headnotes, point-in-time storage for statutes, a lexical search stack with the terms-and-connectors syntax lawyers rely on, and an AI drafting layer that may only say what retrieved passages support. The one idea: the text is searched, but the graph decides what to trust. Closed examples are Westlaw (with CoCounsel), Lexis+ AI, vLex and Bloomberg Law; the open reference is CourtListener with the Free Law Project tools.

Parts:

• Ingest — dockets, opinion PDFs and statute feeds become clean text with positions — PDF text extraction, Optical character recognition (text recognition), Layout analysis (no page yet)
• Citation and entity layer — find every citation, judge, court and statute in the text — Linear-time regular expression matching, Aho-Corasick multi-pattern matching, Named entity recognition, Entity linking
• Citator — typed, dated citation edges plus the court hierarchy — Relation extraction (no page yet), Text classification, Triple store indexing (no page yet), Transitive closure index (no page yet)
• Editorial taxonomy — topic tree and headnotes assigned to passages — Text classification, embedding-inference (no page yet)
• Law over time — which version of a statute, and which treatment, was in force on a date — Bitemporal data (no page yet), Point-in-time join (no page yet), Point-in-time recovery
• Search — terms-and-connectors, natural language and ranking — Inverted index, BM25 ranking, Finite-state transducer (no page yet), Stemming (no page yet), Hybrid search and rank fusion, Learning to rank, Cross-encoder reranking
• Alerts — new decisions pushed to the lawyers who follow them — Boolean expression indexing (no page yet), Realtime query subscriptions (no page yet)
• Grounded drafting — generated text tied to sources and checked against the citator — Citation grounding (no page yet), LLM-as-judge
• Operations — keep a regulated, always-on service alive and testable — Primary-backup replication, Deterministic simulation testing, Coverage-guided fuzzing, Mutual TLS, Graceful draining, Process supervision, HDR histogram (no page yet)

 court feeds, dockets, statute feeds
        │ cursors
        ▼
 ingest ─► text + offsets ─► citation/entity layer ─► citator graph ◄── editors
 (PDF/OCR)      │            (regex, Aho-Corasick,     (typed edges,
                │             NER, linking)             court tree)
                ▼                                           │
        inverted index ◄── topics ◄── taxonomy       bitemporal store
        (positions, FST)                             (statutes, treatments)
                └────► search: connectors ► BM25 ► LTR ► rerank ◄─┘
                              │            (authority, hierarchy, as-of date)
          alerts ◄────────────┴────► drafting: retrieve ► generate ► ground ► citator check

There are two workloads: a write path per new document (minutes are fine, correctness is not negotiable) and an interactive read path that returns results, ranking and good-law flags together.

How the parts fit together

Ingest

Courts publish PDFs, HTML and bulk feeds in inconsistent forms. Born-digital PDFs go through PDF text extraction; pages without a usable text layer (older scans, stamped filings) go to OCR. Layout analysis (no page yet) separates body text from footnotes, running headers and tables: a header repeated on every page corrupts proximity searches. Docket ingest is a polling problem: each court source keeps an incremental sync cursor, and the cursor is saved only after the document is stored.

Contract with the next part: one record per document with its court id, decision date, a docket or reporter identifier, and text plus character offsets that never change afterwards. Citations, headnotes and grounding all store spans against these offsets, so a re-extraction that shifts them creates a new document version.

Citation and entity layer

Legal text is full of rigid-grammar strings: reporter citations, short forms, "id." and "supra", statute sections. The open tool eyecite finds them with a reporter pattern database and an Aho-Corasick prefilter (source: eyecite README). The same split fits here: a cheap multi-pattern pass (Aho-Corasick) finds the few places where any reporter abbreviation occurs, then a linear-time regex engine runs the full patterns only there. Linear-time matching matters because filings are untrusted text and a backtracking engine turns a crafted one into a denial of service.

Judges, courts, parties and agencies are found by Named entity recognition and resolved by Entity linking against a knowledge base of courts and judges (alias table, then context score, then NIL for unknown names). "The Ninth Circuit" and "9th Cir." must land on one court id, because the hierarchy and the citator key on it. A wrong link is worse than no link: threshold, and send doubtful ones to editors.

Citator

The citator turns citation spans into edges: for each (citing, cited) pair, the span, a treatment label and a confidence. Relation extraction (no page yet) is the task: the sentence around the citation says "we decline to follow" or "overruled on other grounds", and a text classifier over that window assigns the label. Climb only as far as the data demands: patterns for explicit phrases ("overruled by"), a fine-tuned encoder for the rest, an LLM prompt for the hard tail, and an editor queue for anything that would raise a negative flag.

Two stores back it. Triple store indexing (no page yet) holds (subject, relation, object) facts with permutation indexes: case cites case with label X, court A appeals to court B, case belongs to topic T. Triples carry the same provenance fields on every fact (source, model version, editor id). Transitive closure index (no page yet) answers the reachability questions that otherwise need a graph walk per query: "is court A above court B?" and "does any negative subsequent history lie on the appeal chain of this decision?". The court tree is small and nearly static, so interval labelling makes each test two integer comparisons.

The rule that combines them: a treatment counts against a decision only if the treating court is above or equal to the decision's court, or is an appellate disposition of this very case. A district court "criticizing" a Supreme Court decision does not flag it. That is why the hierarchy is data, not code.

Editorial taxonomy

Headnotes state one legal point decided in a case, filed under a topic tree. Editors write them; the model triages. Vectors from a pretrained embedding model (embedding-inference (no page yet), run at serving time, no training of its own) for passages and topic descriptions shortlist candidates, a classifier scores them, and the candidate-score-NIL pattern of entity linking attaches the passage to a taxonomy node. The output feeds facets, "same point of law" grouping and the topic feature in ranking. Low-confidence proposals go to an editor, whose answers become training data for the next classifier.

Law over time

Statutes change; decisions apply the version in force when the facts happened. The store is Bitemporal data (no page yet): every row has valid time (when the text was law) and transaction time (when the platform learned it). A section amended in March with the amendment commencing in July is one row with a valid-from in July recorded in March; the UK legislation service exposes the same idea as enacted, dated and prospective versions of one provision (source: legislation.gov.uk developer docs). A citator label corrected by an editor in October keeps its old row, so "what did the platform show the day the brief was filed" stays answerable. Legislative XML such as Akoma Ntoso (an OASIS standard) gives sections stable identifiers for the rows to hang on (source: OASIS spec).

The Point-in-time join (no page yet) ties a question to a version: for each event date (facts, filing), pick the latest valid version at or before it. Point-in-time recovery is a different thing, restoring the database to an instant, and belongs to Operations.

Search

Lawyers search two ways. Terms and connectors is an exact Boolean language with proximity: root expanders and wildcards, "within n words", "same sentence", "same paragraph". It is predictable (the same query returns the same set) and cannot be replaced by embeddings. Natural language is the ranked path.

Both run on one Inverted index that stores positions. Sentence and paragraph connectors need boundary markers: the indexer inserts a boundary token with a position gap, so "same sentence" becomes a window test on positions. The splitter must know legal abbreviations ("v.", "Cir.") or it cuts citations in half. Root expansion ("negligen!") and wildcards use the term dictionary: a Finite-state transducer (no page yet) keeps the sorted terms compact and enumerates prefixes quickly, and a regex engine handles the general wildcard case over the enumerated terms. Stemming (no page yet) is applied for the natural-language path but never silently to a connectors query.

Ranking has stages, cheap to expensive. BM25 scores the lexical candidates. For natural-language queries a vector list from pretrained-model vectors (embedding-inference (no page yet)) is merged by Hybrid search and rank fusion, so a query also finds decisions that never use its words. A Learning-to-rank model then orders the top few hundred with features only a legal platform has: court level and whether it binds the user's jurisdiction, treatment-weighted in-links, recency, headnote topic match, and the decision's flag. Finally a cross-encoder reads the query with each of the top few dozen passages. Ranking may demote a decision; it may never hide the citator's warning.

Alerts

A lawyer follows a topic or saved query and expects a note when a new decision matches. That is the inverse of search: many stored queries, one document. Boolean expression indexing (no page yet) indexes the queries as conjunctions of conditions (court in a set, date at least d, term present) so that one document looks up only queries whose conditions it can satisfy; the search-engine route is a stored-query ("percolator") field, which Elasticsearch offers (source: Elasticsearch reference). Realtime query subscriptions (no page yet) pushes result-set diffs to an open session instead of polling. Alerts are deduplicated per decision, since an opinion is often published twice (slip opinion, then reporter).

Grounded drafting

The generation layer must not be the source of law. Retrieval comes first (filtered by the matter's court and date), generation second, grounding last. Citation grounding (no page yet) splits the draft into claims, attaches each to the passages it relies on, scores whether the passage supports the claim (a classifier or a cross-encoder used as an entailment check), and removes or flags failures. Then two checks only this system can make: every cited decision must exist in the platform (a lookup, not a model call; CourtListener's citation lookup API is described as a guardrail against hallucinated citations, source: Free Law Project wiki), and its citator status as of the matter's date must not be negative.

A preregistered study of commercial RAG legal research tools found that Lexis+ AI and Westlaw's AI tools each hallucinated between 17 % and 33 % of the time (source: Magesh et al. 2024); general-purpose models asked verifiable questions about federal cases hallucinated between 58 % and 88 % of the time (source: Dahl et al. 2024). Quality is measured on a gold set of research questions with an LLM judge calibrated against lawyer labels; the judge never replaces the citator check. LegalBench is a public benchmark of legal reasoning (source: Guha et al. 2023).

Operations

Primary-backup replication keeps one writer for the citator and statute stores, because edits must be ordered. Parsers read hostile input, so coverage-guided fuzzing targets the PDF reader, citation parser and query parser; deterministic simulation testing exercises the ingest cursor and alert delivery under crashes. Services authenticate with Mutual TLS, deploy with graceful draining, restart under process supervision, and record latency percentiles in HDR histogram (no page yet).

Data and control flow

A new opinion arrives (write path):

1. The docket poller reads entries since its cursor and downloads the PDF.
2. Text extraction runs, OCR for pages without a text layer; text and offsets are stored as version 1.
3. The citation pass runs (Aho-Corasick prefilter, then regex); recognition and linking assign court, judges, statutes.
4. Each cited decision becomes an edge; the classifier labels its treatment; negative labels go to the editor queue, the rest are written with a valid-from of the decision date.
5. The topic classifier proposes headnotes; editors accept or correct.
6. The document is indexed with positions; the alert matcher runs the stored queries against it. Time goes mostly to OCR and editor review.

A terms-and-connectors search (read path):

1. The parser builds a tree of terms, expanders and proximity operators; syntax errors are reported with a position.
2. Root and wildcard terms expand through the term dictionary into a bounded set of concrete terms.
3. Filters (court, date, jurisdiction) run first because they are cheap; position lists are then intersected under the window rules.
4. Matches are scored (BM25 plus authority features), cut to the top set, and each gets its citator status as of the chosen date.

A drafted paragraph (AI path): the matter's court and date filter retrieval; the model drafts quoting passage ids; grounding scores each claim against its passages and removes or marks failures; each cited decision is looked up and its status checked; the lawyer sees draft, sources and flags together.

Scaling

In the order you would pull the levers:

1. Editor throughput limits quality first, because negative labels need a person. Lever: confidence thresholds that route only doubtful cases, and a classifier retrained on every correction.
2. Index size and query latency. Positions make the index larger. Lever: shard by court and year (queries nearly always filter on both), keep the term dictionary in memory, run cheap filters first.
3. Ranking cost. Cross-encoders cost one model call per pair. Lever: cut candidates hard first, cache by query, rerank passages rather than full opinions.
4. Ingest backlog. OCR dominates. Lever: parallelize per document, prefer born-digital sources.
5. Alert fan-out. Lever: the expression index, so cost follows matching alerts, not all alerts.
6. Citator reads. Interval labels and a cached good-law flag per decision; recompute when a negative edge arrives.

Failure modes

Real implementations

The alternative: plain full-text search over a document dump with an LLM on top. It is far cheaper and good for orientation, but without the citator and the hierarchy it cannot say whether a decision is binding or still good law, which is where the studies below locate the risk.

Sources

• V. Magesh et al., Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Tools, 2024. paper
• M. Dahl et al., Large Legal Fictions: Profiling Legal Hallucinations in Large Language Models, 2024. paper
• N. Guha et al., LegalBench: A Collaboratively Built Benchmark for Measuring Legal Reasoning in Large Language Models, 2023. paper
• J. Cushman, M. Dahl, M. Lissner, eyecite: A Tool for Parsing Legal Citations, Journal of Open Source Software, 2021. paper
• eyecite README (tokenizers): source
• Free Law Project, Citation Lookup and Verification API: docs
• legislation.gov.uk developer docs, URIs and versions: docs
• Elasticsearch reference, percolator field type: docs
• OASIS Akoma Ntoso (LegalDocML) vocabulary, the XML standard for legislative documents: spec