The case for automating title data extraction is now settled. A system built for recorded instruments can read instrument types, party blocks, legal descriptions and recording references from scanned courthouse images faster and more consistently than a keying team, and at a cost that makes county-scale plant builds practical again. What is not settled, and is often misunderstood, is what the automation cannot do.
It cannot tell you which of its answers to trust. That is not a temporary limitation waiting on a better model. It is structural. A model produces an answer for every field, including the fields where the record is ambiguous, damaged, or written in a convention the model has never seen. The output for a confident, correct field and the output for a confident, wrong one look the same. The only way to tell them apart is to measure the uncertainty and put a person in front of the cases where it is high.
This article sets out where automated extraction fails on title records, why those failures are invisible without human review, and how to design a review workflow that catches them without re-keying the county.
Where machines fail, specifically
General descriptions of "AI error" are not useful to a plant operator. The failures on courthouse records fall into a small number of recurring classes, and each one has a signature a well-designed system can detect.
The first is the low-confidence field in the ordinary sense: a faded stamp, a handwritten margin note, a recording reference on a torn edge, a legal description that runs across a page break. The model reads something. Its own internal uncertainty is high, and if the system exposes that uncertainty as a score, the field can be flagged.
The second is the ambiguous party role. Title records name people in capacities that change what the instrument does: trustee, successor trustee, attorney-in-fact, executor, "et al", "a/k/a", "f/k/a", a married couple named once as "and wife". The words are legible. The question is structural: who is actually conveying, who is receiving, and which name must the instrument be indexed under so a later search will find it? These cases often come back from the model with high character-level confidence and low structural confidence, and a system that only measures the first will miss them.
The third is the county-specific convention. Every recording office has its own habits: how a "see instrument" reference is written, whether a re-recorded document carries both references, how a plat is cited, what a particular clerk's abbreviation means. None of this is documented anywhere a general model could have learned it. A purpose-built extractor can be taught the conventions of a county, but the first time it meets a new one, the correct behavior is to notice it is new and ask.
The fourth is the cross-reference that is not on the page. A release that refers to the deed of trust it releases, a correction deed that refers to the deed it corrects, an assignment that refers to a mortgage recorded under an older book-and-page scheme. The model can extract the reference. Whether the reference resolves to a real instrument in the plant is a question about the plant, not the page, and resolving it is where an experienced reviewer earns their place.
Why these failures are silent
An examiner working from a plant does not re-read the image for every record. That is the point of the plant. So an extraction error that reaches the posted index is not a visible defect; it is a search that returns the wrong answer with the same confidence as a right one. A missing grantor means a chain that looks complete. A mis-typed instrument means an encumbrance filed where no search will look for it. A legal description indexed to the wrong block means a tract that appears to have no history.
Each of these is eventually discovered, usually on a live order, sometimes in a claim. By then the cost of the error has nothing to do with how cheaply it was made. This is the asymmetry that justifies human review: the error is cheap to catch at extraction time and expensive to catch anywhere later, and the review step is the only point in the process where the uncertainty is still visible.
Confidence scoring: finding the cases that need a person
The mechanism that makes targeted review possible is confidence scoring, and it is worth being precise about what it is. For every extracted field, the system records a measure of how certain it is that the value is correct. That measure needs to reflect more than how clearly the characters were read. It needs to incorporate whether the instrument type was recognized, whether the party structure matched a known pattern, whether the legal description parsed into a searchable key, and whether the references the document makes could be resolved.
A field below threshold on any of those dimensions is an exception. Exceptions are routed to a reviewer. Everything else is posted with its score attached, so that the plant records not only the value but how much it was trusted at the time.
The thresholds are a design decision, and they should be set with the plant operator, not for them. A plant that serves a commercial examiner base may set a stricter threshold on legal descriptions than one that serves residential refinance work. A county with a known difficult period of records, a water-damaged volume or a clerk with idiosyncratic handwriting, may warrant a lower threshold for that date range alone. The point of a threshold is to spend reviewer time where the data says the risk is, and the right place to spend it varies by plant.
Source-page traceability: what the review is worth
Review only has value if it leaves a record. A reviewer who corrects a party name and moves on has improved one field. A reviewer whose decision is stored against the field, with a link to the source page and a timestamp, has created evidence.
That evidence is what an underwriter's audit, a regulator's examination or a claims investigation will ask for. The Gramm-Leach-Bliley Act's safeguarding requirements and state privacy statutes such as the California Consumer Privacy Act govern how non-public personal information in these records is handled; the audit trail is what demonstrates that handling was controlled and that each value in the plant can be accounted for. It is also what makes the plant maintainable: when a systematic error is found, lineage identifies every record that shares the condition, instead of leaving the operator to find them one order at a time.
The minimum standard for a traceable field is simple to state. For any value in the plant, the operator should be able to see the image it came from, the confidence it was extracted with, whether a person reviewed it, who, when, and what they changed. If a system cannot produce that for a given field, that field has not been verified, whatever the accuracy statistics say about the batch it came from.
Designing the workflow
Putting the pieces together, a review workflow for a plant build has four parts, and they should be designed before extraction starts, not after.
Automation first. The extractor processes every document and scores every field. No document is sent to a human before the system has had its attempt, because the system's attempt is what tells you whether a human is needed.
An exception queue, not a review of everything. Fields below threshold, and whole documents the system flags as unrecognized, go to reviewers. The queue should be ordered by the cost of the error, not the order of recording: a legal description that failed to parse is more urgent than a low-confidence recording date.
Reviewer workflows with a second level. Most exceptions are resolved by a trained reviewer with the image beside the field. A minority, typically the cross-reference and party-structure cases, need someone with examination experience. Routing those to the right person is part of the design.
Feedback into the rules. Each reviewer decision is a labeled example. A convention that was an exception in one volume should be recognized in the next. The queue should shrink as the build proceeds, and if it does not, that is a signal about the extractor, not the reviewers.
How Quantius Leo does it
Quantius Leo is built around this workflow rather than adding review to it afterwards. Extraction is purpose-built for recorded instruments and scores every field across the dimensions described above, not only character confidence. Fields below threshold, and fields the system identifies as structurally ambiguous, are routed to human reviewers before anything is posted. Every field, reviewed or not, links to its source page and carries its score and its review history. Thresholds are agreed with each plant operator, and reviewer decisions are fed back so the extractor learns each county's conventions as the build progresses.
The output is delivered in the client's own schema as the client's own data, with the lineage included.
Validating it on your own county
None of this should be taken on description. The right way to evaluate an extraction and review workflow is on documents from your own county, compared against your own plant.
A pilot is the usual first step: send a sample of recorded instruments and receive them back indexed, confidence-scored and traceable. Review the output with attention to the scores. Where the system reported low confidence, check whether it was right to; where it reported high confidence, check that it was. Compare the posted fields against your existing index and note where they differ, because those differences are where either your plant or the extractor has something to learn.
A technical walkthrough with your examination and plant staff follows naturally from that. The aim is for the people who will rely on the data to understand how the thresholds were set, what the queue looks like, and how a field's history is read, so that the workflow is theirs rather than ours.