SealbookAccuracy
How good is the reader? Here are the numbers.
We test the reader on labelled sets of made-up project meetings and publish the result, including the hard cases where it misses things.
First, what accuracy changes and what it doesn't
The reader only suggests. Your project lead checks every suggestion against the quote before anything goes to the client, and nothing is agreed until both sides confirm it. So the record is never "85% right": it is exactly what both sides confirmed. Reader accuracy decides how much your project lead has to add by hand.
Rule-based reader (runs offline)
Scores from 2 November 2026. "Held-out" means meetings written after the reader was built and never used to tune it; those are the numbers to trust.
| Test set | Meetings | Found (recall) | Right when it suggests (precision) | Agreed decisions invented |
|---|---|---|---|---|
| Held-out 1: ordinary phrasing | 5 | 89% | 73% | 0 |
| Held-out 2: messy, reversals, cross-talk, an injected instruction | 8 | 50% | 65% | 0 |
Where it misses: agreement in very loose words ("Mm-hm", "Okay, that's fine with us" after a long proposal), and decisions spread over several turns. It never turned one of those into a false agreement; it simply didn't see them.
Claude reader (optional, for real meetings)
The Claude reader is built for the loose, real way people speak in meetings. It is scored on the same test sets with one command, and we publish the result here as it comes out, unrounded. Scores are being run in November 2026.
The tests themselves
- Each test meeting is labelled with the decisions, actions, questions and blockers a careful reviewer would want, and which lines prove each one.
- A suggestion counts as found only if it is the right type and cites the right lines.
- "Invented" counts agreed decisions the reader produced where the label says nothing was agreed. That number matters most, and it has been zero on every set.
- The sets and the scoring script ship with the product, so a buyer or a pilot firm can rerun them.
What would convince us more
Made-up meetings are cleaner than real ones. The evidence that counts is a pilot firm labelling 20 to 30 of its own meetings and scoring the reader on them once. Every pilot includes that.