Tutorial ID | T-21 |
Section | AI & Analysis |
Title | Understanding and Using the Validation Score |
Subtitle | Use the Validation Score to prioritize review, not to decide whether a report is finished. |
Before You Begin
You have run a workflow and generated AI Assistant's Validation Analysis for a report.
You consider the source documents for this case complete, or you know which records are still outstanding.
What You Will Accomplish
By the end of this tutorial, you will know what the Validation Score does and doesn't measure, and how to read it alongside the Statement-by-Statement Review and Overall Assessment. You will be able to prioritize your review time using the score instead of reviewing every statement to the same depth.
Why Does this Tutorial Matter
The Validation Score is a triage tool — it tells you where to look first, not whether a report is finished. Used well, it turns an overwhelming review into a structured one, pointing you to the weakest links first so your attention goes where the report actually needs it.
Used poorly, as a pass-or-fail stamp, it creates false confidence. A high score only means a statement closely matches its source document — it says nothing about whether that document is reliable, whether the statement is relevant, or whether a document is missing from the case altogether.
Info A low score usually points to one of a few common causes: ambiguous or contradictory source text; a handwritten or poorly scanned document; sparse evidence; content that goes beyond what the record actually says; or two documents linked together incorrectly. |
Steps
Open the Overall Assessment and read its Key Issues before looking at the Validation Score.
Open the Statement-by-Statement Review and identify the specific statements pulling the score down.
Sort or filter statements by Validation Score and start your review with the lowest-scoring ones.
Verify each low-scoring statement in full against its linked source document.
Spot-check high-scoring statements rather than skipping them — a high score confirms a match to the source, not that the statement belongs in the report.
Confirm the case's document set is complete before trusting any score, since a score cannot account for a document that was never uploaded.
Correct, verify, or remove every statement that still sits in the low band — do not leave one unresolved before finalizing the report.
After making corrections, rerun the Validation Analysis so the score and Key Issues reflect the current version of the report.
Agree with your team on the minimum score required for inclusion, and on which statements must be verified manually regardless of score.
Tip Never finalize a report while a statement still sits in the low band unaddressed — verify and keep it, correct it, or remove it. |
What to Do if Something Goes Wrong
The list below covers the most common problems teams run into with the Validation Score. Each entry follows the same pattern: what you'll notice, why it likely happened, a real example, and how to fix it.
Problem: A high score results in a statement being accepted without further review.
Likely cause: A high score feels reassuring, so well-scored statements get skipped — even though the score only confirms AI Assistant read the document faithfully, not that the statement is correct or relevant.
Example: A statement scores 10/10 because it matches its source document exactly, but the source document itself contains an outdated diagnosis. The error carries into the report unnoticed.
Fix:
Treat a high score as confirmation that the AI read the document faithfully, not as confirmation that the statement is correct or belongs in the report.
For high-scoring statements, confirm the underlying document is reliable and that the statement is relevant to the case.
Reserve your deepest scrutiny for statements that carry the most weight in the case, regardless of their score.
Problem: A low score is dismissed as noise instead of being reviewed.
Likely cause: A low score is assumed to be a technical artifact — caused by a poor scan or awkward wording — rather than a genuine problem, so it never gets reviewed.
Example: A statement is flagged Partially supported. The reviewer assumes it's just a scanning issue with a handwritten note and moves on, missing that the statement actually misstates the treatment date.
Fix:
Open every low-scoring statement and read it against its linked source.
Determine the cause — a misread, an unsupported assumption, a wrong reference, or a clumsy phrasing of an accurate fact.
Fix the underlying issue by correcting the statement, changing the reference, or removing it, rather than leaving the flag unaddressed.
Problem: There's no clear method for deciding which statements to review first. (a large report gets reviewed unevenly — some sections are checked closely while others are skimmed or skipped).
Likely cause: There's no agreed method for prioritizing statements, so reviewers either spend too long on low-stakes content or run out of time before reaching the statements that need attention most.
Example: A reviewer works through a hundred-statement report from top to bottom and runs out of time before reaching the low-scoring statements near the end — the ones that most needed attention.
Fix:
Sort or filter the report by Validation Score and begin with the lowest-scoring statements.
Within each score band, prioritize statements that affect liability, causation, treatment, or damages.
Work upward through the scores, easing the depth of review as the grounding improves and the stakes fall.
Problem: A statement is relied on solely based on its score, without checking the source.
Likely cause: The score is treated as a substitute for the source document rather than a signal pointing to it, so the underlying record never gets checked.
Example: A statement scores 9/10 and is included without review. The linked document actually supports a related but distinct claim, and the mismatch surfaces only after the report is shared.
Fix:
For any statement you intend to rely on, open its linked reference even when the score is high.
Use AI Chat to ask what specifically in the document supports the statement.
Record your own confirmation of the statement rather than relying on the score alone.
Summary
The Validation Score, the Statement-by-Statement Review, and the Overall Assessment now work together to focus your review on the statements that need it most — with your own reading of the source as the final check, not the score alone.