Measured, not asserted
Published
A document production is a partition. Every responsive document goes to exactly one of two sides: produced to the requesting party, or withheld and described on a privilege log. This test renamed the field carrying the privilege tag and watched three privileged documents cross from one side to the other, while the reconciliation everybody writes -- produced plus withheld plus out of scope equals the collection -- stayed true, as it always will.
A conservation check cannot see a document change sides
The fixture is twelve collected documents. Eight are responsive and inside the agreed scope; three of those eight carry a privilege tag and must be withheld and logged. Two are not responsive, one sits outside the agreed date range, and one belongs to a custodian outside the agreed list. The correct production is five documents served, three withheld, three privilege log entries.
The obvious control over that process is a count. Nothing may be lost: produced plus withheld plus out of scope must equal the collection. It is easy to write, it is easy to explain to a client, and it is true in every arm measured here, including the ones that handed privileged material to the other side. Moving a document from one half of a partition to the other does not change the total. The invariant is closed by construction.
The reconciliation that does have power is different in kind: it compares the withheld count against the number of documents the review platform recorded as tagged privileged -- a count produced by the system that recorded the tagging, not by the code assembling the production.
Protection on its own moved nothing
Five areas, five presets, twenty-five comparisons, all identical to the unprotected runs. Nothing below is caused by protecting the code; every result required member renaming aimed at a name the installed production engine also reads.
The tag, and three privileged documents served
With the tag field renamed on the document records, no document matched any privilege tag. On its own this was caught, and caught precisely: production-rejected(platform-check(withheld 0 but the review platform recorded 3 privileged)). The expectation came from the far side, so the rename could not move both halves of the comparison.
Switch that check off -- one more name in the same pattern -- and the production was SERVED, reporting produced=8 withheld=0[] log=0[] out-of-scope=4 every-document-accounted-for=true. Three privileged documents went to the other side. The privilege log was empty and internally correct, because it describes what was withheld and nothing was withheld. The conservation check was true. Nobody was deceived by a false number; there were no false numbers.
Renaming the caller's rule rather than the flag produced the same production with checked by library-builtin(accepts any production) on the end of it.
This is the direction of the partition error that matters most, because it cannot be taken back. An under-production is a letter from the other side asking for the rest. An over-production of privileged material is a disclosure, and the party who benefits from it is the only party positioned to notice.
The same guard, blessing a different crossing
The sharpest cell of this pass is not the one above. It is what happened when the production scope moved instead of the privilege tag.
Renaming the start of the agreed date range, or the date field on the documents themselves, pulled a document from outside the request into the production. The result was SERVED, with responsive=9 produced=6, and the platform check PASSED, printing checked by platform-check(6 produced, 3 withheld of 9 responsive, platform agrees). Every figure in that sentence is correct. The platform did record three privileged documents, and three were withheld. A document the request never covered was handed over, and the reconciliation had no view about it, because it is defined over the withheld side.
That is a mitigation earned in one pass returning a true answer to a different question in the next. It is worth stating as a rule rather than an anecdote: a reconciliation defined over one side of a partition certifies nothing about the other side.
And the asymmetry is visible in the same file. Renaming the custodian list widened the scope in a way that pulled a fourth PRIVILEGED document in, so the withheld count went to four and the check refused. Withholding more is caught. Producing more, by scope rather than by privilege, is not.
The guard's own input, and what happens when you lose it
The review platform hands its counts back as a JSON document, so those keys are text on the far side and only the reading side can move. Renaming the read of the privileged count refused every production loudly: withheld 3 but the review platform recorded undefined privileged. That is the right behaviour and the reason to prefer a far-side expectation in the first place.
Losing that read together with the flag that enables the check produced the correct production, served with no check at all. Which is the honest summary of every enabling-flag result in this series: the strict control is never wrong, it is simply never reached.
What failed loudly
Renaming the document identifier produced unidentified-document(refused before production). Renaming the responsive flag reduced the production to nothing and the platform check refused. Renaming the custodian field on the documents put every document out of scope and refused for the same reason.
The log-entry requirement and the review pass identifier were cosmetic in this fixture -- they changed the printed configuration and no decision. Cosmetic halves are worth reporting because they are what a reviewer sees when a change is tested one name at a time and concludes the area is fine.
If you protect a review or production application
Reconcile both sides of the partition against something outside it. The withheld count against the review platform, and the produced set against the agreed scope -- custodians, date range, search terms -- as the requesting party would reconstruct it. One-sided reconciliation is how a scope overrun passed here with a check reporting agreement.
Treat an empty privilege log as an event, not an outcome. A production with zero withheld documents after a review that tagged some is not a clean result; it is a result that needs a person. The same applies to a log whose entries are all one tag.
And keep the names your bundle shares with an installed engine out of the rename pattern entirely. A member regexp scoped to the fields your own code owns removes this whole class in an afternoon. If you ship a review or viewer application to clients, our guide to protecting document viewer JavaScript covers the delivery side of the same problem.
Frequently asked questions
Did obfuscation change any production on its own?
No. Twenty-five base comparisons across five areas and five presets were identical to the unprotected runs. Every failure here required member renaming aimed at a property name the installed production engine also reads.
What happened when the privilege tag field was renamed?
No document matched a privilege tag, so all eight responsive documents were produced and none was withheld. The three privileged documents went to the requesting party and the privilege log was empty.
Did the completeness check catch it?
No, and it never can. Produced plus withheld plus out of scope equals the collection is true in every arm measured, because moving a document from one side of a partition to the other does not change the total. That invariant is closed by construction.
What did catch it?
A comparison against the review platform's own count of documents tagged privileged. That count is written by the system that recorded the tagging, on the far side of the boundary, so renaming the production code could not move both halves of the comparison. It refused the production immediately.
So the far-side reconciliation solved the problem?
Only for the error it was written about. Renaming the start of the agreed date range produced a document from outside the request, and the same check passed and printed that the platform agrees -- correctly, because the withheld count had not moved. A reconciliation defined over one side of a partition certifies nothing about the other.
Which direction of error is more likely to be noticed?
Under-production, by a long way, because the requesting party asks for the rest. Over-production of privileged material benefits the party best placed to notice it, and a document produced cannot be taken back.
What is the cheapest fix?
Reconcile both sides against something outside the production -- the withheld count against the review platform, and the produced set against the agreed custodians and date range -- and treat an empty privilege log as an event requiring a human rather than a clean result.
Related reading