Measured Behaviour
Published
A self-exclusion screen has obvious failures everybody tests for: a register that will not load, an applicant that will not save, a match that throws. It has one almost nobody tests for, because it is not a property of the list - the COMPARISON that decides two spellings are one person. We measured what a rename does to it.
The list is not the control
A self-exclusion register is the one control a gambling operator cannot argue with. A person has said, in writing, that they must not be allowed to open an account. Loading the list is easy. The hard part is that the register holds the name somebody gave a call handler in 2023 and the application holds the name they typed into a form this morning, and something has to decide those are the same person.
We built eight real applicants against a five-row register. One is spelled MacDonald on the register and Macdonald on the form. One has an apostrophe on the register and none on the form. One has a date of birth transcribed a day out. One has an accent she no longer types. One matches exactly. Three are not on the register at all, and one of those three shares a surname with a person who is.
The operator supplies a normaliser: fold accents, drop punctuation and hyphens, reconcile the Mac and Mc prefix, strip a postcode of its space. It matches on surname and date of birth, allows a single day of drift, and refuses at a score of 0.85. In the correct run all five registered people are refused, the three others are admitted, and nobody on the register gets an account.
Protected on five profiles with default settings, the behaviour was identical every time, and the base column was measured twice and diffed identical.
What renaming the normaliser actually did
The library has a normaliser of its own. It has to: a caller who supplies none still needs matching to work, so it lower-cases and trims and calls it a day. That builtin is a real normaliser and it really runs. It is simply much weaker than the one a licensed operator writes.
Renaming the property the operator's normaliser is passed under does not delete the function. The function is still in the bundle, still correct, still reachable. It is absent from the PROPERTY the library looks it up under, so the library substitutes its own.
Nothing broke. Applicants stayed at eight, screened stayed at eight, the register still reported five people on it, and every applicant was still compared against every row. Refusals went from five to three. Admissions went from three to five. Two people who had excluded themselves - the one with the apostrophe and the one with the accent - were admitted at a score of 0.50, because their surnames no longer normalise to the register's spelling.
On the book we modelled, 25% of screened applicants being on the register and admitted projects to 12,000 admissions a year. Both renaming presets produced byte-identical output.
The one case where the configuration screen tells you
This area differs from the others we measured this pass, and the difference is worth naming rather than glossing over.
The library reads the normaliser off the options object every time it is called, so its own configuration line moved too: it went from caller-supplied to library-builtin (lower-cases and trims). Anybody dumping the effective configuration would see it.
That is a real difference and it is the mitigation, not an accident. A library that captures a caller-supplied function once at construction and then reports the flag it captured would print caller-supplied forever, and this defect would be invisible from the configuration. Ask your libraries to report the collaborator they ACTUALLY consulted on each call, and assert it at start-up against what you passed.
A prediction that failed, and why that is the finding
We pre-registered a second prediction before measuring: that losing the normaliser AND the minimum match score together would be worse than losing either alone, because a weaker comparison and a stricter floor look like two independent routes to the same miss.
It was wrong. The union is byte-identical to losing the normaliser alone on every counter - same three refusals, same five admissions, same two registered people admitted - and the only thing that moved was the printed floor, from 0.85 to 1.00.
The reason is the useful part. Once the comparison is weakened, the surviving matches score exactly 1.00 and the misses score exactly 0.50. There is nothing between the two floors for a floor to bite on. Two options only compound when they act on different parts of the distribution, and here one of them collapsed the distribution the other was supposed to cut.
The general lesson for reading any matrix like this: a union that is identical to one of its halves is telling you the halves are not independent, and which one is load-bearing.
Why a test suite would not have caught it
We re-ran the same eight applicants with the five registered ones spelled EXACTLY as the register spells them, leaving the three genuine non-matches untouched so the control could not manufacture a hit that does not exist.
In that fixture the arm is inert. All five are still refused, no registered person is admitted, and the check still passes. Two identical strings are equal under any normaliser, including one that does nothing at all, so the only thing that moves is the reported normaliser label.
That is the fixture most people build, because test data for a matching control gets written by copying the register rows into the applicant rows. It is precisely the data that cannot see the defect, and the reason is not subtle once stated: a matching test built from exact copies is not testing matching.
What to do about it
Three things, and the first two are worth doing whatever you build with.
Make the normaliser's identity observable. Have the library report which comparison it actually used on each call, and assert it against the one you supplied when the process starts. A guard that cannot say whose implementation it ran is a guard you cannot audit.
Build the fixture out of the spellings real people use. Every variant in our set - the Mac/Mc case, the apostrophe, the accent, the transcribed date - is a variant a register genuinely contains. A screening test whose applicants are copies of the register rows measures string equality and nothing else.
On the build side, scope the renaming. Member renaming is opt-in and takes a pattern, so excluding the names you hand to an installed library keeps the rest of your bundle renamed; the mechanics are on the RenameMembers documentation. And because a self-exclusion register is a list of identified people with a lawful basis attached, the handling rules around it are the same ones our GDPR guidance covers.
Frequently asked questions
Does obfuscation break self-exclusion screening by default?
No. We protected the same screening bundle on five profiles - the ES5 target, the modern target, both emit-gate configurations and the string-encoding profile - and every one produced behaviour identical to the unprotected run, measured twice. The failure in this article comes from member renaming, which is opt-in and takes a pattern you choose.
What broke when the normaliser was renamed?
Nothing threw. The register still loaded with five people on it, all eight applicants were still screened against every row, and every score was still computed and compared. The library simply substituted its own weaker builtin normaliser, which lower-cases and trims but folds no accents and reconciles no name prefixes.
How many people got through?
Two of the five registered applicants were admitted - the one whose surname carries an apostrophe on the register and the one whose surname carries an accent. Refusals went from five to three and admissions from three to five. On the book we modelled that rate projects to 12,000 admissions a year.
Would a configuration review have caught this one?
Yes, and that makes this area the exception rather than the rule. The library reads the normaliser off the options object on every call, so its reported configuration flipped from caller-supplied to library-builtin. A library that captured the function once at construction would have kept reporting caller-supplied and the defect would have been invisible from the configuration.
Does raising the minimum match score help?
No, and we measured it. Losing the normaliser and the minimum match score together produced output byte-identical to losing the normaliser alone, apart from the printed floor. Once the comparison is weakened the surviving matches score 1.00 and the misses score 0.50, so there is nothing between the floors for a floor to act on.
Why did our test data not show this?
We re-ran the set with the registered applicants spelled exactly as the register spells them. The arm is inert there: all five are still refused and the check still passes, because two identical strings are equal under any normaliser. Test data built by copying register rows into applicant rows measures string equality, not matching.
How do we keep the protection and avoid the defect?
Assert at start-up that the comparison the library reports is the one you passed it, and build the fixture from the spellings real people use rather than from copies of the register. On the build side, scope RenameMembers so the names you hand to installed libraries are excluded - see the RenameMembers documentation.
Related reading