Measured Behaviour
Published
A competence system has an obvious failure everybody tests for: the ticket that has expired. It has a second one almost nobody tests for, because it is not a property of the ticket at all - the ticket that was taken away. We measured what a rename does to each, and they are not the same kind of failure.
A ticket that is real, in date, and no longer valid
We built a morning dispatch the way one actually arrives: twenty assignments across twelve workers, a table saying which task needs which ticket, a table giving each ticket its stated life in months, an eighteen-line roster of certificates, and a withdrawal register carrying four entries.
Each of those four is worth describing, because the whole article turns on them. A high-voltage authorised person suspended pending a switching-error investigation. A confined-space entrant withdrawn on medical review. A rigger revoked for a falsified lift log. A hot-work operator suspended after a fire-watch failure. Every one of those workers still holds a certificate that is genuine, was issued by the body named on it, on the day named on it, and is comfortably inside its stated life. There is nothing wrong with the paper. The paper is simply no longer worth anything.
The correct run examines twenty assignments, clears twelve and refuses eight. Four of those refusals are the withdrawn workers, each stopped at the gate for the one task their withdrawn ticket covers. Three hold nothing at all for the task they were sent to. One is a genuinely lapsed high-voltage ticket, caught by ordinary date arithmetic.
We then protected the file with default settings on five profiles: the ES5 target, the modern target, both emit-gate configurations and the string-encoding profile. All five produced identical behaviour. That is the first result and it deserves stating plainly - protection on its own did not change a single dispatch decision.
The failures start when member renaming reaches the names on the qualification object or on the register's own records, which is what a broad RenameMembers pattern over a configuration literal actually does. If you have not yet decided how far your member pattern reaches, the mechanics are in Protect Members.
What the register loss actually did
We wrote the prediction down before measuring, because an expectation formed after the fact is worth nothing. The prediction was that the number examined would not move, that every worker holding nothing would still be refused in the same words, that the lapsed ticket would still be caught, that the withdrawal counter would read zero, and that the site's own check would pass.
Every part of it landed. Examined stayed at twenty. Cleared went from twelve to sixteen. Refused fell from eight to four - and the four that remain are exactly the three workers holding nothing and the one lapsed ticket, refused for the same reasons in the same words. The counter whose entire job is to report this - the number stopped by a withdrawal - went from four to zero.
And the site's competence check passed, printing: twenty assignments examined, sixteen cleared, four refused, zero stopped by a withdrawal. Every figure in that sentence is true. Twenty assignments really were examined. Sixteen workers really were cleared, and every one of them really does hold a genuine, in-date, correctly-matched certificate. Zero really were stopped by a withdrawal, because as far as the running code was concerned there were no withdrawals.
This is the part that separates this failure from the ones we have measured before. In every earlier shape something in the answer is wrong and can in principle be pointed at: a value, a unit, a population, a total, a date. Here nothing in the answer is wrong. Every record the system can see is correct. What is wrong is that four of those records should not exist any more, and the records that said so are the ones that went missing.
Five different names, one identical outcome
We drove the same area five ways, and they all land on the same twenty-sixteen-four-zero. Renaming the register itself. Renaming the flag that says whether to honour it. Renaming any one of the three fields on a withdrawal record - the worker it names, the certificate it names, or the day it took effect. Five distinct names, five distinct mechanisms, one identical set of people on site who should not be there.
What differs is only how much of a trace each leaves, and the ranking is not the one you would guess. Renaming the register makes the configuration screen read empty register (nobody is withdrawn), which at least looks like something. Renaming a field on the withdrawal records leaves four entries on screen with a visible undefined in one column. But renaming the honour flag leaves the register display completely intact - all four names, all four certificates, all four dates, every character correct - while nobody is stopped by any of them.
So a populated register screen is evidence that entries exist. It is never evidence that anything reads them. That is worth remembering the next time an audit is satisfied by a screenshot of a configuration page.
The union of the register and the check that was supposed to catch it produces the same figures again, with the check line reading unchecked. The rule was contributing nothing to remove, because it was never able to see this.
The loud half, one field away
The contrast is on the same records. Rename the issue day, or the certificate code, and cleared collapses from twelve to one: nineteen of twenty assignments refused because no certificate can be dated or matched. Nobody works. Somebody telephones within the hour.
That is the pattern this series keeps finding, now for the eighteenth area running. Losing a VALUE fails closed and loudly. Losing a MEANING - a policy, a unit, a scope, and now a retraction - fails open and silently. The failure that stops the site gets fixed on the day it happens. The failure that puts a revoked rigger back on a crane does not announce itself at all.
It is also worth naming who is affected, because it is not a random sample. The workers this moves are precisely the ones whose qualification was taken away, usually for cause, usually after an incident. They are the population the control exists for, and they are the only population it stops protecting.
Why the obvious test cannot see it
We ran the same twenty assignments against a roster in which nobody has ever been withdrawn - a register that is empty because there is genuinely nothing in it. That run clears sixteen and refuses four.
Those are the same numbers. The broken run and the clean-history run are byte-identical in every counter we print. There is no assertion over the output that separates them, because there is nothing to separate: a system that ignores four withdrawals and a system with no withdrawals to ignore produce the same day's work.
That matters more than it sounds, because constructed test data has no history. When you build a fixture you invent workers and give them tickets; you do not usually invent a worker, give them a ticket, and then take it away six months later for falsifying a log. So the fixture most teams would write is blind to this entire class by construction, and it will stay green through every arm above.
The fix is not a stricter assertion over the dispatch. It is to make the retraction visible in the output rather than implicit in the absence of a refusal: report how many register entries were READ and matched this run, and fail if a register you believe is loaded contributes nothing. And check the count against something outside the deployment - the awarding body's own suspension feed, which is where the four notices came from in the first place.
What to do about it
None of this is an argument against protecting the bundle. The base column was clean on all five profiles: nothing here was caused by obfuscation on its own. Every result came from member renaming reaching names that carry meaning across a boundary.
Scope the member pattern so it cannot reach configuration objects or the records you exchange with a competence provider. Use MemberRegexp to name the members you want renamed rather than accepting a pattern broad enough to include an options literal, and keep a reserved list for anything that crosses a process boundary. The same page linked above, Protect Members, documents the matching rules.
And if you are deciding where obfuscation belongs in a plant or industrial control estate at all, IEC 62443 and your industrial HMI JavaScript is the place to start, because the answer depends on which side of the boundary the decision is taken.
Frequently asked questions
Did obfuscation change any dispatch decision on its own?
No. On all five profiles we tried - the ES5 target, the modern target, both emit-gate configurations and the string-encoding profile - the protected bundle produced identical behaviour to the original. Every difference came from member renaming reaching an options object or a withdrawal record.
What did renaming the withdrawal register do?
It sent four workers whose tickets had been suspended or revoked onto the exact jobs those tickets cover. Assignments examined stayed at twenty, cleared rose from twelve to sixteen, and the counter for workers stopped by a withdrawal fell from four to zero.
Did the site's competence check catch it?
No, and it passed on true figures. It printed twenty assignments examined, sixteen cleared, four refused, zero stopped by a withdrawal. Every one of those numbers is correct - the check is defined over the dispatch, and the dispatch is not what moved.
Is renaming the register records safer than the settings object?
No. Five names produce byte-identical outcomes: the register option, the honour flag, and each of the three fields on a withdrawal record - the worker it names, the certificate it names and the day it took effect. All five give twenty examined, sixteen cleared, four refused.
Which arm leaves the least trace?
The honour flag. Renaming the register makes the screen read empty, and renaming a record field leaves a visible undefined in one column, but renaming the flag leaves all four register entries displayed correctly while none of them stops anybody.
Why does a normal test fixture miss this?
Because constructed test data has no history. We ran the same twenty assignments against a roster where nobody had ever been withdrawn and got the same sixteen cleared and four refused. A system ignoring four withdrawals and a system with none to ignore produce identical output.
What should we assert instead?
Report how many register entries were actually read and matched on each run, and fail when a register you believe is loaded contributes nothing. Reconcile that count against the awarding body's own suspension feed rather than against your own configuration.
Related reading