Measured Behaviour
Published
Every failure we have measured in this series so far could, in principle, be found by looking at the records it damaged. A containment test breaks that assumption, because its correct answer for almost every record in any real fleet is no - and a screen that has stopped working answers no as well, correctly, about most of them.
A predicate whose correct answer is usually no
We built one spring week for three vessels. Sixteen positions, every one really transmitted, at the minute and on the day recorded, plotting exactly where the vessel was. Two published closed areas drawn as boxes, a one-mile margin around every boundary, a spring window on the spawning box, a depth closure below six hundred metres, and a transit exemption for a vessel steaming through with its gear stowed.
Twelve of the sixteen positions are in open water and entirely legal. That is not a convenience of the fixture; it is what a real track looks like, and it is the whole difficulty of the area. The correct run flags four: a vessel towing a third of the way into the sandeel box, one towing six tenths of a mile outside the eastern boundary and therefore inside the margin, one towing in the spawning box during the closed window, and one towing at 720 metres. It records one legitimate transit. The compliance check fails, which is what it is for.
Protection alone changed nothing. Five profiles - ES5, modern, both emit-gate configurations, string encoding - identical output, measured twice. The failures start when member renaming reaches the closure configuration, which is what a broad RenameMembers pattern over a configuration literal does. The mechanics are in Protect Members.
The prediction we wrote down, and why it was wrong
We pre-registered this: renaming the latitude field would leave every position plotted and every count intact, drive incursions from four to zero, and turn the authority's own check from fail to pass. A containment test that cannot read a coordinate cannot answer yes, and not inside is a perfectly ordinary result.
Most of that landed. Positions stayed at sixteen, tested stayed at sixteen, fishing positions stayed at thirteen, and three of the four incursions vanished. Compliance went from 75.00 per cent to 93.75 per cent.
But the check still failed, and the reason is the useful part. The depth closure is not a spatial predicate. It compares one number to one threshold, it never touches a coordinate, and it survived intact - so the vessel at 720 metres was still flagged, and one incursion was enough to keep the alarm on. Renaming the closed-area list itself gave exactly the same result for exactly the same reason.
That is worth stating as a general point, because it is cheap and it works: a second closure criterion expressed over a different kind of value is what keeps the alarm on when the first one goes quiet. It did not have to be depth. It had to be something that is not a pair of coordinates.
The arm that did silence it
The arm that took incursions to zero was not about coordinates at all. This authority's transit exemption requires the gear to be stowed; the vendor's default does not. Renaming that one key left every boundary correct, every coordinate readable, every position plotted - and reclassified all five positions inside a closure - the one that really was legitimate passage and the four that were towing - as transit.
Incursions went from four to zero. Transits went from one to five. Compliance went from 75.00 per cent to 100.00 per cent. The projected annual figure went from 52,500 incursions to none. The authority's own check passed, printing 16 position(s), 16 plotted and screened, 13 fishing, 0 inside any closure - and every figure in that sentence is true.
Here is the measurement that makes this area different from the other four in the pass. We ran the same week against a negative control fleet - the same sixteen positions moved into open water, well clear of both boxes and off the deep ground. The compliance line the defective run produces is byte-identical to the line the clean run produces over the compliant fleet: compliance 100.00% (0 of 16 position(s) flagged); at 210000 fleet position(s)/yr that projects 0 incursion(s)/yr.
The figure a fleet manager reads cannot distinguish a broken screen over a real fleet from a working screen over a fleet that did nothing wrong. Two counters do distinguish them - the transit count and the depth-hit count are 5 and 1 in the defective run against 0 and 0 in the control - and neither is a number anybody puts on a dashboard.
The wider pattern was quieter than the narrower one
We also pre-registered a union, and this one held exactly. The transit-allowed flag and the gear-stowed requirement sit on the same object and both concern the same exemption, so a sloppy broad pattern reaches both.
Alone, the transit flag fails closed and loudly. The vendor has no transit exemption at all, so the vessel legitimately steaming through the sandeel box with its gear in is flagged as an incursion. Incursions went from four to five, compliance to 68.75 per cent, and somebody has to explain to a skipper why passage is being written up.
Alone, the gear requirement fails open and silently, as above.
Together they are byte-identical to the transit flag alone: five incursions, 68.75 per cent, no transits recorded. The half that fails loudly is consulted first and the quiet half never gets a say. So the broad, careless pattern produces the self-correcting failure, and the narrow, careful one produces the silent one. If you test only the wide pattern because it is the realistic one, this cell is invisible.
The buffer and the season window are the other two ways a true position escapes a true boundary, and they go in opposite directions. Losing the buffer took incursions from four to three: the vessel six tenths of a mile outside the line is now outside it, which is exactly what the margin exists to prevent. Losing the season window took them from four to five, because the spawning box became closed all year and a vessel fishing it legally in July was written up.
What to do about it
Scope the member pattern away from configuration keys and record field names. Coordinates are record field names.
Then add the criterion that is not a coordinate. A depth limit, a gear-type restriction, a time-of-day rule - anything expressed over a different kind of value will keep failing when the spatial test goes quiet, and it is the only thing in our matrix that did.
Then treat a compliance rate as an output rather than evidence. The rate went up in three of the arms above and it went up because the screen stopped working. What does distinguish the runs is the counters nobody reports: how many positions were actually tested against a boundary, how many transits were granted, how many were within the margin rather than inside the line. Report those beside the rate and the defective arms separate immediately.
Finally, make a missing coordinate an error rather than a point that is nowhere. A containment test has three possible answers - inside, outside, and unknown - and collapsing the third into the second is the entire failure. This is worth fixing regardless of obfuscation, because a dropped field over a serialisation hop produces the same result.
Vessel-side software runs on hardware with its own constraints; protecting JavaScript on embedded devices covers what is and is not available there.
Frequently asked questions
Does obfuscation break closed area restrictions?
Not on its own. Five protection profiles produced identical output on the same fixture, twice. What broke the screen was member renaming reaching the closure configuration and the position records.
Which key silenced the screen completely?
The requirement that gear be stowed during transit. Renaming it reclassified all five positions inside a closure - the one that really was legitimate passage and the four that were towing - as transit. Incursions went from four to zero and compliance from 75.00% to 100.00%.
Did renaming a coordinate silence it too?
Not completely, and that is the useful result. Losing latitude took three of the four incursions away, but the depth closure compares one number to one threshold and never touches a coordinate, so it survived and kept the check failing. A second criterion of a different kind is what keeps the alarm on.
Why is a spatial test harder to audit than other rules?
Because its correct answer for almost every record is no. Twelve of our sixteen positions really were in open water, so a broken screen is right about most of the fleet. Sampling the output cannot separate a broken screen from a compliant fleet.
Was the compliance rate really identical to a clean run?
Yes, byte for byte. The defective run over the real week and the clean run over a control fleet that never approached a boundary both print 'compliance 100.00% (0 of 16 position(s) flagged); at 210000 fleet position(s)/yr that projects 0 incursion(s)/yr'.
Was the broader renaming pattern worse?
No - it was quieter, and we pre-registered that. Renaming the transit flag alone fails closed and flags legitimate passage. Renaming both it and the gear requirement is byte-identical to the flag alone, because the loud half is consulted first. The careless pattern self-corrects and the careful one does not.
How do we protect vessel monitoring software safely?
Scope the member pattern away from configuration and record names, add a closure criterion that is not a coordinate, make an unreadable position an error rather than a point outside every area, and report tested-position and transit counts beside the compliance rate. Protect Members documents how the regexp is applied and what a safe scope looks like.
Related reading