Measured, not asserted
Published
This test is not a comparison between two numbers. It is a comparison between two populations, and the two rules that decide who belongs to the smaller one look exactly like defence in depth: a pay threshold and an ownership stake, either sufficient on its own. Renaming either one changed the group composition, changed the reported averages, and left the pass or fail verdict completely unchanged. Renaming both, together with the check written to refuse a test with an empty group, certified a plan that owes corrective distributions as having passed.
Two routes into one group, and a verdict that depends on the group
The actual deferral percentage test compares how much the highly compensated group defers with how much everybody else does. If the first average exceeds what the second permits, the plan fails and money has to come back out of highly compensated accounts.
Membership of that group has two independent routes. Pay above a threshold in the year the plan tests on is one. An ownership stake is the other. Either is sufficient, which is exactly what redundancy looks like from inside the code.
The fixture has eight participants. Two are highly compensated by prior-year pay, at $214,000 and $168,000 against a $155,000 threshold. One more is highly compensated ONLY by ownership: an 18% owner paid $121,000, comfortably under the threshold. That third participant is the whole point of the area, because they are the proof the two routes are not redundant at all -- each catches somebody the other misses.
Correctly computed, three highly compensated participants average 7.60% against five others averaging 3.10%, which permits 5.10%. The plan fails and owes corrective distributions.
Protection on its own moved nothing
Twenty-five comparisons across five areas and five presets, all byte-identical to the unprotected run, with every original run twice before anything was compared against it.
Each half moves the numbers and not the answer
Renaming the pay threshold reverts it to a default under which no salary makes anybody highly compensated. The ownership route survives, so the group becomes one person: groups=1 HCE / 7 NHCE adp-hce=9.40% adp-nhce=4.13% allowed=6.13% => ADP TEST FAILED.
Renaming the ownership threshold instead reverts that route and leaves the pay route: groups=2 HCE / 6 NHCE adp-hce=6.70% adp-nhce=4.15% allowed=6.15% => ADP TEST FAILED.
Look at what a reviewer sees in each. The verdict is unchanged. The plan failed before and the plan fails now. Every figure on the line is different, and every figure is a well-formed percentage to two decimals that could plausibly be the result of a recalculation. There is no undefined, no NaN, and no error.
That is a sharper version of a finding this series has recorded before. A half-arm that moves only a display label reads as cosmetic. These half-arms move the group composition and both averages, and still read as cosmetic -- because the only thing anybody checks is whether the test passed.
Both halves, and then the guard
Renaming both thresholds in one pattern empties the highly compensated group entirely. The plan's own rule of record catches it, and this is the mitigation five passes in this series have recommended, stated over the SHAPE of the answer rather than its value: test-rejected(population-check(the test has 0 HCE and 8 NHCE, so no two groups were compared)). Nothing certifies, somebody investigates.
Add the enabling flag in front of that rule to the same pattern and the plan certifies: groups=0 HCE / 8 NHCE hce=[] adp-hce=NaN% adp-nhce=4.79% allowed=6.79% => ADP TEST PASSED, unchecked.
Three highly compensated participants who owe corrective distributions are now not highly compensated, the comparison has one side, NaN > 6.79 is false, and the engine reports a pass. The empty bracket after hce= is the only visible tell, and it is on a line whose verdict is the reassuring one.
Two things fell together here that are usually discussed separately. Two controls that were genuinely independent -- different rules, different fields, catching different people -- were reached by one pattern because they live on the same object in the same file. And the guard that would have caught the collapse was switched off by the same edit, because it sits behind a flag that the same pattern reaches. Redundancy is not independence when one pattern reaches both layers, and a guard is only loud while it is still reached.
The denominator nobody checks
A deferral percentage is a ratio: the year's deferrals over the year's plan compensation. Renaming the compensation field makes every rate NaN, so both averages and the permitted figure are NaN, the comparison is false, and the plan reads adp-hce=NaN% adp-nhce=NaN% allowed=NaN% => ADP TEST PASSED.
The plan's population check passes too, and correctly, because it is defined over the group counts and the group counts are still 3 and 5. The mitigation was written about who is being compared and this failure is about what they are being compared on.
Renaming the deferred amount does the same thing from the numerator side. Both operands of a ratio behave this way: nothing accumulates them, so no finiteness check downstream ever sees them. That is the general form of a lesson two earlier passes recorded in narrower shapes -- a weighted average protects both operands because both accumulate, and a ratio protects neither.
For completeness, the two statutory alternatives behave as a pair rather than as two controls. Renaming either the additive spread or the multiple leaves the plan failing; renaming both sets the permitted figure to infinity and the plan passes with its true 7.60% and 3.10% printed correctly beside it.
What this means if you protect a plan testing front end
Keep member renaming off the boundary with a MemberRegexp scoped to your internal names, and exclude every option key and record field an installed testing engine reads.
Then take the placement lesson seriously, because it is the one that survives past obfuscation. A second control only adds anything if it lives somewhere the first control's failure cannot reach. Two membership rules on one options object in one bundle are one control with two names. If the classification genuinely matters, classify server-side and let the front end display the result.
And assert the shape of the answer, not just its value: refuse a test with an empty group on either side, refuse a comparison whose operands are not finite, and put those refusals somewhere that cannot be disabled by the same edit that causes the problem. Group membership is also personal data under GDPR requirements for JavaScript, which is a second reason to keep the classification off the client. The protected build classified all eight participants correctly on all five presets.
Frequently asked questions
Did obfuscation change any test result on its own?
No. Five areas across five presets, every protected output byte-identical to the unprotected run. Every failure here required member renaming reaching a name the installed testing engine also reads.
Why did renaming one membership rule change nothing?
Because the other route still caught people. The group composition and both averages changed -- one arm reported 1 highly compensated participant averaging 9.40%, another reported 2 averaging 6.70% -- and the plan still failed in both. The only thing anybody checks is the verdict, and the verdict did not move.
What happened when both membership rules went at once?
The highly compensated group emptied, and the plan's own population check caught it and refused to certify. Adding the flag that enables that check to the same pattern produced ADP TEST PASSED with zero highly compensated participants and an NaN average.
Is that a failure of the mitigation this series recommends?
Not of the rule itself -- the population check caught the collapse when it was reached. It is a failure of placement. A guard behind an enabling flag that the same rename pattern can reach is not independent of the thing it guards.
What did renaming the compensation field do?
It made every deferral rate NaN, so both averages and the permitted figure were NaN, the comparison was false, and the plan passed. The population check still passed correctly, because it counts people and the counts were right.
Why do ratios fail worse than weighted averages here?
Because neither operand of a ratio accumulates into anything a finiteness check inspects. In a weighted average both operands are summed, so one check on the accumulators covers both. Knowing which of your operands accumulate tells you which are protected.
What is the cheapest test that would have caught all of this?
Run one known census end to end and assert the group counts, both averages and the permitted percentage -- not just the pass or fail verdict. That fails in every arm in this article, including the two halves that leave the verdict alone.
Related reading