Bot Defence
Published
A CAPTCHA token is not a verdict. It is a receipt, and a receipt is only worth what you check about it: which site it was solved on, which action it was solved for, how human the solver looked, how long ago it happened, whether it has been spent already, and whether anybody signed it. Every one of those checks is a property name on an options object, and the library reading them was installed rather than built. We protected a file that configures one, renamed the names a group at a time, and counted the logins that got through.
What the sample actually does
The file configures server-side token verification the way a careful team configures it. The expected hostname is pinned to the application's own domain. The expected action is pinned to the login form specifically. A minimum score is set, so a solver the challenge service scored as almost certainly automated is refused. A maximum token age is set, so a receipt from twenty minutes ago is stale. Single use is switched on, so one solve buys one attempt. Failing open on a service error is switched off. And the application supplies its own token check, which verifies the signature the challenge widget put on the token.
Each of those is then exercised against a request only that guard can refuse. The token solved on a phishing proxy is genuine in every other respect - real signature, real human, high score, seconds old, never spent - so nothing but the hostname check can stop it. The token harvested from the newsletter box is equally genuine, and only the action check knows it was solved somewhere else on the same site. The headless browser scores badly and nothing else about it is wrong. Without that separation the first guard to refuse hides every guard behind it, and an arm that is never reached reports no change - truthfully and uselessly.
Before any of the protected arms were read, each option was deleted from the sample's own configuration and the output compared. Every option moved the result except the one deliberately pinned equal to the library's default, which is the control. That step is cheap and it is the only reason the arms below mean anything.
The verification helper is copied into the measurement directory unprotected, because that is the shape of the real thing. Your bundle is rebuilt when you protect it; the package in node_modules is not, and it keeps reading the property names it has always read. Its defaults are the permissive ones this family of helpers ships when handed an empty configuration: no hostname check, no action check, no score floor, no freshness check, no replay protection, allow on service error, and a token test that asks whether the token is a non-empty string.
Protection alone was applied first, on five presets covering both output targets, the gate profile and the compressed profile. All five behaved identically to the unprotected file. Nothing below is caused by protection on its own. Every result required member renaming aimed at the names, which is what the MemberRegexp option exists to scope.
A token solved on the attacker's own domain
Renaming the expected-hostname option took the measured line from solved-on-attacker-domain=refused(wrong-hostname(app-example-com.attacker.test)) to solved-on-attacker-domain=ALLOWED(verified), and the reported policy from the application's own domain to null.
This is the arm that matters most, because the hostname field is the only thing in the whole exchange that says where the challenge was solved. A phishing proxy that serves your login page from its own domain can embed your widget, let the victim solve it honestly, and forward the resulting token to you. The signature is real. The human is real. The score is excellent. The receipt is simply for a transaction that happened somewhere else.
The same shape appeared one option along. Renaming the expected-action check let a token solved at the newsletter signup - a box any visitor can solve as often as they like, with no rate limit and no consequence - be presented at the login form and accepted. A challenge is only a barrier where it is expensive, and an action binding is what stops an attacker doing their solving somewhere cheap.
Renaming both together produced both outcomes at once. Neither option protects the other; they narrow different fields, and one pattern over an options object reaches them both.
One paid solve, five logins
Replay protection produced the sharpest quantitative result. In the unprotected run the measured line reads logins-one-solve-bought=1/5: the first attempt verifies and the next four are refused as already spent. Renaming the single-use option took it to 5/5.
That number is the entire economics of a credential-stuffing run. A solving service charges per solve. If one receipt is good for one attempt, the attacker pays for every guess and most campaigns stop being worth running. If one receipt is good for as many attempts as they care to make, the challenge has become a one-off toll at the entrance to an unlimited buffet, and the cost per guess falls to approximately nothing.
The freshness check failed the same way and compounds it. Renaming the maximum token age reverted the policy to 0(freshness not checked), and a token solved forty minutes earlier by a human solving farm verified. Stale tokens are what a solving farm sells: a batch is solved in bulk, warehoused, and resold hours later. An age limit is what makes that inventory perishable.
The score floor moved in the same direction. Renaming it reverted the minimum to zero, and a session the challenge service had scored at 0.1 - its way of saying this is almost certainly automated - verified as cleanly as a human would have.
The outage that is not an outage
One option in this area decides what happens when the challenge service itself is unreachable, and its library default is to allow the request. Renaming it moved the reported policy from fail-closed to fail-open, and the measured line from checker-unreachable=refused(service-error-refused) to checker-unreachable=ALLOWED(service-error-allowed).
Read that against the rest of the article. Every other arm here needs a specific request shape to exploit. This one needs the attacker to make the verification endpoint unreachable, which for a busy origin is a much smaller ask than it sounds, and every request during that window is unverified. Nothing in your logs distinguishes it from a quiet period, because the requests succeed.
It is also the arm least likely to be noticed in testing, because a fail-open default is only visible in the failure case, and the failure case does not occur in a test suite that mocks the verification call.
The verifier was not switched off, it was downgraded
Renaming the option that carries the application's own token check did not disable checking. It substituted the library's builtin, which asks whether the token is a non-empty string. A token typed by hand into the request - no signature, no solve, nothing - verified. The reported verifier moved from caller-supplied(checks the signature on the token) to library-builtin(presence-only), which is the honest reading and the only line in the capture that names the substitution.
This is the fifteenth library across seven passes in which a caller-supplied implementation of a guard was replaced by a weaker builtin rather than removed. The pattern is worth internalising because it defeats the obvious audit: token verification is still configured, still enabled, still running, and still logging that it ran. It is answering a different question. Any check of the form "is verification turned on?" returns true.
The whole-block arm - renaming every option in the group at once, which is what a broad pattern actually does - combined all of the above. A token solved on a look-alike domain, for a different action, by a browser scored as automated, forty minutes ago, presented five times, with a signature nobody wrote, was accepted every time.
Which direction fails loudly
Renaming the token's own field names - hostname, action, score, signature - fails in the opposite direction and is easy to catch. Those values are read and then compared, so a missing one compares unequal and everything is refused: the measured line reads genuine-login=refused(wrong-hostname(undefined)) for every request, including the real customer's. That is an outage, somebody notices within the hour, and nobody gets in who should not.
The fraud log went the quiet way instead. Renaming the fields on the row this bundle writes for the fraud team took the stored field list from action,ip,score,verdict to _0x1,_0x2,action,score, and the readback to verdict=MISSING ip=MISSING. Every decision was unchanged and correct. The record of those decisions stopped describing them, and the team that would investigate a stuffing campaign lost the two fields they would sort by.
The generalisation held across all five areas measured this pass. A name flowing inward - your options into a dependency - fails silently, because a library has to tolerate keys it does not recognise. A name flowing outward - a result you read and compare - fails loudly. A name written into a record that leaves the process fails silently and does not surface until somebody needs the record.
What to do about it
None of this argues against protecting the code. It argues against pointing member renaming at an options object without scoping it. The option is RenameMembers, the scope is MemberRegexp, and the smallest fix is an exclusion covering the option names and result fields of anything installed rather than built.
Prefer a policy shape that fails closed when a name moves. A minimum score, an expected hostname and an expected action are all narrowing rules whose absence means "accept everything", which is why losing them is silent. If your framework lets you express the check as an assertion that must pass rather than a filter that may apply, take it.
Set the service-error policy explicitly, in code, next to the option that sets it. It is the one setting in this area whose default is the dangerous value, and the one least likely to be exercised by a test.
Then verify with a request, not with a configuration read. Send a token with the wrong hostname and confirm it is refused. Send the same token twice and confirm the second attempt fails. Those two probes take a minute and they check the thing that actually matters, which is what the deployed bundle does rather than what its source says it intends.
The client-side half of this problem is a separate subject: the anti-automation script you ship runs on the machine whose trustworthiness is in question. That is covered in its own article, linked below.
Frequently asked questions
Does protecting my JavaScript break CAPTCHA verification on its own?
Not in this measurement. The sample was protected on five profiles covering both output targets, the gate profile and the compressed profile, and all five behaved identically to the unprotected file. Every result in this article required member renaming pointed at property names.
What was the worst result?
Renaming the expected-hostname option. A token solved on a phishing proxy's own domain was accepted at the real login form. The signature, the human and the score were all genuine; the hostname field is the only thing in the exchange that says where the solve happened.
How much does losing replay protection actually cost?
It was measured directly. One solve bought one login attempt before, and five after, from an identical five-attempt run. That ratio is the economics of a stuffing campaign: a solving service charges per solve, so a reusable receipt drives the cost per guess towards nothing.
Is the fail-open default really that common?
It is common enough to be worth setting explicitly. Renaming that option moved an unreachable challenge service from a refusal to an acceptance, and a fail-open default is only visible in the failure case, which a test suite that mocks the verification call never reaches.
Did the signature check get switched off?
No, it was downgraded. The application's own check was replaced by the library's builtin, which confirms the token is a non-empty string. A hand-written token verified, and every check of the form "is verification enabled?" still returned true.
Which arms fail loudly enough to catch in testing?
The ones that rename the token's own field names. Those are read and compared, so they fail closed and refuse everybody including the real customer. The dangerous direction is inward, where a renamed option name is silently ignored and the library's permissive default applies.
What is the smallest change that prevents all of this?
Scope RenameMembers with a MemberRegexp that excludes the option names and result fields of installed dependencies, then confirm with two live probes: a token with the wrong hostname must be refused, and the same token presented twice must fail the second time.
Related reading