Scheduling and background work
Published
A job scheduler is configured almost entirely by narrowing its defaults: the timezone the schedule is written in, a refusal to start a second copy while the first is alive, a decision about the ticks that were missed while the box was down, a cap on how long one run may take, and an admission test only your application can express. All of those are property names on an options object, and the scheduler reading them was installed rather than built. We protected a file that configures one, renamed the names a group at a time, and counted the invoices.
What the sample actually does
The file configures a nightly billing job the way a careful team configures one. The schedule is written in the company's own timezone because the finance team chose an hour when nobody is trading. Overlapping runs are forbidden. Ticks missed during an outage are skipped rather than replayed. A run may take a minute and no longer. And an admission test refuses to start on a replica that is not the leader, or during a maintenance window - two conditions the scheduler has no way to know about.
It then exercises each of those exactly once, in a scenario only that guard can decide. The overlap scenario uses a tick the admission test allows, so the admission test is not what refuses it. The admission-test scenario uses a tick with nothing in flight, so the overlap rule is not what refuses it. A second job with a deliberately different misfire policy exists purely so that the catch-up cap is reachable at all - in the first job the skip policy decides first, and the cap is never read.
The scheduler is copied into the measurement directory unprotected, because that is the shape of the real thing. Your bundle is rebuilt when you protect it; the package in node_modules is not, and it keeps reading the property names it has always read. Its defaults are the permissive ones this family of libraries ships: UTC, overlapping runs allowed, every missed tick replayed, no cap on a run, and an admission test that knows only whether a run is already in flight.
Protection alone was applied first, on five presets covering both output targets, the gate profile and the compressed profile. All five behaved identically to the unprotected file. Nothing below is caused by protection on its own. Every result required member renaming aimed at the names, which is what the MemberRegexp option exists to scope.
The timezone is one word and it moves the whole night
Renaming the timezone option reverted it to UTC. The billing run's reported firing hour moved from 06:00 UTC to 02:00 UTC, which in the office's own timezone is ten o'clock the previous evening.
Nothing failed. The job ran, it ran once, it billed the right customers the right amounts, and every log line looks correct. It simply ran four hours earlier than the day it belongs to, which means the last four hours of trading fell into tomorrow's run instead of today's. For a billing job that is a month-end boundary error that only appears on the first of the month. For a job that closes a trading day, or expires a promotion, or cuts a statement, it is the same error with a different name on the invoice.
This is the arm that best explains why the whole class is hard to find. There is no exception, no wrong count, no failed assertion. The only symptom is that a number belongs to the wrong day, and it takes a person who knows what the number should have been to notice.
Two copies of the billing job, and every customer billed twice
The refusal to start a second copy while the first is alive is a single boolean. Renaming it reverted it to the library default, which is to start the tick regardless.
Last night's run was still going when tonight's tick arrived, both ran, and the invoice count went from three to six. The run's own summary line, which reads false in the unprotected run, read true: double billed. The reported policy flipped from enabled to disabled on the same line, which is the one piece of evidence available - and it is only available because the sample asks the scheduler what policy it is running under, rather than reading back the value it wrote itself.
That distinction matters more than it looks. Had the sample printed its own option object, it would have printed the configured value in every arm, because the write and the read move together when both are inside the protected file. The only reading with any evidential value is the one taken from the far side of the boundary.
The missed ticks, and the cap that was unreachable
The application skips ticks missed during an outage, because replaying five nights of billing is worse than missing them. Renaming the misfire option reverted it to replay-everything: five missed nights fired, and fifteen invoices were emitted for work that had already been done or deliberately abandoned.
The catch-up cap deserves its own paragraph, because in its first form this arm measured inert and the reason was the fixture rather than the engine. With the skip policy in force the cap is never read - the policy decides first and returns. An arm that cannot be reached reports no change, truthfully and uselessly. A second job was added whose misfire policy is deliberately replay-everything, which is what a metrics roll-up legitimately wants, so that the cap is the only thing bounding the burst. On that job, renaming the cap took the replay from two ticks to five and the work from six units to fifteen.
This is the ninth consecutive pass in this series where an arm that measured inert turned out to be structurally blind rather than safe. The rule it bought is simple: when an arm reports no change, check whether the code it governs was reachable before concluding anything.
The admission test, and an assertion that passed for the wrong reason
The application's own admission test refuses to run on a replica that is not the leader. The scheduler cannot express that; it only knows whether a run is in flight. Renaming the option that carries the test replaced it with the library's builtin.
A tick arrived on a replica and started. Three invoices were emitted by a machine that is not supposed to emit any. The reported gate moved from caller-supplied to the library's in-flight check, which is the honest reading, and the only one that names the substitution.
The interesting part is what happened to a different line in the same arm. The overlap scenario still refused its second tick - the printed reason simply changed from overlap-prevented to refused-by-gate, because the builtin's in-flight check happens to refuse the same tick for a different reason. An observer watching only for "did the second tick start?" would conclude that overlap protection survived this edit intact. It did not survive; it was replaced by something that agrees with it in one scenario and not in the one that matters. A check that passes for a different reason than the one you think is not evidence.
The rest of the arms, and the direction each one fails in
The run cap went from a minute to uncapped, and the wedged export ran for the full four hours it wanted instead of being killed at sixty seconds. The kill count went from one to zero. That is a job holding its locks all morning, and it is the arm most likely to be noticed - by everything else in the system slowing down, not by anything the job itself reports.
The arm that renamed the job's own bookkeeping fields is the one worth studying. The scheduler writes a row keyed on the job name, and the row belongs to the scheduler's store, not to the application. Renaming those field names did not raise: the write was accepted and reported four fields written. But the row landed under a key the store had never seen, so looking the job up by name afterwards returned no row at all, and its status read undefined. The bookkeeping that tells an operator whether last night's billing succeeded quietly stopped describing any job that exists.
The arms that rename the values coming BACK from the scheduler - the started flag, the reason, the count of ticks fired - fail in the opposite direction. Every one read undefined, and the run reported a tick count of undefined out of five. Names flowing into a dependency fail silently; names flowing out of one fail loudly. That has been the pattern in every area this series has measured.
Two arms measured no change at all. One is an option pinned to a value identical to the library default on purpose, as a control - renaming it changes which name carries the value, and since the value is what the library would have used anyway, nothing moves. That prediction has now held for ten consecutive passes. The other is the context object the admission test reads, which this file writes and this file's own function consumes, so both sides moved together. A name your own code owns on both sides is genuinely safe - right up until it is serialised into something that leaves the process.
What to do about it
The mechanism is not specific to schedulers and it is not a defect in the obfuscator. Member renaming rewrites property names inside the code it is given. An installed scheduler is not inside that code, so a renamed option name is a name it has never heard of, and every mature library ignores what it does not recognise and applies its documented default. The defaults are the permissive ones, because a library that shipped strict defaults would break on installation.
The practical control is scoping. RenameMembers takes a MemberRegexp, and the options object handed to the scheduler belongs outside it, along with the shape of any record the scheduler stores on your behalf. If you would rather not maintain an exclusion list, build both with quoted string keys and read them with bracket access using literal strings you wrote.
The verification that catches this whole area is an assertion at start-up rather than a test. Ask the scheduler what timezone, overlap policy and misfire policy it is actually running under, compare them to the values you intended, and refuse to boot if they differ. Every arm in this article except the two silent ones is caught by four lines of that. Read the values from the scheduler, never from the object you wrote.
For the two that survive a start-up assertion - the bookkeeping row and the admission test - the check is behavioural. Have a test that runs a tick on a non-leader and asserts nothing happened, and have the job's own completion record be read back by name after it is written. A functional test that runs the job and sees correct output passes on every arm in this article, because on almost all of them the job does exactly what it was asked, at the wrong time, on the wrong machine, or twice.
Frequently asked questions
Does protecting my JavaScript break scheduled jobs on its own?
Not in this measurement. The sample was protected on five profiles covering both output targets, the gate profile and the compressed profile, and all five behaved identically to the unprotected file. Every result in this article required member renaming pointed at property names.
What was the worst result?
Two of them tie. Renaming the overlap option billed every customer twice, and renaming the admission test let a replica that should never start a billing run start one. Neither raised an error and both produced complete, well-formed output.
Why did losing the timezone matter if the job still ran?
Because the run moved four hours earlier, so the last four hours of a day's activity fell into the next day's run. There is no exception and no wrong total on the night itself - only a boundary in the wrong place, which shows up as a month-end discrepancy weeks later.
One of the options seemed unaffected. Was it safe?
No, it was unreachable. The catch-up cap sits behind the misfire policy, and the configured policy returns before the cap is read. A second job with the opposite policy was added so the cap could be measured, and on that job renaming it took a five-tick replay from two to five.
Why did the overlap check still appear to work in one arm?
Because the library's builtin admission test happens to refuse a tick that is already running, for its own reasons. The configured protection was gone; something else agreed with it in that one scenario. A check that passes for a different reason than the one you assume is not evidence that the check is present.
Which arms fail loudly?
The ones that rename values coming back from the scheduler. Those read undefined immediately and the run reports nonsense, so they surface in the first test. The dangerous direction is inward: a renamed option name is silently ignored and the library's default applies.
What is the smallest change that prevents all of this?
Scope RenameMembers with a MemberRegexp that excludes the scheduler's options object, then assert at start-up that the timezone, overlap policy and misfire policy the scheduler reports match what you configured. Read those values from the scheduler, not from your own object, or the assertion will pass in every arm.
Related reading