Parsing untrusted input
Published
A markup parser is the one dependency in a normal application whose defaults were chosen decades before anybody was feeding it hostile input. Entity resolution, external entity resolution and an unbounded expansion budget are all on unless a caller turns them off, so a parser configuration is almost entirely a list of subtractions. We protected a file that hands three hardened profiles to an installed parser, renamed the option names one group at a time, and read back what came off the disk.
What the sample actually does
The file builds three parser profiles, because a single hardened profile makes most of the guards unreachable. Refusing document type declarations outright short-circuits the entity budget, the external-entity switch and the resolve switch, and all three would then report no change for a reason that has nothing to do with renaming. The first version of this fixture did exactly that, and three of the six option arms were blind.
The profiles are an untrusted-upload profile that tolerates a document type declaration but substitutes nothing from it, a partner-feed profile that does substitute entities but with a budget of 64 and no disk access, and a public-form profile that refuses declarations altogether. Six documents run through them, arranged so that every guard fires exactly once in the unprotected run: the resolve switch on a document that names /etc/passwd, the depth cap on a 400-level document, the schema on a document carrying a language-specific tag, the external-entity switch on a document that names the process environment, the expansion budget on a nested entity bomb, and the declaration switch on the public form.
The parser is copied into the measurement directory unprotected. That is the whole design: an installed parser is a dependency, it is not rebuilt when you protect your own bundle, and it keeps looking up the property names it has always looked up. Its defaults are the permissive ones a real parser ships with - declarations allowed, entities resolved, external entities dereferenced, no expansion budget, no depth budget, and a schema that constructs language-specific tags.
Protection alone was applied first, on five presets covering both output targets and the compressed profile. All five produced behaviour identical to the unprotected file. Nothing in this article is caused by protection on its own; every result below required member renaming pointed at the option names.
One name turns a parser into a file-read primitive
Renaming externalEntities alone was the sharpest result in this area. The partner feed's document, which names file:///proc/self/environ, went from resolving that entity to an empty string to actually dereferencing it: feed-disk-reads moved from (none) to the path, and feed-secret-in-text moved from false to true. The environment variable we planted in that file, a database password, was sitting in the parsed output.
The detail worth keeping is what happened on the other profile at the same time. The untrusted-upload profile does not substitute entities at all - its resolve switch is off, and that switch was not renamed in this arm. It still read /etc/passwd off the disk. The entity table is built when the declaration is processed, before anything decides whether to substitute it, so the file was opened and its contents loaded into memory even though they never reached the parsed document.
That distinction matters when you are trying to detect this. If you look only at parser output you will see nothing on that profile, because the content genuinely does not appear there. What you would see is a process opening files it has no reason to open, which is visible in audit logs, in container security tooling and in latency if the path happens to be a network location rather than a local one. The read is the primitive; the output is only one way of getting the bytes back.
Renaming resolveEntities on its own is the milder half. The declared entity started being substituted - expansions went from zero to one - but because external entities were still refused it substituted to nothing, so no secret appeared. Rename both together and the upload profile joins the feed profile: upload-secret-in-text goes from false to true as well. Each half moves a number; the pair moves the file contents.
The expansion budget and the depth cap
The nested entity bomb in the sample is a small one by design: five levels of ten-fold nesting, which the partner-feed profile stops at 68 expansions and 563 bytes with a clear entity-budget refusal. Renaming maxEntityExpansions took it to 11,111 expansions and 100,013 bytes, and the refusal became an acceptance. A document of a couple of hundred bytes produced a hundred kilobytes of parsed text, and the budget that was there to stop it was simply not found.
Real bombs are written with more levels, and the arithmetic is the point of the attack: each level multiplies. The measured jump here is roughly 180-fold in bytes from removing one option name, on a document deliberately kept small enough to finish quickly. The mechanism does not have a ceiling that the parser imposes; the ceiling is the memory of the process.
Renaming maxDepth is the same shape with a different resource. The 400-level document went from a depth-limit refusal to being parsed. Deep nesting is the cheapest denial-of-service payload there is to write, because it needs no cleverness at all, and a depth cap is the only thing standing between a recursive-descent parser and its own stack.
Renaming allowDtd affects the strictest profile, the one in front of a public form that has no business declaring entities. Its refusal disappeared and the document was parsed. On its own that changes an error into a success rather than leaking anything, because that profile still substitutes nothing - but it removes the outermost of three layers, and the arms above are what the two inner layers were holding.
The schema arm, which is a different kind of problem
Renaming schema took the constructed-tag list from (none) to the language-specific tag our sample document carries. In the YAML world this is the difference between a schema that treats an unknown tag as inert text and one that constructs whatever the tag names, which is the mechanism behind every deserialization advisory that library has ever had.
This is not a size limit or a rate limit. It is the parser being asked to build objects a document chose, and the reason the safe schema exists is that document authors are not always the people you think they are. An application that pinned a safe schema and then lost the option name is running the permissive schema with no other change in behaviour, and the only visible difference is in what the parsed document contains.
Worth being precise about the boundary: our sample records that the tag was constructed, it does not execute a subprocess. What is measured is that the safe-schema decision was reversed by renaming one option name. What follows from that depends on your parser and the gadgets available in your process, which is exactly the analysis the safe schema exists to let you skip.
The control arm, and why it is not reassuring
One option was pinned to a value identical to the library default on purpose: boolean attributes are disallowed either way. Renaming it produced no change at all, on both output targets. That is the expected result, and across the whole pass every one of the six options pinned this way behaved the same, which is now the sixth consecutive time that prediction has held.
The caveat is the useful part. An audit asking "which of my options would change behaviour if they disappeared" is really asking "which of my options carry a value the library would not have chosen", and that is your exclusion list. It is a property of your configuration rather than of the parser, and it changes the day somebody edits a value to disagree with the default. An application that had deliberately allowed boolean attributes for a legacy feed would find this same rename breaking it.
The two directions, once more
Renaming the fields of the document objects the sample passes in - the declaration flag, the depth, the entity table, the body and the tag list - crashed the parser with a type error on the first document. That is the loud direction, and it is loud because the parser reaches into the structure it was handed and finds nothing there.
Renaming the fields of the result the parser hands back is the quiet direction and the more interesting one. Every guard still worked perfectly: the bomb was refused, the deep document was refused, the public form was refused. The application could not read any of it. bomb-refused, deep-refused and form-refused all reported undefined, and the reasons all reported (none).
So a parser that did its job correctly reports, one layer up, as a parser that found nothing wrong. If the calling code's next step is "if it was not refused, use it", three refusals have just been converted into three acceptances without a single error being raised. This is now the eleventh area in this series where the out direction degrades quietly while the in direction crashes.
What to do about it
Member renaming is off unless you turn it on, and when it is on it only touches names your expression matches. The fix is a scoped pattern, not avoiding protection. Exclude the option names your parser reads and the field names on anything it hands back.
Keeping parser configuration in a JSON file rather than an object literal removes the exposure entirely, because keys loaded from JSON are string data and string literals are not renamed. For parser hardening specifically that is a reasonable architectural choice anyway: it makes the security-relevant settings reviewable without reading code.
Whatever you choose, test the decision rather than the presence. A test that feeds your parser a document declaring an external entity and asserts that no file was opened catches every arm in this article. A test that asserts the parser is installed catches none of them. If you can afford one integration test in this area, make it the entity one - it is the arm that reached the disk.
Frequently asked questions
Does obfuscation change how an XML or YAML parser behaves?
Not by itself. Protection alone, on all five presets we tested including both output targets and the compressed profile, produced behaviour identical to the unprotected file. Everything in this article required member renaming pointed at the option names the parser reads.
Why does a lost parser option make the parser less strict rather than more?
Because a renamed key is indistinguishable from a key you never supplied, and the parser then applies its own default. Markup parser defaults predate hostile input as a design consideration, so entity resolution, external entity resolution and unbounded expansion are all on unless a caller turns them off.
Which parser option was the most dangerous to rename in your measurement?
The external-entity switch. It took a document naming a file path from resolving to an empty string to actually reading that file, and the planted secret appeared in the parsed output. On a second profile the file was read off disk even though entity substitution was still disabled there, because the entity table is built before anything decides whether to substitute it.
Would the file read show up anywhere if the contents never reached the output?
Not in parser output, which is the point worth planning around. It shows up as a process opening paths it has no reason to open, so audit logging, container security tooling and unexpected latency on network paths are where you would find it rather than in the parsed document.
How much did losing the expansion budget change?
On a deliberately small nested-entity document, from 68 expansions and 563 bytes to 11,111 expansions and 100,013 bytes, with the refusal becoming an acceptance. The multiplication is the attack, so a document written to be hostile rather than to finish quickly would go considerably further.
What happens if the parser's result object gets renamed instead?
The guards still work and the application stops being able to read the answer. In our run the entity bomb, the deep document and the public form were all still refused, while the calling code read undefined for every refusal flag and an empty reason for every reason. Nothing threw.
What should I exclude from member renaming here?
The option names your parser reads, the field names on any document or node structure you hand it, and the property names on the result. In our sample that was allowDtd, resolveEntities, externalEntities, maxEntityExpansions, maxDepth and schema, plus the refused and reason fields on the result.
Related reading