Compatibility
Published
Text encoding is the layer nobody looks at until it produces a replacement character in front of a customer. It is also, unusually, a place where the difference between correct and incorrect output is a single option in a dictionary -- fatal decides whether malformed input throws or is quietly patched, and stream decides whether a character split across two chunks is reassembled or destroyed. Both are names, and names are rename sites.
What was measured
The sample encodes a string containing an accented character and an em dash, checks the byte length, and uses encodeInto to write into a preallocated buffer and read back the result object the platform returns. It then decodes deliberately malformed bytes through both a validating decoder and a permissive one, decodes a byte order mark with the option to keep it and the option to drop it, and finally decodes a three-byte character split across two chunks using the streaming option.
That last one is the case that matters in production. A UTF-8 character can span up to four bytes, and a network or file read can end in the middle of one. The stream option is what tells the decoder to hold the incomplete tail and finish it on the next call. Without it, each call is treated as a complete unit and the fragments are replaced.
Protected in five configurations -- the default target, the modern target, both gate profiles and the string-transform profile -- every line matched. The encoded length stayed twelve bytes. The validating decoder still threw on malformed input and the permissive one still produced replacement characters. The byte order mark was still preserved by one decoder and stripped by the other. The split character was still reassembled correctly across the two streamed calls.
That result covers the case people most often worry about, which is whether the string transforms disturb non-ASCII content. They do not. Moving and encoding a string literal changes where it is stored, not what it decodes to, and the sample deliberately contains characters outside ASCII to make that concrete.
A strict decoder that quietly becomes permissive
The member column opens with the most consequential result. A pattern matching fatal and ignoreBOM produced a protected build that ran to completion, and in which a decoder configured to reject malformed input accepted it instead.
Measured: where the source printed that the strict decoder threw, the protected build printed that it did not. The platform looked for a property called fatal in the options dictionary, found a generated name, and applied the default, which is permissive. Malformed bytes that should have raised a TypeError were silently replaced with the replacement character and handed back as an ordinary string.
That inversion is worth dwelling on because of who chooses fatal in the first place. Nobody sets it by accident. It is set precisely when silently mangled text is unacceptable -- decoding a signed payload, a file the user will get back, a record that will be stored and later compared. Turning it off does not produce an error; it produces data that is subtly and permanently wrong, and the code that would have caught it has been disabled.
The same pattern dropped ignoreBOM, and the byte order mark that the source preserved was stripped instead. Reading the flags back confirmed the cause directly: fatal and ignoreBOM both read undefined on the constructed decoder, because the code was asking a platform-built object for properties under generated names.
A character destroyed between two chunks
The streaming arm is the one that will look most familiar to anyone who has debugged a text pipeline. A pattern matching a single name, stream, corrupted a character that the unprotected build reassembled correctly.
The sample splits a three-byte em dash after its second byte. In the source, the first call returns an empty string -- the decoder is holding an incomplete sequence -- and the second call returns the complete character. In the protected build both calls returned a replacement character: the first because the trailing fragment was treated as a complete and invalid unit, and the second because the remaining byte was orphaned.
The failure has the specific character of all chunk-boundary bugs. It depends on where the boundary falls, so it is invisible for ASCII input, invisible for short inputs that arrive in one piece, and invisible in tests that decode a whole string at once. It appears when a response is large enough to be split, and it appears at a different place in the text every time.
The encodeInto arm fails more visibly. The platform returns a result object with fields named read and written, and a pattern matching those names produced read=undefined written=undefined. Since the normal use of those numbers is to advance an offset into a buffer, undefined arithmetic turns into NaN immediately, which is the kind of failure that surfaces quickly.
What the string transforms do and do not touch
It is worth separating two things that are easy to conflate, because this is the article where they meet. The string transforms move literals into a table and encode them, and a reader seeing an encoded string table sometimes concludes that text handling is at risk. This measurement says otherwise, on non-ASCII input, in five configurations.
The reason is that the transform is lossless with respect to the value. A literal is stored differently and decoded back to the same sequence of code units before anything uses it. That is why the em dash, the accented character and the byte order mark all survived the string-transform profile unchanged.
What is at risk is not the values but the names, and encoding is a domain unusually rich in names that look like configuration and are contracts. fatal, ignoreBOM, stream, read and written are all read or written by the platform. A label such as 'utf-8' is a value and is safe; the key it sits next to is not.
The control arm confirms the boundary: a record of the sample's own with label and size fields measured identical on both targets, and the platform method names failed loudly, TypeError: enc._0x1 is not a function on the first call. Loud is the good case here, as everywhere in this series.
Where this shows up in real pipelines
Three places, in rough order of how often they appear. The first is any code that reads a response body incrementally rather than calling a convenience method that buffers the whole thing -- a progress indicator, a large download, a server-sent event stream, a log tail. All of those decode in chunks, and all of them depend on stream.
The second is binary protocol work. Code that mixes typed arrays and text -- reading a length prefix, decoding a field, moving an offset -- uses encodeInto and its result fields precisely because it is trying to avoid allocation. That is the arm where undefined offsets turn into NaN and the parse goes wrong immediately.
The third is validation. Anywhere a decoder is constructed with fatal set, someone decided that malformed input must be rejected rather than patched. That decision is expressed entirely as a property name, and it is silently reversible by a member pattern that was written with a completely different part of the application in mind.
None of this argues against member renaming. It argues for scoping it, which is the same conclusion every article in this series reaches, and which is cheap to apply once you know which names belong to the platform rather than to you.
How to check your own build
Two assertions cover the practical risk, and both run without any I/O. Decode a deliberately malformed byte sequence through a decoder constructed with fatal set, and assert that it throws. If it returns a string instead, the option never reached the platform and every validating decoder in the build has become permissive.
Then take any multi-byte character, split its bytes across two arrays, decode them with the streaming option, and assert that the concatenated result equals the original character. If either call returns a replacement character, the streaming option was renamed and chunked decoding is corrupting text at every boundary.
If you want a single diagnostic rather than a pair of assertions, construct the decoder and log it serialised as JSON. The flags the platform reports are the ones it actually applied, so a decoder that reports permissive behaviour when your source asked for validation identifies the cause immediately.
As with the rest of this series, the mitigation is to anchor the member pattern to names your own application owns. Encoding labels and the text itself are values and are never at risk. The option keys and the platform's result fields are names, and those are the ones to keep out of the pattern.
Frequently asked questions
Does obfuscation break TextEncoder and TextDecoder?
Not on its own. Encoding an accented character and an em dash, decoding malformed bytes through both a validating and a permissive decoder, preserving and stripping a byte order mark, and reassembling a character split across two streamed chunks all produced identical output across five protection configurations.
Do the string transforms corrupt non-ASCII text?
No. The sample was written with an accented character, an em dash and a byte order mark specifically to test that, and the string-transform profile produced identical output. Moving a literal into a table and encoding it changes where the value is stored, not what it decodes to.
Why does my decoder stop rejecting malformed input after protection?
Because member renaming reached the fatal key of the options dictionary. The platform finds no property by that name and applies its default, which is permissive, so malformed bytes are replaced rather than throwing. Reading the flags back on the decoder returns undefined, which confirms the cause.
Why do I get replacement characters only on large responses?
That is the signature of a renamed stream option. A character whose bytes are split across two chunks is reassembled only when the decoder is told the input is a stream. Rename that key and each chunk is treated as complete, so the fragments become replacement characters. Short inputs that arrive in one piece are unaffected, which is why it looks size-dependent.
Are the read and written fields from encodeInto affected?
Yes, if your member pattern matches them. The platform builds that result object with those names, so renaming them leaves your code reading undefined. Because those numbers are normally used to advance a buffer offset, the arithmetic becomes NaN quickly, which makes this one of the faster failures to notice.
What is the cheapest check that my text pipeline is unchanged?
Two assertions with no I/O. Decode malformed bytes through a decoder constructed with fatal set and assert that it throws. Then split a multi-byte character across two arrays, decode both with the streaming option, and assert the result equals the original character. Those cover validation and chunk boundaries, which are the two failures measured here.
Related reading