Why language models break document structure when you trust them with editing

Editing documents with a language model appears to be an obvious solution: insert text, give an instruction, get a result without hours of manual work. In practice, however, the returned document often turns out to be different — with shifted headings, missing paragraphs, or reworded fragments that no one asked to touch. Instead of editing, invisible degradation of structure occurs.

Generation versus editing

The key contradiction lies in the architecture of transformers themselves. Language models are trained to predict the next token based on context — they do not "edit" in the sense that a text processor does. When a model receives the instruction "fix the grammar in the third section," it does not apply a precise rule to lines — it generates what it believes the updated text should look like in its entirety. The boundary "edit only this" is fundamentally blurred for an LLM: the model evaluates the entire context and generates a new version rather than applying atomic edits. This is especially noticeable in large documents — the more "material" there is, the stronger the drift from the source.

The lost-in-the-middle phenomenon

Even models with a context window of 200k tokens create structural problems with long documents. Research consistently identifies the "lost in the middle" effect: models retain information well from the beginning and end of context, but systematically lose details from its middle. The longer the document, the stronger the effect. For real files, this means that structural elements in middle sections are most likely to be corrupted. Typical casualties:

• Nested lists — nesting levels flatten to one
• Tables — lose column alignment or convert to prose
• Cross-references — turn into plain text without anchors
• YAML frontmatter and custom tags — are deleted as "unnecessary clutter"
• Section numbering — gets disrupted after any insertion or deletion of content

Formatting as tokens without semantics

Markdown, HTML, LaTeX — the model sees all of this as ordinary tokens, not as syntax with rules. To a neural network, the symbols `##` are simply two hash marks, not "a second-level heading aligned with the table of contents." Formatting patterns are reproduced by statistical analogy with training data, not by explicit syntactic rules. The result is predictable: inconsistent indentation, jumbled heading levels, broken link anchors, random switching between formats within a single document. Each of these violations is barely noticeable on its own, but together they render the document unusable for automated processing or publication.

"The model does not understand your document — it understands the

probability distribution of the next token in its context."

Competition between instruction and patterns

Another factor is the competition between the user's explicit instruction and the statistical patterns learned during training. If standard Markdown predominated in the training data, the model will be drawn to it even against explicit prohibition. Non-standard corporate templates, specific technical standard markup, author-specific structural style — all of this is vulnerable to the "memory" of the training corpus. Moreover, long instructions "blur" as generation progresses. By the end of a large document, attention to constraints from the original prompt weakens — and structural drift accumulates.

What this means

For practical work, the conclusion is straightforward: LLMs are better at point tasks than at comprehensive editing of an entire document in a single pass. For complex structured files — technical specifications, legal contracts, academic papers with cross-references — it is worth breaking the document into isolated sections and working with each separately. Instructions should be as explicit as possible: "change only this paragraph, leave everything else alone." After any LLM editing, explicit verification of structural elements is required — these are the first to suffer.