Neural networks are bad calculators: why LLMs fail at arithmetic and what to do about it

A neural network fails even at simple math, confidently producing an incorrect result for multiplying six-digit numbers—correct digits on the edges, error in the middle. This is not a bug or shortcoming of a particular model: a transformer predicts tokens based on statistics, not executing a multiplication algorithm as a calculator would.

Token versus algorithm: what's the difference

Column multiplication is a procedure with digits, carries, intermediate sums, and strict order of operations. A processor executes it. A schoolchild executes it. A transformer in one pass—no: it generates text resembling the right answer, based on statistics from training data.

Short arithmetic works exactly like memory, not computation:

• Multiplication tables appear in training data millions of times
• "7 × 8 = 56" the model does not calculate—it remembers
• With two or three digit numbers, familiar patterns are still sufficient
• With six-digit numbers—there are almost no familiar examples; the model starts to "continue plausibly"
• Confidence when giving an answer is unrelated to accuracy of the result

Why does the error end up in the middle of the number?

The outer digits of the result can be derived from the first and last digits of the factors—this is a pattern the model learns. Middle digits require an honest chain of carries, which a transformer does not construct in one autoregressive pass.

Characteristic handwriting: the right edge (units, tens) and left edge (high-order digits) are often correct, middle digits—invented. Meanwhile, the model has no internal signal "I am not confident"—the token predictor simply continues the sequence as plausibly as possible.

How does the industry solve this

The solution came not from within the architecture, but from outside: tools. Code Interpreter in ChatGPT, Python blocks in Claude, calculator calls via function calling—the model delegates computation to someone who knows how to compute.

"Does it, and quite successfully, just not where it seems."

Progress is not in teaching LLM to calculate independently, but in teaching it to call the right tool.

Chain-of-thought (explicit recording of intermediate steps) partially helps: accuracy on multi-digit numbers increases. But each token in the chain is itself a prediction, so this technique does not provide 100% reliability.

In parallel, research continues on specialized fine-tuning for mathematics—models trained predominantly on scientific texts better understand tasks. But even they do not replace a calculator: the value of such models is in correctly posing the task for a tool, not in self-computation.

What this means

Expecting LLM to behave as a calculator is an architectural misunderstanding. The correct pattern: the model understands the task, constructs logic, delegates numbers to a tool, interprets the output. Trusting unit economics directly in a chat without tools—means checking not the model, but luck.

Frequently asked questions

Why doesn't the model warn that it cannot calculate?

A language model has no internal signal of uncertainty when generating a number. It outputs the next token with maximum probability—and with equal confidence outputs both correct and incorrect results. This is a fundamental property of autoregressive generation, not a code error.

Does chain-of-thought help with complex arithmetic?

Yes, but limitedly. Step-by-step recording of carries increases accuracy because the model "sees" intermediate results as context for the next prediction. However, each step remains a prediction, and errors accumulate—for critical calculations, an external tool is needed.

Why does a neural network fail at math?

A transformer predicts tokens based on statistics from training data, not executing a mathematical algorithm like a processor or human does.