LLMs Covertly Rewrite African American Speech into Standard English: New Research

A new study on arXiv has revealed a large-scale problem: six modern LLMs (ranging from 14B to 70B parameters) systematically rewrite African American English (AAE) to standard American English (SAE), even when context requires preserving the dialect.

What bias was discovered

African American English is not an error, but a full-fledged dialect spoken by more than 30 million people in the United States. However, modern LLMs treat it as "incorrect" text that needs to be corrected to the standard variant.

The authors tested six instruction-tuned LLMs. In all cases, the models preferred SAE continuation even when the context clearly required AAE. Effectively, LLMs rewrite users' speech, automatically and invisibly replacing dialect with standard English.

• 6 instruction-tuned models (14B–70B parameters)
• 30+ million AAE speakers
• Negative concord ("ain't nobody") — universal bias trigger across all models
• Syntactic constructions consistently trigger rewrites

How it was discovered and measured

The authors developed a systematic audit method: the conditional Dialect Group Invariance (cDGI) indicator, which separates true model bias from external artifacts. Analysis at the feature level revealed which specific AAE markers cause bias most frequently.

For validation, researchers created REAL-AAE — a dataset of 17,479 AAE/SAE/AAE_back pairs from natural tweets. This is 2–6 times larger than all previous AAE resources. The dataset was validated automatically (BERTScore F1 = 0.95) and manually by three AAE speakers (83% semantic agreement). Such double validation guarantees that the dataset reflects real dialect nuances.

Activation steering: fixing without retraining

Researchers proposed activation steering — the first application of this method to dialect bias. This is a training-free method working only at test time, without any model weight changes.

How it works: causal tracing extracts dialect directions from model activations, then these directions are injected into layers responsible for generation. Result: the model preserves AAE in contexts where it is appropriate.

Results are impressive: activation steering reduces dialect bias by 5–20 times better than prompting approaches, while maintaining fluent standard English when it is truly needed. This proves that a solution is possible without full model retraining.

Why this is critical

Dialect bias is a form of hidden cultural assimilation. LLMs embedded in education systems, legal platforms, HR, and voice assistants can silently correct users' speech. A user enters text in their dialect, the system outputs it in standard English, erasing linguistic identity. The study shows that this can be detected and fixed with a method that does not require retraining.

Frequently asked questions

Why do LLMs rewrite AAE?

Because they are trained primarily on standard English texts. Models interpret AAE as a grammatical error that needs to be fixed in accordance with the dominant pattern in training data.

Is this a problem only for English?

This study focuses on AAE/SAE, but bias is typical for any dialect-standard language pair. Similar bias likely exists for other languages where minority dialects are underrepresented in training data.

When will commercial models get activation steering?

This is currently a research method. Integration into LLMs depends on providers. However, the study proves that a solution is technically possible without deep retraining.