Structured LLM Pruning Without Accuracy Loss: Method Verified on Llama-3-8B and Vicuna

In July 2026, a new method of structural pruning of large language models was published on arXiv: researchers described three systemic issues when transferring AFR technique to structural pruning and proposed a unified approach combining power transformation, sign-preserving aggregation and percentile clipping of outliers. Experiments on Llama-3-8B, Vicuna-v1.5-13B and LLaVA-v1.5-13B showed accuracy on par with unstructured pruning while achieving real inference speedup.

Why is structural pruning better for deployment

Pruning is one of the basic methods of neural network compression: insignificant parameters are removed, the model becomes smaller and faster. There are two fundamentally different approaches.

Unstructured pruning zeros out individual weights, resulting in sparse matrices. Accuracy suffers less, but this provides almost no practical speedup on standard GPUs — specialized kernels or hardware unavailable to most teams are needed.

Structural pruning removes entire neurons, attention heads or layers. The resulting model of standard density runs faster on regular hardware without additional tricks. The problem is that quality traditionally fell short of unstructured methods. The authors of this paper tackled exactly this gap.

Three problems in adapting AFR

AFR (Adaptive Feature Retention) is an unstructured method that preserves "important" weights taking into account gradient information. The authors attempted to transfer it to the structural level and encountered three obstacles:

• Distribution mismatch: importance estimates from different layers have incomparable scales — direct averaging gives distorted results
• Loss of sign information: the sign of the estimate shows whether the direction of weight update matches the gradient of optimization; most aggregation methods lose this sign
• Effect of outliers: extreme values dominate aggregation and force deletion of structures not based on actual importance

Each of these problems individually reduces final quality; together they make direct transfer of AFR unworkable.

How the proposed solution works

The authors proposed three techniques applied sequentially in a unified aggregation scheme.

Power transformation equalizes distributions of estimates in a nonlinear way. Linear scaling poorly handles heavy tails characteristic of transformer activations. A power function with tunable exponent makes estimates from different sources comparable.

Sign-preserving aggregation combines transformed estimates without losing directional information. If a neuron receives an estimate with a certain sign, this is taken into account when selecting candidates for deletion — instead of simply averaging absolute values.

Percentile clipping of outliers removes extreme values before aggregation. The threshold is set in percentiles, not absolute numbers, making the method robust to architecture or task changes.

Results on Llama, Vicuna and LLaVA

The method was tested on three open models of different classes:

• Llama-3-8B — accuracy after structural pruning is comparable to the unstructured AFR version
• Vicuna-v1.5-13B — dialogue model maintains answer quality while reducing parameter count
• LLaVA-v1.5-13B — multimodal model also showed no significant degradation

Authors emphasize "practical inference speedup": compressed models run faster on standard GPUs without special libraries for sparse matrices. This is what makes structural pruning desirable for production deployments, where computational costs are calculated directly.

What this means

The work closes a specific technical gap: adapting unstructured importance metrics to structural pruning previously required a compromise on accuracy or expensive post-pruning fine-tuning. Three targeted fixes — distribution equalization, sign preservation, percentile outlier removal — allow simultaneous accuracy preservation and real speedup without post-pruning tricks.