Fine-tuning Gemma-3 on Mathematics Using GRPO and LoRA Adapters

Researchers published on July 5, 2026, a detailed workflow for fine-tuning Gemma-3 from Google on the mathematical dataset GSM8K using the GRPO algorithm, LoRA adapters, and the Tunix framework.

What is GRPO and how does it differ from classical methods

GRPO (Group Relative Policy Optimization) is an algorithm from the reinforcement learning family, specially adapted for language models. Unlike classical PPO, which requires a separate reference critic model, GRPO compares multiple generated answer variants within a single group and updates the policy in favor of more successful solutions. This reduces memory consumption and makes training more stable.

For mathematical tasks, this approach is particularly convenient: the answer is either correct or not — the reward signal is computed automatically and unambiguously. GSM8K (Grade School Math 8K) is a standard benchmark with 8,500 school mathematics problems requiring multi-step reasoning. It requires the model not just to output a number, but to reproduce a chain of reasoning, making it an ideal testing ground for GRPO.

• Base model — Gemma-3 from Google
• Dataset — GSM8K (8,500 school problems)
• Algorithm — GRPO (Group Relative Policy Optimization)
• Framework — Tunix on top of Hugging Face Transformers
• Fine-tuning — LoRA adapters without changing base weights

How the pipeline is built step by step

The workflow is divided into logical stages. First, the environment is configured and authentication is performed on the Hugging Face Hub — this is where Gemma-3 weights are loaded from. Then each example from GSM8K is wrapped in a "reasoning + answer" template: the model sees the expected output structure from the first training steps and learns to follow it.

The central element consists of two reward functions. The first checks format compliance: are there reasoning blocks and numerical results in the answer? The second evaluates numerical accuracy — does the final number match the benchmark? GRPO optimizes both signals simultaneously, generating groups of variants and selecting the best ones.

To avoid fine-tuning the entire model, LoRA adapters are connected to Gemma-3 — compact low-rank matrices that are trained instead of "frozen" base weights. The result: the number of trainable parameters drops sharply, and video memory requirements decrease so much that the workflow becomes implementable on consumer hardware.

After training, the authors suggest measuring accuracy with adapters and comparing it to the base checkpoint. Optionally, merge adapters with base weights and export the final model for deployment.

Why this matters for developers

"LoRA adapters make the entire process feasible without a data

center," the workflow description states.

Just a few years ago, fine-tuning large models required tens of gigabytes of video memory and access to professional clusters. The GRPO + LoRA combination changes this: the algorithm is already more memory-efficient than classical PPO, and LoRA additionally reduces the number of trainable parameters 10–100x depending on adapter rank.

Practical consequence: a developer with one modern graphics card can take an open model like Gemma-3, tune it for a specific domain — mathematics, law, medicine — and get a specialized model with measurable improvement on the target benchmark. Tunix serves as a convenient wrapper that automates GRPO configuration and adapter management; a similar approach is applicable to any dataset with verifiable answers.

What this means

GRPO fine-tuning with LoRA on domain datasets is moving from academic papers into practical guides. The authors showed the complete cycle — from loading to export — on a real model and public benchmark, lowering the entry barrier for developers who want to improve an open model for a specific task without expensive infrastructure.