Hugging Face trains harness adapters to optimize Qwen3.8-27B performance

Hugging Face published a recipe on November 20, 2026, demonstrating that reinforcement learning (RL) can produce harness-specific LoRA adapters that improve both accuracy and efficiency for Qwen3.8-27B when run inside real agent harnesses.

Training inside real harnesses

The team trained one LoRA adapter per harness — pi, Hermes Agent and OpenCode — using TRL’s AsyncGRPOTrainer, OpenEnv and Harbor. Training and rollouts ran on the Hugging Face Hub. The target harness versions were Pi 1.5.2, OpenCode 2.2.7 and Hermes Agent v0.22.6, with vLLM used as a proxy to capture exact tokens produced by each harness.

Hugging Face reported a concrete example: Qwen3.8-27B scored 39% on Terminal-Bench 2.1 under the pi harness while using about 2.9K tokens per task. Freezing the model and adding a harness-specific RL-trained LoRA adapter raised pass@1 to 55% and reduced average token usage to 1.5K, driven largely by less reasoning and fewer tool calls.

Data pipeline, validation and training

The dataset combined three sources: SWE-smith (injected bugs), SWE-Gym (TaskTrove), and Tasksmith-generated tasks from merged pull requests. From roughly 1,127 Hub candidates, 858 passed openenv validate; the pipeline produced a dataset view of 871 accepted tasks (35 repositories) and a qualified subset of 171 tasks for GRPO training.

Training choices included a small LR sweep (1e-5, 5e-5, 1e-4), with Hugging Face selecting 2e-5 for final runs after observing instability at higher rates. Each harness adapter was trained on NVIDIA 2xB200 GPUs (one GPU for vLLM, one for training). The final data mixture used 750 tasks from harness-adapter RL environments and 300 tasks from Termgrade.

Safeagents was used to block misbehavior during training; Hugging Face reported a steady decline in blocked tool calls as training progressed.

Adapters were merged into a single “super-adapter” by concatenating the three LoRAs (rank 96). Cosine similarity between adapters was about 0.01–0.02. Hugging Face evaluated scaling factors 1/3, 1/2 and 1 and found the 1/2 weighting provided the best balance across harnesses; the full sum shortened agents and over-applied updates (13.4 tool calls per task in one trial, with Hermes reaching 62.8% and pi dropping back).

All artifacts — adapters, GGUFs, RL tasks and the safeagents plugin — are available through the Hugging Face collection accompanying the publication.


Original source: Hugging Face Blog

Leave a Comment