Repository navigation

Issues

New issue

[Tracking] Move off liger-kernel and make the memory-efficient loss the default
#7063 · qgallouedec opened on Sep 4, 2026
8
[Tracking] TRL on AWS Trainium (Neuron)
#7658 · qgallouedec opened on Oct 13, 2026
[Tracking] Promote AsyncGRPO to the stable API
#7841 · qgallouedec opened on Oct 20, 2026
2

[Tracking] Update the DeepSeek-V3 and DeepSeek-R1-Distill Hub chat templates for transformers 5.23
huggingface/trl#8280 · opened on Nov 12, 2026
1
SFT with assistant_only_loss trains the generation prompt's \n on Qwen2.5
huggingface/trl#8278 · opened on Nov 12, 2026
CI fails for chat template drift check: meta-models/Muse-Spark-Open-816B-A42B ships a chat template not stored in trl/chat_templates/
huggingface/trl#8273 · opened on Nov 12, 2026
1
CI fails for chat template drift check: HuggingFaceTB/SmolVLM2-2.2B-Instruct ships a chat template not stored in trl/chat_templates/
huggingface/trl#8270 · opened on Nov 12, 2026
1
The tiny model revision overrides in tests/conftest.py don't reach the scripts that tests call in-process, which pass revision="main"
huggingface/trl#8263 · opened on Nov 12, 2026
CI fails with latest vLLM for GRPO structured outputs: AssertionError: assert 0.6377995014190674 < 0.1
huggingface/trl#8260 · opened on Nov 12, 2026
1
CI fails for latest TRL release with dev dependencies for GPT-OSS and DeepSeek-V3: ValueError: You cannot perform fine-tuning on purely quantized models
huggingface/trl#8259 · opened on Nov 12, 2026
1
CI fails for latest TRL release with dev dependencies for FLOPs without head_dim: AssertionError: assert 10456320 == 19900416
huggingface/trl#8258 · opened on Nov 12, 2026
1
GOLDTrainer: no pad_token fallback for plain tokenizers
huggingface/trl#8255 · opened on Nov 11, 2026
KTOTrainer: train_sampling_strategy="group_by_length" always fails (no input_ids column)
huggingface/trl#8254 · opened on Nov 11, 2026
DistillationTrainer + FSDP2: teacher stays on CPU, step 1 fails with "Expected all tensors to be on the same device"
huggingface/trl#8253 · opened on Nov 11, 2026
1
SFT evaluation fails when compute_loss returns a tensor requiring gradients
huggingface/trl#8246 · opened on Nov 10, 2026
[Tracking] Support more model types in compute_flops_per_token
huggingface/trl#8237 · opened on Nov 10, 2026
2
Regenerating a trl-internal-testing tiny model in place changes it for every repo that loads it
huggingface/trl#8234 · opened on Nov 10, 2026
Support Nemotron-H in AsyncGRPO
huggingface/trl#8233 · opened on Nov 10, 2026
Support Granite 4.0 hybrid models in AsyncGRPO
huggingface/trl#8232 · opened on Nov 10, 2026
Support GPT-OSS in AsyncGRPO
huggingface/trl#8231 · opened on Nov 10, 2026
DistillationTrainer: the Python API ignores teacher_model_name_or_path
huggingface/trl#8213 · opened on Nov 10, 2026
2
AsyncGRPO: prefix cache keeps KV from old weights across weight syncs
huggingface/trl#8212 · opened on Nov 10, 2026
2
[Tracking] Support tokenizers 1.0
huggingface/trl#8203 · opened on Nov 9, 2026
SFT with assistant_only_loss trains the pre-filled thinking block on chat templates shipped with generation tags (Qwen3.6, Qwen3.8)
huggingface/trl#8182 · opened on Nov 9, 2026
3
AsyncGRPO with top_k/top_p crashes on vLLM >= 0.32
huggingface/trl#8178 · opened on Nov 8, 2026
3
GRPOTrainer hangs in multi-process training when tool responses return images on some ranks but not others (log_multimodal gather is rank-conditional)
huggingface/trl#8177 · opened on Nov 8, 2026
1
DPOTrainer: SFT loss weighting changes with gradient accumulation for unequal-length completions
huggingface/trl#8176 · opened on Nov 8, 2026
1
Add UDPO and URM losses for noisy preferences (ICML 2026)
huggingface/trl#8175 · opened on Nov 8, 2026
1

Navigation Menu

Repository navigation

Issues

Issues

[Tracking] Update the DeepSeek-V3 and DeepSeek-R1-Distill Hub chat templates for transformers 5.23

SFT with assistant_only_loss trains the generation prompt's `\n` on Qwen2.5

CI fails for chat template drift check: meta-models/Muse-Spark-Open-816B-A42B ships a chat template not stored in trl/chat_templates/

CI fails for chat template drift check: HuggingFaceTB/SmolVLM2-2.2B-Instruct ships a chat template not stored in trl/chat_templates/

The tiny model revision overrides in tests/conftest.py don't reach the scripts that tests call in-process, which pass revision="main"

CI fails with latest vLLM for GRPO structured outputs: AssertionError: assert 0.6377995014190674 < 0.1

CI fails for latest TRL release with dev dependencies for GPT-OSS and DeepSeek-V3: ValueError: You cannot perform fine-tuning on purely quantized models

CI fails for latest TRL release with dev dependencies for FLOPs without head_dim: AssertionError: assert 10456320 == 19900416

GOLDTrainer: no pad_token fallback for plain tokenizers

KTOTrainer: train_sampling_strategy="group_by_length" always fails (no input_ids column)

DistillationTrainer + FSDP2: teacher stays on CPU, step 1 fails with "Expected all tensors to be on the same device"

SFT evaluation fails when compute_loss returns a tensor requiring gradients

[Tracking] Support more model types in `compute_flops_per_token`

Regenerating a trl-internal-testing tiny model in place changes it for every repo that loads it

Support Nemotron-H in AsyncGRPO

Support Granite 4.0 hybrid models in AsyncGRPO

Support GPT-OSS in AsyncGRPO

DistillationTrainer: the Python API ignores teacher_model_name_or_path

AsyncGRPO: prefix cache keeps KV from old weights across weight syncs

[Tracking] Support tokenizers 1.0

SFT with assistant_only_loss trains the pre-filled thinking block on chat templates shipped with generation tags (Qwen3.6, Qwen3.8)

AsyncGRPO with top_k/top_p crashes on vLLM >= 0.32

GRPOTrainer hangs in multi-process training when tool responses return images on some ranks but not others (log_multimodal gather is rank-conditional)

DPOTrainer: SFT loss weighting changes with gradient accumulation for unequal-length completions

Add UDPO and URM losses for noisy preferences (ICML 2026)