Skip to content

Issues: huggingface/trl

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

47 Open 895 Closed
47 Open 895 Closed
Author
Filter by author
Label
Filter by label
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Milestones
Filter by milestone
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

FSDP Must flatten tensors with uniform dtype but got torch.bfloat16 and torch.float32
#1723 opened Jun 11, 2024 by qZhang88
1
KTO train_loss = 0.0
#1722 opened Jun 10, 2024 by yaoxiao1999
1
Distributed fine-tuning using KTO, QLorA and FSDP throws ValueError You cannot perform fine-tuning on purely quantized models...
#1719 opened Jun 9, 2024 by nubloso
Problem handling of response masks
#1717 opened Jun 9, 2024 by mertsayar8
1
trl CLI doesn't work
#1716 opened Jun 7, 2024 by nikhil-tensorwave
1
Feature Request: Self-Improving Robust Preference Optimization (SRPO) enhancement New feature or request good second issue
#1714 opened Jun 7, 2024 by duyvuleo
Got an abnormally high loss when training Gemma-7B.
#1709 opened Jun 6, 2024 by smartliuhw
6
Feature: IterableDataset support for SFTTrainer
#1695 opened Jun 5, 2024 by yash4242
1
stf Example not working
#1693 opened Jun 5, 2024 by TATOAO
1
RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:7 and cpu! (values = values * mask)
#1691 opened Jun 3, 2024 by lzl65825
1
vsft_llava: ValueError: Expected input batch_size (78528) to match target batch_size (41728).
#1685 opened May 31, 2024 by kishan-character
1
SFTrainer with FSDP on a model that doens't fit in GPU memory
#1681 opened May 29, 2024 by tambulkar
1
Why compute IPO loss using average_log_prob=Ture？
#1677 opened May 28, 2024 by AIR-hl
2
ppov2 & rloo trainer doesn't correctly compute reward if reward tokenizer is different from policy tokenizer
#1674 opened May 28, 2024 by TheBlackCat22
3
Path for support generative eval in DPOTrainer
#1671 opened May 27, 2024 by prompteus
1
ImportError: cannot import name 'SFTScriptArguments' from 'trl.commands.cli_utils' & --report_to flag
#1669 opened May 26, 2024 by alielfilali01
1
Llama 3 Unsloth Fixes
#1668 opened May 26, 2024 by lhl
OOM with DPO Trainer on A100 GPU
#1667 opened May 25, 2024 by JhonDan1999
1
FineTuning issue with Gemma-2B-IT model using the SFTTrainer
#1665 opened May 23, 2024 by AvisP
3
Can we use SFTTrainer for pre-training?
#1657 opened May 23, 2024 by wennycooper
6
DPOTrainer deepspeed.initialize cause ref_model not fixed
#1652 opened May 22, 2024 by shuoYan97
2
how to save v_head
#1650 opened May 20, 2024 by zyzhang1130
2
Adapter name for SFT trainer
#1649 opened May 18, 2024 by para-zhou
1
Set seed
#1648 opened May 17, 2024 by user799595
1
How to save and resume a checkpoint from PPOTrainer
#1643 opened May 14, 2024 by paraGONG
ProTip! Type g i on any issue or pull request to go back to the issue listing page.