Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix dtype parsing #97

Merged
merged 2 commits into from
Jun 18, 2023
Merged

Fix dtype parsing #97

merged 2 commits into from
Jun 18, 2023

Conversation

minosvasilias
Copy link
Contributor

@minosvasilias minosvasilias commented Jun 16, 2023 •
edited

This fixes the issue noted in #94 , as well as other --dtype arguments that may not have been parsed correctly.

  • Ignore dtype model arg if 8bit specified as --dtype argument (only set load_in_8bit instead)
  • Use getattr to parse string value (turning "float16" into torch.float16 etc.)

@minosvasilias
Fix dtype parsing
5328e23 
- Ignore dtype model arg if 8bit specified
- Use getattr to parse string argument
@minosvasilias minosvasilias mentioned this pull request Jun 16, 2023
Closed
@minosvasilias
Copy link
Contributor Author

I also added support for the load_in_4bit argument introduced recently: huggingface/transformers#23479

@lbeurerkellner
Copy link
Collaborator

Awesome, thanks a lot.

@lbeurerkellner lbeurerkellner merged commit 6aca8e5 into eth-sri:main Jun 18, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
2 participants
@minosvasilias @lbeurerkellner