ENH: Better DoRA check in mixed adapter batch inference #2089
+23
−2
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This is a bit of an edge case, but I noticed this while working on something else. PEFT allows mixed batch adapter inference, i.e. when predicting, the same batch can use different adapters by passing the adapter_names argument. However, this is not supported for DoRA (yet), so there is a check that raises an error if DoRA is used. Previously, this check would check all adapters for DoRA, even if those adapters are not being used in adapter_names. This was unnecessarily strict and with this PR, we only check the adapters that are actually being used.
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
sayakpaul
reviewed
Sep 23, 2024
src/peft/tuners/lora/layer.py
Outdated
| @@ -344,7 +344,8 @@ def _check_forward_args(self, x, *args, **kwargs): | |||
| msg = "Cannot pass `adapter_names` when there are merged adapters, please call `unmerge_adapter` first." | |||
| raise ValueError(msg) | |||
|
|
|||
| unique_adapters = set(self.active_adapters) | |||
| # DoRA is not supported (yet), check that it's not being used | |||
There was a problem hiding this comment.
Is there a need to log an information here?
There was a problem hiding this comment.
Not sure what. If DoRA is used by this layer, an error is raised, otherwise nothing further happens.
BenjaminBossan
commented
Sep 23, 2024
src/peft/tuners/lora/layer.py
Outdated
| @@ -344,7 +344,8 @@ def _check_forward_args(self, x, *args, **kwargs): | |||
| msg = "Cannot pass `adapter_names` when there are merged adapters, please call `unmerge_adapter` first." | |||
| raise ValueError(msg) | |||
|
|
|||
| unique_adapters = set(self.active_adapters) | |||
| # DoRA is not supported (yet), check that it's not being used | |||
There was a problem hiding this comment.
Not sure what. If DoRA is used by this layer, an error is raised, otherwise nothing further happens.
sayakpaul
approved these changes
Sep 24, 2024
BenjaminBossan
deleted the
enh-mixed-batch-inference-dont-raise-if-dora-not-included
branch
Guy-Bilitski
pushed a commit
to Guy-Bilitski/peft
that referenced
this pull request
May 13, 2025
…2089) This is a bit of an edge case, but I noticed this while working on something else. PEFT allows mixed batch adapter inference, i.e. when predicting, the same batch can use different adapters by passing the adapter_names argument. However, this is not supported for DoRA (yet), so there is a check that raises an error if DoRA is used. Previously, this check would check all adapters for DoRA, even if those adapters are not being used in adapter_names. This was unnecessarily strict and with this PR, we only check the adapters that are actually being used.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is a bit of an edge case, but I noticed this while working on something else.
PEFT allows mixed batch adapter inference, i.e. when predicting, the same batch can use different adapters by passing the
adapter_namesargument. However, this is not supported for DoRA (yet), so there is a check that raises an error if DoRA is used.Previously, this check would check all adapters for DoRA, even if those adapters are not being used in
adapter_names. This was unnecessarily strict and with this PR, we only check the adapters that are actually being used.