Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

improve llamacpp embeddings #12972

Merged

Conversation

streamnsight
Copy link
Contributor

  • Description:
    Improve llamacpp embedding class by adding the device parameter so it can be passed to the model and used with gpu, cpu or Apple metal (mps).
    Improve performance by making use of the bulk client api to compute embeddings in batches.

  • Dependencies: none

  • Tag maintainer:
    @hwchase17

Copy link

vercel bot commented Nov 7, 2023

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Skipped Deployment
Name Status Preview Comments Updated (UTC)
langchain ⬜️ Ignored (Inspect) Visit Preview Aug 31, 2024 6:16pm

@dosubot dosubot bot added Ɑ: embeddings Related to text embedding models module 🤖:improvement Medium size change to existing code to handle new use-cases labels Nov 7, 2023
@streamnsight
Copy link
Contributor Author

@hwchase17

@hwchase17 hwchase17 closed this Jan 30, 2024
@baskaryan baskaryan reopened this Jan 30, 2024
@dosubot dosubot bot added the size:XS This PR changes 0-9 lines, ignoring generated files. label Feb 1, 2024
@streamnsight
Copy link
Contributor Author

@baskaryan
Should be mergable now. This branch has been sitting around for too long and things changed a lot, but that was an easy fix.

@streamnsight
Copy link
Contributor Author

@hwchase17 fixed the PR

@@ -57,6 +57,9 @@ class LlamaCppEmbeddings(BaseModel, Embeddings):
verbose: bool = Field(True, alias="verbose")
"""Print verbose output to stderr."""

device: Optional[str] = Field(None, alias="device")
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should this be added for the LlamaCpp LLM as well?

@ccurme ccurme added the community Related to langchain-community label Jun 18, 2024
@dosubot dosubot bot added the lgtm PR looks good. Use to confirm that a PR is ready for merging. label Aug 31, 2024
@ccurme ccurme enabled auto-merge (squash) August 31, 2024 18:16
@ccurme ccurme merged commit 654da27 into langchain-ai:master Aug 31, 2024
27 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
community Related to langchain-community Ɑ: embeddings Related to text embedding models module 🤖:improvement Medium size change to existing code to handle new use-cases lgtm PR looks good. Use to confirm that a PR is ready for merging. size:XS This PR changes 0-9 lines, ignoring generated files.
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

None yet

4 participants