Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add nvidia nim rerank support #13178

Merged
merged 28 commits into from
May 3, 2024

Conversation

mattf
Copy link
Contributor

@mattf mattf commented Apr 30, 2024

Description

this adds the llama-index-postprocessor-nvidia-rerank package for interacting w/ ranking models hosted on ai.nvidia.com.

from llama_index.postprocessor.nvidia_rerank import NVIDIARerank
from llama_index.core.schema import NodeWithScore, Document

texts = [
    "two roads diverged in a yellow wood, and sorry i could not travel both and be one traveler, long i stood and looked down one as far as i could to where it bent in the undergrowth;",
    "then took the other, as just as fair, and having perhaps the better claim because it was grassy and wanted wear, though as for that the passing there had worn them really about the same,",
    "and both that morning equally lay in leaves no step had trodden black. oh, i marked the first for another day! yet knowing how way leads on to way i doubted if i should ever come back.",
    "i shall be telling this with a sigh somewhere ages and ages hense: two roads diverged in a wood, and i, i took the one less traveled by, and that has made all the difference."
]

nodes = [NodeWithScore(node=Document(text=text)) for text in texts]

rerank = NVIDIARerank()
rerank.postprocess_nodes(nodes=nodes, query_str="which way should i go?")

New Package?

Did I fill in the tool.llamahub section in the pyproject.toml and provide a detailed README.md for my new integration or package?

  • Yes
  • No

Version Bump?

Did I bump the version in the pyproject.toml file of the package I am updating? (Except for the llama-index-core package)

  • Yes
  • No
  • N/A - new package

Type of Change

Please delete options that are not relevant.

  • New feature (non-breaking change which adds functionality)
  • This change requires a documentation update

How Has This Been Tested?

Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration

  • Added new unit/integration tests
  • Added new notebook (that tests end-to-end)
  • I stared at the code and made sure it makes sense

Suggested Checklist:

  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have added Google Colab support for the newly added notebooks.
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes
  • I ran make format; make lint to appease the lint gods

Note

Co-authored with Zenodia Charpy

Zenodia and others added 26 commits April 30, 2024 10:03
Signed-off-by: Zenodia Charpy <zcharpy@nvidia.com>
Signed-off-by: Zenodia Charpy <zcharpy@nvidia.com>
Signed-off-by: Zenodia Charpy <zcharpy@nvidia.com>
Signed-off-by: Zenodia Charpy <zcharpy@nvidia.com>
Signed-off-by: Zenodia Charpy <zcharpy@nvidia.com>
Signed-off-by: Zenodia Charpy <zcharpy@nvidia.com>
Signed-off-by: Zenodia Charpy <zcharpy@nvidia.com>
…list

Signed-off-by: Zenodia Charpy <zcharpy@nvidia.com>
…works

Signed-off-by: Zenodia Charpy <zcharpy@nvidia.com>
… verify batching works

Signed-off-by: Zenodia Charpy <zcharpy@nvidia.com>
Signed-off-by: Zenodia Charpy <zcharpy@nvidia.com>
Signed-off-by: Zenodia Charpy <zcharpy@nvidia.com>
Signed-off-by: Zenodia Charpy <zcharpy@nvidia.com>
Signed-off-by: Zenodia Charpy <zcharpy@nvidia.com>
Signed-off-by: Zenodia Charpy <zcharpy@nvidia.com>
Signed-off-by: Zenodia Charpy <zcharpy@nvidia.com>
Signed-off-by: Zenodia Charpy <zcharpy@nvidia.com>
@dosubot dosubot bot added the size:XXL This PR changes 1000+ lines, ignoring generated files. label Apr 30, 2024
Copy link

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

@mattf mattf marked this pull request as draft April 30, 2024 14:19
@mattf mattf marked this pull request as ready for review April 30, 2024 20:53
Untitled.ipynb Outdated Show resolved Hide resolved
ids = [DEFAULT_MODEL]
return [Model(id=id) for id in ids]

def mode(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yea again, this feels like a function that isn't needed? It could be just part of the init right?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i agree, this could be part of init. there's low potential for a user to want to create an instance and then mode switch it more than once. this is the design flow we're using consistently across communities. it's also up for review.

@dosubot dosubot bot added size:XL This PR changes 500-999 lines, ignoring generated files. and removed size:XXL This PR changes 1000+ lines, ignoring generated files. labels May 2, 2024
@dosubot dosubot bot added the lgtm This PR has been approved by a maintainer label May 3, 2024
@logan-markewich logan-markewich merged commit f79e186 into run-llama:main May 3, 2024
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
lgtm This PR has been approved by a maintainer size:XL This PR changes 500-999 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants