Enhance metadata handling in GoogleVectorStore #11018
Merged
+61
−12
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This update introduces several enhancements to the `GoogleVectorStore` class, focusing on improved handling and flexibility of metadata during document retrieval. The changes are designed to provide users with more control over which metadata to include in the query results, aligning with the requirements of various use cases.
### Changes:
1. **Metadata Inclusion Control**: Added the ability to specify whether to include custom metadata in query results through the `include_metadata` parameter. This boolean parameter allows users to decide if metadata should be part of the query response, enhancing the versatility of data retrieval.
2. **Selective Metadata Filtering**: Introduced the `metadata_keys` parameter, enabling users to specify a list of metadata keys that should be included in the query results. This feature allows for the selective inclusion of metadata, ensuring that only relevant information is returned, which can be particularly useful for reducing payload sizes or focusing on specific data aspects.
3. **Constructor and Factory Method Updates**: Modified the `from_corpus` factory method of the `GoogleVectorStore` class to accept `include_metadata` and `metadata_keys` parameters. These updates allow the configuration of metadata handling preferences at the instance creation stage, providing a seamless and intuitive setup process.
4. **Query Method Enhancement**: Updated the `query` method to leverage the `include_metadata` and `metadata_keys` attributes for filtering metadata based on user preferences. The method now conditionally includes metadata in the query results, based on the instance configuration, and filters the included metadata according to the specified keys.
### Usage Example:
```python
store = GoogleVectorStore.from_corpus(
corpus_id="my-corpus-id",
include_metadata=True,
metadata_keys=['file_name', 'creation_date']
)
```
This enhancement improves the GoogleVectorStore's flexibility and usability, providing users with precise control over metadata handling in their applications. By allowing selective metadata inclusion and filtering, it caters to a wider range of use cases and data processing requirements.
dosubot
bot
added
the
size:M
|
@logan-markewich, @nerdai, I'm not entirely certain if these changes align with your current plans and vision for the project structure. Could you please review and let me know if this should be integrated differently? |
Dominastorm
pushed a commit
to uptrain-ai/llama_index
that referenced
this pull request
Feb 28, 2024
anoopshrma
pushed a commit
to anoopshrma/llama_index
that referenced
this pull request
Mar 2, 2024
Izukimat
pushed a commit
to Izukimat/llama_index
that referenced
this pull request
Mar 29, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This update introduces several enhancements to the
GoogleVectorStoreclass, focusing on improved handling and flexibility of metadata during document retrieval. The changes are designed to provide users with more control over which metadata to include in the query results, aligning with the requirements of various use cases.Type of Change
Changes:
Metadata Inclusion Control: Added the ability to specify whether to include custom metadata in query results through the
include_metadataparameter. This boolean parameter allows users to decide if metadata should be part of the query response, enhancing the versatility of data retrieval.Selective Metadata Filtering: Introduced the
metadata_keysparameter, enabling users to specify a list of metadata keys that should be included in the query results. This feature allows for the selective inclusion of metadata, ensuring that only relevant information is returned, which can be particularly useful for reducing payload sizes or focusing on specific data aspects.Constructor and Factory Method Updates: Modified the
from_corpusfactory method of theGoogleVectorStoreclass to acceptinclude_metadataandmetadata_keysparameters. These updates allow the configuration of metadata handling preferences at the instance creation stage, providing a seamless and intuitive setup process.Query Method Enhancement: Updated the
querymethod to leverage theinclude_metadataandmetadata_keysattributes for filtering metadata based on user preferences. The method now conditionally includes metadata in the query results, based on the instance configuration, and filters the included metadata according to the specified keys.Usage Example:
This enhancement improves the GoogleVectorStore's flexibility and usability, providing users with precise control over metadata handling in their applications. By allowing selective metadata inclusion and filtering, it caters to a wider range of use cases and data processing requirements.
How Has This Been Tested?