Google Cloud Enterprise Search retriever #7857
Conversation
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎ 1 Ignored Deployment
|
dosubot
bot
added
the
🤖:enhancement
|
This PR adds a LangChain retriever that encapsulates Enterprise Search on Google Cloud Gen App Builder. |
| """Filter expression.""" | ||
| get_extractive_answers: bool = False | ||
| """If True the retriever will return Extractive Answers. Otherwise Extractive Segments.""" | ||
| max_documents: int = Field(default=5, ge=1, le=100) |
There was a problem hiding this comment.
most retriever / doc store interfaces refer to this as top_k, could we change for consistency?
and out of curiosity, are the bounds enforced on the server side as well? if not, do we need them?
There was a problem hiding this comment.
This setting is a little bit different than top_k as the underlying retrieval mechanism is a little bit different than in many other vector stores . The max_documents, max_extractive_answer_count, and max_extractive_segment_count settings together decide how many matching "chunks" will be returned. The max_documents controls how many top ranked documents - as ingested by Enterprise Search engine - will be used to prepare extractive answers or extractive segments. When you populate Enterprise Search you ingest full document (rather than chunks) and the engine does chunking and embedding generations in the backend.
For example, if you set max_documents to 3 and max_extractive_answers_count to 3 you can get up to 9 LangChain documents.
When we have the retriever in the main, we are planning to publish a number of code samples that show how to use different LangChain retrieval chains and agents with Enterprise Search. In these code samples we plan to go into pretty deep detail in describing different strategies for ingesting/quering both structured and unstructured data with Enterprise Search.
|
lgtm, thanks @jarokaz!! |
|
@jarokaz - I just spoke to Kristopher Overholt, and he suggested I talk to you about having me work on this for LangChainJS if work isn't already being done and anything I should be aware of in doing so. |
No description provided.