Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

safe format prompt variables in strings with JSON #15734

Merged
merged 15 commits into from
Sep 22, 2024

Conversation

jerryjliu
Copy link
Collaborator

@jerryjliu jerryjliu commented Aug 30, 2024

This fixes the edge case where the prompt template also contains json, and you don't want to treat the JSON string as template variables

Thanks @erik-megarad

cr
cr
cr
cr
@dosubot dosubot bot added the size:L This PR changes 100-499 lines, ignoring generated files. label Aug 30, 2024
Copy link

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

@jerryjliu jerryjliu changed the title fix escape json safe format prompt variables in strings with JSON Aug 30, 2024
@jerryjliu jerryjliu changed the base branch from main to jerry/force_openai_struct_output August 30, 2024 06:37
@dosubot dosubot bot added size:XXL This PR changes 1000+ lines, ignoring generated files. and removed size:L This PR changes 100-499 lines, ignoring generated files. labels Aug 30, 2024
cr
@dosubot dosubot bot added size:M This PR changes 30-99 lines, ignoring generated files. and removed size:XXL This PR changes 1000+ lines, ignoring generated files. labels Aug 30, 2024
cr
Base automatically changed from jerry/force_openai_struct_output to main September 6, 2024 22:07
@dosubot dosubot bot added size:L This PR changes 100-499 lines, ignoring generated files. and removed size:M This PR changes 30-99 lines, ignoring generated files. labels Sep 6, 2024
@dosubot dosubot bot added size:M This PR changes 30-99 lines, ignoring generated files. and removed size:L This PR changes 100-499 lines, ignoring generated files. labels Sep 22, 2024
@dosubot dosubot bot removed the size:M This PR changes 30-99 lines, ignoring generated files. label Sep 22, 2024
@dosubot dosubot bot added the size:L This PR changes 100-499 lines, ignoring generated files. label Sep 22, 2024
@logan-markewich logan-markewich merged commit 7d9bd0f into main Sep 22, 2024
9 of 10 checks passed
@logan-markewich logan-markewich deleted the jerry/fix_escape_json branch September 22, 2024 23:20
barduinor added a commit to box-community/llama_index that referenced this pull request Sep 23, 2024

Partially verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
We cannot verify signatures from co-authors, and some of the co-authors attributed to this commit require their commits to be signed.
* v0.11.5 (run-llama#15834)

* Box bug fix 202409 (run-llama#15836)

* fix document matadata for box file

* fix BoxSearchOptions class initialization

* bump versions to 0.2.1

* Add oreilly course cookbooks (run-llama#15845)

* correct course name (run-llama#15859)

* clone llama-deploy docs during docs builds (run-llama#15794)

* chore(deps-dev): bump cryptography from 43.0.0 to 43.0.1 in /llama-index-core (run-llama#15840)

* Refresh Opensearch index after delete operation (run-llama#15854)

* Update ImageReader file loading logic (run-llama#15848)

* Minor bug in SQQE (run-llama#15861)

* v0.11.6 (run-llama#15863)

* remove widgets from recent notebooks (run-llama#15864)

* Fix image document deserialization issue (run-llama#15857)

* Bugfix: Remove warnings during data insertion for Hybrid Search due to duplicated columns in the schema (run-llama#15846)

* Add TablestoreVectorStore. (run-llama#15657)

* Removed unused `llama-index-llms-anthropic` dependency from Bedrock Converse (run-llama#15869)

Removed unused `llama-index-llms-anthropic` dependency. Incremented to `0.3.0`.

* Update bedrock.py (run-llama#15879)

Fix minor issue in error message

* Add tool calling example in NVIDIA notebook (run-llama#15842)

* NVIDIA: Completion fixes (run-llama#15820)

* Fix PGVectorStore with latest pydantic, update pydantic imports (run-llama#15886)

* add type ignore for streaming agents (run-llama#15887)

* [wip] force openai structured output (run-llama#15706)

* feature: Make SentenceSplitter's secondary_chunking_regex optional (run-llama#15882)

* Bug fix for KuzuPropertyGraphStore: Allow upserting relations even when chunks are absent (run-llama#15889)

* Missing tablestore (run-llama#15890)

* v0.11.7 (run-llama#15891)

* nit: extend workflow tutorial (run-llama#15901)

* Modify generate_qa_embedding_pairs to use LLM from Settings (run-llama#15904)

* docs: document how to disable global timeout along with more docs (run-llama#15912)

document how to disable global timeout along with more docs

* fix: model_name property and pydantic v2 in Azure Inference package (run-llama#15907)

* fix: model_name property and pydantic v2

* fix: tests

* Add InternalServerError to retry decorator (run-llama#15921)

* Add InternalServerError to retry decorator

* Bump version

* docs: fix broken link (run-llama#15924)

* Adding vector store for Azure Cosmos DB NoSql (run-llama#14158)

* Initial changes

* initial changes fixed with example

* Initial commit with all the code and test changes

* fixing test cases

* adding jupyter notebook

* cleaning files

* resolving comments

* fix linting

* more linting

* add a basic readme

---------

Co-authored-by: Aayush Kataria <aayushkataria3011@gmail.com>
Co-authored-by: Massimiliano Pippi <mpippi@gmail.com>

* update docs for custom embeddings (run-llama#15929)

* Add doc.id to Langchain format conversions (run-llama#15928)

* Update RankLLM with new rerankers (run-llama#15892)

* Feature Request run-llama#15810 :: Add DynamoDBChatStore (run-llama#15917)

* Adding support for `MetadataFilters` to WordLift Vector Store (run-llama#15905)

* Repair and route colab links to official repo (run-llama#15900)

* feat: Add a retry policy to steps (run-llama#15757)

* Fix error handling in sharepoint reader (run-llama#15868)

* feat: add llama-index llms alibabacloud_aisearch integration (run-llama#15850)

* Opensearch Serverless filtered query support using knn_score script (run-llama#15899)

* Mistral AI LLM Integration fixes (run-llama#15906)

* Fix RagCLI (run-llama#15931)

* v0.11.8 (run-llama#15933)

* docs: document retry policy (run-llama#15938)

* docs: update AI21 docs and readme (run-llama#15937)

* docs: update ai21 docs and readme with jamba 1.5 and tool calling

* docs: update ai21 docs and readme with jamba 1.5 and tool calling

* docs: update ai21 docs and readme with jamba 1.5 and tool calling

* Retry retriable errors in neo4j integrations (run-llama#15915)

* Add four alibabacloud-aisearch llama-index integrations: rerank, node_parser, readers, embeddings (run-llama#15934)

* Differentiate sync and async calls in OpenSearchVectorClient (run-llama#15945)

* feat(postgres): add support for engine parameters (run-llama#15951)

* feat(postgres): add support for engine parameters
- Introduced engine_params to support passing parameters to create_engine.
- Updated create_engine and create_async_engine calls to include engine_params.
- Initialized engine_params in the constructor.

* style(lint): reformat for readability

* refactor(postgres): rename engine_params to create_engine_kwargs

* refactor(postgres): rename engine_params to create_engine_kwargs

* chore: bump version to 0.2.3

* fix(postgres): rename engine_params to create_engine_kwargs

* update falkordb client (run-llama#15940)

* update falkordb client

* bump version

* update version

* fix

* fix: Error when parsing output if tool name contains non-English characters (run-llama#15956)

* fix attribute error in PGVectorStore (run-llama#15961)

* Catch nest_asyncio errors (run-llama#15975)

* NUDGE (run-llama#15954)

* Add support for o1 openai models (run-llama#15979)

* force temp to 1.0 for o1 (run-llama#15983)

* Update concepts.md - fix link for Structured Data Extraction page (run-llama#15982)

* Fix the import path example for SimpleMongoReader (run-llama#15988)

* Do not pass system prompt from fn calling runner to fn calling worker (run-llama#15986)

* Add callback manager to retriever query engine from args (run-llama#15990)

* v0.11.9 (run-llama#15992)

* Fix: get all documents from Elasticsearch KVStore (run-llama#16006)

* Fix Pydantic numeric validation in openai/base.py (run-llama#15993)

* feat: add quip reader (run-llama#16000)

* [docs/example] Human in loop workflow example (run-llama#16011)

* start choose own adventure hitl nb

* hitl example

* note on alternative implementation

* add module guides and run prepare_for_build

* fix: removed author info from quip reader (run-llama#16012)

fix: removed author info

* Fix Pydantic models definition (run-llama#16008)

* chore(deps): bump litellm from 1.43.7 to 1.44.8 in /llama-index-integrations/embeddings/llama-index-embeddings-litellm (run-llama#16013)

* Hotfix: Fix Citations Text (run-llama#16015)

* Jacques/opik integration (run-llama#16007)

* update llamacloud index with image nodes (run-llama#15996)

* chore: add o1 models pinned versions (run-llama#16025)

* add sparse embedding abstraction (run-llama#16018)

* Implement `get_nodes` on `PGVectorStore` (run-llama#16026)

* Implement `get_nodes` on `PGVectorStore`

* Bump version num in pyproject.toml

* Update docstring in base.py in MilvusVectorStore adding COSINE as available similarity metric (run-llama#16031)

Update docstring in base.py adding COSINE as similarity metric available

According to Milvus Documentation, **COSINE** as similarity metric is supported (Both Milvus and Milvus Lite) but in Llama-Index docs was missing.

[Link to Milvus official docs](https://milvus.io/docs/metric.md?tab=floating#Similarity-Metrics)

I've checked in the [code](https://github.com/run-llama/llama_index/blob/723c2533ed4b7b43b7d814c89af1838f0f1994c2/llama-index-integrations/vector_stores/llama-index-vector-stores-milvus/llama_index/vector_stores/milvus/base.py#L256) , so indeed COSINE is supported, so no more further changes are needed.

* Fix: unnecessary warning issue in HuggingFace LLM when tokenizer is provided as argument(run-llama#16035) (run-llama#16037)

* Attempt #3 of context/result refactor (run-llama#16036)

* temporarily limit lancedb version (run-llama#16045)

* fix: new Data Connector adaption for DashVector (run-llama#16028)

* v0.11.10 (run-llama#16046)

* Fix serde issue for huggingface inference API embedding (run-llama#16053)

* wip

* wip

* fix: fix regression in OctoAI llm provider after 0.11 (run-llama#16002)

* fix OctoAI llm provider

* adjust code to the latest client

* resolve conflicts

* fix message format conversion

* do not pass max_tokens if None

* add chat test

* bump version

* update LanceDB integration (run-llama#16057)

* Adding docstring inside preprocess base code. (run-llama#16060)

* Add default to optional args for BedrockEmbedding (run-llama#16067)

* wip

* wip

* Fix optional type for Cohere embedding (run-llama#16068)

wip

* fix vertex pydantic arguments (run-llama#16069)

* Fix result order (run-llama#16078)

* Fix result order

* Bump version

* fix elasticsearch embedding async function (run-llama#16083)

* docs: add decorators to the api reference (run-llama#16081)

* fix incorrect parameters in VertexAIIndex (run-llama#16080)

* feat: update JinaEmbedding for v3 release (run-llama#15971)

* Mistralai enable custom endpoint from env (run-llama#16084)

* Implement async for multi modal ollama (run-llama#16091)

* Remove circular package dep (run-llama#16070)

* [chore] fix incorrect `handler.context` for `handler.ctx` in docs (run-llama#16101)

fix

* fix bug missing import (run-llama#16096)

* Add more workflow references to docs (run-llama#16102)

* fix cohere tests (run-llama#16104)

* Use response synthesizer in context chat engines (run-llama#16017)

* Fix mongodb hybrid search, also pass hybrid_top_k in vector retriever (run-llama#16105)

* bump openai agent deps (run-llama#16112)

* Fixed up test_vectorstore.  (run-llama#16113)

* Issue 16071: wordpress requires username, password (run-llama#16072)

* Issue 16071: wordpress requires username, password

* Adding changes suggested in PR template

* Use Optional typing keyword

* feat: add configurable base_url field in rerank (run-llama#16050)

* add base_url

* version bump

* add default

---------

Co-authored-by: Massimiliano Pippi <mpippi@gmail.com>
Co-authored-by: Logan <logan.markewich@live.com>

* Enhance Pandas Query Engine Output Processor (run-llama#16052)

* Enhance output processor to temporarily adjust display options.

* Make format changes.

* Update pyproject.toml

---------

Co-authored-by: Logan <logan.markewich@live.com>
Co-authored-by: Massimiliano Pippi <mpippi@gmail.com>

* fix workflow docs (run-llama#16117)

cr

* Update OpenVINO LLM pyproject.toml (run-llama#16130)

* Wordpress: Allow control of whether Pages and/or Posts are retrieved (run-llama#16128)

* Async achat put operation (run-llama#16127)

* [fix] handler.stream_events() doesn't yield `StopEvent` (run-llama#16115)

* Improved TLM Rag cookbook (run-llama#16109)

* Update chat message class for multi-modal (run-llama#15969)

* Add support for Path for SimpleDirectoryReader (run-llama#16108)

* Add TopicNodeParser based on MedGraphRAG paper (run-llama#16131)

* Sql markdown response (run-llama#16103)

* v0.11.11 (run-llama#16134)

* Correct Pydantic warning(s) issed for llama-index-llms-ibm (run-llama#16141)

Fix Pydantic warnings in llam-index-llms-ibm

* feat: add drive link to google drive reader (run-llama#16156)

* Introducing new VoyageAI models (run-llama#16150)

* Add `required_exts` option to SharePoint reader (run-llama#16152)

* User-defined schema in MilvusVectorStore (run-llama#16151)

* safe format prompt variables in strings with JSON (run-llama#15734)

* account for tools in prompt helper (run-llama#16157)

* v0.11.12 (run-llama#16159)

---------

Co-authored-by: Logan <logan.markewich@live.com>
Co-authored-by: Ravi Theja <ravi03071991@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Yannic Spreen-Ledebur <35889034+spreeni@users.noreply.github.com>
Co-authored-by: Adnan Alkattan <Adnankattan9@gmail.com>
Co-authored-by: Laurie Voss <github@seldo.com>
Co-authored-by: saipjkai <84132316+saipjkai@users.noreply.github.com>
Co-authored-by: ScriptShi <xunjian.sl@alibaba-inc.com>
Co-authored-by: Bryce Freshcorn <26725654+brycecf@users.noreply.github.com>
Co-authored-by: Graham Tibbitts <grahamtt@users.noreply.github.com>
Co-authored-by: Rashmi Pawar <168514198+raspawar@users.noreply.github.com>
Co-authored-by: Jerry Liu <jerryjliu98@gmail.com>
Co-authored-by: Caroline Binley <39920563+carolinebinley@users.noreply.github.com>
Co-authored-by: Prashanth Rao <35005448+prrao87@users.noreply.github.com>
Co-authored-by: Asi Greenholts <88270351+TupleType@users.noreply.github.com>
Co-authored-by: Massimiliano Pippi <mpippi@gmail.com>
Co-authored-by: Facundo Santiago <fasantia@microsoft.com>
Co-authored-by: gsa9989 <117786401+gsa9989@users.noreply.github.com>
Co-authored-by: Aayush Kataria <aayushkataria3011@gmail.com>
Co-authored-by: Ryan Nguyen <96593302+xpbowler@users.noreply.github.com>
Co-authored-by: David Riccitelli <david@wordlift.io>
Co-authored-by: GICodeWarrior <GICodeWarrior@gmail.com>
Co-authored-by: Javier Torres <javierandrestorresreyes@gmail.com>
Co-authored-by: 才胜 <1392197444@qq.com>
Co-authored-by: George Dittmar <georgedittmar@gmail.com>
Co-authored-by: Tarun Jain <jaintarun.abd17@gmail.com>
Co-authored-by: miri-bar <160584887+miri-bar@users.noreply.github.com>
Co-authored-by: Tomaz Bratanic <bratanic.tomaz@gmail.com>
Co-authored-by: kobiche <56874660+kobiche@users.noreply.github.com>
Co-authored-by: Arthur Moura Carvalho <141850116+armoucar-neon@users.noreply.github.com>
Co-authored-by: Avi Avni <avi.avni@gmail.com>
Co-authored-by: Cheese <11363971+cheese-git@users.noreply.github.com>
Co-authored-by: Zac Wellmer <9603276+zacwellmer@users.noreply.github.com>
Co-authored-by: Ragul Kachiappan <ragul.kachiappan@techjays.com>
Co-authored-by: Vinicius <53495782+Vractos@users.noreply.github.com>
Co-authored-by: Bidimpata-Kerim Aramyan-Tshimanga <bk.tshimanga@gmail.com>
Co-authored-by: Laura Ceconi <laura.cec@hotmail.com>
Co-authored-by: N1eo <zzhsaga@gmail.com>
Co-authored-by: Chirag Agrawal <chirag.agrawal93@gmail.com>
Co-authored-by: Andrei Fajardo <92402603+nerdai@users.noreply.github.com>
Co-authored-by: David Oplatka <david.oplatka@vectara.com>
Co-authored-by: Jacques Verré <jverre@gmail.com>
Co-authored-by: Harpinder <harpinderjot36@gmail.com>
Co-authored-by: Richard Liu <richardliu7896@gmail.com>
Co-authored-by: Jorge Barrachina Gutiérrez <1333901+ntkog@users.noreply.github.com>
Co-authored-by: Sourabh <Sourabh72101@gmail.com>
Co-authored-by: OceanPresent <oceanpresent@163.com>
Co-authored-by: Simon Suo <simonsdsuo@gmail.com>
Co-authored-by: Ayush Chaurasia <ayush.chaurarsia@gmail.com>
Co-authored-by: preprocess-co <137915090+preprocess-co@users.noreply.github.com>
Co-authored-by: guodong <songoodong@163.com>
Co-authored-by: polarbear567 <269739606@qq.com>
Co-authored-by: Aaron Ji <127167174+DresAaron@users.noreply.github.com>
Co-authored-by: enrico-stauss <73635664+enrico-stauss@users.noreply.github.com>
Co-authored-by: Selim Çavaş <92586913+selimcavas@users.noreply.github.com>
Co-authored-by: Casey Clements <caseyclements@users.noreply.github.com>
Co-authored-by: Jonathan Springer <jonpspri@gmail.com>
Co-authored-by: Anirudh31415926535 <anirudh@cohere.com>
Co-authored-by: Matin Khajavi <58955268+MatinKhajavi@users.noreply.github.com>
Co-authored-by: Ethan Yang <ethan.yang@intel.com>
Co-authored-by: Matthew Turk <mturk24@users.noreply.github.com>
Co-authored-by: Andrew Kim <59037923+andrewwkimm@users.noreply.github.com>
Co-authored-by: José Henrique Luckmann <133380513+JoseLuckmann@users.noreply.github.com>
Co-authored-by: Emanuel Ferreira <contatoferreirads@gmail.com>
Co-authored-by: fzowl <160063452+fzowl@users.noreply.github.com>
Co-authored-by: João Martins <11438285+jl-martins@users.noreply.github.com>
Co-authored-by: Petros Mitseas <petros_94@icloud.com>
barduinor added a commit to box-community/llama_index that referenced this pull request Sep 23, 2024

Partially verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
We cannot verify signatures from co-authors, and some of the co-authors attributed to this commit require their commits to be signed.
* v0.11.5 (run-llama#15834)

* Box bug fix 202409 (run-llama#15836)

* fix document matadata for box file

* fix BoxSearchOptions class initialization

* bump versions to 0.2.1

* Add oreilly course cookbooks (run-llama#15845)

* correct course name (run-llama#15859)

* clone llama-deploy docs during docs builds (run-llama#15794)

* chore(deps-dev): bump cryptography from 43.0.0 to 43.0.1 in /llama-index-core (run-llama#15840)

* Refresh Opensearch index after delete operation (run-llama#15854)

* Update ImageReader file loading logic (run-llama#15848)

* Minor bug in SQQE (run-llama#15861)

* v0.11.6 (run-llama#15863)

* remove widgets from recent notebooks (run-llama#15864)

* Fix image document deserialization issue (run-llama#15857)

* Bugfix: Remove warnings during data insertion for Hybrid Search due to duplicated columns in the schema (run-llama#15846)

* Add TablestoreVectorStore. (run-llama#15657)

* Removed unused `llama-index-llms-anthropic` dependency from Bedrock Converse (run-llama#15869)

Removed unused `llama-index-llms-anthropic` dependency. Incremented to `0.3.0`.

* Update bedrock.py (run-llama#15879)

Fix minor issue in error message

* Add tool calling example in NVIDIA notebook (run-llama#15842)

* NVIDIA: Completion fixes (run-llama#15820)

* Fix PGVectorStore with latest pydantic, update pydantic imports (run-llama#15886)

* add type ignore for streaming agents (run-llama#15887)

* [wip] force openai structured output (run-llama#15706)

* feature: Make SentenceSplitter's secondary_chunking_regex optional (run-llama#15882)

* Bug fix for KuzuPropertyGraphStore: Allow upserting relations even when chunks are absent (run-llama#15889)

* Missing tablestore (run-llama#15890)

* v0.11.7 (run-llama#15891)

* nit: extend workflow tutorial (run-llama#15901)

* Modify generate_qa_embedding_pairs to use LLM from Settings (run-llama#15904)

* docs: document how to disable global timeout along with more docs (run-llama#15912)

document how to disable global timeout along with more docs

* fix: model_name property and pydantic v2 in Azure Inference package (run-llama#15907)

* fix: model_name property and pydantic v2

* fix: tests

* Add InternalServerError to retry decorator (run-llama#15921)

* Add InternalServerError to retry decorator

* Bump version

* docs: fix broken link (run-llama#15924)

* Adding vector store for Azure Cosmos DB NoSql (run-llama#14158)

* Initial changes

* initial changes fixed with example

* Initial commit with all the code and test changes

* fixing test cases

* adding jupyter notebook

* cleaning files

* resolving comments

* fix linting

* more linting

* add a basic readme

---------

Co-authored-by: Aayush Kataria <aayushkataria3011@gmail.com>
Co-authored-by: Massimiliano Pippi <mpippi@gmail.com>

* update docs for custom embeddings (run-llama#15929)

* Add doc.id to Langchain format conversions (run-llama#15928)

* Update RankLLM with new rerankers (run-llama#15892)

* Feature Request run-llama#15810 :: Add DynamoDBChatStore (run-llama#15917)

* Adding support for `MetadataFilters` to WordLift Vector Store (run-llama#15905)

* Repair and route colab links to official repo (run-llama#15900)

* feat: Add a retry policy to steps (run-llama#15757)

* Fix error handling in sharepoint reader (run-llama#15868)

* feat: add llama-index llms alibabacloud_aisearch integration (run-llama#15850)

* Opensearch Serverless filtered query support using knn_score script (run-llama#15899)

* Mistral AI LLM Integration fixes (run-llama#15906)

* Fix RagCLI (run-llama#15931)

* v0.11.8 (run-llama#15933)

* docs: document retry policy (run-llama#15938)

* docs: update AI21 docs and readme (run-llama#15937)

* docs: update ai21 docs and readme with jamba 1.5 and tool calling

* docs: update ai21 docs and readme with jamba 1.5 and tool calling

* docs: update ai21 docs and readme with jamba 1.5 and tool calling

* Retry retriable errors in neo4j integrations (run-llama#15915)

* Add four alibabacloud-aisearch llama-index integrations: rerank, node_parser, readers, embeddings (run-llama#15934)

* Differentiate sync and async calls in OpenSearchVectorClient (run-llama#15945)

* feat(postgres): add support for engine parameters (run-llama#15951)

* feat(postgres): add support for engine parameters
- Introduced engine_params to support passing parameters to create_engine.
- Updated create_engine and create_async_engine calls to include engine_params.
- Initialized engine_params in the constructor.

* style(lint): reformat for readability

* refactor(postgres): rename engine_params to create_engine_kwargs

* refactor(postgres): rename engine_params to create_engine_kwargs

* chore: bump version to 0.2.3

* fix(postgres): rename engine_params to create_engine_kwargs

* update falkordb client (run-llama#15940)

* update falkordb client

* bump version

* update version

* fix

* fix: Error when parsing output if tool name contains non-English characters (run-llama#15956)

* fix attribute error in PGVectorStore (run-llama#15961)

* Catch nest_asyncio errors (run-llama#15975)

* NUDGE (run-llama#15954)

* Add support for o1 openai models (run-llama#15979)

* force temp to 1.0 for o1 (run-llama#15983)

* Update concepts.md - fix link for Structured Data Extraction page (run-llama#15982)

* Fix the import path example for SimpleMongoReader (run-llama#15988)

* Do not pass system prompt from fn calling runner to fn calling worker (run-llama#15986)

* Add callback manager to retriever query engine from args (run-llama#15990)

* v0.11.9 (run-llama#15992)

* Fix: get all documents from Elasticsearch KVStore (run-llama#16006)

* Fix Pydantic numeric validation in openai/base.py (run-llama#15993)

* feat: add quip reader (run-llama#16000)

* [docs/example] Human in loop workflow example (run-llama#16011)

* start choose own adventure hitl nb

* hitl example

* note on alternative implementation

* add module guides and run prepare_for_build

* fix: removed author info from quip reader (run-llama#16012)

fix: removed author info

* Fix Pydantic models definition (run-llama#16008)

* chore(deps): bump litellm from 1.43.7 to 1.44.8 in /llama-index-integrations/embeddings/llama-index-embeddings-litellm (run-llama#16013)

* Hotfix: Fix Citations Text (run-llama#16015)

* Jacques/opik integration (run-llama#16007)

* update llamacloud index with image nodes (run-llama#15996)

* chore: add o1 models pinned versions (run-llama#16025)

* add sparse embedding abstraction (run-llama#16018)

* Implement `get_nodes` on `PGVectorStore` (run-llama#16026)

* Implement `get_nodes` on `PGVectorStore`

* Bump version num in pyproject.toml

* Update docstring in base.py in MilvusVectorStore adding COSINE as available similarity metric (run-llama#16031)

Update docstring in base.py adding COSINE as similarity metric available

According to Milvus Documentation, **COSINE** as similarity metric is supported (Both Milvus and Milvus Lite) but in Llama-Index docs was missing.

[Link to Milvus official docs](https://milvus.io/docs/metric.md?tab=floating#Similarity-Metrics)

I've checked in the [code](https://github.com/run-llama/llama_index/blob/723c2533ed4b7b43b7d814c89af1838f0f1994c2/llama-index-integrations/vector_stores/llama-index-vector-stores-milvus/llama_index/vector_stores/milvus/base.py#L256) , so indeed COSINE is supported, so no more further changes are needed.

* Fix: unnecessary warning issue in HuggingFace LLM when tokenizer is provided as argument(run-llama#16035) (run-llama#16037)

* Attempt #3 of context/result refactor (run-llama#16036)

* temporarily limit lancedb version (run-llama#16045)

* fix: new Data Connector adaption for DashVector (run-llama#16028)

* v0.11.10 (run-llama#16046)

* Fix serde issue for huggingface inference API embedding (run-llama#16053)

* wip

* wip

* fix: fix regression in OctoAI llm provider after 0.11 (run-llama#16002)

* fix OctoAI llm provider

* adjust code to the latest client

* resolve conflicts

* fix message format conversion

* do not pass max_tokens if None

* add chat test

* bump version

* update LanceDB integration (run-llama#16057)

* Adding docstring inside preprocess base code. (run-llama#16060)

* Add default to optional args for BedrockEmbedding (run-llama#16067)

* wip

* wip

* Fix optional type for Cohere embedding (run-llama#16068)

wip

* fix vertex pydantic arguments (run-llama#16069)

* Fix result order (run-llama#16078)

* Fix result order

* Bump version

* fix elasticsearch embedding async function (run-llama#16083)

* docs: add decorators to the api reference (run-llama#16081)

* fix incorrect parameters in VertexAIIndex (run-llama#16080)

* feat: update JinaEmbedding for v3 release (run-llama#15971)

* Mistralai enable custom endpoint from env (run-llama#16084)

* Implement async for multi modal ollama (run-llama#16091)

* Remove circular package dep (run-llama#16070)

* [chore] fix incorrect `handler.context` for `handler.ctx` in docs (run-llama#16101)

fix

* fix bug missing import (run-llama#16096)

* Add more workflow references to docs (run-llama#16102)

* fix cohere tests (run-llama#16104)

* Use response synthesizer in context chat engines (run-llama#16017)

* Fix mongodb hybrid search, also pass hybrid_top_k in vector retriever (run-llama#16105)

* bump openai agent deps (run-llama#16112)

* Fixed up test_vectorstore.  (run-llama#16113)

* Issue 16071: wordpress requires username, password (run-llama#16072)

* Issue 16071: wordpress requires username, password

* Adding changes suggested in PR template

* Use Optional typing keyword

* feat: add configurable base_url field in rerank (run-llama#16050)

* add base_url

* version bump

* add default

---------

Co-authored-by: Massimiliano Pippi <mpippi@gmail.com>
Co-authored-by: Logan <logan.markewich@live.com>

* Enhance Pandas Query Engine Output Processor (run-llama#16052)

* Enhance output processor to temporarily adjust display options.

* Make format changes.

* Update pyproject.toml

---------

Co-authored-by: Logan <logan.markewich@live.com>
Co-authored-by: Massimiliano Pippi <mpippi@gmail.com>

* fix workflow docs (run-llama#16117)

cr

* Update OpenVINO LLM pyproject.toml (run-llama#16130)

* Wordpress: Allow control of whether Pages and/or Posts are retrieved (run-llama#16128)

* Async achat put operation (run-llama#16127)

* [fix] handler.stream_events() doesn't yield `StopEvent` (run-llama#16115)

* Improved TLM Rag cookbook (run-llama#16109)

* Update chat message class for multi-modal (run-llama#15969)

* Add support for Path for SimpleDirectoryReader (run-llama#16108)

* Add TopicNodeParser based on MedGraphRAG paper (run-llama#16131)

* Sql markdown response (run-llama#16103)

* v0.11.11 (run-llama#16134)

* Correct Pydantic warning(s) issed for llama-index-llms-ibm (run-llama#16141)

Fix Pydantic warnings in llam-index-llms-ibm

* feat: add drive link to google drive reader (run-llama#16156)

* Introducing new VoyageAI models (run-llama#16150)

* Add `required_exts` option to SharePoint reader (run-llama#16152)

* User-defined schema in MilvusVectorStore (run-llama#16151)

* safe format prompt variables in strings with JSON (run-llama#15734)

* account for tools in prompt helper (run-llama#16157)

* v0.11.12 (run-llama#16159)

---------

Co-authored-by: Logan <logan.markewich@live.com>
Co-authored-by: Ravi Theja <ravi03071991@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Yannic Spreen-Ledebur <35889034+spreeni@users.noreply.github.com>
Co-authored-by: Adnan Alkattan <Adnankattan9@gmail.com>
Co-authored-by: Laurie Voss <github@seldo.com>
Co-authored-by: saipjkai <84132316+saipjkai@users.noreply.github.com>
Co-authored-by: ScriptShi <xunjian.sl@alibaba-inc.com>
Co-authored-by: Bryce Freshcorn <26725654+brycecf@users.noreply.github.com>
Co-authored-by: Graham Tibbitts <grahamtt@users.noreply.github.com>
Co-authored-by: Rashmi Pawar <168514198+raspawar@users.noreply.github.com>
Co-authored-by: Jerry Liu <jerryjliu98@gmail.com>
Co-authored-by: Caroline Binley <39920563+carolinebinley@users.noreply.github.com>
Co-authored-by: Prashanth Rao <35005448+prrao87@users.noreply.github.com>
Co-authored-by: Asi Greenholts <88270351+TupleType@users.noreply.github.com>
Co-authored-by: Massimiliano Pippi <mpippi@gmail.com>
Co-authored-by: Facundo Santiago <fasantia@microsoft.com>
Co-authored-by: gsa9989 <117786401+gsa9989@users.noreply.github.com>
Co-authored-by: Aayush Kataria <aayushkataria3011@gmail.com>
Co-authored-by: Ryan Nguyen <96593302+xpbowler@users.noreply.github.com>
Co-authored-by: David Riccitelli <david@wordlift.io>
Co-authored-by: GICodeWarrior <GICodeWarrior@gmail.com>
Co-authored-by: Javier Torres <javierandrestorresreyes@gmail.com>
Co-authored-by: 才胜 <1392197444@qq.com>
Co-authored-by: George Dittmar <georgedittmar@gmail.com>
Co-authored-by: Tarun Jain <jaintarun.abd17@gmail.com>
Co-authored-by: miri-bar <160584887+miri-bar@users.noreply.github.com>
Co-authored-by: Tomaz Bratanic <bratanic.tomaz@gmail.com>
Co-authored-by: kobiche <56874660+kobiche@users.noreply.github.com>
Co-authored-by: Arthur Moura Carvalho <141850116+armoucar-neon@users.noreply.github.com>
Co-authored-by: Avi Avni <avi.avni@gmail.com>
Co-authored-by: Cheese <11363971+cheese-git@users.noreply.github.com>
Co-authored-by: Zac Wellmer <9603276+zacwellmer@users.noreply.github.com>
Co-authored-by: Ragul Kachiappan <ragul.kachiappan@techjays.com>
Co-authored-by: Vinicius <53495782+Vractos@users.noreply.github.com>
Co-authored-by: Bidimpata-Kerim Aramyan-Tshimanga <bk.tshimanga@gmail.com>
Co-authored-by: Laura Ceconi <laura.cec@hotmail.com>
Co-authored-by: N1eo <zzhsaga@gmail.com>
Co-authored-by: Chirag Agrawal <chirag.agrawal93@gmail.com>
Co-authored-by: Andrei Fajardo <92402603+nerdai@users.noreply.github.com>
Co-authored-by: David Oplatka <david.oplatka@vectara.com>
Co-authored-by: Jacques Verré <jverre@gmail.com>
Co-authored-by: Harpinder <harpinderjot36@gmail.com>
Co-authored-by: Richard Liu <richardliu7896@gmail.com>
Co-authored-by: Jorge Barrachina Gutiérrez <1333901+ntkog@users.noreply.github.com>
Co-authored-by: Sourabh <Sourabh72101@gmail.com>
Co-authored-by: OceanPresent <oceanpresent@163.com>
Co-authored-by: Simon Suo <simonsdsuo@gmail.com>
Co-authored-by: Ayush Chaurasia <ayush.chaurarsia@gmail.com>
Co-authored-by: preprocess-co <137915090+preprocess-co@users.noreply.github.com>
Co-authored-by: guodong <songoodong@163.com>
Co-authored-by: polarbear567 <269739606@qq.com>
Co-authored-by: Aaron Ji <127167174+DresAaron@users.noreply.github.com>
Co-authored-by: enrico-stauss <73635664+enrico-stauss@users.noreply.github.com>
Co-authored-by: Selim Çavaş <92586913+selimcavas@users.noreply.github.com>
Co-authored-by: Casey Clements <caseyclements@users.noreply.github.com>
Co-authored-by: Jonathan Springer <jonpspri@gmail.com>
Co-authored-by: Anirudh31415926535 <anirudh@cohere.com>
Co-authored-by: Matin Khajavi <58955268+MatinKhajavi@users.noreply.github.com>
Co-authored-by: Ethan Yang <ethan.yang@intel.com>
Co-authored-by: Matthew Turk <mturk24@users.noreply.github.com>
Co-authored-by: Andrew Kim <59037923+andrewwkimm@users.noreply.github.com>
Co-authored-by: José Henrique Luckmann <133380513+JoseLuckmann@users.noreply.github.com>
Co-authored-by: Emanuel Ferreira <contatoferreirads@gmail.com>
Co-authored-by: fzowl <160063452+fzowl@users.noreply.github.com>
Co-authored-by: João Martins <11438285+jl-martins@users.noreply.github.com>
Co-authored-by: Petros Mitseas <petros_94@icloud.com>
jerryjliu added a commit to run-llama/llamacloud-demo that referenced this pull request Sep 23, 2024

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
- do chunk-level retrieval and document-level retrival to fetch relevant
bits for apple/tesla
- do custom LLM prompt to insert chunks to generate report
- use structured output abstractions to generate report

depends on this PR: run-llama/llama_index#15734
raspawar pushed a commit to raspawar/llama_index that referenced this pull request Oct 7, 2024
raspawar pushed a commit to raspawar/llama_index that referenced this pull request Oct 7, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
size:L This PR changes 100-499 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants