community[patch]: Add streaming for HuggingFace Inference (also add the support of stop-sequences) #3932
+98
−25
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
dosubot
bot
added
the
size:L
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎
|
dosubot
bot
added
the
auto:improvement
bracesproul
previously requested changes
Jan 8, 2024
There was a problem hiding this comment.
Largely looks good to me, just have one comment about stop sequences API, and this should have a section in the docs. Doesn't need to be complex, but should show off that it's possible.
thank you for doing this!!
|
I think the param HuggingFace expects is actually called Otherwise looks good! |
jacoblee93
changed the title
jacoblee93
added
question
|
It's hard to pin down but I believe the parameter is indeed https://huggingface.co/docs/api-inference/detailed_parameters#question-answering-task |
|
Thank you! |
nicolewhite
referenced
this pull request
in autoblocksai/javascript-sdk
Jan 22, 2024
[](https://renovatebot.com) This PR contains the following updates: | Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | [langchain](https://togithub.com/langchain-ai/langchainjs) | [`0.1.3` -> `0.1.5`](https://renovatebot.com/diffs/npm/langchain/0.1.3/0.1.5) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | | [openai](https://togithub.com/openai/openai-node) | [`4.24.7` -> `4.25.0`](https://renovatebot.com/diffs/npm/openai/4.24.7/4.25.0) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | | [prettier](https://prettier.io) ([source](https://togithub.com/prettier/prettier)) | [`3.2.2` -> `3.2.4`](https://renovatebot.com/diffs/npm/prettier/3.2.2/3.2.4) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | | [ts-jest](https://kulshekhar.github.io/ts-jest) ([source](https://togithub.com/kulshekhar/ts-jest)) | [`29.1.1` -> `29.1.2`](https://renovatebot.com/diffs/npm/ts-jest/29.1.1/29.1.2) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | --- ### Release Notes <details> <summary>langchain-ai/langchainjs (langchain)</summary> ### [`v0.1.5`](https://togithub.com/langchain-ai/langchainjs/releases/tag/0.1.5) [Compare Source](https://togithub.com/langchain-ai/langchainjs/compare/0.1.4...0.1.5) #### What's Changed - langchain\[patch]: Release 0.1.4 by [@​jacoblee93](https://togithub.com/jacoblee93) in [https://github.com/langchain-ai/langchainjs/pull/4072](https://togithub.com/langchain-ai/langchainjs/pull/4072) - Add Pinecone integration by [@​zackproser](https://togithub.com/zackproser) in [https://github.com/langchain-ai/langchainjs/pull/4062](https://togithub.com/langchain-ai/langchainjs/pull/4062) - langchain\[patch], community\[patch]: Allow Anthropic models to handle function messages by [@​jacoblee93](https://togithub.com/jacoblee93) in [https://github.com/langchain-ai/langchainjs/pull/4076](https://togithub.com/langchain-ai/langchainjs/pull/4076) - docs\[minor]: Autogenerate typedoc config, add to gitignore. by [@​bracesproul](https://togithub.com/bracesproul) in [https://github.com/langchain-ai/langchainjs/pull/4079](https://togithub.com/langchain-ai/langchainjs/pull/4079) - all\[patch]: Run prettier on scripts folders too by [@​bracesproul](https://togithub.com/bracesproul) in [https://github.com/langchain-ai/langchainjs/pull/4080](https://togithub.com/langchain-ai/langchainjs/pull/4080) - community\[patch]: Add fallback for missing metadata in Redis documents by [@​jacoblee93](https://togithub.com/jacoblee93) in [https://github.com/langchain-ai/langchainjs/pull/4085](https://togithub.com/langchain-ai/langchainjs/pull/4085) - core\[minor]: Buffer the first chunk on .stream() calls to catch streaming errors immediately on startup by [@​jacoblee93](https://togithub.com/jacoblee93) in [https://github.com/langchain-ai/langchainjs/pull/4083](https://togithub.com/langchain-ai/langchainjs/pull/4083) - langchain\[minor]: update sql_utils: add schema for mssql by [@​therajatg](https://togithub.com/therajatg) in [https://github.com/langchain-ai/langchainjs/pull/4006](https://togithub.com/langchain-ai/langchainjs/pull/4006) - langchain\[patch]: Correcting token text-splitting behavior by [@​matthewdparker](https://togithub.com/matthewdparker) in [https://github.com/langchain-ai/langchainjs/pull/4078](https://togithub.com/langchain-ai/langchainjs/pull/4078) - core\[patch]: Release 0.1.17 by [@​jacoblee93](https://togithub.com/jacoblee93) in [https://github.com/langchain-ai/langchainjs/pull/4088](https://togithub.com/langchain-ai/langchainjs/pull/4088) - community\[patch]: Release 0.0.19 by [@​jacoblee93](https://togithub.com/jacoblee93) in [https://github.com/langchain-ai/langchainjs/pull/4089](https://togithub.com/langchain-ai/langchainjs/pull/4089) - langchain\[patch]: Increase remote runnable default timeout by [@​jacoblee93](https://togithub.com/jacoblee93) in [https://github.com/langchain-ai/langchainjs/pull/4090](https://togithub.com/langchain-ai/langchainjs/pull/4090) - all\[patch]: add prettier config files by [@​bracesproul](https://togithub.com/bracesproul) in [https://github.com/langchain-ai/langchainjs/pull/4087](https://togithub.com/langchain-ai/langchainjs/pull/4087) - scripts\[major]: Add LangChain scripts package by [@​bracesproul](https://togithub.com/bracesproul) in [https://github.com/langchain-ai/langchainjs/pull/4093](https://togithub.com/langchain-ai/langchainjs/pull/4093) - scripts\[patch]: Revert version back to 0 by [@​bracesproul](https://togithub.com/bracesproul) in [https://github.com/langchain-ai/langchainjs/pull/4101](https://togithub.com/langchain-ai/langchainjs/pull/4101) - Release `@langchain/scripts` 0.0.1 by [@​bracesproul](https://togithub.com/bracesproul) in [https://github.com/langchain-ai/langchainjs/pull/4102](https://togithub.com/langchain-ai/langchainjs/pull/4102) - fixed issue [#​4097](https://togithub.com/langchain-ai/langchainjs/issues/4097) by [@​ritvikPuranik](https://togithub.com/ritvikPuranik) in [https://github.com/langchain-ai/langchainjs/pull/4098](https://togithub.com/langchain-ai/langchainjs/pull/4098) - langchain\[patch]: Allow headers to be passed in remote runnable requests, docs by [@​jacoblee93](https://togithub.com/jacoblee93) in [https://github.com/langchain-ai/langchainjs/pull/4107](https://togithub.com/langchain-ai/langchainjs/pull/4107) - langchain\[patch]: Release 0.1.5 by [@​jacoblee93](https://togithub.com/jacoblee93) in [https://github.com/langchain-ai/langchainjs/pull/4108](https://togithub.com/langchain-ai/langchainjs/pull/4108) #### New Contributors - [@​therajatg](https://togithub.com/therajatg) made their first contribution in [https://github.com/langchain-ai/langchainjs/pull/4006](https://togithub.com/langchain-ai/langchainjs/pull/4006) - [@​matthewdparker](https://togithub.com/matthewdparker) made their first contribution in [https://github.com/langchain-ai/langchainjs/pull/4078](https://togithub.com/langchain-ai/langchainjs/pull/4078) - [@​ritvikPuranik](https://togithub.com/ritvikPuranik) made their first contribution in [https://github.com/langchain-ai/langchainjs/pull/4098](https://togithub.com/langchain-ai/langchainjs/pull/4098) **Full Changelog**: langchain-ai/langchainjs@0.1.4...0.1.5 ### [`v0.1.4`](https://togithub.com/langchain-ai/langchainjs/releases/tag/0.1.4) [Compare Source](https://togithub.com/langchain-ai/langchainjs/compare/0.1.3...0.1.4) #### What's Changed - langchain\[patch]: Release 0.1.3 by [@​jacoblee93](https://togithub.com/jacoblee93) in [https://github.com/langchain-ai/langchainjs/pull/4031](https://togithub.com/langchain-ai/langchainjs/pull/4031) - core\[minor]: Add ability for runnable passthrough to call a func by [@​bracesproul](https://togithub.com/bracesproul) in [https://github.com/langchain-ai/langchainjs/pull/3998](https://togithub.com/langchain-ai/langchainjs/pull/3998) - core\[patch]: Allow runnable func to have async by [@​bracesproul](https://togithub.com/bracesproul) in [https://github.com/langchain-ai/langchainjs/pull/4033](https://togithub.com/langchain-ai/langchainjs/pull/4033) - core\[patch]: Fix return type of .pipe() by [@​nfcampos](https://togithub.com/nfcampos) in [https://github.com/langchain-ai/langchainjs/pull/4032](https://togithub.com/langchain-ai/langchainjs/pull/4032) - core\[patch]: Release 0.1.14 by [@​jacoblee93](https://togithub.com/jacoblee93) in [https://github.com/langchain-ai/langchainjs/pull/4040](https://togithub.com/langchain-ai/langchainjs/pull/4040) - Populate streamed_output for all runs handled by transformStreamWithConfig by [@​jacoblee93](https://togithub.com/jacoblee93) in [https://github.com/langchain-ai/langchainjs/pull/4042](https://togithub.com/langchain-ai/langchainjs/pull/4042) - all\[patch]: Fix `yarn lint:fix` everywhere, add to turbo by [@​bracesproul](https://togithub.com/bracesproul) in [https://github.com/langchain-ai/langchainjs/pull/4045](https://togithub.com/langchain-ai/langchainjs/pull/4045) - core\[minor]: Fix configs not getting passes into all batch invocations by [@​bracesproul](https://togithub.com/bracesproul) in [https://github.com/langchain-ai/langchainjs/pull/4047](https://togithub.com/langchain-ai/langchainjs/pull/4047) - core\[patch]: Suppress run map warnings by [@​jacoblee93](https://togithub.com/jacoblee93) in [https://github.com/langchain-ai/langchainjs/pull/4051](https://togithub.com/langchain-ai/langchainjs/pull/4051) - core\[patch]: Refactor patchConfig method and add ensureConfig by [@​jacoblee93](https://togithub.com/jacoblee93) in [https://github.com/langchain-ai/langchainjs/pull/4048](https://togithub.com/langchain-ai/langchainjs/pull/4048) - core\[patch]: Use new batch maxConcurrency parameter in runnable config by [@​jacoblee93](https://togithub.com/jacoblee93) in [https://github.com/langchain-ai/langchainjs/pull/4052](https://togithub.com/langchain-ai/langchainjs/pull/4052) - core\[patch]: Release 0.1.15 by [@​jacoblee93](https://togithub.com/jacoblee93) in [https://github.com/langchain-ai/langchainjs/pull/4053](https://togithub.com/langchain-ai/langchainjs/pull/4053) - community\[patch]: Add streaming for HuggingFace Inference (also add the support of stop-sequences) by [@​ljhskyso](https://togithub.com/ljhskyso) in [https://github.com/langchain-ai/langchainjs/pull/3932](https://togithub.com/langchain-ai/langchainjs/pull/3932) - Add the Milvus partition feature by [@​Mista-Shelby](https://togithub.com/Mista-Shelby) in [https://github.com/langchain-ai/langchainjs/pull/3950](https://togithub.com/langchain-ai/langchainjs/pull/3950) - Bump Zod & Zod-to-Json-Schema, fix imports by [@​StefanTerdell](https://togithub.com/StefanTerdell) in [https://github.com/langchain-ai/langchainjs/pull/4041](https://togithub.com/langchain-ai/langchainjs/pull/4041) - fix [#​4027](https://togithub.com/langchain-ai/langchainjs/issues/4027) loadFromPython failed by [@​ewfian](https://togithub.com/ewfian) in [https://github.com/langchain-ai/langchainjs/pull/4036](https://togithub.com/langchain-ai/langchainjs/pull/4036) - community\[patch]: Add neo4j timeout option by [@​tomasonjo](https://togithub.com/tomasonjo) in [https://github.com/langchain-ai/langchainjs/pull/4057](https://togithub.com/langchain-ai/langchainjs/pull/4057) - core\[patch]: Add fake embeddings classes to core by [@​jacoblee93](https://togithub.com/jacoblee93) in [https://github.com/langchain-ai/langchainjs/pull/4066](https://togithub.com/langchain-ai/langchainjs/pull/4066) - Update runnable config ensure and merge to match py by [@​nfcampos](https://togithub.com/nfcampos) in [https://github.com/langchain-ai/langchainjs/pull/4067](https://togithub.com/langchain-ai/langchainjs/pull/4067) - langchain\[minor]: remote runnable stream log, additional serialization updates by [@​bracesproul](https://togithub.com/bracesproul) in [https://github.com/langchain-ai/langchainjs/pull/3555](https://togithub.com/langchain-ai/langchainjs/pull/3555) - community\[minor]: feat: turbopuffer vector store by [@​mattzcarey](https://togithub.com/mattzcarey) in [https://github.com/langchain-ai/langchainjs/pull/3780](https://togithub.com/langchain-ai/langchainjs/pull/3780) - docs\[minor], core\[minor], langchain\[minor]: Adds tool use use case docs by [@​jacoblee93](https://togithub.com/jacoblee93) in [https://github.com/langchain-ai/langchainjs/pull/4068](https://togithub.com/langchain-ai/langchainjs/pull/4068) - core\[patch]: Release 0.1.16 by [@​jacoblee93](https://togithub.com/jacoblee93) in [https://github.com/langchain-ai/langchainjs/pull/4070](https://togithub.com/langchain-ai/langchainjs/pull/4070) - community\[patch]: Change recommended entrypoint, bump versions by [@​jacoblee93](https://togithub.com/jacoblee93) in [https://github.com/langchain-ai/langchainjs/pull/4069](https://togithub.com/langchain-ai/langchainjs/pull/4069) - community\[patch]: Release 0.0.18 by [@​jacoblee93](https://togithub.com/jacoblee93) in [https://github.com/langchain-ai/langchainjs/pull/4071](https://togithub.com/langchain-ai/langchainjs/pull/4071) #### New Contributors - [@​ljhskyso](https://togithub.com/ljhskyso) made their first contribution in [https://github.com/langchain-ai/langchainjs/pull/3932](https://togithub.com/langchain-ai/langchainjs/pull/3932) - [@​Mista-Shelby](https://togithub.com/Mista-Shelby) made their first contribution in [https://github.com/langchain-ai/langchainjs/pull/3950](https://togithub.com/langchain-ai/langchainjs/pull/3950) - [@​StefanTerdell](https://togithub.com/StefanTerdell) made their first contribution in [https://github.com/langchain-ai/langchainjs/pull/4041](https://togithub.com/langchain-ai/langchainjs/pull/4041) **Full Changelog**: langchain-ai/langchainjs@0.1.3...0.1.4 </details> <details> <summary>openai/openai-node (openai)</summary> ### [`v4.25.0`](https://togithub.com/openai/openai-node/blob/HEAD/CHANGELOG.md#4250-2024-01-21) [Compare Source](https://togithub.com/openai/openai-node/compare/v4.24.7...v4.25.0) Full Changelog: [v4.24.7...v4.25.0](https://togithub.com/openai/openai-node/compare/v4.24.7...v4.25.0) ##### Features - **api:** add usage to runs and run steps ([#​640](https://togithub.com/openai/openai-node/issues/640)) ([3caa416](https://togithub.com/openai/openai-node/commit/3caa4166b8abb5bffb4c8be1495834b7f16af32d)) ##### Bug Fixes - allow body type in RequestOptions to be null ([#​637](https://togithub.com/openai/openai-node/issues/637)) ([c4f8a36](https://togithub.com/openai/openai-node/commit/c4f8a3698dc1d80439131c5097975d6a5db1b4e2)) - handle system_fingerprint in streaming helpers ([#​636](https://togithub.com/openai/openai-node/issues/636)) ([f273530](https://togithub.com/openai/openai-node/commit/f273530ac491300842aef463852821a1a27805fb)) - **types:** accept undefined for optional client options ([#​635](https://togithub.com/openai/openai-node/issues/635)) ([e48cd57](https://togithub.com/openai/openai-node/commit/e48cd57931cd0e81a77b55653cb1f663111dd733)) ##### Chores - **internal:** debug logging for retries; speculative retry-after-ms support ([#​633](https://togithub.com/openai/openai-node/issues/633)) ([fd64971](https://togithub.com/openai/openai-node/commit/fd64971612d1d7fcbd8a63885d333485bff68ab1)) - **internal:** update comment ([#​631](https://togithub.com/openai/openai-node/issues/631)) ([e109d40](https://togithub.com/openai/openai-node/commit/e109d40a5c02c5bf4586e54d92bf0e355d254c1b)) </details> <details> <summary>prettier/prettier (prettier)</summary> ### [`v3.2.4`](https://togithub.com/prettier/prettier/blob/HEAD/CHANGELOG.md#324) [Compare Source](https://togithub.com/prettier/prettier/compare/3.2.3...3.2.4) [diff](https://togithub.com/prettier/prettier/compare/3.2.3...3.2.4) ##### Fix incorrect parser inference ([#​15947](https://togithub.com/prettier/prettier/pull/15947) by [@​fisker](https://togithub.com/fisker)) Files like `.eslintrc.json` were incorrectly formatted as JSONC files. <!-- prettier-ignore --> ```jsx // Input prettier --file-info .eslintrc.json { "ignored": false, "inferredParser": "jsonc" } // Prettier 3.2.4 prettier --file-info .eslintrc.json { "ignored": false, "inferredParser": "json" } ``` ### [`v3.2.3`](https://togithub.com/prettier/prettier/blob/HEAD/CHANGELOG.md#323) [Compare Source](https://togithub.com/prettier/prettier/compare/3.2.2...3.2.3) [diff](https://togithub.com/prettier/prettier/compare/3.2.2...3.2.3) ##### Throw errors for invalid code ([#​15881](https://togithub.com/prettier/prettier/pull/15881) by [@​fisker](https://togithub.com/fisker), [@​Josh-Cena](https://togithub.com/Josh-Cena), [@​auvred](https://togithub.com/auvred)) <!-- prettier-ignore --> ```ts // Input 1++; // Prettier 3.2.2 1++; // Prettier 3.2.3 SyntaxError: Invalid left-hand side expression in unary operation (1:1) > 1 | 1++; | ^ ``` <!-- prettier-ignore --> ```ts // Input try {} catch (error = 1){} // Prettier 3.2.2 try { } catch (error) {} // Prettier 3.2.3 SyntaxError: Catch clause variable cannot have an initializer. (1:23) > 1 | try {} catch (error = 1){} | ^ ``` ##### Fix parser inference ([#​15927](https://togithub.com/prettier/prettier/pull/15927) by [@​fisker](https://togithub.com/fisker)) <!-- prettier-ignore --> ```console // Prettier 3.2.2 prettier --file-info tsconfig.json { "ignored": false, "inferredParser": "json" } // Prettier 3.2.3 prettier --file-info tsconfig.json { "ignored": false, "inferredParser": "jsonc" } ``` </details> <details> <summary>kulshekhar/ts-jest (ts-jest)</summary> ### [`v29.1.2`](https://togithub.com/kulshekhar/ts-jest/blob/HEAD/CHANGELOG.md#2912-2024-01-22) [Compare Source](https://togithub.com/kulshekhar/ts-jest/compare/v29.1.1...v29.1.2) ##### Bug Fixes - calculated cache key based on `supportsStaticESM` ([a5d6f2d](https://togithub.com/kulshekhar/ts-jest/commit/a5d6f2d4f4bfa1c7c217f6faf9310958797b436f)) - correct error handling in `processAsync` ([e7be4bf](https://togithub.com/kulshekhar/ts-jest/commit/e7be4bf6a863cd76efec28f40fdd5193b2f50bad)), closes [#​4207](https://togithub.com/kulshekhar/ts-jest/issues/4207) - use `Config.ProjectConfig` ([918312b](https://togithub.com/kulshekhar/ts-jest/commit/918312bee22a795ec6bb347f95df4c1ff0a054de)), closes [#​4028](https://togithub.com/kulshekhar/ts-jest/issues/4028) </details> --- ### Configuration 📅 **Schedule**: Branch creation - "before 4am on Monday" in timezone America/Chicago, Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 👻 **Immortal**: This PR will be recreated if closed unmerged. Get [config help](https://togithub.com/renovatebot/renovate/discussions) if that's undesired. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://developer.mend.io/github/autoblocksai/javascript-sdk). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy4xMzUuMCIsInVwZGF0ZWRJblZlciI6IjM3LjEzNS4wIiwidGFyZ2V0QnJhbmNoIjoibWFpbiJ9--> Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
](https://renovatebot.com){kind=link}
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes # (issue)