Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding streaming tool use support for groq #1653

Draft
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

colemccracken
Copy link

No description provided.

@colemccracken colemccracken marked this pull request as draft May 19, 2024 22:43
@colemccracken colemccracken marked this pull request as ready for review May 20, 2024 15:15
@colemccracken colemccracken marked this pull request as draft May 20, 2024 22:43
@lgrammel lgrammel marked this pull request as ready for review May 27, 2024 12:53
@lgrammel lgrammel marked this pull request as draft May 27, 2024 12:55
@lgrammel
Copy link
Collaborator

@colemccracken i tried the stream object example with groq but it didn't work. what am i missing?

@colemccracken
Copy link
Author

@lgrammel sorry for the delay. I realized there were a couple issues.

One is that Groq was sending an initial chunk with content: "", which was tripping up the sdk. Theres no reason for this and openai doesnt do it, so going to deploy this fix on Groq's side, should land tomorrow.

The second is that Groq sends the entire tool call payload in a single delta, which I realized wasnt supported in openai's handler. I pushed a commit to the ai sdk that handles this- but understand if you dont want to have this logic given this is an openai specific sdk. Let me know what you think.

Anyways, Ill add another comment on this pr once we've deployed the Groq fix to production and the example should work.

@colemccracken colemccracken changed the title Adding streaming tool use support to groq docs Adding streaming tool use support for groq Jun 2, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants