-
-
Notifications
You must be signed in to change notification settings - Fork 2.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix triton streaming completions bug #8386
Fix triton streaming completions bug #8386
Conversation
The latest updates on your projects. Learn more about Vercel for Git ↗︎
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
pls add a test in llm translation
bump on this @minwhoo |
Added streaming test |
Thanks @minwhoo can you share a screenshot of it passing. Should be good to merge once that's up! 👍 Appreciate your work on this |
@krrishdholakia Sure thing. Managed to catch a mistake I made. Would this screenshot suffice? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
Hi @minwhoo would you be open to working with us on LiteLLM as a founding engineer ? (I'm impressed by your work on this) If this sounds interesting here's my cal, please book some time: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat |
@ishaan-jaff Hey, thanks for the offer—really appreciate it! But at the moment I ’m unable to take on any new commitments. Really great product though. Best of luck on your startup! |
Title
Reimplements triton streaming handling code that got lost during refactoring.
Relevant issues
Fixes #8362
Type
🐛 Bug Fix