-
Notifications
You must be signed in to change notification settings - Fork 16k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Experimental]: Async agenerate method ollama functions #21682
[Experimental]: Async agenerate method ollama functions #21682
Conversation
The latest updates on your projects. Learn more about Vercel for Git ↗︎ 1 Ignored Deployment
|
@baskaryan, @efriis, @eyurtsev, @hwchase17 please review |
…generate method
@@ -354,6 +357,86 @@ def _generate( | |||
generations=[ChatGeneration(message=response_message_with_functions)] | |||
) | |||
|
|||
async def _agenerate( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Isn't this duplicating the implementation of the sync method? I thought BaseModel already takes care of this automatically under the hood?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I can't tell if this code is doing anything that the default base implementation isn't doing already.
It looks like the original issue has to do with serialization and that something is not serializable. Have you verified that adding an async implementation makes a difference?
system_message = system_message_prompt_template.format( | ||
tools=json.dumps(functions, indent=2) | ||
) | ||
response_message = await super()._agenerate( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@eyurtsev this is the difference, I am calling the async generate method which solves the serialization issue.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also I tested the async method on my machine and it solves the issue: #2222
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You can try it on your machine as well!!!
@eyurtsev please see |
what is the release schedule for this? |
- **Description:** : Added Async method for Generate for OllamaFunctions which was missing and was raising errors for the users. - **Issue:** #21422
Description: :
Added Async method for Generate for OllamaFunctions which was missing and was raising errors for the users.
Issue:
OllamaFunctions returning type Error when using with_structured_output #21422