Update tool callbacks to send the actual response from the tool

[eyurtsev]

Privileged issue

• I am a LangChain maintainer, or was asked directly by a LangChain maintainer to create an issue here.

Issue Content

Reproducible issue. on_tool_end logs the string representation of the tool. All tool callbacks need to be updated to accommodate

from langchain_core.tools import tool
from langchain_core.documents import Document

def foo(x):
    return {
        'x': 5
    }

@tool
def get_docs(x: int):
    """Hello"""
    return [Document(page_content='hello')]

chain = foo | get_docs

async for event in chain.astream_events({}, version='v1'):
    if event['event'] == 'on_tool_end':
        print(event)

[eyurtsev]

#18694

[eyurtsev]

This is a bug in the astream_events API and an enhancement for existing tool callback handlers

[keenborder786]

@eyurtsev I am taking this and working on it.

[eyurtsev]

@keenborder786 thanks for taking a look at this! The fix is actually more involved since it needs to be backwards compatible with existing callback handlers. I'm going to scope it out today

[keenborder786]

@eyurtsev oh okay. Can you please explain a little about the backwards compatibility in this case so I can help you as well. All the existing handlers that are using on_tool_end need to work properly??? Correct.

[eyurtsev]

Any code that relies on the output from the handlers should not break

[keenborder786]

Okay. Get Outlook for Android<https://aka.ms/AAb9ysg>

…

________________________________ From: Eugene Yurtsev ***@***.***> Sent: Friday, March 8, 2024 8:59:08 PM To: langchain-ai/langchain ***@***.***> Cc: Mohammad Mohtashim Khan ***@***.***>; Mention ***@***.***> Subject: Re: [langchain-ai/langchain] Update tool callbacks to send the actual response from the tool (Issue #18760) CAUTION: This email has originated from outside of LUMS. Do not click links or open attachments unless you recognize sender’s email address and know the content is safe. Any code that relies on the output from the handlers should not break — Reply to this email directly, view it on GitHub<#18760 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AKZFN65Z4NUICCEXMS3C6ITYXHN4ZAVCNFSM6AAAAABELY4UW2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSOBVHE2TAMZVHE>. You are receiving this because you were mentioned.Message ID: ***@***.***>

[eyurtsev]

I'm looking through to figure out what can be affected. For example, one thing is BaseTracer which is used to power both astream_log and astream_events. If we change the output of the on_tool_end, we will fix the output from astream_events, but we also might cause an unexpected breaking change (e.g., if users are sending the output of these APIs from the backend to the front end suddenly the code might stop serializing properly).

it's a bit tricky in some of these cases to figure out whether to make a breaking change (and keep the code simple) or make a more involved change so that we don't break user code

[eyurtsev]

OK @keenborder786 I think we can proceed with the PR that you made! I'm going to outline a few places to propagate information to so that we break the minimum amount of code

[eyurtsev]

PR merged will be available in next release

[alex-dreo-persefoni]

@eyurtsev I don't see this issue (#18760) referenced in the latest release: https://github.com/langchain-ai/langchain/releases/tag/v0.1.12

When I test the original code that output a str(Document), I still see the same problematic output:

Can you confirm that this issue was not fixed by the latest release?

[alex-dreo-persefoni]

It looks like this PR might have to be merged before this issue can actually be corrected in a release. Is that correct?