Question

I understand tool call results are sent as ToolReturnPart objects.

In the docs, under Function Tool Output:

Some models (e.g. Gemini) natively support semi-structured return values, while some expect text (OpenAI) but seem to be just as good at extracting meaning from the data. If a Python object is returned and the model expects a string, the value will be serialized to JSON.

However I want to have some metadata associated with the tool response that I can reference. An example use-case is sending back str formatted tool response to the model but retaining structured output in the messages.

Langchain handles this like so.

Is there a way to achieve this currently?

Additional Context

No response