Thanks @tongyu0924 for taking this up. Though I may need to walk back from the original issue description. There a several things I want to first understand before adding the field.

1. For OpenAI, Ollama, Anthropic, and Gemini, do they all have request IDs in their responses? Looking at OpenAI's doc, it seems the field is id: https://platform.openai.com/docs/api-reference/chat/object. There isn't an ID field in ollama's response: https://github.com/ollama/ollama/blob/main/docs/api.md#generate-a-chat-completion. What about Gemini and Anthropic? We need to investigate before we add this field.
2. Is having an id or request_id field sufficient? Is there any benefit in including the raw response as a dictionary from the endpoint in the CreateResult? This way, we aren't bound to a specific model provider.

@abhinav-aegis you raised this issue originally, what's your thought?

Not all LLM APIs do have an ID in the completion (equivalent API). I know Anthropic does have an ID field: https://docs.anthropic.com/en/api/messages#response-id and Ollama does not. I have not used Gemini but a quick look through their API shows that Gemini does not support a response ID. There are two types of issues:

1. The API/LLM does not include anything that could be considered an ID.
2. The ID is called something else.

In the first case, not much can really be done in terms of building audit trails for security. In the later case, we can of course use the model client implementation to do the mapping. In either situation, I am not sure how storing the raw dictionary helps the case of tracking IDs for the purpose of security/audit trails.

I do feel that as the ecosystem matures, having a response ID in the LLM API would start to become standard. Also, LLM proxies/gateways like LiteLLM do have support for a response ID field. Applications would probably introduce proxies in general and introduce their own ID's in the completion response API allowing them to use any LLM like Gemini or ollama in the background.

For the time being, having an optional Response ID ensures that AgentChat supports what is needed and it becomes the responsibility of the application/LLM vendor to provide/use an ID.

There maybe separate usecases for storing the raw dictionary and I can't comment on that - but for the purpose of audit trails/security having an optional field of Response Id would be the best approach. I don't see the two items as mutually exclusive anyways.

For the time being, having an optional Response ID ensures that AgentChat supports what is needed and it becomes the responsibility of the application/LLM vendor to provide/use an ID.

It is good thing to have a unique identifier though we can just adopt a vendor-specific identifier scheme. I am more inclined to just include the raw model response as a new entry in CreateResult so if the application assumes a specific model vendor, the application itself can understand the raw response.

For the time being, having an optional Response ID ensures that AgentChat supports what is needed and it becomes the responsibility of the application/LLM vendor to provide/use an ID.

It is good thing to have a unique identifier though we can just adopt a vendor-specific identifier scheme. I am more inclined to just include the raw model response as a new entry in CreateResult so if the application assumes a specific model vendor, the application itself can understand the raw response.

This is okay too - we can work with this option as long as the ID is available somewhere. Probably useful in general to have the raw response.

Having said that, having a standard Agentchat "Response ID" field to track all LLM responses would make the life of the security team a lot easier to track what is going on in the system. If it is model specific, it will start to become very complicated - particularly across applications. Will leave it to you to decide how best to proceed.

Thanks! I will close this issue for now.

@tongyu0924 you can create a separate PR for adding raw model response to CreateResult.

Thanks! I will close this issue for now.

@tongyu0924 you can create a separate PR for adding raw model response to CreateResult.

Thanks for the clarification!
I'll go ahead and create a PR to add the raw model response to CreateResult.