Add support for StreamingChatResponseHandler

[Plawn]

Today when using

import dev.langchain4j.model.chat.response.StreamingChatResponseHandler;
import dev.langchain4j.service.MemoryId;
import dev.langchain4j.service.SystemMessage;
import dev.langchain4j.service.UserMessage;


public interface Agent {
    @SystemMessage("""
       <message>
            """)
    String chat(@MemoryId String memoryId, @UserMessage String message, StreamingChatResponseHandler handler);
}

the StreamingChatResponseHandler is not used / bound

cf: https://docs.langchain4j.dev/tutorials/response-streaming

[geoand]

Currently you can leverage streaming by returning Multi<String> from an AiService

[Plawn]

Yes !
I wanted to use the OnToolCall handler in order to provide nice responsive UI telling the user, what is happening :)

[datenschauer]

Well, you could use TokenStream like so:

TokenStream chat(@MemoryId int memoryId, @UserMessage String userMessage);

And use it in your application:

TokenStream stream = model.chat(chatId, userMessage);

    stream.onPartialResponse(token -> {
          // UPDATE your UI partially
          })
        .onCompleteResponse(response -> {
          // handle complete response ...
        })
        .onError(error -> logger.error(error.getMessage()))
        .start();

This works fine for me.