azure - 我应该怎么做才能使 azure openai 返回 java 11 的流事件

Question

在使用 azure openai api 时，我使用此代码返回流响应:

public void getChatCompletion(SseEmitter emitter, String prompt, String azureOpenaiKey) {
        String endpoint = "https://xxxx.openai.azure.com/";
        String deploymentOrModelId = "xxx-ai";
        OpenAIClient client = new OpenAIClientBuilder()
                .endpoint(endpoint)
                .credential(new AzureKeyCredential(azureOpenaiKey))
                .buildClient();
        List<ChatMessage> chatMessages = new ArrayList<>();
        chatMessages.add(new ChatMessage(ChatRole.SYSTEM).setContent("You are a helpful assistant."));
        chatMessages.add(new ChatMessage(ChatRole.USER).setContent(prompt));
        ChatCompletionsOptions options = new ChatCompletionsOptions(chatMessages);
        options.setStream(true);
        options.setModel("gpt-3.5-turbo-0613");
        IterableStream<ChatCompletions> chatCompletions = client.getChatCompletionsStream(deploymentOrModelId, options);
        chatCompletions.forEach(completions->{
            try {
                emitter.send(completions);
            } catch (Exception e) {
                log.error("send emit message error", e);
            }
        });
    }

这段代码可以工作，但我发现它不是流结果，它仍然看起来像一个http请求，发送一个请求并返回完整的响应一次，是否可以逐字返回响应？我已经看过官方demo:https://learn.microsoft.com/en-us/java/api/overview/azure/ai-openai-readme?view=azure-java-preview#chat-completions 。我错过了什么吗？这是我使用 gradle 的 java 依赖项:

implementation'com.azure:azure-ai-openai:1.0.0-beta.2'

Answer 1

最后我找到了更好的返回流响应的方法，将客户端更改为异步以获得快速响应:

public void getAsyncChatCompletion(SseEmitter emitter, String prompt, String azureOpenaiKey) {
        String endpoint = "https://xxx.openai.azure.com/";
        String deploymentOrModelId = "xxx-ai";
        OpenAIAsyncClient client = new OpenAIClientBuilder()
                .endpoint(endpoint)
                .credential(new AzureKeyCredential(azureOpenaiKey))
                .buildAsyncClient();
        List<ChatMessage> chatMessages = new ArrayList<>();
        chatMessages.add(new ChatMessage(ChatRole.SYSTEM).setContent("You are a helpful assistant."));
        chatMessages.add(new ChatMessage(ChatRole.USER).setContent(prompt));
        ChatCompletionsOptions options = new ChatCompletionsOptions(chatMessages);
        options.setStream(true);
        options.setModel("gpt-3.5-turbo-0613");
        Flux<ChatCompletions> chatCompletions = client.getChatCompletionsStream(deploymentOrModelId, options);
        chatCompletions.subscribe(chatCompletion -> {
                    try {
                        SseChatService.getInstance().appendMsg(chatCompletion, RequestContext.getRequestId());
                        emitter.send(chatCompletion);
                    } catch (Exception e) {
                        log.error("send emit message error", e);
                    }
                },
                throwable -> {
                    log.error("flux response facing error:", throwable.getMessage());
                },
                () -> {
                    emitter.complete();
                });
    }