@Override
    public CompletableFuture<Completion.FinishReason> completeAsync(
            Prompt prompt, InferenceParameters parameters, Consumer<Completion> consumer) {
        var preparedParameters = prepareParameters(parameters);
        String apiKey = preparedParameters.getApiKey().orElse(DEFAULT_API_KEY);
        String endpoint = preparedParameters.getEndpoint().orElse(DEFAULT_ENDPOINT);
        
        OpenAIClientAsync client = getAsyncClient(apiKey, endpoint);
                
        ChatCompletionCreateParams createParams = getChatCompletionCreateParams(preparedParameters, prompt);
        
        final Completion.FinishReason[] lastFinishReasonHolder = new Completion.FinishReason[]{Completion.FinishReason.stop};
        CompletableFuture<Completion.FinishReason> future = new CompletableFuture<>();
                
        // Use streaming API
        client.chat()
                .completions()
                .createStreaming(createParams)
                .subscribe(completion -> completion.choices().stream()
                    .flatMap(choice -> {
                        // Capture the finish reason if present
                        choice.finishReason().ifPresent(fr -> {
                            lastFinishReasonHolder[0] = mapFinishReason(fr.toString());
                        });
                        // Process delta content
                        return choice.delta().content().stream()
                            .map(content -> new Completion(content, 
                                choice.finishReason().map(fr -> mapFinishReason(fr.toString())).orElse(Completion.FinishReason.none)
                            ));
                    })
                    .forEach(consumer))
                .onCompleteFuture()
              .thenAccept(unused -> {
                  // When the stream completes, resolve the future with the last known finish reason
                  future.complete(lastFinishReasonHolder[0]);
              })
              .exceptionally(e -> {
                  future.completeExceptionally(e);
                  return null;
              });
        
        return future;
    }
A configurable OpenAI client that extends the {@link ConfigurableLanguageModel} class.
Uses Official OpenAI java client (https://github.com/openai/openai-java)
Supports basic completion and structured JSON output. Extensible to support Embedding, Tool Calling and Moderations.
Will reuse clients for Completions using same endpoint and API key to reduce connection overhead for multiple requests to the same endpoint with the same API key.
@author lesters
@author glebashnik
@author thomasht86
model-integration/src/main/java/ai/vespa/llm/clients/OpenAI.java