@Test
    public void testParallelGenerationWithLargeContext() {
        var llmConfig = new LlmLocalClientConfig.Builder()
                .parallelRequests(3)
                .contextSize(3 * 1024)
                .maxQueueSize(2)
                .seed(42)
                .contextOverflowPolicy(LlmLocalClientConfig.ContextOverflowPolicy.NONE)
                .model(ModelReference.valueOf(MEDIUM_LLM_PATH));

        var llm = new LocalLLM(llmConfig.build());
        var executor = Executors.newFixedThreadPool(5);
        var futures = new ArrayList<CompletableFuture<Void>>();
        
        try {
            for (var task : TASKS) {
                var future = CompletableFuture.runAsync(() -> 
                        new CompletionTest(llm, task.input)
                                .expectOutput(task.output)
                                .expectFinishReason(Completion.FinishReason.stop)
                                .test()
                        , executor);
                futures.add(future);
            }

            for (var future : futures) {
                future.join();
            }

            executor.shutdown();
        } finally {
             llm.deconstruct();
        }
    }
Tests for LocalLLM.
Tests are disabled to avoid running them automatically as part of the CI/CD.
The reason for this is the long time it takes to download (2-5 min) and run LLMs (10-60 seconds).
Still, these tests can be triggered manually and are useful for debugging runtime issues.

@author lesters
@author glebashnik
model-integration/src/test/java/ai/vespa/llm/clients/LocalLLMTest.java