@Test
    public void testParallelGenerationWithSmallContextOverflowPolicyDiscard() {
        var llmConfig = new LlmLocalClientConfig.Builder()
                .parallelRequests(5)
                .contextSize( 5 * 100)
                .maxTokens(50)
                .seed(42)
                .contextOverflowPolicy(LlmLocalClientConfig.ContextOverflowPolicy.DISCARD)
                .model(ModelReference.valueOf(MEDIUM_LLM_PATH));

        var llm = new LocalLLM(llmConfig.build());
        var executor = Executors.newFixedThreadPool(5);
        var futures = new ArrayList<CompletableFuture<Void>>();

        try {
            for (var i : List.of(0, 2, 3, 4, 5)) {
                futures.add(CompletableFuture.runAsync(
                        () -> new CompletionTest(llm, TASKS.get(i).input)
                                .expectOutput(TASKS.get(i).output)
                                .expectFinishReason(Completion.FinishReason.stop)
                                .test()
                ));
            }

            for (var i : List.of(1, 6)) {
                futures.add(CompletableFuture.runAsync(
                        () -> new CompletionTest(llm, TASKS.get(i).input)
                                .expectOutput("")
                                .expectFinishReason(Completion.FinishReason.discard)
                                .test()
                ));
            }
            
            for (var future : futures) {
                future.join();
            }

            executor.shutdown();
        } finally {
            llm.deconstruct();
        }
    }
Tests for LocalLLM.
Tests are disabled to avoid running them automatically as part of the CI/CD.
The reason for this is the long time it takes to download (2-5 min) and run LLMs (10-60 seconds).
Still, these tests can be triggered manually and are useful for debugging runtime issues.

@author lesters
@author glebashnik
model-integration/src/test/java/ai/vespa/llm/clients/LocalLLMTest.java