@Test
    public void testAsyncGeneration() {
        var sb = new StringBuilder();
        var tokenCount = new AtomicInteger(0);
        var config = new LlmLocalClientConfig.Builder()
                .parallelRequests(1)
                .model(ModelReference.valueOf(model));
        var llm = new LocalLLM(config.build());

        try {
            var future = llm.completeAsync(prompt, defaultOptions(), completion -> {
                sb.append(completion.text());
                tokenCount.incrementAndGet();
            }).exceptionally(exception -> Completion.FinishReason.error);

            assertFalse(future.isDone());
            var reason = future.join();
            assertTrue(future.isDone());
            assertNotEquals(Completion.FinishReason.error, reason);

        } finally {
            llm.deconstruct();
        }
        assertTrue(tokenCount.get() > 0);
        System.out.println(sb);
    }
Tests for LocalLLM.
Tests are disabled to avoid running them automatically as part of the CI/CD.
The reason for this is the long time it takes to download (2-5 min) and run LLMs (10-60 seconds).
Still, these tests can be triggered manually and are useful for debugging runtime issues.

@author lesters
@author glebashnik
model-integration/src/test/java/ai/vespa/llm/clients/LocalLLMTest.java