synchronized HttpException createUpstreamUnavailableException() {
        int busyUpstreamsCount = 0;
        for (UpstreamState upstreamState : upstreamStates) {
            if (upstreamState.getStatus() == HttpStatus.TOO_MANY_REQUESTS) {
                busyUpstreamsCount++;
            }
        }
        if (busyUpstreamsCount == upstreamStates.size()) {
            long replyAfter = -1;
            for (UpstreamState upstreamState : upstreamStates) {
                if (upstreamState.getStatus() == HttpStatus.TOO_MANY_REQUESTS
                        && upstreamState.getSource() == UpstreamState.RetryAfterSource.UPSTREAM) {
                    if (replyAfter == -1) {
                        replyAfter = upstreamState.getRetryAfter();
                    } else {
                        replyAfter = Math.min(replyAfter, upstreamState.getRetryAfter());
                    }
                }
            }
            String errorMessage = "Service is not available";
            if (replyAfter == -1) {
                // no upstreams with the requested source
                // we don't provide reply-after header
                return new HttpException(HttpStatus.SERVICE_UNAVAILABLE, errorMessage);
            } else {
                // according to the spec: A non-negative decimal integer indicating the seconds to delay after the response is received.
                long replyAfterInSeconds = Math.max(0, TimeUnit.MILLISECONDS.toSeconds(replyAfter - System.currentTimeMillis()));
                return new HttpException(HttpStatus.SERVICE_UNAVAILABLE, errorMessage,
                        Map.of(HttpHeaders.RETRY_AFTER.toString(), Long.toString(replyAfterInSeconds)));
            }
        }
        // default error - no route
        return new HttpException(HttpStatus.BAD_GATEWAY, "No route");
    }
Tiered load balancer. Each next() call returns an available upstream from the highest tier (lowest tier value in config).
If the whole tier (highest) is unavailable, balancer start routing upstreams from next tier (lower) if any.
server/src/main/java/com/epam/aidial/core/server/upstream/TieredBalancer.java