@Nullable
    synchronized Upstream next(Set<Upstream> usedUpstreams) {
        for (RandomizedWeightedBalancer tier : tiers) {
            Upstream upstream = tier.next();
            if (upstream != null) {
                return upstream;
            }
        }
        // fallback
        for (Predicate<UpstreamState> p : predicates) {
            UpstreamState candidate = upstreamStates.stream().filter(p)
                    .filter(upstreamState -> !usedUpstreams.contains(upstreamState.getUpstream()))
                    .min(Comparator.comparingLong(UpstreamState::getRetryAfter)).orElse(null);
            if (candidate != null) {
                usedUpstreams.add(candidate.getUpstream());
                return candidate.getUpstream();
            }
        }
        return null;
    }
Tiered load balancer. Each next() call returns an available upstream from the highest tier (lowest tier value in config).
If the whole tier (highest) is unavailable, balancer start routing upstreams from next tier (lower) if any.
server/src/main/java/com/epam/aidial/core/server/upstream/TieredBalancer.java