Circuit Breaker and Replica Failover

Question

Implement a circuit breaker that protects a dependency after repeated failures, then use it to route work across database replicas.

```python
class CircuitBreaker:
    def __init__(self, failure_threshold: int, recovery_timeout: float, clock=None): ...
    def state(self) -> str: ...
    def call(self, supplier): ...
```

## Requirements

- `CLOSED` permits calls. A success resets the consecutive-failure count.
- After `failure_threshold` consecutive supplier failures, transition to `OPEN` and propagate the failure.
- While `OPEN`, reject calls without invoking the supplier.
- Raise the provided `CircuitOpenError` for breaker-generated denial; propagate supplier errors unchanged. An error message containing the word `OPEN` is not a denial signal.
- After `recovery_timeout`, permit one `HALF_OPEN` probe. Success closes the breaker; failure reopens it and restarts the timeout.
- The optional monotonic `clock` returns the current time and enables deterministic testing.
- State transitions must be safe if several callers arrive concurrently.

Suppliers run outside the state lock. Concurrent closed-state completions update failure counts in completion order. Once failures open a new epoch, older in-flight completions still return or raise to their callers but must not alter that newer state. Only one half-open supplier may be in flight; other callers are rejected without invoking their suppliers. A probe failure, including an interrupted supplier, releases the probe slot and restarts the open timeout. `state()` may expose `HALF_OPEN` once the timeout elapses, before any probe starts. Assume positive integer thresholds, nonnegative finite timeouts and a monotonic clock.

Then implement:

```python
class ReplicaRouter:
    def __init__(self, recovery_timeout: float, clock=None): ...
    def execute(self, callbacks: dict): ...
```

Try `primary`, then replicas `a` and `b`, each with threshold `2`. Return `(name, value)` from the first successful callback; raise `RuntimeError` if none can serve the request.

## Follow-ups

1. Prevent a thundering herd of half-open probes.
2. Add metrics and distinguish timeout, overload, and application failures.
3. Discuss retry budgets, backoff, and whether every replica should share one breaker.