Reported interview periodApr 2026–Aug 2026
Online softmax entropy
Implement numerically stable softmax, log-softmax, and entropy for a vector of logits. The standard entropy is -Σ p log p.
Practice focusThen implement a block-wise variant that processes the logits in bounded-size chunks without losing numerical stability. The block-wise result must match the full-vector computation within floating-point tolerance. Handle very large and very small logits, empty inputs according to the provided contract, and both supported entropy-sign conventions. Explain why subtracting the maximum logit prevents overflow and how partial statistics combine across blocks. Follow-ups 1. Return gradients with respect to the logits. 2.…