Question
Implement numerically stable softmax, log-softmax, and entropy for a vector of logits. The standard entropy is `-Σ p log p`. Then implement a block-wise variant that processes the logits in bounded-size chunks without losing numerical stability. The block-wise result must match the full-vector computation within floating-point tolerance. Handle very large and very small logits, empty inputs according to the provided contract, and both supported entropy-sign conventions. Explain why subtracting the maximum logit prevents overflow and how partial statistics combine across blocks. ## Follow-ups 1. Return gradients with respect to the logits. 2. Merge partial statistics computed independently on several devices. 3. Compare two-pass and online algorithms for accuracy and memory use.