Question
Implement numerically stable cross-entropy for class logits, with optional masking and label smoothing.
```python
def cross_entropy(
logits: np.ndarray,
targets: np.ndarray,
mask: np.ndarray | None = None,
epsilon: float = 0.0,
) -> float: ...
```
## Requirements and assumptions
- `logits` has shape `(N,C)` or `(N,T,C)`; integer targets and mask match its leading dimensions.
- Compute mean negative log probability only over positions where the mask is nonzero; without a mask, include all positions.
- Label smoothing uses target distribution `(1-epsilon) * one_hot + epsilon/C`.
- Use log-sum-exp; do not form unstable exponentials or probabilities unnecessarily.
- Validate shapes, target range, `0 <= epsilon < 1`, and define the all-masked result explicitly (raise `ValueError`).
- Preserve input dtype where sensible while accumulating safely.
## Follow-ups
1. Return gradients with respect to logits.
2. Add class weights and ignore-index semantics.
3. Compare smoothing with a KL-divergence objective.
4. Implement without materializing one-hot targets.