Reported interview periodOct 2025–Apr 2026
Debug a GRPO Training Step
A small PyTorch implementation of Group Relative Policy Optimization runs without exceptions but learns poorly. Inspect and correct its sampling, advantage normalization, and clipped policy loss while preservin…
Practice focusRequired behavior - Convert logits to probabilities with softmax before sampling tokens. - For each prompt group, compute A i = (r i - mean(r)) / (std(r) + eps) using a positive epsilon so zero-variance groups remain finite. - Compute the per-token importance ratio as exp(current logprob - old logprob) . - Exclude masked tokens from both the loss sum and averaging denominator. - Compute the negative mean of the minimum between the unclipped and clipped ratio times advantage. - Return a finite zero loss if every token is masked.…