Question
Implement forward and backward matrix multiplication across `world` workers. Split `A` evenly by rows, replicate `B`, compute each local `A_i @ B`, and concatenate the local outputs in row order. For the backward pass, compute each local `dA_i = dY_i @ B.T`, concatenate the `dA_i` blocks, and compute the global `dB` as the sum of `A_i.T @ dY_i` across workers. Return results that match an unsharded NumPy implementation. Validate matrix shapes and explain the communication required by the forward and backward passes. ## Follow-ups 1. Shard `B` instead and compare communication costs. 2. Add uneven row partitions and preserve output ordering. 3. Explain how numerical tolerance and collective failures affect debugging.