Question
Compute the product of two matrices on a group of devices, two ways, and compare both results with a product computed on one device. ## Requirements - The device count divides both dimensions. One device is the local product. - Data-parallel: each device multiplies its row slice of the left matrix by the full right matrix and sends that block to rank zero, which stacks the rows. - Sharded: the left matrix is split by rows and the right matrix by columns. Over as many rounds as there are devices, each device passes its right-hand shard to the neighbor and adds that partial into the matching columns of the piece it keeps. - The communicator is a set of blocking queues. Do not have every device send before any device receives. - Each rank must know which column slice it currently holds. Send the owner index with the shard. - Accumulate into an array with the same numeric type as the expected product. ## Review checklist 1. Does the one-device case match the local product exactly? 2. Does rank zero's stacked result match the local product for the row-split form? 3. After every rotation, does each shard still land in the column range named by its owner index? 4. Can a send and a receive on the same rank overlap so the queues do not fill and stall? ## Staged follow-ups 1. Implement the row-split form and the comparison with the local product. 2. Implement the row-and-column shard form and the rotation. 3. Change the queue use so a receive is posted before the matching send. 4. State which rank owns the final assembly in each form.