Reported interview periodNot available
Split matrix product
Compute the product of two matrices on a group of devices, two ways, and compare both results with a product computed on one device.
Practice focusRequirements - The device count divides both dimensions. One device is the local product. - Data-parallel: each device multiplies its row slice of the left matrix by the full right matrix and sends that block to rank zero, which stacks the rows. - Sharded: the left matrix is split by rows and the right matrix by columns. Over as many rounds as there are devices, each device passes its right-hand shard to the neighbor and adds that partial into the matching columns of the piece it keeps. - The communicator is a set of blocking queues.…