Transfer Churn: Stage the Data Once, Not Per Row

The starter copies 64 CPU rows to CUDA one at a time, computes each row sum, and returns a CUDA tensor. Stage the full input once and perform the equivalent row reductions on the device.

Languages: PyTorch

Lesson

Host-to-device (H2D) copies cross the PCIe bus, which is far slower than on-device memory. A loop that calls `.to('cuda')` on each row issues one H2D transfer per iteration, and the transfers - not the math - set the pace. Moving the entire input to the GPU once (`data.to('cuda')`) and then reducing it with a batched op (`(g * 2 + 1).sum(dim=1)`) does the identical work with a single H2D transfer. GPUFlight measures the H2D transfer count inside the judged window, so the staged version passes while the per-row copy loop does not.

Return a CUDA float32 tensor of shape [64]. CPU results and lists are rejected. Stage data once and keep reductions on the GPU. Both H2D and D2H counts are measured inside judged. This is contract validation, not a security proof of how arbitrary code computed the result.

What you will learn

Recognize `.to('cuda')` inside a loop as a per-iteration PCIe transfer.
Distinguish transfer-bound from compute-bound work.
Stage an input to the device once and keep the computation device-resident.
Read host-to-device transfer count as a signal of transfer churn.

Starter code

    out = torch.empty(data.shape[0], device='cuda')
    for i in range(data.shape[0]):
        row = data[i].to('cuda')
        out[i] = (row * 2.0 + 1.0).sum()
    return out