Hidden Sync: Stop Reading Scalars Off the GPU Every Step
For a CUDA tensor, .item() makes the calling host wait until the scalar is available. Repeated reads can interrupt enqueueing and reduce overlap. Other independent GPU work may continue. Accumulate on the device and return one Python float after the final read.
Sign in to run this lab. GPU availability and credits are shown in the app.
Languages: PyTorch
Lesson
The starter reads a scalar each step. Keep the accumulator on CUDA and call .item() when the result is needed on the host. Fewer reads do not guarantee a proportional speedup: compare D2H counts and synchronized elapsed time. solve must return a finite Python float within judged; returning a CUDA scalar for the harness to read later is rejected. This repeated-input warm-up deliberately accepts algebraically equivalent GPU reductions; the newer input-pipeline labs vary data between batches.
What you will learn
- Recognize `.item()` / `float()` / `print()` on a GPU tensor as a D2H sync.
- See how a per-step read serializes the loop against the host.
- Accumulate into a device tensor and read the result back once.
- Read device-to-host transfer count as a signal of hidden synchronization.
Starter code
total = 0.0
for _ in range(steps):
total += x.sum().item()
return total