I Added torch.compile. Why Is It Still Slow?

Lesson

The starter compiles a preprocessing chain with an intentional scalar extraction. Rewrite the chain so the mean stays on the device, compile once, and reuse the callable. Cold latency includes compilation; five paired trials compare warmed execution against the eager scalar-reading baseline. COMPILER_DIAGNOSTICS comes from a separate torch._dynamo.explain run and names graph-break causes; a GPU gap alone cannot establish them. Inspect launches and allocation peaks separately. The worker dependency lock pins PyTorch; ENVIRONMENT records the actual runtime version. Internal compiler diagnostics are version-sensitive.

Measurement contract: the real read-only driver defines WORKLOAD. Two warmups precede five paired trials with alternating baseline/candidate order. Each trial uses fresh deterministic inputs. Synchronized host time includes solve and completion, excludes input construction and reference verification. METRICS reports every sample, median and IQR; memory includes persistent buffers and allocator state. One additional solve is isolated in judged for Trace. No speedup is promised. Run the unchanged starter first.

What you will learn

• Identify the bottleneck from the appropriate evidence.
• Preserve the output contract on fresh inputs.
• Compare paired measurements and numerical error.

Starter code

    state = case['_state']
    if 'compiled' not in state:
        state['compiled'] = torch.compile(broken)
    return state['compiled'](case['x'])