Reduction 1: CPU Sum Baseline
Add 4,194,304 floats into a single number on one CPU core, and record how long it takes. Every attention softmax, every LayerNorm, and every loss value in a Transformer starts with a reduction like this one.
Sign in to run this lab. GPU availability and credits are shown in the app.
Languages: C++, Python, Rust
Lesson
The editor contains one host function and no CUDA. Fill in the loop so the function returns the sum of the first N values of X, with N = 4,194,304:
```
sum = X[0] + X[1] + ... + X[N - 1]
```
The shape is different from vector add, and the difference is the whole point of this path. Vector add produced **one output per input index**: element 5 wrote to slot 5 and never looked at slot 4. A reduction produces **one output for the entire input**, so every element has to reach the same destination. On one core that is easy, because the loop visits them in order and there is only one accumulator. It is exactly this that forces CUDA threads to coordinate in the next stage.
GPUFlight allocates the input, checks your returned value against a trusted reference, and reports the elapsed time of your function. That time is evidence for the comparison ahead, not a target to beat.
Accumulation is FP32. Parallel regrouping changes rounding, so compare using the runner tolerance rather than bitwise equality; the read-only driver specifies the check.
What you will learn
- Write a serial accumulation loop that folds an array into one value.
- Explain why a reduction has a cross-element dependency that an elementwise operation does not.
- Establish a CPU time for the same element count the CUDA stages will use.
Starter code
// One running total for the whole array, not one output slot per element.
float total = 0.0f;
for (int i = 0; i < N; ++i) {
// TODO: add X[i] into total.
}
return total;