Elementwise 1: CPU Vector Add Baseline
Add two arrays element by element on one CPU core, and record how long it takes. This is the same shape a Transformer uses when it adds an attention or MLP output back into the residual stream.
Sign in to run this lab. GPU availability and credits are shown in the app.
Languages: C++, Python, Rust
Lesson
The editor contains one host function and no CUDA. Fill in the loop so that C[i] = A[i] + B[i] for every element, with N = 16,777,216.
Nothing here depends on anything else: element 5 does not need element 4. That independence is the whole reason this operation can be split across thousands of CUDA threads in the next stage, and it is worth noticing now while the code is still a single loop.
GPUFlight allocates the inputs, checks hidden output locations, and reports the elapsed time of your function. That time is evidence for the comparison ahead, not a target to beat.
For a tiny example, [1,2] + [3,4] = [4,6]. This timing is a single-core scalar reference, not a comparison with optimized CPU vector libraries.
What you will learn
- Write an elementwise loop over a flat array.
- Recognize that no element depends on another, which is what makes the operation parallel.
- Establish a CPU time for the same element count the CUDA stage will use.
Starter code
// Walk every element once. Nothing here depends on any other element.
for (int i = 0; i < N; ++i) {
// TODO: write the sum of A[i] and B[i] into C[i].
}