1. What decides whether a workload fits a CPU or a GPU?
Show answer
B — CPUs suit general control flow, preprocessing, and modest workloads. GPUs suit dense tensor operations with high parallelism and bandwidth demand. Parameter count and file size are consequences, not the deciding property.
2. Why is autoregressive decode harder to parallelise than prefill?
Show answer
B — Prefill processes all prompt positions at once and is highly parallel. Decode emits one token position at a time, each conditioned on the previous output, so the sequence cannot be unrolled in advance.
3. A 7B-parameter model is stored at 16 bits per weight. What is the first weight-memory estimate?
Show answer
C — 7e9 x 16 / 8 = 1.4e10 bytes, roughly 14 GB. The formula is parameters x bits per weight / 8, and 16 bits is 2 bytes per weight.
4. The same 7B model is quantized to 4 bits. What does the estimate become?
Show answer
B — 7e9 x 4 / 8 = 3.5e9 bytes, roughly 3.5 GB. Halving the bits per weight halves this estimate, but it says nothing about total resident memory.
5. Which of these is NOT covered by the parameters x bits / 8 estimate?
Show answer
B — The estimate covers weight storage only. KV cache, activations, allocator overhead, runtime buffers, metadata, and unquantized layers all sit outside it.
6. Does a smaller model file guarantee lower latency?
Show answer
B — A quantized format needs kernels that handle it efficiently on the target hardware. If the runtime dequantizes on the fly or falls back to a generic path, the smaller artifact can be slower than the larger one.
7. You quantized weights from 16 bits to 4 bits. What happened to the KV cache?
Show answer
C — Weight precision and KV-cache precision are independent settings. Lowering one does not lower the other, and at long context or high concurrency the cache can dominate the budget.
8. What must be recorded for a quantization release gate?
Show answer
C — The gate compares a quantized candidate with a higher-precision baseline on representative tasks, and it is only reproducible if hardware, runtime, and artifact identity are recorded alongside the performance and quality numbers.
9. You compare a CPU build and a GPU build, but each used a different prompt set. What is the status of the result?
Show answer
C — Workload, prompts, runtime, and quality rubric must be held fixed while the hardware or precision varies. Otherwise the difference cannot be attributed to the variable under test.
10. Which phase is typically the more memory-bandwidth-bound one at batch size 1?
Show answer
B — At batch size 1 decode reads the full weight set to produce a single token, so it is usually limited by memory bandwidth. Prefill amortises that read across many positions and tends to be compute-bound.