You asked how they work differently. Here is the honest version, with the question that makes it interesting: if a GPU can run thousands of things at once, why don't we just use GPUs for everything? By the end of this page you will be able to answer that — with a formula you already know.
A CPU and a GPU are both made of the same stuff — transistors switching. The difference is what they spend those transistors on.
The task decides the winner. Give the kitchen "make one wedding cake, adjusting as you go" and the eight chefs win easily — the 4,000 cooks would be standing around waiting to be told what to do. Give it "chop 4,000 carrots identically" and it is not close.
On a CPU die, a large share of the area is spent on things that are not arithmetic at all: branch predictors, out-of-order scheduling, deep caches, prefetchers. All of that exists to make one instruction stream go fast when it is unpredictable. A GPU spends that same area on more arithmetic units and wide memory paths, and it pays for that by demanding that its threads mostly do the same thing. Neither is "better engineered". They bought different things with the same budget.
Two jobs, same two machines. Press start and watch. (The timings here are a fair caricature, not a benchmark — they are set to the right ratio to make the mechanism visible. Real numbers come from real measurement, which is what your own 5090 experiment is for.)
pick a task
Task A is sequential and branchy: step 7 depends on how step 6 turned out, so nothing can be done ahead of time. Task B is data-parallel: all 4,000 carrots are independent, so 4,000 workers can start at once.
GPU cores are not independent. They are herded in groups (NVIDIA calls a group of 32 a warp) that must execute the same instruction at the same time. So when your code says if (x > 0) and half the group goes one way, the hardware does not run both halves at once — it runs the "true" branch with half the lanes switched off, then the "false" branch with the other half switched off. Two passes instead of one. This is called divergence, and it is why a GPU can be humiliated by code a CPU finds trivial.
You already derived this. Here it is with a slider on it, because the shape is the point.
Speedup(N) = 1 / ( S + (1 − S)/N ) → as N → ∞, Speedup → 1/S
S = the fraction of the work that is stuck being serial · N = how many workers you throw at the rest
This is WHY both chips exist. Every real program is a mixture: a serial part that needs a chip which is brilliant at one thing at a time, and a parallel part that wants an army. Your PC has both because your work is both.
And the strategic line, which is the part worth keeping: optimisation is not adding N. It is finding and shrinking S. Doubling your core count moves you along a curve that is already flattening. Cutting S in half moves the ceiling itself.
Amdahl's law assumes the problem stays the same size while you add workers. Gustafson pointed out that in practice nobody buys 21,760 cores to run yesterday's problem faster — they run a bigger problem in the same time (a larger simulation, a higher resolution, a bigger model). Under that assumption the ceiling looks completely different.
Both are correct. They answer different questions. Amdahl: "how much faster is THIS job?" Gustafson: "how much more job fits in the same hour?" When someone quotes you a speedup number, the first thing to ask is which question they were answering — and whether the baseline they divided by was a fair one.
One instruction, stepping through the machine. Click Step and read what happens at each stage.
Instruction fetch — the CPU reads the next instruction. Everything starts here.
Because distance costs time. Here is the ladder, in the only unit that matters — how many clock cycles you wait:
| Where the data is | Wait (clock cycles, approximate) | In human terms, if 1 cycle = 1 second |
|---|---|---|
| Register (inside the core) | ~1 | reaching into your pocket — 1 second |
| L1 cache | ~4 | a drawer in your desk — 4 seconds |
| L2 cache | ~14 | a shelf across the room — 14 seconds |
| L3 cache | ~40 | walking down the corridor — 40 seconds |
| Main RAM | ~200 – 300 | going out to another building — 4 minutes |
Approximate and machine-dependent — the exact numbers differ per chip. The ratios are the durable fact, and the ratios are brutal: RAM is roughly 200× further away than a register.
So cache is not a luxury. Without it, a modern core would spend most of its life standing still, waiting. Every trick in the chapter above — out-of-order execution, prefetching, speculation — exists mostly to have something useful to do during that wait.
Signals in a chip travel at a large fraction of the speed of light, but not instantly. Light covers about 30 cm in one nanosecond; a signal in copper on silicon does rather less. At 4 GHz, one clock cycle is 0.25 ns — in which light itself moves about 7.5 cm. A memory chip sitting centimetres away on the motherboard is therefore several clock cycles away by the laws of physics alone, before you add the cost of the memory's own lookup.
That is a rare and wonderful thing in engineering: a limit you cannot design around, only design with. Cache exists because c is finite.
You have met binary and logic gates in CS1. Here is what they were for. Addition is not a feature the hardware was given. It is what falls out for free once you wire five gates together, four times in a row. Nothing below is a simplified stand-in: this is an adder, four bits wide instead of sixty-four.
Click any bit to flip it. Bit 3 is on the left, exactly the way you write a number down.
Two independent routes to the same number. The left side is what the gates produce, bit by bit, with no arithmetic anywhere in the code that drew it. The right side is ordinary decimal addition. If they ever disagree, one of them is lying — and it will not be the gates.
Step the carry above and this diagram follows it. A wire carrying 1 is drawn gold and thick; a wire carrying 0 is thin and grey. Watch carry-in arrive from the bit to its right — that single wire is the entire reason this thing has to be done in order.
sum = (A XOR B) XOR carry-in
carry = (A AND B) OR ((A XOR B) AND carry-in)
Read the second line in English: a carry comes out if both inputs were 1, or if exactly one of them was 1 and a carry came in. There is no third way to reach 2. Wires that cross without a dot are not connected — that is the standard convention, and this diagram uses it once.
Eight rows, because three one-bit inputs have 2³ = 8 combinations. This table is not typed into the page. It is produced by running those five gates over all eight inputs, so it cannot quietly drift away from the diagram above. The last column checks the gates against plain arithmetic.
Now the cost, which is the interesting part. The carry into bit 3 cannot be known until bit 2 has settled, which needed bit 1, which needed bit 0. That is what ripple means, and it makes this design's delay grow linearly with width. A 64-bit ripple adder would be 64 gate-delays deep — and at 4 GHz your entire clock cycle is 250 picoseconds long. There is no room.
So real ALUs do not ripple. They use carry-lookahead: extra gates that work out in parallel whether each bit position will generate a carry (A AND B) or propagate one (A XOR B), then combine those in a tree. It costs a great many more transistors to buy a delay that grows like log(width) instead of width. That trade — spend area and power to shorten the critical path — is the same trade being made everywhere else on the die, including in the pipeline two sections down.
Section 4 named the stages. This section shows you the bits. Below is a complete little machine: four registers, eight operations, and instructions exactly one byte wide. The encoding is invented — no real chip uses it — but nothing about it is faked. Every byte shown is computed from its fields, and every register value is computed by actually running the operation.
bits 7 6 5 = opcode (3 bits → 8 operations)
bits 4 3 = destination register (2 bits → R0–R3)
bits 2 1 0 = operand (3 bits → a number 0–7 for LDI and SHL, or a register number in the low two bits for everything else)
That is the entire instruction set architecture. Three fields, eight bits, no keywords, no syntax, no whitespace. A CPU has never seen a letter in its life — the mnemonics below exist only so that you can read it.
Four micro-steps per instruction: fetch the byte, decode it into control signals, run the ALU, write the answer back. Section 4's six-stage diagram is the same idea with memory split out.
A decoder is a lump of gates whose inputs are instruction bits and whose outputs are control lines — real wires running to the rest of the core carrying statements like register file: write, yes or no; ALU: take your second input from the instruction, or from a register; ALU: do this operation, not that one. Nothing is "interpreted" and nothing is looked up in a dictionary. The bit pattern is wired straight into the machine's behaviour, which is why decoding costs a fraction of a cycle rather than a visit to memory.
In this toy the opcode is the ALU operation, which is why the table looks so tidy. On an x86 core it is anything but: a variable-length instruction — anywhere from 1 to 15 bytes — is cracked by the decoder into one or more fixed-length internal operations, and it is those, not the instruction you wrote, that the pipeline actually schedules. That translation layer is a good part of why x86 chips are complicated and hot. It is also why they can still run a program compiled in 1995, which is a feature nobody wants to give up.
A car factory does not finish one car before starting the next. Neither does a CPU. Split the work into five stages and five instructions can be inside the machine at once, each in a different stage — the chip finishes one instruction per cycle without any single instruction getting faster.
Here are the same five instructions, scheduled three ways. The third way contains the thing that ruins it.
F = Fetch · D = Decode and read registers · X = Execute (ALU) · M = Memory · W = Write back · ◦ = stalled, doing nothing
Look at where the ceiling came from. A 5-stage pipeline can never exceed 5×, because the first instruction still has to walk through all five stages before anything finishes, and the last one still has to drain. Those fill-and-drain cycles are serial. You have met this shape before, two sections back, under a different name — and the cure is the same one: the interesting question is not "add more stages", it is "what is the serial part, and can I shrink it?"
The stall exists because I3 reads R1 while I2 is still computing it. Under the assumption printed on the diagram — no operand forwarding, and a register file that can be written in the first half of a cycle and read in the second half — I3's register read cannot succeed until cycle 6, the cycle in which I2 writes back. So I3 sits in Decode through cycles 4, 5 and 6. Two cycles do no work, and everything behind I3 waits too: 11 cycles instead of 9.
Forwarding (also called bypassing) removes almost all of it. I2's answer physically exists at the output of the ALU at the end of cycle 4. I3 needs it at the input of the ALU at the start of cycle 5. Run a wire directly from the one to the other, skipping the register file entirely, and the stall vanishes — straight back to 9 cycles. Every modern core is threaded with these wires.
One hazard survives anyway. If I2 is a load, its value does not exist until the end of the Memory stage, cycle 5 — and I3 wants it at the start of Execute, also cycle 5. No wire can send data backwards in time. That "load-use" hazard costs exactly one bubble, permanently, on every pipelined machine ever built. Compilers know it, and spend real effort rearranging instructions so that something useful sits in that slot.
And the last trap: a branch. The pipeline has to fetch something in the cycle after a conditional jump, but it does not yet know which way the jump went. So it guesses, and keeps going. If the guess is wrong, every instruction fetched since must be thrown away. That is exactly what the branch predictor in section 1 is buying, and it is why deeper pipelines are a gamble: the Pentium 4 stretched to 20 and then 31 stages chasing clock speed, and paid for every misprediction with a longer flush.
A transistor is a switch with no moving parts. A small voltage on one terminal decides whether current may flow between the other two. That is all. Wire a few together and you can build a gate that computes AND, OR, NOT; wire billions together and you can build something that plays a game or trains a model. There is no additional magic ingredient at any level — it is switches, all the way down.
The clock is a metronome. A crystal oscillator produces a steady pulse, and on every tick the whole chip advances one step in lockstep. 4 GHz means four billion ticks per second. The clock is what keeps billions of independent switches from disagreeing about what time it is.
Switching costs energy, and energy becomes heat. Each flip charges and discharges a tiny capacitance; the power roughly follows P ≈ C·V²·f. Look at that expression: power grows linearly with frequency f but with the square of voltage V. Pushing the clock higher normally needs more voltage to switch cleanly — so the power bill climbs faster than the speed does.
And that is why your chip has many cores instead of one very fast one. Around 2005 the industry hit the wall: clock speeds stopped climbing because the heat could not be removed. The escape route was sideways — more cores at a sane clock instead of one heroic core. Which is exactly why parallelism stopped being exotic and became the default, and why Amdahl's law went from a footnote to the thing that shapes every design meeting.
The paragraph above says the power bill climbs faster than the speed. Put sliders on it and find out how much faster. Move V and watch the readout move by the square; move f and watch it move in a straight line. Then notice that in real silicon you cannot move f without dragging V along behind it — which is the whole problem.
Assumptions, printed so that you can argue with them. C = 30 nF stands in for one core's effective switched capacitance — a made-up number of realistic size, because the real one is a manufacturing secret. The dashed curve uses a rule of thumb V(f) = 0.75 + 0.10·f, with f in GHz, because a faster clock needs more voltage to switch cleanly in the time available. Both are stand-ins. The shape — linear in f, square in V, so roughly cubic when V has to follow f — is not a stand-in. That shape is the wall.
This is the entire 2005 decision in one comparison, and both rows are computed from the same formula and the same V(f) rule, so it is a fair fight. Two cores at half the clock do the same amount of parallel work per second as one core at full clock — and because the halved clock runs at a lower voltage, and voltage is squared, they do it for noticeably less power. The catch is hiding in the word "parallel", and you already know its name. It is S.
So the industry did not stop making chips faster because it ran out of ideas. It ran out of watts. Every gain since has had to come from doing more things at once, or from doing each thing more cleverly — which is why the two chapters of this page are really one chapter.
One layer of a neural network, in full. Press the button and watch all nine multiplications happen at once.
ready
That is the whole secret. A neural network layer is a matrix multiplied by a vector, then a simple function applied to each result. Stack a hundred of those and you have a model. The reason AI needed GPUs is not that GPUs are clever — it is that this particular piece of mathematics is embarrassingly parallel, and somebody had already spent thirty years building hardware that does exactly it, for video games.
Three ingredients, none of them mysterious: a maths structuremassive parallelismhardware that happens to fit
A 3×3 multiply needs 9 multiplies and 6 adds — and it needs to fetch 9 matrix numbers and 3 vector numbers. At that size, the fetching dominates. This ratio has a name, arithmetic intensity: operations performed per byte moved. When intensity is low, your expensive arithmetic units sit idle waiting for memory — the "memory wall".
This is why real GPU code obsesses over tiling and reuse rather than over the multiplies themselves, and why a chip advertised at 100 TFLOPS routinely delivers a fraction of that. When you run your own 5090 experiment, watch for exactly this: the point where making the matrix bigger stops making things slower per element, because you finally have enough arithmetic to hide the fetching behind.
Five questions. Instant feedback, and an explanation either way — a wrong answer here costs nothing and teaches more than a right one.
Every race timing, core count and latency number here is a caricature with honest ratios — chosen so the mechanism is visible, not measured on any real machine. This page cannot tell you how your 5090 behaves, how your game's frame budget is actually spent, or whether a given piece of code is 3% serial or 30%. Those are measurements, and measurements need a stopwatch and a named baseline, not a diagram.
Two things here are exact rather than caricatured, and you can check both by hand: the four-bit adder in section 5 (its output comes from the gates, and the page prints the arithmetic beside it so you can catch it lying) and the one-byte machine in section 6 (every byte is computed from its fields). The pipeline diagram in section 7 is exact too — but only for the assumption printed on it, which is the point: cycle counts are meaningless until someone tells you what they assumed about forwarding.
That is exactly what the 20-Microsecond Hunt is for, and why it makes you write a prediction before it will let you press run. Read this page to know what to look for; run that one to find out what is true in your house.
Why don't we use GPUs for everything? Because every real program has a serial spine — decisions, dependencies, steps that cannot begin until the previous one has finished. A GPU core is deliberately bad at that, and Amdahl's law says the serial spine sets the ceiling no matter how many cores you buy. The interesting engineering question is never "which chip is faster". It is "what fraction of this work is actually independent — and can I make that fraction bigger?"