Inside the Encoder

Level 1 of 7 · new decision: how much precision to destroy scaffold present

The quantizer — where the loss happens

One frame, no prediction at all: just transform, quantize, count bits. The quantizer divides every DCT coefficient by a step and rounds. Anything smaller than two-thirds of a step becomes exactly zero and costs almost nothing to send. QP sets that step, and the step doubles every six QP — so QP is a logarithmic dial, which is why the quality curve has a cliff in it.

Level 2 of 7 · new decision: what to throw the precision away from scaffold inverted

The transform — why the DCT comes first

Quantizing is just rounding. You could round the pixels directly and skip the transform entirely. Both paths below use the same quantizer and the same entropy coder, and the page searches for a step size that makes them spend the same number of bits. Only one thing differs: whether the rounding happens on pixels or on DCT coefficients.

Commit a prediction first — the answer is hidden until you do.

A natural picture is mostly smooth, so in the DCT domain nearly all its energy lands in a handful of low-frequency coefficients and the rest is near zero — free to send. In the pixel domain every one of the 64 values is equally important, so an equal budget buys you 64 coarse numbers instead of 6 good ones. This is energy compaction, and it is the whole reason a transform is in the pipeline.

Level 3 of 7 · new decision: what to predict from scaffold present

Prediction — send the surprise, not the picture

So far every block has been coded from scratch. Real codecs almost never do that. They guess the block first — from the decoded pixels directly above and left of it, or from the same place in the previous frame — and then transmit only the residual: guess minus truth. A good guess makes the residual nearly flat, and a flat residual quantizes to almost nothing.

Five blocks are marked on the frame. They do not all want the same guess: one sits on a smooth gradient, one on a hard vertical edge, one on a moving object. Pick a mode for each.

Level 4 of 7 · new decision: where in the past frame to look scaffold present

Motion — the surprise is somewhere else

The camera is panning and three objects are moving. Predicting a block from the same place in the previous frame is then a bad guess, but predicting it from the place that content came from is an excellent one. That offset is a motion vector, and the encoder must transmit it.

Drag the two sliders until the residual goes flat grey. Watch both bars: the orange residual bar falls as your aim improves, and the purple motion bar grows as the vector gets further from zero. The vector is not free.

Level 5 of 7 · new decision: how finely to carve the picture scaffold degraded

Partitioning — where one vector is not enough

Up to now every block had one vector. Real pictures are not that tidy: a 16×16 macroblock lying across the edge of a moving object contains two different motions, and no single vector describes both. So the encoder may split it into four 8×8 partitions, each with its own vector — four times the accuracy, four times the vectors to transmit.

Sixty macroblocks. Click the ones you think need splitting. Your live readouts are gone: you get the totals only after you press Encode, so you have to reason about where the motion disagrees instead of watching a number.

Level 6 of 7 · new decision: the price of a bit scaffold inverted

J = D + λR — the one rule under all of it

Quantizer, prediction mode, vector, partition: four separate levers, and you have been choosing each by feel. An encoder does not. It scores every candidate with one number —

J = D + λ·R   distortion, plus λ times the bits

— and keeps the lowest. λ is the exchange rate between squared error and bits. Raise it and bits get expensive, so the encoder starts refusing to pay for detail and falls back on cheaper modes. Lower it and it will buy anything. Changing λ does not change the quantizer at all; it changes what the encoder is willing to spend.

Level 7 of 7 · everything at once scaffold absent

Ship the shot — rate control, frame types, and a channel that will not wait

Twenty-four frames, going out over a link that delivers a fixed 3,200 bits per frame interval and no more. The decoder plays from a buffer. Spend more than arrives and the buffer drains; empty it and playback stalls — the failure every real rate controller exists to prevent.

You choose, one frame at a time and with no take-backs:

• I — code it from scratch. Expensive, but it owes nothing to the past and lets a viewer join here.
• P — predict from the previous decoded frame. Usually several times cheaper.
• B — predict from the previous and the next frame, averaged. Cheaper still, but the encoder must code the future frame first, so it commits two frames at once and adds a frame of delay. You have three frames of delay for the whole shot and no more.
• QP — the quantizer, as in level 1.

No live readouts. You find out what a frame cost after it is sent. Somewhere in this shot the camera cuts to a different scene; you will not be told when.

Debrief

What you now know, and what killed the thing you believed

The transferable rules

1. Every decision is J = D + λR. Mode, vector, partition, quantizer, frame type — all of it is one comparison, repeated millions of times a second. If you only remember one thing, remember that an encoder is a search algorithm with a price list.
2. Transmit surprise, never content. Predict as well as you can, then send only the error. Everything else in the pipeline exists to make the error cheap.
3. Side information is information. Vectors, modes and split flags are paid for in the same currency as picture detail, which is why more flexibility is not automatically better.
4. The encoder must live inside the decoder's head. It predicts from the reconstruction, never the original. That single constraint produces drift, error propagation, and the need for I frames.
5. Rate control is a buffer problem, not an average. Hitting a target bitrate on average while emptying the buffer at the scene cut is still a stall.
6. Newer standards are bigger menus, not smaller files. HEVC and AV1 add partition shapes, prediction modes and transform sizes — more candidates for the same J comparison. The gain is real and it is paid for in encoding time, which is why presets exist.

What this model leaves out

The fidelity contract, stated plainly, because a caricature presented as the truth would quietly miscalibrate you.

Simulated honestly: the separable 8×8 DCT-II; a deadzone quantizer on H.264's logarithmic step ladder (the step doubles every 6 QP); an Exp-Golomb run/level entropy coder of the CAVLC family; intra DC, vertical and horizontal prediction from genuinely reconstructed neighbours; integer-pel rate-constrained full-search motion estimation against reconstructed references; bidirectional prediction by averaging two references; macroblock partitioning; Skip; the J = D + λR mode decision with λ = 0.85·2^(QP−12)/3; a leaky-bucket decoder buffer. Error propagation and drift are not scripted — they emerge, because the references really are the reconstructions.

Not here at all: colour — this is luma only, and chroma is roughly another 25% of the bits and its own prediction machinery. CABAC, whose context-adaptive arithmetic coding is worth 10–15% over the CAVLC-style coder used here. Sub-pixel motion at half and quarter resolution with interpolation filters, worth a great deal more than you might guess. Deblocking and SAO filters, which is why your reconstructions look blockier than real ones at the same bitrate. Multiple reference frames, weighted prediction, intra modes beyond three of the nine in H.264, transform sizes other than 8×8, trellis quantization, adaptive quantization by perceptual importance, and rate–distortion optimised quantization. Everything the last fifteen years added in HEVC, VP9, AV1 and VVC: quadtree and multi-type-tree partitioning, 35-to-67 intra directions, compound and warped motion, transform skip, film grain synthesis. Real sequences, real camera noise, and real perceptual metrics — PSNR is used here because it is honest arithmetic on the signal, but it is a poor model of what an eye objects to, and no serious codec comparison relies on it alone. And the frames are 160×96 synthetic images with exact integer motion, which makes motion estimation far easier than it is in life.

What survives all of that is the shape of the system, and the shape is what you came for.

Where to go next

• Run ffmpeg -i in.mp4 -c:v libx264 -crf 23 -preset veryslow out.mp4 against -preset ultrafast at the same CRF. Same standard, same quantizer, very different sizes: that difference is entirely the search you just did by hand.
• ffprobe -show_frames will print pict_type and pkt_size per frame. The spikes are I frames and scene cuts — level 7, on real video.
• CRF is not a bitrate. It is approximately a fixed λ: a constant price for a bit, letting the bitrate go wherever the content needs. That is why it holds quality steady and why its file sizes are unpredictable.