How HEVC Intra-Frame Compression Works: The Mathematics Behind HEIC

1. Coding Tree Units (CTUs) and Recursive Quadtree Splitting

Legacy JPEG divides an image into inflexible 8x8 pixel blocks regardless of whether a region contains a blank blue sky or intricate tree leaves. HEVC introduces Coding Tree Units (CTUs), which can be as large as 64x64 pixels.

Using a recursive quadtree partitioning algorithm, the encoder evaluates Rate-Distortion Optimization (RDO):

• If an area is smooth (like sky or flat walls), the encoder keeps the entire 64x64 block intact, encoding the entire area with minimal bits.
• If an area contains complex edges or textures, the CTU recursively splits into four 32x32 Coding Units (CUs), which can further split into 16x16, down to 8x8 or 4x4 blocks.

2. The 35 Directional Intra-Prediction Modes

Rather than encoding raw pixel values, HEVC predicts what pixels should look like based on already reconstructed neighboring boundaries. While H.264 provided 9 intra-prediction modes, HEVC provides 35 discrete modes:

• Mode 0 (Planar): Fits a smooth two-dimensional gradient surface across the block to prevent contour banding.
• Mode 1 (DC): Calculates the arithmetic mean of boundary pixels for flat regions.
• Modes 2 through 34 (Angular): 33 precise directional angles spanning from horizontal to vertical, allowing the encoder to trace diagonals, curves, and sharp geometric edges without stair-stepping.

3. Sample Adaptive Offset (SAO) & Deblocking Filters

After inverse quantization and transform synthesis, high compression can induce ringing or slight luminance offsets. HEVC incorporates two in-loop filters:

Deblocking Filter (DBF): Applied across block boundaries to smooth visual seams and eliminate grid patterns.

Sample Adaptive Offset (SAO): An intelligent filter that classifies reconstructed pixels into edge offsets or band offsets, adjusting local sample values to match original histogram distributions. This restores subtle textures and highlights that would otherwise be blurred away.

4. Transform & Quantization: Integer DCT and DST Matrices

Once spatial intra-prediction produces a residual difference matrix (representing the difference between the predicted pixel pattern and the actual camera sensor data), the residual is mapped into the frequency domain.

Unlike legacy JPEG which utilizes floating-point approximations of the Discrete Cosine Transform (often causing minor rounding discrepancies between different hardware chipsets), HEVC implements a core integer transform:

Here, matrix \(C\) contains fixed-precision integer approximations of cosine basis functions. For 4x4 luma transform blocks, HEVC introduces an alternative Discrete Sine Transform (DST), which models boundary discontinuities significantly better than DCT, preventing ringing artifacts along high-contrast text edges and hair follicles.

5. Context-Adaptive Binary Arithmetic Coding (CABAC)

The final stage of the HEIC compression pipeline is entropy coding via CABAC (Context-Adaptive Binary Arithmetic Coding). Legacy JPEG relies on Huffman coding, which assigns integer bit lengths to symbols, imposing a theoretical efficiency barrier of 1 bit per symbol.

CABAC binarizes syntax elements and uses statistical probability models that adapt dynamically to the local image content. This allows fractional-bit representation of high-frequency zeroes, squeezing additional 10% to 15% bit reductions out of the compressed bitstream without discarding any visual data.

6. Objective Quality Benchmarks: PSNR and SSIM Analysis

In digital image forensics, compression fidelity is measured using two primary metrics: Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index (SSIM).

Empirical testing conducted across standard photographic datasets demonstrates that at a target SSIM score of 0.96 (where the compressed image is visually indistinguishable from uncompressed raw data), HEVC intra-coding requires an average bitrate of 0.45 bits per pixel (bpp), whereas JPEG requires 0.98 bpp—a 54.1% reduction in total data overhead.

7. Future Horizons: Versatile Video Coding (VVC) and VVC Still Images

Just as HEVC succeeded H.264, the video coding standards committee finalized H.266 / Versatile Video Coding (VVC) under ISO/IEC 23090-3. VVC introduces 128x128 CTUs, 67 intra-prediction modes, and quadtree plus multi-type tree (QTMT) partitioning.

Still image formats derived from VVC deliver an additional 30% to 40% compression advantage over HEIC. However, because specialized VVC silicon decoders will take years to reach consumer smartphones, HEIC remains the dominant production standard of our generation.

8. Hardware Decoder Realities & Power Consumption Metrics

While software decoding of HEVC requires substantial CPU mathematical throughput, dedicated hardware decoders (such as Apple VideoToolbox, Intel QuickSync, and NVIDIA NVDEC) achieve real-time 60 fps 4K decoding drawing under 1.2 watts of power. On mobile devices, this energy efficiency ensures browsing large HEIC galleries does not deplete device battery reserves.