Sampling and Quantization

Sampling and quantization are the 2 independent choices made when converting an analog image to a digital image. Sampling sets the grid size m×nm \times n. Quantization sets the number of levels L=2kL = 2^k. Together they fix the storage size m×n×km \times n \times k bits per channel.

Coarse choices in either one cause visible artifacts.

Sampling

Sampling fixes the spatial resolution. Resolution is the number of samples per unit length, set by the size of the image sensor element.

Coarse sampling means large grid cells, so a small m×nm \times n. It has 3 visible effects: loss of information, pixelation, and aliasing.

Sampling Rate

Number of samples per unit distance, one per grid cell.

Loss of Information

Each cell keeps only one value for the region it covers, so any variation finer than a cell is averaged away and cannot be recovered. Worse with large cells and scenes carrying fine detail.

Pixelation

Also known as the checkerboard effect or blocking. Each grid cell becomes one flat square of intensity, so a smooth or diagonal edge is replaced by a staircase of these squares. Most visible on high-contrast boundaries such as text against a plain background.

Pixelation from coarse sampling

Left: Original. Right: Sampled on a grid 8 times coarser, then shown at the original size. Blocky cells replace fine detail.
Generated from a photo by Hannes Röst, CC BY-SA 3.0.

Worked Example

Nyquist Sampling Rate

A stripe pattern has 120120 cycles per millimetre. Find the minimum sampling rate that prevents aliasing.

Solution Reveal Hide
fs≥2fmax⁡=2⋅120=240 samples/mmf_s \ge 2f_{\max} = 2 \cdot 120 = 240\text{ samples/mm}

The sampling grid needs at least 240 samples/mm240\text{ samples/mm}.

Aliasing

Aliasing happens when a pattern in the scene repeats faster than the sampling grid can follow. The samples then fit a different, lower-frequency pattern that was never in the scene. That false pattern is the alias.

  • Capturing a repeating pattern needs at least 2 samples per cycle of that pattern. This is the Nyquist criterion.
  • A pattern with more than 1 cycle per 2 cells is undersampled.
  • The alias frequency is the true frequency folded about half the sampling rate. A higher scene frequency gives a lower alias frequency.
  • On fine repeating textures the alias shows up as broad curved bands, called moire fringes.

Anti-aliasing removes detail finer than the grid can represent, by optical blur or a Gaussian low-pass filter, before the image is sampled. A Gaussian filter weights nearby pixels with smoothly decreasing coefficients. This costs high-frequency detail, which is less noticeable when sub-sampling.

Aliasing

Left: Original image. Right: Aliasing effect. Image by Wikipedia.

Moire fringes

Two overlapping railings on a bridge. The beat between the two picket spacings shows up as broad moire bands where the grid is undersampled. Image by Hidalgo944, CC BY-SA 4.0.

Quantization

Quantization fixes the grey-level depth. Grey-level depth is the number of discrete intensity levels LL, set by the resolution of the analog to digital converter.

Coarse quantization means a small LL and a low bit depth kk. The gaps between levels are large, so nearby intensities collapse to the same level. This causes false contouring.

False Contouring

Also known as Banding. Across a smooth gradient, a band of neighbouring intensities collapses onto one level and renders as a flat region. The jump to the next level shows up as a sharp edge that is not in the scene. These false edges resemble contour lines, most visible in skies and other large smooth areas.

Color banding

Color banding visible in sky. Image by Steve F, CC BY-SA 2.0.

Written by October 28, 2026 3 min read
Was this helpful?