A photo-sensor produces a weak electrical signal proportional to incident light.
- Signal is analog, a few microvolts, and usually noisy
- Conditioned for DC bias and noise, then amplified through a gain-controlled amplifier
- Converted to digital format by a fast analog to digital converter
Image Sensor Types
An image sensor is a 2D array of photo-sensors. The 2 main types differ in how the signal leaves the array.
CCD
Also known as charge coupled device. Photons create charged particles, forming a potential field between 2 terminals. The field integrates over time, so a longer exposure increases the response. The charge is read by the amplifier as a voltage, and must be neutralized before the next frame. Sensitive to illumination, not color.
- Charge is shifted pixel by pixel to a single output amplifier
- Every pixel goes through the same amplifier, so the response is uniform
- Requires a dedicated manufacturing process
CMOS
Provides a non-integrating dynamic response. Each element gives an addressable single output. Requires relatively larger circuitry than CCD.
- Each pixel has its own amplifier
- Pixels are read through row and column addressing
- Built with the standard chip manufacturing process, so the ADC and processing logic can sit on the same chip
Comparison
| Property | CCD | CMOS |
|---|---|---|
| Response | Integrating | Non-integrating |
| Readout | Shifted to 1 amplifier | Addressed per pixel |
| Noise | Low and uniform | Higher, varies between pixels |
| Power use | High | Low |
| Readout speed | Slower | Faster |
| Cost | Higher | Lower |
| Light-sensitive area | Larger | Smaller |
| Shutter | Can hold charge to read later | Rolling shutter is common |
Frame Capture
Each photo-receptor must be read fast enough that the captured scene can be considered stationary, making shutter control important.
- CMOS sensors require direct on-sensor reading through complex multiplexing
- CCD sensors retain charge like memory, even after light is removed, and can transfer it to a holding memory to be read in the background
Delayed reading causes motion blur or the rolling shutter effect.
Rolling Shutter
A rolling shutter records rows at different times. Motion during the row scan distorts the recorded geometry. The effect is not limited to rotational motion.
CMOS sensors commonly use rolling shutter readout. CCD sensors can also produce the effect when their rows are read at different times.
Blooming
Blooming occurs when a sensor element saturates and excess charge leaks into neighbouring elements. It creates bright streaks or enlarged bright regions.
CCD sensors are more susceptible because charge is transferred through neighbouring elements during readout.
Strobe Lighting
Strobe lighting illuminates a moving subject with a short pulse. The short exposure interval limits the distance travelled during capture, freezing motion in the image.
Machine Vision Illumination
Illumination determines the contrast available to a machine-vision system. A feature cannot be recovered reliably when its reflected light is indistinguishable from its background.
- Uniformity
Even lighting reduces brightness variation caused by position in the scene. - Direction
A light angle can reveal surface relief or suppress shadows and specular reflections. - Spectrum
A selected wavelength can increase contrast between materials with different spectral reflectance. - Intensity
Sufficient light keeps the sensor response above its noise floor without saturation.
Vibrating cello strings captured with a rolling shutter. Each sensor row is read at a slightly different time, so the straight strings record as waves. Image by Ephramac, CC BY-SA 4.0.
Sensitivity and SNR
An adjustable aperture regulates incident light by controlling the exposed lens surface area. CCD sensors, due to their integrating response, can also regulate the collected signal through exposure time.
Exposure control keeps the sensor response in its optimum sensitivity region.
- Aperture
A larger opening admits more light. It also reduces depth of field. - Exposure time
A longer exposure collects more light. Motion during the exposure causes blur. - Sensor gain
Electronic gain brightens the signal after capture. It also amplifies sensor noise.
The response curve, output signal against incident light, has 3 regions.
- Non-sensitive region
Light is too weak to lift the signal above the sensor’s noise floor, so nothing usable is recorded. - Optimum sensitivity
Output tracks incident light and stays well above the noise floor, giving the best signal to noise ratio. - Saturated region
Wells are full, so output stops responding to further light. Separating detail needs high-gain amplification, which also amplifies noise and gives a low signal to noise ratio.
Capturing Color
By default an image sensor captures only illumination, not color. Color is recovered by separating the light into its primary colors before it is recorded. There are 2 approaches.
Beam Splitting
The incident light is split into multiple components. Each component is read by a sensor. Structure must be rigid and properly aligned.
A dichroic prism splits incoming light into primary colors.
Color and illumination have the same resolution.
Color Filter Array
Also known as CFA. An optical filter allows only one color per sensing element. Other color components are estimated from nearby sensors.
The most common arrangement is the Bayer pattern. It has twice as many green filters as red or blue, since the human eye is most sensitive to green. The missing 2 color values at each pixel are interpolated from neighbouring pixels, a step called demosaicing.
Color resolution is lower than illumination resolution.
