A photo-sensor produces a weak electrical signal proportional to incident light.
- Signal is analog, a few microvolts, and usually noisy
- Conditioned for DC bias and noise, then amplified through a gain-controlled amplifier
- Converted to digital format by a fast analog to digital converter
Image Sensor Types
CCD
Aka. charge coupled device. Photons create charged particles, forming a potential field between 2 terminals. The field integrates over time, so a longer exposure increases the response. The charge is read by the amplifier as a voltage, and must be neutralized before the next frame. Sensitive to illumination, not color.
CMOS
Provides a non-integrating dynamic response. Each element gives an addressable single output. Requires relatively larger circuitry than CCD.
Frame Capture
Each photo-receptor must be read fast enough that the captured scene can be considered stationary, making shutter control important.
- CMOS sensors require direct on-sensor reading through complex multiplexing
- CCD sensors retain charge like memory, even after light is removed, and can transfer it to a holding memory to be read in the background
Delayed reading causes motion blur or the rolling shutter effect
Vibrating cello strings captured with a rolling shutter. Each sensor row is read at a slightly different time, so the straight strings record as waves. Image by Ephramac, CC BY-SA 4.0.
Sensitivity and SNR
An adjustable aperture regulates incident light by controlling the exposed lens surface area. CCD sensors, due to their integrating response, can also regulate the collected signal through exposure time.
The response curve, output signal against incident light, has 3 regions.
- Non-sensitive region
Light is too weak to lift the signal above the sensor’s noise floor, so nothing usable is recorded. - Optimum sensitivity
Output tracks incident light and stays well above the noise floor, giving the best signal to noise ratio. - Saturated region
Wells are full, so output stops responding to further light. Separating detail needs high-gain amplification, which also amplifies noise and gives a low signal to noise ratio.
Capturing Color
By default an image sensor captures only illumination, not color.
Beam Splitting
- A dichroic prism splits incoming light into primary colors
- A separate image sensor captures each separated beam
- Structure must be rigid and properly aligned
- Color and illumination have the same resolution
Color Filter Array
Aka. CFA.
- An optical filter allows only one color per sensing element
- Other color components are estimated from nearby sensors
- Color resolution is lower than illumination resolution