A simulation tool for evaluating digital camera image quality

Joyce E. Farrell Feng Xiao Peter B. Catrysse Brian A. Wandell

Stanford University, Stanford, CA ImagEval Consulting, Palo Alto, CA

Imaging Systems
Closed imaging system

Consumer cameras

Cellular phones

PDAs

Multifunction devices have the ability to capture, process and display images

Imaging Systems

Imaging Systems Evaluation Toolbox ISET

Simulate the entire imaging system in order to understand how components interact and influence the final displayed image quality ISET Scene database vCamera vDisplay

Digital Camera Simulation

ISET: vCamera

ISET: vCamera

Scene Optics Sensor Processor

Scene

Optics

Sensor

Processor

Graphical User Interface

Online Help

Scene

Scene Database
Spectral radiance at each pixel in the sampled scene
photons/wavelength/sec/solid angle/meter2

Synthetic Scenes
Macbeth Color Checker Spatial frequency sweep patterns Intensity ramps and uniform fields

Natural Scenes
High Dynamic range, High-resolution, MultiSpectral images Nikon D100, 2 additional color filters, exposure bracketing Linear models for surfaces and illuminants Natural scene dynamic range < 10,000:1 See www.imageval.com

Scene Data
Spectral radiance at each pixel in the sampled scene
photons/wavelength/sec/solid angle/meter2

Scene Analysis

Optics

Optics
Plot and Analyze Optical Properties e.g. Pointspread function (PSF)

Sensor

Sensor

Sensor

Sensor
Pixel Properties
Pixel height, width Fill factor Dark current Read noise Conversion gain Voltage swing

Color Properties
Color filter, transmittance

Photodetector QE Infrared filter transmittance Color filter array pattern

Circuit Properties
Analog-to-digital conversion Correlated Double Sampling

Array Properties
DSNU and PRNU Pixel spacing Number of pixels

Future features
Microlens array

Sensor
Design and customize color filter arrays

Sensor Analysis
SNR
Pixel and sensor Volts and lux-sec

Dynamic Range ISO Speed ROI statistics

pixels, rows, columns, area Mean, sd, histograms

Processor

Processor
Image Processing
Demosaicing
Nearest Neighbor Bilinear (Cok, 1987) Laplacian Adaptive Laplacian (Laroche, 1994)

Display
SPD of primaries Gamma Luminance and chromaticity of white point

Color Conversions (3x3 customized) Color spaces (Sensor, XYZ, Bradford, etc.) Color Balancing
None GrayWorld (Evans, 1959) WhiteWorld Customized Matrices

Direct Access
to Data and Commands

Example: Choice of Color Filter Array

(Baer et al, SPIE 1999, Barnhoefer et al, SPIE 2003)

Other examples: www.imageval.com

Color Filter Array

Sensor Images Color Balanced in XYZ space Rendered in RGB space YCMY

YCMY

GRBG

GRBG

Image Quality Evaluation

Visual observation
Subjective judgments
Can I see the difference? Which image do I prefer?

Metrics
Quantify visibility of distortions introduced by the optics, sensor, image processing or display rendering
e.g. Sensor noise, Quantization artifacts,

Quantify the perceived difference between two renderings

Color Filter Array

Processor
Difference Image

Conclusions
Simulation of entire imaging system
HDMS images vCamera vDisplay Image quality metrics

ISET improves collaboration between people with different types of expertise Open-source software architecture Qualitative and quantitative methods for image quality evaluation