Computer Vision Forensics Research

Adam Steinberger
steinz08@gmail.com http://www.amsteinberger.com/ Summer 2011

The following files are saved on UVEA in the visionmatlab home directory in /students/home/visionmatlab/workspace/. Runnable files include: Bayer2JPGtest4.m, cropImages.m, demosaicTest.m, demosaicTest2.m, findErrEdges.m, findErrEdgeSmooth.m, findErrNew.m, findError.m, getBadImgStats.m, getDemoStats.m, getImageStats.m, getImageStatsEdge.m, getRAW8bit.m, getRawDemoStats.m, graphImageStats.m, graphImageStats2.m, graphImageStat3.m.
Bayer2JPGtest4.m (108 lines)
Dependencies: ForensicImage.m, imgEdges.m, camCrop2.jpg Read image file /students/home/visonmatlab/workspace/camCrop2.jpg and create Bayer patterns of the given JPEG. Then for each Bayer pattern image, get the color channel intensity gradients and demosaicing patterns for smooth and edge areas and convert image back to JPEG accordingly. Outputs difference between original JPEG and new JPEG images.

cropImages.m (130 lines)

Dependencies: ForensicImage.m, getAllFiles.m Search recursively through all subdirectories of /students/home/semistatic/summer2011/photos/ for JPEG images, and crop each image at its center to 512 by 512 square pixels. Cropped images are saved to /students/home/semistatic/summer2011/crops/. If file path is complete in crops folder, this code will build up the directories that mirror the original image path.

demosaicTest.m (184 lines)

Dependencies: ForensicImage.m, getAllFiles.m, camCrop.jpg Read image file /students/home/visonmatlab/workspace/camCrop.jpg and create Bayer patterns of the given JPEG. Run edge detection for each Bayer pattern. Then for each Bayer pattern image, get the color channel intensity gradients and demosaicing patterns for smooth and edge areas (including chirality) and convert image back to JPEG accordingly. Outputs difference between original JPEG and new JPEG images. 08/05/2011 1

demosaicTest2.m (136 lines)

Dependencies: ForensicRawImage.m, getAllFiles.m, rawTest.dng Read RAW image file /students/home/visonmatlab/workspace/rawTest.dng and crop it to 512 by 512 square pixels. Convert this cropped image from 16 bit to 8 bit. Run edge detection on the 8 bit RAW image. Then for each Bayer pattern image, get the color channel intensity gradients and demosaicing patterns for smooth and edge areas (including chirality) and convert image back to JPEG accordingly. Outputs difference between original JPEG and new JPEG images.

findErrEdges.m (390 lines)

Dependencies: ForensicImage.m, imgStats.m, bassCrop.jpg Read image file /students/home/visonmatlab/workspace/bassCrop.jpg and gather different statistics for the error values of the pixels using a 3 by 3, 5 by 5, and 7 by 7 square pixel windows. This implementation uses edge detection to focus only on processing edge pixels.

findErrEdgeSmooth.m (699 lines)

Dependencies: ForensicImage.m, imgStats.m, bassCrop.jpg Read image file /students/home/visonmatlab/workspace/bassCrop.jpg and gather different statistics for the error values of the pixels using a 3 by 3, 5 by 5, and 7 by 7 square pixel windows. This implementation uses edge detection to process on edge pixels differently than smooth pixels.

findErrNew.m (91 lines)

Dependencies: ForensicImage.m, imgStats.m, bassCrop.jpg Read image file /students/home/visonmatlab/workspace/bassCrop.jpg and gather different statistics for the error values of the pixels using a 3 by 3 square pixel window. This implementation is FAST because it uses medfilt2, but this includes center pixels.

findError.m (115 lines)

Dependencies: ForensicImage.m, imgStats.m, bassCrop.jpg Read image file /students/home/visonmatlab/workspace/bassCrop.jpg and gather different statistics for the error values of the pixels using a 3 by 3 square pixel window. This implementation is VERY SLOW!

ForensicImage.m (2003 lines)

Dependencies: imgStats.m, imgEdges.m, imgChiral.m, medfilt2new.m, medfilt2edge.m A color image file with forensics functions attached. ForensicImage(f): creates forensic image where f = filename. getColorCh(): stores each color channel of image in separate variable. crop512(): crop image to 512 by 512 square pixels. getnb(r,c,ch,win): Get image pixel neighbors at r = row, c = column, ch = 08/05/2011 2

color_channel, where win = window_size. The window is of size win by win square pixels, where win is an odd integer greater than 1. Currently, this function grabs all pixels in this window; getAltColorBins(): separates image color channels by (R+G+B)/3, (R-B)/2, (2G-RB)/4. imgStatsACB(): returns an imgStats objects for image statistics based on the image's alternate color bins. Uses imgStats.m to return image statistics. AvgPixel(): Get average pixel of image. imgStats(): returns three imgStats objects for image statistics using 3 by 3, 5 by 5, and 7 by 7 square pixel windows. Uses imgStats.m to return image statistics. Also uses medfilt2new.m to get the median of pixel neighbors. imgStatsEdge(th1,th2,cam,pic): returns three imgStats objects for image edge pixel statistics using 3 by 3, 5 by 5, and 7 by 7 square pixel windows, where th1 = threshold for edge detection, th2 = threshold for median filter, cam = camera number (when used by getImageStatsEdge.m), and pic = photo number (when used by getImageStatsEdge.m). Uses imgStats.m to return image statistics. Also uses medfilt2edge.m to get the median of pixel neighbors. ImgStatsSmooth(th1): returns three imgStats objects for image smooth pixel statistics using 3 by 3, 5 by 5, and 7 by 7 square pixel windows, where th1 = threshold for edge detection. Uses imgStats.m to return image statistics. Also uses medfilt2new.m to get the median of pixel neighbors. getBayer(): turns an image into four different Bayer Color Filter Array patterns, and each pattern is saved as a new image. getImgEdge(bNum): return imgEdge of channel intensity gradients and demosaicing patterns for smooth and edge areas for the Bayer pattern image where bNum = 1 uses bayerR, bNum = 2 uses bayerGR, bNum = 3 uses bayerGB, and bNum = 4 uses bayerB. getColorPats(bNum): return four images in which each pixel is the mean of an appropriate neighborhood to get bayer color patterns, where bNum = 1 uses bayerR, bNum = 2 uses bayerGR, bNum = 3 uses bayerGB, and bNum = 4 uses bayerB. demoBayerR(x,imgVB,imgHB,imgD1,imgD2): Demosaic bayer red pattern images (used with block process), where imgVB, imgHB, imgD1, and imgD2 are the images returned from getColorPats(). demoBayerG1(x,imgVB,imgHB,imgD1,imgD2): Demosaic bayer green1 pattern images (used with block process), where imgVB, imgHB, imgD1, and imgD2 are the images returned from getColorPats(). demoBayerG2(x,imgVB,imgHB,imgD1,imgD2): Demosaic bayer green 2 pattern images (used with block process), where imgVB, imgHB, imgD1, and imgD2 are the images returned from getColorPats(). demoBayerB(x,imgVB,imgHB,imgD1,imgD2): Demosaic bayer blue pattern images (used with block process), where imgVB, imgHB, imgD1, and imgD2 are the images returned from getColorPats(). getImgChr(bNum,iEdge,iTh): Get chirality images for bayer pattern image, where bNum = 1 uses bayerR, bNum = 2 uses bayerGR, bNum = 3 uses bayerGB, and bNum = 4 uses bayerB. Also, iEdge is imgEdges for image with edge types, and iTh 08/05/2011 3

is image with edge types. chr2jpg(bNum,img,imgChr,iCV,iCH,iCD1,iCD2): Convert edges in bayer image to JPEG, where bNum = 1 uses bayerR, bNum = 2 uses bayerGR, bNum = 3 uses bayerGB, and bNum = 4 uses bayerB. Also, img = 3 color channel image, imgChr = image with edge type chiralities, iCV = imgChiral for vertical edge chiral patterns, iCH = imgChiral for horizontal edge chiral patterns, iCD1 = imgChiral for forward diagonal edge chiral patterns, and iCD2 = imgChiral for backward diagonal edge chiral patterns. getChrPats(bNum): get chiral patterns from a bayer image, where bNum = 1 uses bayerR, bNum = 2 uses bayerGR, bNum = 3 uses bayerGB, and bNum = 4 uses bayerB. demoStats(th1): Demosaic image, where th1 = threshold for edge detection.

ForensicRawImage.m (876 lines)

Dependencies: imgStats.m, imgEdges.m, imgChiral.m, medfilt2new.m, medfilt2edge.m A RAW Digital Negative (DNG) image file with forensics functions attached. ForensicRawImage(f): creates forensic image where f = filename. Image must be a Digital Negative (DNG) file, which are created from proprietary RAW image files with Adobe Photoshop. crop512(): crop image to 512 by 512 square pixels. getnb(r,c,win): Get image pixel neighbors at r = row, c = column, where win = window_size. The window is of size win by win square pixels, where win is an odd int greater than 1. Currently, this function grabs all pixels in this window. getCurve(): get the linearization table for a RAW image if it has one. hasCurve(): check if the RAW image has a linearization table or not. imraw8(): converts 16-bit RAW image into an 8-bit image, which can be saved to disk as a PNG file. getColorPats(bNum): return four images in which each pixel is the mean of an appropriate neighborhood to get bayer color patterns, where bNum = 1 uses bayerR, bNum = 2 uses bayerGR, bNum = 3 uses bayerGB, and bNum = 4 uses bayerB. demoBayerR(x,imgVB,imgHB,imgD1,imgD2): Demosaic assumed bayer red pattern images (used with block process), where imgVB, imgHB, imgD1, and imgD2 are the images returned from getColorPats(). demoBayerG1(x,imgVB,imgHB,imgD1,imgD2): Demosaic assumed bayer green1 pattern images (used with block process), where imgVB, imgHB, imgD1, and imgD2 are the images returned from getColorPats(). demoBayerG2(x,imgVB,imgHB,imgD1,imgD2): Demosaic assumed bayer green 2 pattern images (used with block process), where imgVB, imgHB, imgD1, and imgD2 are the images returned from getColorPats(). demoBayerB(x,imgVB,imgHB,imgD1,imgD2): Demosaic bayer assumed blue pattern images (used with block process), where imgVB, imgHB, imgD1, and imgD2 are the images returned from getColorPats(). getImgEdge(bNum): return imgEdge of channel intensity gradients and demosaicing patterns for smooth and edge areas for the Bayer pattern image where bNum = 1 08/05/2011 4

uses bayerR, bNum = 2 uses bayerGR, bNum = 3 uses bayerGB, and bNum = 4 uses bayerB. getImgChr(bNum,iEdge,iTh): Get chirality images for bayer pattern image, where bNum = 1 uses bayerR, bNum = 2 uses bayerGR, bNum = 3 uses bayerGB, and bNum = 4 uses bayerB. Also, iEdge is imgEdges for image with edge types, and iTh is image with edge types. getChrPats(bNum): get chiral patterns from a bayer image, where bNum = 1 uses bayerR, bNum = 2 uses bayerGR, bNum = 3 uses bayerGB, and bNum = 4 uses bayerB. chr2jpg(bNum,img,imgChr,iCV,iCH,iCD1,iCD2): Convert edges in bayer image to JPEG, where bNum = 1 uses bayerR, bNum = 2 uses bayerGR, bNum = 3 uses bayerGB, and bNum = 4 uses bayerB. Also, img = 3 color channel image, imgChr = image with edge type chiralities, iCV = imgChiral for vertical edge chiral patterns, iCH = imgChiral for horizontal edge chiral patterns, iCD1 = imgChiral for forward diagonal edge chiral patterns, and iCD2 = imgChiral for backward diagonal edge chiral patterns. demosaicImg(th1): Demosaic raw image, where th1 = threshold for edge detection.

getAllFiles.m (26 lines)

Recursively find all files in a directory. getAllFiles(dirName): Search recursively through all subdirectories of dirName for files, return fileList.

getBadImgStats.m (136 lines)

Dependencies: ForensicImage.m, getAllFiles.m The 3 colors for all four corners and the center pixel from JPEG files in /students/home/semistatic/summer2011/crops/ for each camera are saved to text files. The file /students/home/visionmatlab/workspace/iStatBad.log keeps track of which image file was last read, so if the program is killed or breaks it will start up where it last stopped. The data files are saved in /students/home/visionmatlab/stats/08-04-bad/ by appending new data to the end of each file, so to restart this program from scratch you must delete the old data files or move them to a different folder. To restart the program from scratch, you must also edit iStatBad.log so that it reads: filename 1 0. Because the data is read by other programs with spaces in between each data entry, any spaces in image filenames will be replaced with underscores in the data file.

getDemoStats.m (166 lines)

Dependencies: ForensicImage.m, getAllFiles.m, imgStats.m, medfilt2new.m, medfilt2edge.m Image statistics from demosaicing JPEG files in /students/home/semistatic/summer2011/crops/ for each camera are saved to text files. The file /students/home/visionmatlab/workspace/iStatDemo.log keeps track of which image file was last read, so if the program is killed or breaks it will start up where it last stopped. The data files are saved in 08/05/2011 5

/students/home/visionmatlab/stats/07-30-demo/ by appending new data to the end of each file, so to restart this program from scratch you must delete the old data files or move them to a different folder. To restart the program from scratch, you must also edit iStatDemo.log so that it reads: filename 1 0. Because the data is read by other programs with spaces in between each data entry, any spaces in image filenames will be replaced with underscores in the data file.

getImageStats.m (206 lines)

Dependencies: ForensicImage.m, getAllFiles.m, imgStats.m, medfilt2new.m Image statistics from JPEG files in /students/home/semistatic/summer2011/crops/ for each camera are saved to text files. The file /students/home/visionmatlab/workspace/iStat.log keeps track of which image file was last read, so if the program is killed or breaks it will start up where it last stopped. The data files are saved in /students/home/visionmatlab/ by appending new data to the end of each file, so to restart this program from scratch you must delete the old data files or move them to a different folder. To restart the program from scratch, you must also edit iStat.log so that it reads: filename 1 0. Because the data is read by other programs with spaces in between each data entry, any spaces in image filenames will be replaced with underscores in the data file.

getImageStatsEdge.m (329 lines)

Dependencies: ForensicImage.m, getAllFiles.m, imgStats.m, medfilt2new.m Image statistics for both edge and smooth pixels from JPEG files in /students/home/semistatic/summer2011/crops/ for each camera are saved to text files. The file /students/home/visionmatlab/workspace/iStatEdge.log keeps track of which image file was last read, so if the program is killed or breaks it will start up where it last stopped. The data files are saved in /students/home/visionmatlab/stats/07-26-edge/ by appending new data to the end of each file, so to restart this program from scratch you must delete the old data files or move them to a different folder. To restart the program from scratch, you must also edit iStatEdge.log so that it reads: filename 1 0. Because the data is read by other programs with spaces in between each data entry, any spaces in image filenames will be replaced with underscores in the data file.

getRAW8bit.m (90 lines)

Dependencies: ForensicRawImage.m, getAllFiles.m Search recursively through all subdirectories of /students/home/semistatic/summer2011/raws/ for Digital Negative (DNG) images, and convert each image from 16-bit RAW to 8-bit grayscale PNG.

getRawDemoStats.m (165 lines)

Dependencies: ForensicRawImage.m, getAllFiles.m, imgStats.m, medfilt2new.m, medfilt2edge.m 08/05/2011 6

Image statistics from demosaicing RAW DNG files in /students/home/semistatic/summer2011/raws/ for each camera are saved to text files. The file /students/home/visionmatlab/workspace/iStatRawDemo.log keeps track of which image file was last read, so if the program is killed or breaks it will start up where it last stopped. The data files are saved in /students/home/visionmatlab/stats/07-28-rawdemo/ by appending new data to the end of each file, so to restart this program from scratch you must delete the old data files or move them to a different folder. To restart the program from scratch, you must also edit iStatRawDemo.log so that it reads: filename 1 0. Because the data is read by other programs with spaces in between each data entry, any spaces in image filenames will be replaced with underscores in the data file.

graphImageStats.m (129 lines)

Dependencies: getAllFiles.m Search recursively through all subdirectories of /students/home/visionmatlab/stats/07-11/ for data files for each camera, and read each data file line by line. Then scatter plots of data for red color channel from a 3 by 3 square pixel window are displayed on screen. Plots are for mean, standard deviation, skew and kurtosis.

graphImageStats2.m (145 lines)

Dependencies: ForensicImage.m, getAllFiles.m Image statistical error values from JPEG files in /students/home/semistatic/summer2011/crops/ for each image from each camera are plotted as a histogram with a normal distribution curve fitted to it.

graphImageStats3.m (126 lines)

Dependencies: ForensicImage.m Image statistical error values from the following JPEG files are plotted as a histogram with a normal distribution curve fitted to it: /students/home/semistatic/summer2011/crops/12-camera-canon-eos50d/indoor/IMG_0201.JPG /students/home/semistatic/summer2011/crops/13-camera-kodak-easysharez981/indoor/Picture218.jpg /students/home/semistatic/summer2011/crops/14-camera-nikond40/indoor/DSC_7724.JPG /students/home/semistatic/summer2011/crops/15-camera-olympus-sp570uz/indoor/PA310209.JPG /students/home/semistatic/summer2011/crops/16-camera-panasonic-lumix-dmcfz35/indoor/P1000211.JPG /students/home/semistatic/summer2011/crops/17-camera-sony-alphadslra500l/indoor/DSC00204.JPG

imgChiral.m (137 lines)

Hold chiral patterns for bayer images. 08/05/2011 7

imgChiral(imgHBneg,imgVBneg,imgD1neg,imgD2neg,imgHBpos,imgVBpos,imgD1p os,imgD2pos): Hold imgHBneg = horizontal edge negative, imgVBneg = vertical edge negative, imgD1neg = Diag1 edge negative, imgD2neg = Diag2 edge negative, imgHBpos = horizontal edge positive, imgVBpos = vertical edge positive, imgD1pos = Diag1 edge positive, and imgD2pos = Diag2 edge positive images.

imgEdges.m (264 lines)

Hold edge gradients and demosaicing patterns for Bayer images. imgEdges(imgHdiff,imgVdiff,imgD1diff,imgD2diff,imgHpat,imgVpat,imgD1pat,imgD 2pat): Hold imgHdiff = horizontal_gradient_image, imgVdiff = vertical_gradient_image, imgD1diff = diagonal1_gradient_image, imgD2diff = diagonal2_gradient_image, imgHpat = horizontal_pattern_image, imgVpat = vertical_pattern_image, imgD1pat = diagonal1_pattern_image, and imgD2pat = diagonal2_pattern_image. getEdgeTypes(imgBW): get edge types for grayscale image imgBW. Outputs have 0 for smooth edges, 1 for vertical edges, 2 for horizontal edges, 4 for Diag 2 edges, and 8 for Diag1 edges. getChiral(img,imgTh,cType,s1,s2,win): adjust signs of thresholded image for chirality, where img = grayscale image, imgTh = image with edge types, cType = edge type, s1 = side1 based on matrix found in comments, s2 = side2 based on matrix found in comments, win = window size.

imgStats.m (183 lines)

Hold statistics for images. imgStats(avg,sd,skew,kurt,entropy,energy,pixels,meds,errors): Hold avg = average, sd = standard_deviation, skew = skewness, kurt = kurtosis, entropy = entropy, energy = energy, pixels = original_pixels, meds = pixel_neighborhood_medians, errors = pixel_errors for an image. clean(): Clean imgStats data so all NaNs become 0s.

medfilt2edge.m (272 lines)

Median filter without middle pixel for images. medfilt2edge(image,imgBW,wn,thr): Take median filter of image at edge pixels for pixel neighborhoods in window of size win by win square pixels, where imgBW = edge detection image, thr = threshold for edge filter.

medfilt2new.m (49 lines)

Median filter without middle pixel for images. medfilt2new(image,win): Take median filter of image excluding the middle pixel for pixel neighborhoods in window of size win by win square pixels.

08/05/2011