Вы находитесь на странице: 1из 41

Computer Vision

Spring 2006 15-385,-685


Instructor: S. Narasimhan
Wean 5403
T-R 3:00pm 4:20pm
Lecture #12

Midterm March 9
Syllabus until and including Lightness and Retinex
Closed book, closed notes exam in class.
Time: 3:00pm 4:20pm
Midterm review class next Tuesday (March 7)
(Email me by March 6 specific questions)
If you have read the notes and readings, attended
all classes, done assignments well, it should be
a walk in the park

Mechanisms of Reflection
source
surface
reflection
surface
incident
direction
body
reflection

Body Reflection:
Diffuse Reflection
Matte Appearance
Non-Homogeneous Medium
Clay, paper, etc

Surface Reflection:
Specular Reflection
Glossy Appearance
Highlights
Dominant for Metals
Image Intensity = Body Reflection + Surface Reflection

Example Surfaces
Body Reflection:
Diffuse Reflection
Matte Appearance
Non-Homogeneous Medium
Clay, paper, etc
Surface Reflection:
Specular Reflection
Glossy Appearance
Highlights
Dominant for Metals
Many materials exhibit
both Reflections:

Diffuse Reflection and Lambertian BRDF
viewing
direction
surface
element
normal
incident
direction
i

n
v
s


d
r r i i
f ) , ; , (
Lambertian BRDF is simply a constant :
albedo
Surface appears equally bright from ALL directions! (independent of )
Surface Radiance :
v
Commonly used in Vision and Graphics!
s n I I L
d
i
d
. cos


source intensity
source intensity I

Diffuse Reflection and Lambertian BRDF

White-out: Snow and Overcast Skies
CANT perceive the shape of the snow covered terrain!
CAN perceive shape in regions
lit by the street lamp!!
WHY?

Diffuse Reflection from Uniform Sky
Assume Lambertian Surface with Albedo = 1 (no absorption)
Assume Sky radiance is constant
Substituting in above Equation:
Radiance of any patch is the same as Sky radiance !! (white-out condition)


2 /
0
sin cos ) , ; , ( ) , ( ) , (
i i i i r r i i i i
src
r r
surface
d d f L L
sky
i i
src
L L ) , (


1
) , ; , (
r r i i
f
sky
r r
surface
L L ) , (

Specular Reflection and Mirror BRDF
source intensity I
viewing
direction
surface
element
normal
incident
direction
n
v
s
r
specular/mirror
direction
) , (
i i

) , (
v v

) , (
r r

Mirror BRDF is simply a double-delta function :
Valid for very smooth surfaces.
All incident light energy reflected in a SINGLE direction (only when = ).
Surface Radiance :
) ( ) (
v i v i s
I L +
v r
) ( ) ( ) , ; , (
v i v i s v v i i
f +
specular albedo

Combing Specular and Diffuse: Dichromatic Reflection
Observed Image Color = a x Body Color + b x Specular Reflection Color
R
G
B
Klinker-Shafer-Kanade 1988
Color of Source
(Specular reflection)
Color of Surface
(Diffuse/Body Reflection)
Does not specify any specific model for
Diffuse/specular reflection

Diffuse and Specular Reflection
diffuse specular diffuse+specular

Photometric Stereo
Lecture #12

Image Intensity and 3D Geometry

Shading as a cue for shape reconstruction

What is the relation between intensity and shape?

Reflectance Map

Surface Normal
N
surface normal
y
z
x
Equation of plane 0 + + + D Cz By Ax
0 + + +
C
D
z y
C
B
x
C
A
or
Let
p
C
A
x
z

q
C
B
y
z

( ) 1 , , 1 , , q p
C
B
C
A

,
_

N
Surface normal

Surface Normal
X
Y
Z
X
Y
X p
Y q

Gradient Space
y
z
x
1 z
q
p
1
s
n
N
( )
1
1 , ,
2 2
+ +

q p
q p
N
N
n
( )
1
1 , ,
2 2
+ +

S S
S S
q p
q p
S
S
s
Normal vector
Source vector
i

( )
1 1
1
cos
2 2 2 2
+ + + +
+ +

S S
S S
i
q p q p
qq pp
s n
1 z
plane is called the Gradient Space (pq plane)
Every point on it corresponds to a particular surface orientation
S

Reflectance Map

Relates image irradiance I(x,y) to surface orientation (p,q)


for given source direction and surface reflectance

Lambertian case:
( ) y x I ,
s
v
n
i

: source brightness
: surface albedo (reflectance)
: constant (optical system)
k

c
Image irradiance:
s n kc kc I
i

cos
Let
1 kc

then s n
i
I cos

Lambertian case
( )
( ) q p R
q p q p
qq pp
I
S S
s s
i
,
1 1
1
cos
2 2 2 2

+ + + +
+ +
s n
Reflectance Map
(Lambertian)
cone of constant
i

Iso-brightness contour
Reflectance Map

Lambertian case
0 . 1
3 . 0
0 . 0
9 . 0
8 . 0
( ) 7 . 0 , q p R
p
q

90
i

( ) 0 1 + +
S S
qq pp
( )
S S
q p ,
iso-brightness
contour
Note: is maximum when ( ) q p R , ( ) ( )
S S
q p q p , ,
Reflectance Map

Glossy surfaces (Torrance-Sparrow reflectance model)


( ) ( ) q p R G p
kc
kc I
r
s
i
d
,
cos
cos +

diffuse term specular term


p
q
( ) 5 . 0 , q p R
( )
S S
q p ,
Diffuse peak
Specular peak
Reflectance Map

Shape from a Single Image?

Given a single image of an object with known surface


reflectance taken under a known light source, can we
recover the shape of the object?
Given R(p,q) ( (p
S
,q
S
) and surface reflectance) can we
determine (p,q) uniquely for each image point?
NO
p
q

Solution

Take more images

Photometric stereo

Add more constraints

Shape-from-shading (next class)



Photometric Stereo
p
q
( )
1 1
,
S S
q p
( )
2 2
,
S S
q p
( )
3 3
,
S S
q p

We can write this in matrix form:


Image irradiance:

,
_

kc
1 1
s n I
1
s
n
v
2
s
2 2
s n I
3
s
3 3
s n I
n
s
s
s
1
1
1
]
1

1
1
1
]
1

T
T
T
I
I
I
3
2
2
2
1
1

Lambertian case:
s n

i
kc I cos
Photometric Stereo

Solving the Equations
n
s
s
s

1
1
1
]
1

1
1
1
]
1

T
T
T
I
I
I
3
2
2
2
1
1
I S n
~
1 3 3 3 1 3
I S n
1
~

n
~

n
n
n
n
~
~
~

inverse

More than Three Light Sources

Get better results by using more lights


n
s
s

1
1
1
]
1

1
1
1
]
1

T
N
T
N
I
I

1
1

Least squares solution:


n S I
~

n S S I S
~
T T

( ) I S S S n
T T
1
~

Solve for as before n ,


Moore-Penrose pseudo inverse
( ) ( ) 1 3 3 1 N N

Color Images

The case of RGB images

get three sets of equations, one per color channel:

Simple solution: first solve for using one channel

Then substitute known into above equations to get

Or combine three channels and solve for


Sn I
G G

Sn I
B B

Sn I
R R

( )
B G R
, ,
n
n
Sn I I I I + +
2 2 2
B G R
n

Computing light source directions

Trick: place a chrome sphere in the scene


the location of the highlight tells you the source direction

For a perfect mirror, light is reflected about N


Specular Reflection - Recap

'

otherwise 0
if r v
i
e
R
R

We see a highlight when

Then is given as follows:


n
v
s
r v
s
( ) r n r n s 2
r
i


Computing the Light Source Direction

Can compute N by studying this figure


Hints:
use this equation:
can measure c, h, and r in the image
N
rN
C
H
c
h
Chrome sphere that has a highlight at position h in the image
image plane
sphere in 3D

Depth from Normals
Get a similar equation for V
2
Each normal gives us two linear constraints on z
compute z values by solving a matrix equation
V
1
V
2
N

Limitations

Big problems

Doesnt work for shiny things, semi-translucent things

Shadows, inter-reflections

Smaller problems

Camera and lights have to be distant

Calibration requirements
measure light source directions, intensities
camera response function

Trick for Handling Shadows

Weight each equation by the pixel brightness:

Gives weighted least-squares matrix equation:

Solve for as before


n
s
s

1
1
1
]
1

1
1
1
]
1

T
N N
T
N
I
I
I
I

1
1
2
2
1
n ,
( ) ( )
i i i i
I I I s n

Original Images

Results - Shape
Shallow reconstruction
(effect of interreflections)
Accurate reconstruction
(after removing interreflections)

Results - Albedo
No Shading Information

Original Images

Results - Shape

Results - Albedo

Results
1. Estimate light source directions
2. Compute surface normals
3. Compute albedo values
4. Estimate depth from surface normals
5. Relight the object (with original texture and uniform albedo)

Next Class

Shape from Shading

Reading: Horn, Chapter 11.