Академический Документы
Профессиональный Документы
Культура Документы
So far...
So far...
So far...
S-sparse x
S-sparse x
S-sparse x
Suppose =
pn
m
R F
Suppose =
pn
m
R F
Suppose =
pn
m
R F
s = y = x
s = y = x
Assume 2S <
2 1 = .414. Then
kx? xk`2 .
kx xS k`1
s = y = x
Assume 2S <
2 1 = .414. Then
kx? xk`2 .
kx xS k`1
s = y = x
Assume 2S <
2 1 = .414. Then
kx? xk`2 .
kx xS k`1
x = x0
x = x0
21
Formal equivalence
min ksk`0 ,
sRn
s = y
min ksk`1 ,
sRn
s = y
Formal equivalence
min ksk`0 ,
sRn
s = y
min ksk`1 ,
sRn
s = y
Equivalence
If 2S < 1, solution to (P0 ) is unique
If 2S < 2 1, the solutions to (P0 ) and (P1 ) are unique and the same!
s. t.
y = s = (x)
s. t.
y = s = (x)
s. t.
y = s = (x)
kx + hk`1 =
X
iT0
X
iT0
|xi + hi | +
|xi + hi |
iT0c
|xi | |hi | +
|hi | |xi |
iT0c
Implication
khk`1 (T0c ) khk`1 (T0 ) + 2kxk`1 (T0c ) = khk`1 (T0 ) + 2kx xS k`1
Implication
khk`1 (T0c ) khk`1 (T0 ) + 2kxk`1 (T0c ) = khk`1 (T0 ) + 2kx xS k`1
<1
()
Implication
khk`1 (T0c ) khk`1 (T0 ) + 2kxk`1 (T0c ) = khk`1 (T0 ) + 2kx xS k`1
<1
This gives
khk`1 (T0c )
2
kx xS k`1
1
()
Implication
khk`1 (T0c ) khk`1 (T0 ) + 2kxk`1 (T0c ) = khk`1 (T0 ) + 2kx xS k`1
<1
This gives
khk`1 (T0c )
2
kx xS k`1
1
and
kx? xk`1 2
1+
kx xS k`1
1
()
Implication
khk`1 (T0c ) khk`1 (T0 ) + 2kxk`1 (T0c ) = khk`1 (T0 ) + 2kx xS k`1
<1
This gives
khk`1 (T0c )
2
kx xS k`1
1
and
kx? xk`1 2
We are done!
1+
kx xS k`1
1
()
Implication
khk`1 (T0c ) khk`1 (T0 ) + 2kxk`1 (T0c ) = khk`1 (T0 ) + 2kx xS k`1
<1
This gives
khk`1 (T0c )
2
kx xS k`1
1
and
kx? xk`1 2
1+
kx xS k`1
1
()
By Cauchy-Schwarz
khk`1 (T0 )
S khk`2 (T0 )
By Cauchy-Schwarz
khk`1 (T0 )
S khk`2 (T0 )
By Cauchy-Schwarz
khk`1 (T0 )
S khk`2 (T0 )
X
j2
hTj
By Cauchy-Schwarz
khk`1 (T0 )
S khk`2 (T0 )
hTj
j2
j2
For each j 2
khTj k`2
1
S0 khTj k` khTj1 k`1
S0
For each j 2
khTj k`2
and thus
X
j2
1
S0 khTj k` khTj1 k`1
S0
khT c k`1
1
khTj k`2 (khT1 k`1 + khT2 k`1 + . . .) 0
0
S
S0
For each j 2
khTj k`2
and thus
X
j2
1
S0 khTj k` khTj1 k`1
S0
khT c k`1
1
khTj k`2 (khT1 k`1 + khT2 k`1 + . . .) 0
0
S
S0
In short
s
khk`2 (T0 T1 )
1 + S0 khT0c k`1
1 S+S0
S0
For each j 2
khTj k`2
and thus
X
j2
1
S0 khTj k` khTj1 k`1
S0
khT c k`1
1
khTj k`2 (khT1 k`1 + khT2 k`1 + . . .) 0
0
S
S0
In short
s
khk`2 (T0 T1 )
1 + S0 khT0c k`1
1 S+S0
S0
S khk`2 (T0 )
2 =
S 1 + S 0
S0 1 S+S0
For each j 2
khTj k`2
and thus
X
j2
1
S0 khTj k` khTj1 k`1
S0
khT c k`1
1
khTj k`2 (khT1 k`1 + khT2 k`1 + . . .) 0
0
S
S0
In short
s
khk`2 (T0 T1 )
1 + S0 khT0c k`1
1 S+S0
S0
S khk`2 (T0 )
1 1 + 2S
<1
2 1 3S
iff
2S + 3S < 1
2 =
S 1 + S 0
S0 1 S+S0
Slight improvement
Lemma
khT0 k`1
2 2S
khT0c k`1
1 2S
A slight improvement
Definition (Restricted orthogonality numbers)
S,S0 : smallest quantity for which
|hx, x0 i| S,S0 kxk`2 kx0 k`2
x, x0 supported on disjoint subsets T, T 0 {1, . . . , n} with |T| S, |T 0 | S0
A slight improvement
Definition (Restricted orthogonality numbers)
S,S0 : smallest quantity for which
|hx, x0 i| S,S0 kxk`2 kx0 k`2
x, x0 supported on disjoint subsets T, T 0 {1, . . . , n} with |T| S, |T 0 | S0
Suppose x and x0 are unit vectors as above. Then
2(1 S+S0 ) kx + x0 k2`2 2(1 + S+S0 )
2(1 S+S0 ) kx x0 k2`2 2(1 + S+S0 )
Parallelogram identity
|hx, x0 i| =
1
kx + x0 k2`2 kx x0 k2`2 S+S0
4
A slight improvement
Definition (Restricted orthogonality numbers)
S,S0 : smallest quantity for which
|hx, x0 i| S,S0 kxk`2 kx0 k`2
x, x0 supported on disjoint subsets T, T 0 {1, . . . , n} with |T| S, |T 0 | S0
Suppose x and x0 are unit vectors as above. Then
2(1 S+S0 ) kx + x0 k2`2 2(1 + S+S0 )
2(1 S+S0 ) kx x0 k2`2 2(1 + S+S0 )
Parallelogram identity
|hx, x0 i| =
Therefore S,S0 S+S0
1
kx + x0 k2`2 kx x0 k2`2 S+S0
4
gives
khT0 T1 k2`2 = hhT0 T1 ,
X
j2
hTj i = hhT0 ,
hTj i hhT1 ,
j2
hTj i
j2
X
j2
khTj k`2
gives
khT0 T1 k2`2 = hhT0 T1 ,
hTj i = hhT0 ,
j2
hTj i hhT1 ,
j2
hTj i
j2
khTj k`2
j2
2khT0 T1 k`2 ,
X
j2
khTj k`2
gives
khT0 T1 k2`2 = hhT0 T1 ,
hTj i = hhT0 ,
j2
hTj i hhT1 ,
j2
hTj i
j2
khTj k`2
j2
2khT0 T1 k`2 ,
X
j2
and thus
khT0 T1 k`2 0
X
j2
khTj k`2 ,
0 =
2 S,S
1 2S
khTj k`2
khk`1 = 2
1+
kx xS k`1
1
khk`1 = 2
1+
kx xS k`1
1
Since
2 2S
12S ,
we have
1 + ( 2 1)2S
1
1+
1 (1 + 2)2S
and
kx? xk`1 2
1 + ( 2 1)2S
kx xS k`1
1 (1 + 2)2S
khk`1 = 2
1+
kx xS k`1
1
Since
2 2S
12S ,
we have
1 + ( 2 1)2S
1
1+
1 (1 + 2)2S
and
kx? xk`1 2
1 + ( 2 1)2S
kx xS k`1
1 (1 + 2)2S
khT0c k`1
k
)c )
k2`2
kh
T0c
k2`1
n
X
khT0c k2`1
1
2
k
S
k=S+1
khT0c k`1
k
)c )
k2`2
kh
T0c
k2`1
n
X
khT0c k2`1
1
2
k
S
k=S+1
With
2 2S
12S ,
khT0c k`1
k
)c )
k2`2
kh
T0c
k2`1
n
X
khT0c k2`1
1
2
k
S
k=S+1
With
2 2S
12S ,
And thus
khT0c k`1
1 + kx xS k`1
2
1
S
S
This is exactly the same constant!
khk`2 (1 + )
Gaussian matrices
Random projections
Binary matrices
Bounded orthonormal systems
Gaussian matrices
Gaussian matrices
Condition holds if
m & S log(n/S)
Random projections
Random projections
Condition holds if
m & S log(n/S)
Binary matrices
Binary matrices
Condition holds if
m & S log(n/S)
Binary matrices
Condition holds if
m & S log(n/S)
DeVore et al., Pajor et. al.
Binary matrices
Condition holds if
m & S log(n/S)
DeVore et al., Pajor et. al.
Same result is true for large classes of distributions, i.e. subGaussian
distributions
n sup |Uj,k |
j,k
n sup |Uj,k |
j,k
n sup |Uj,k |
j,k
n sup |Uj,k |
j,k
Sparse in basis 1
Undersample in basis 2
System
U = 2 1
Coherence
(U) =
n sup |Uj,k | =
j,k
(1)
(2)
n sup |hj , k i| = (1 , 2 )
j,k
Sparse in basis 1
Undersample in basis 2
System
U = 2 1
Coherence
(U) =
n sup |Uj,k | =
j,k
(1)
(2)
n sup |hj , k i| = (1 , 2 )
j,k
Condition holds if
m & 2 (1 , 2 ) S (log n)5
subject to f = y
subject to f = y
Same performance!
(Sketch) If m & S log(n/S),
kf ? f k kf fS k