Академический Документы
Профессиональный Документы
Культура Документы
Naveen N. Narisetty
Department of Statistics
2 Computation
Large Matrix Computations
Scalable Gibbs Sampler
Introduction Scalable Computation
Introduction
Graphical Models
···
The number of parameters in most of the statistical models
explodes – brings computational and theoretical challenges!
#{j : βj 6= 0} (p ∧ n)
#{j : βj 6= 0} (p ∧ n)
Bayesian Framework
Place priors on βj as
and on Z = (Z1 , . . . , Zp )
2 → 0 (faster than 1 )
Spike prior shrinks: τ0,n n
– Does not miss “small” non-zero coefficients
2+
2 → ∞ (as ( pn ∨ 1))
Slab prior diffuses: τ1,n n
– Acts as a penalty to drive inactive covariates under Z = 0
Outline
2 Computation
Large Matrix Computations
Scalable Gibbs Sampler
Computation
Introduction Scalable Computation
Posterior Computation
Computation 11/ 28
Introduction Scalable Computation
Posterior Computation
Computation 11/ 28
Introduction Scalable Computation
Posterior Computation
Computation 11/ 28
Introduction Scalable Computation
V = (X 0 X + Dz )−1 ,
−2 −2
where Dz = Diag (Z τ1,n + (1 − Z )τ0,n )
Strategy:
– To sparsify the precision matrix X 0 X + Dz so that the
βj ’s can somehow be sampled independently
V = (X 0 X + Dz )−1 ,
−2 −2
where Dz = Diag (Z τ1,n + (1 − Z )τ0,n )
Strategy:
– To sparsify the precision matrix X 0 X + Dz so that the
βj ’s can somehow be sampled independently
−2
XA0 XA + τ1n
!
I 0
S −1 :=
−2
0 (n + τ0n )I
−2
XA0 XA + τ1n
!
I 0
S −1 :=
−2
0 (n + τ0n )I
We can recover!
We can recover!
Skinny Gibbs
qn φ(βj , 0, σ 2 τ1,n
2 )
exp σ −2 βj Xj0 (Y − XAj βAj ) ,
Rj = 2 )
(1 − qn ) φ(βj , 0, σ 2 τ0,n
Skinny Gibbs
qn φ(βj , 0, σ 2 τ1,n
2 )
exp σ −2 βj Xj0 (Y − XAj βAj ) ,
Rj = 2 )
(1 − qn ) φ(βj , 0, σ 2 τ0,n
−2
(XA0 XA + τ1n I)−1
!
0
S=
−2 −1
0 (n + τ0n ) I
Rj
P[Zj = 1 | Z−j , Rest] =
1 + Rj
Example:
– Logistic Distribution
Li ∼ Logistic(xi β) ⇐⇒ Li | si ∼ N(xi β, si2 ); si /2 ∼ FKS ,
FKS is the Kolmogorov-Smirnov distribution (Stefanski, 1991)
Example:
– Logistic Distribution
Li ∼ Logistic(xi β) ⇐⇒ Li | si ∼ N(xi β, si2 ); si /2 ∼ FKS ,
FKS is the Kolmogorov-Smirnov distribution (Stefanski, 1991)
Simulation Study
n = 100, p = 250.
Simulation Results
TP FP Z =t Z =t Zs = t
True Positives False positives Selection Inclusion Top 4 Selection
Simulation Results
TP FP Z =t Z =t Zs = t
True Positives False positives Selection Inclusion Top 4 Selection
Computational Time
Figure: CPU time (in seconds) for BASAD and Skinny Gibbs for n = 100
and p varies.
Y o = (Y ∨ c)
Example: Carotid artery plaque thickness in stroke patients
(Gardener et al., 2014)
Conditional mean of Y is not identifiable but (most)
conditional quantiles are!
τ th conditional quantile of Y : Qτ (Y | X ) = X βτ
Y o = (Y ∨ c)
Example: Carotid artery plaque thickness in stroke patients
(Gardener et al., 2014)
Conditional mean of Y is not identifiable but (most)
conditional quantiles are!
τ th conditional quantile of Y : Qτ (Y | X ) = X βτ
Non-convexity
Conclusions