Conjugate prior relationships

The following diagram summarizes conjugate prior relationships for a number of common
sampling distributions. Arrows point from a sampling distribution to its conjugate prior
distribution. The symbol near the arrow indicates which parameter the prior is unknown.
These relationships depends critically on choice of parameterization, some of which are
uncommon. This page uses the parameterizations that make the relationships simplest to state,
not necessarily the most common parameterizations. See footnotes below. Other diagrams on this
site:

Probability distribution relationships

Modes of convergence

Gamma and related function identities

Special function diagram

Bessel function relationships

Click on a distribution to see its parameterization. Click on an arrow to see posterior

parameters.

Parameterizations
Let C(n, k) denote the binomial coefficient(n, k).
The geometric distribution has only one parameter, p, and has PMF f(x) = p (1-p)x.
The binomial distribution with parameters n and p has PMF f(x) = C(n, x) px(1-p)n-x.
The negative binomial distribution with parameters r and p has PMF f(x) = C(r + x - 1, x) pr(1p)x.
The Bernoulli distribution has probability of success p.
The beta distribution has PDF f(p) = ( + ) p-1(1-p)-1 / (() ()).

The exponential distribution parameterized in terms of the rate has PDF f(x) = exp(- x).
The gamma distribution parameterized in terms of the rate has PDF f(x) = x-1exp(- x) / ().
The Poisson distribution has one parameter and PMF f(x) = exp(-) x/ x!.
The normal distribution parameterized in terms of precision ( = 1/2) has PDF f(x) = (/2)1/2
exp( -(x - )2/2 ).
The lognormal distribution parameterized in terms of precision has PDF f(x) = (/2)1/2 exp(
-(log(x) - )2/2 ) / x.

Posterior parameters
For each sampling distribution, assume we have data x1, x2, ..., xn.
If the sampling distribution for x is binomial(m, p) with m known, and the prior distribution is
beta(, ), the posterior distribution for p is beta( + xi, + mn - xi). The Bernoulli is the
special case of the binomial with m = 1.
If the sampling distribution for x is negative binomial(r, p) with r known, and the prior
distribution is beta(, ), the posterior distribution for p is beta( + nr, + xi). The geometric
is the special case of the negative binomial with r = 1.
If the sampling distribution for x is gamma(, ) with known, and the prior distribution on is
gamma(0, 0), the posterior distribution for is gamma(0 + n, 0 + xi). The exponential is a
special case of the gamma with = 1.
If the sampling distribution for x is Poisson(), and the prior distribution on is gamma(0, 0),
the posterior on is gamma(0 + xi, 0 + n).
If the sampling distribution for x is normal(, ) with known, and the prior distribution on is
normal(0, 0), the posterior distribution on is normal((0 0 + xi)/(0 + n), 0 + n0).
If the sampling distribution for x is normal(, ) with known, and the prior distribution on is
gamma(, ), the posterior distribution on is gamma( + n/2, (n-1)S2) where S2 is the sample
variance.
If the sampling distribution for x is lognormal(, ) with known, and the prior distribution on
is normal(0, 0), the posterior distribution on is normal((0 0 + xi)/(0 + n), 0 + n0).
If the sampling distribution for x is lognormal(, ) with known, and the prior distribution on
is gamma(, ), the posterior distribution on is gamma( + n/2, (n-1)S2) where S2 is the
sample variance.
http://www.johndcook.com/conjugate_prior_diagram.html

RELATIONSHIPS AMONG DISTRIBUTIONS

Dotted lines represent limiting cases of a distribution. (From D. Zwillinger and
S. Kokoska, Standard Probability and Statistics Tables and Formulae, Chapman &
Hall/CRC, Boca Raton, Florida, 2000. With permission.)

Source : CRC Standard Mathematical Tables and Formulae__32nd_Ed (p 531) AAA

(From D. Zwillinger and S. Kokoska, Standard Probability and Statistics Tables and Formulae,
Chapman & Hall/CRC, Boca Raton, Florida, 2000. With permission.)

The chart above is adapted from the chart originally published by Lawrence Leemis in 1986
(Relationships Among Common Univariate Distributions, American Statistician 40:143-146.)
Leemis published a larger chart in 2008 which is available online.

Parameterizations

The precise relationships between distributions depend on parameterization. The relationships

detailed below depend on the following parameterizations for the PDFs.
Let C(n, k) denote the binomial coefficient(n, k) and B(a, b) = (a) (b) / (a + b).
Geometric: f(x) = p (1-p)x for non-negative integers x.
Discrete uniform: f(x) = 1/n for x = 1, 2, ..., n.
Negative binomial: f(x) = C(r + x - 1, x) pr(1-p)x for non-negative integers x. See notes on the
negative binomial distribution.
Beta binomial: f(x) = C(n, x) B( + x, n + - x) / B(, ) for x = 0, 1, ..., n.
Hypergeometric: f(x) = C(M, x) C(N-M, K - x) / C(N, K) for x = 0, 1, ..., N.
Poisson: f(x) = exp(-) x/ x! for non-negative integers x. The parameter is both the mean and
the variance.
Binomial: f(x) = C(n, x) px(1 - p)n-x for x = 0, 1, ..., n.
Bernoulli: f(x) = px(1 - p)1-x where x = 0 or 1.
Lognormal: f(x) = (22)-1/2 exp( -(log(x) - )2/ 22) / x for positive x. Note that and 2 are not
the mean and variance of the distribution.
Normal : f(x) = (2 2)-1/2 exp( - ((x - )/)2 ) for all x.
Beta: f(x) = ( + ) x-1(1 - x)-1 / (() ()) for 0 x 1.
Standard normal: f(x) = (2)-1/2 exp( -x2/2) for all x.
Chi-squared: f(x) = x-/2-1 exp(-x/2) / (/2) 2/2 for positive x. The parameter is called the
degrees of freedom.
Gamma: f(x) = - x-1 exp(-x/) / () for positive x. The parameter is called the shape and
is the scale.
Uniform: f(x) = 1 for 0 x 1.
Cauchy: f(x) = /(( (x - )2 + 2) ) for all x. Note that and are location and scale parameters.
The Cauchy distribution has no mean or variance.
Snedecor F: f(x) is proportional to x(1 - 2)/2 / (1 + (1/2) x)(1 + 2)/2 for positive x.
Exponential: f(x) = exp(-x/)/ for positive x. The parameter is the mean.

Student t: f(x) is proportional to (1 + (x2/))-( + 1)/2 for positive x. The parameter is called the
degrees of freedom.
Weibull: f(x) = (/) x-1 exp(- x/) for positive x. The parameter is the shape and is the
scale.
Double exponential : f(x) = exp(-|x-|/) / 2 for all x. The parameter is the location and
mean; is the scale.
For comparison, see distribution parameterizations in R/S-PLUS and Mathematica.

Relationships
In all statements about two random variables, the random variables are implicitly independent.
Geometric / negative binomial: If each Xi is geometric random variable with probability of
success p then the sum of n Xi's is a negative binomial random variable with parameters n and p.
Negative binomial / geometric: A negative binomial distribution with r = 1 is a geometric
distribution.
Negative binomial / Poisson: If X has a negative binomial random variable with r large, p near
1, and r(1-p) = , then FX FY where Y is a Poisson random variable with mean .
Beta-binomial / discrete uniform: A beta-binomial (n, 1, 1) random variable is a discrete
uniform random variable over the values 0 ... n.
Beta-binomial / binomial: Let X be a beta-binomial random variable with parameters (n, , ).
Let p = /( + ) and suppose + is large. If Y is a binomial(n, p) random variable then FX
FY.
Hypergeometric / binomial: The difference between a hypergeometric distribution and a
binomial distribution is the difference between sampling without replacement and sampling with
replacement. As the population size increases relative to the sample size, the difference becomes
negligible.
Geometric / geometric: If X1 and X2 are geometric random variables with probability of success
p1 and p2 respectively, then min(X1, X2) is a geometric random variable with probability of
success p = p1 + p2 - p1 p2. The relationship is simpler in terms of failure probabilities: q = q1 q2.
Poisson / Poisson: If X1 and X2 are Poisson random variables with means 1 and 2 respectively,
then X1 + X2 is a Poisson random variable with mean 1 + 2.
Binomial / Poisson: If X is a binomial(n, p) random variable and Y is a Poisson(np) distribution
then P(X = n) P(Y = n) if n is large and n$ is small. For more information, see Poisson
approximation to binomial.

Binomial / Bernoulli: If X is a binomial(n, p) random variable with n = 1, X is a Bernoulli(p)

random variable.
Bernoulli / Binomial: The sum of n Bernoulli(p) random variables is a binomial(n, p) random
variable.
Poisson / normal: If X is a Poisson random variable with large mean and Y is a normal
distribution with the same mean and variance as X, then for integers j and k, P(j X k) P(j 1/2 Y k + 1/2). For more information, see normal approximation to Poisson.
Binomial / normal: If X is a binomial(n, p) random variable and Y is a normal random variable
with the same mean and variance as X, i.e. np and np(1-p), then for integers j and k, P(j X k)
P(j - 1/2 Y k + 1/2). The approximation is better when p and when n is large. For more
information, see normal approximation to binomial.
Lognormal / lognormal: If X1 and X2 are lognormal random variables with parameters (1, 12)
and (2, 22) respectively, then X1 X2 is a lognormal random variable with parameters (1 + 2, 12
+ 22).
Normal / lognormal: If X is a normal (, 2) random variable then eX is a lognormal (, 2)
random variable. Conversely, if X is a lognormal (, 2) random variable then log X is a normal
(, 2) random variable.
Beta / normal: If X is a beta random variable with parameters and equal and large, FX FY
where Y is a normal random variable with the same mean and variance as X, i.e. mean /( + )
and variance /((+)2( + + 1)). For more information, see normal approximation to beta.
Normal / standard normal: If X is a normal(, 2) random variable then (X - )/ is a standard
normal random variable. Conversely, If X is a normal(0,1) random variable then X + is a
normal (, 2) random variable.
Normal / normal: If X1 is a normal (1, 12) random variable and X2 is a normal (2, 22) random
variable, then X1 + X2 is a normal (1 + 2, 12 + 22) random variable.
Gamma / normal: If X is a gamma(, ) random variable and Y is a normal random variable
with the same mean and variance as X, then FX FY if the shape parameter is large relative to
the scale parameter . For more information, see normal approximation to gamma.
Gamma / beta: If X1 is gamma(1, 1) random variable and X2 is a gamma (2, 1) random
variable then X1/(X1 + X2) is a beta(1, 2) random variable. More generally, if X1 is gamma(1,
1) random variable and X2 is gamma(2, 2) random variable then 2 X1/(2 X1 + 1 X2) is a
beta(1, 2) random variable.
Beta / uniform: A beta random variable with parameters = = 1 is a uniform random variable.

Chi-squared / chi-squared: If X1 and X2 are chi-squared random variables with 1 and 2

degrees of freedom respectively, then X1 + X2 is a chi-squared random variable with 1 + 2
degrees of freedom.
Standard normal / chi-squared: The square of a standard normal random variable has a chisquared distribution with one degree of freedom. The sum of the squares of n standard normal
random variables is has a chi-squared distribution with n degrees of freedom.
Gamma / chi-squared: If X is a gamma (, ) random variable with = /2 and = 2, then X is
a chi-squared random variable with degrees of freedom.
Cauchy / standard normal: If X and Y are standard normal random variables, X/Y is a
Cauchy(0,1) random variable.
Student t / standard normal: If X is a t random variable with a large number of degrees of
freedom then FX FY where Y is a standard normal random variable. For more information, see
normal approximation to t.
Snedecor F / chi-squared: If X is an F(, ) random variable with large, then X is
approximately distributed as a chi-squared random variable with degrees of freedom.
Chi-squared / Snedecor F: If X1 and X2 are chi-squared random variables with 1 and 2 degrees
of freedom respectively, then (X1/1)/(X2/2) is an F(1, 2) random variable.
Chi-squared / exponential: A chi-squared distribution with 2 degrees of freedom is an
exponential distribution with mean 2.
Exponential / chi-squared: An exponential random variable with mean 2 is a chi-squared
random variable with two degrees of freedom.
Gamma / exponential: The sum of n exponential() random variables is a gamma(n, ) random
variable.
Exponential / gamma: A gamma distribution with shape parameter = 1 and scale parameter
is an exponential() distribution.
Exponential / uniform: If X is an exponential random variable with mean , then exp(-X/) is a
uniform random variable. More generally, sticking any random variable into its CDF yields a
uniform random variable.
Uniform / exponential: If X is a uniform random variable, - log X is an exponential random
variable with mean . More generally, applying the inverse CDF of any random variable X to a
uniform random variable creates a variable with the same distribution as X.
Cauchy reciprocal: If X is a Cauchy (, ) random variable, then 1/X is a Cauchy (/c, /c)
random variable where c = 2 + 2.

Cauchy sum: If X1 is a Cauchy (1, 1) random variable and X2 is a Cauchy (2, 2), then X1 +
X2 is a Cauchy (1 + 2, 1 + 2) random variable.
Student t / Cauchy: A random variable with a t distribution with one degree of freedom is a
Cauchy(0,1) random variable.
Student t / Snedecor F: If X is a t random variable with degree of freedom, then X2 is an
F(1,) random variable.
Snedecor F / Snedecor F: If X is an F(1, 2) random variable then 1/X is an F(2, 1) random
variable.
Exponential / Exponential: If X1 and X2 are exponential random variables with mean 1 and 2
respectively, then min(X1, X2) is an exponential random variable with mean 1 2/(1 + 2).
Exponential / Weibull: If X is an exponential random variable with mean , then X1/ is a
Weibull(, ) random variable.
Weibull / Exponential: If X is a Weibull(1, ) random variable, X is an exponential random
variable with mean .
Exponential / Double exponential: If X and Y are exponential random variables with mean ,
then X-Y is a double exponential random variable with mean 0 and scale
Double exponential / exponential: If X is a double exponential random variable with mean 0
and scale , then |X| is an exponential random variable with mean .
http://www.johndcook.com/distribution_chart.html