Course Peru Salas Obey All

1
Riesgo Hidrolgico de Eventos

Extremos en Condiciones
No-Estacionarias
Jose D. Salas
Colorado State University, USA
J. Obeysekera
South Florida Water Management District, USA
Laboratorio Nacional deHidrulica,UniversidadNacional deIngeniera,
LimaPer
LectureOutline
Introduction. Causes of change in extreme events
Review if basic concepts of probability and statistics
Models of extreme events for stationary and non-
stationary conditions
Parameter estimation and model selection for
Other model alternatives for extreme events
Introduction to software R for modeling extreme
events
2
2
LectureOutline(cont.)
Introduction to software R for modeling extreme
events
Parameter estimation and model selection for non-
Return period and risk for stationary and non-
Examples of analysis of extreme events based on
stationary and non-stationary conditions
Hands on experience in using software R for
analyzing extreme events under stationary and non-
3
Resources
4
3
No-Estacionarias
Jose D. Salas
J. Obeysekera
LimaPer
Introduction:Causesofchangesinextremeevents
Courseobjectives
Whatisexpectedtobelearnedfromthecourse
Summaryofthecoursecontent
Format
Materialsandreferences
Others
Introduction
4
Outline
Extremehydrologicevents
Typesofextremevaluedata
Typesofchanges(nonstationarity)inhydrologicdata
Causesofnonstationarity
Introduction
ExtremeHydrologicEvents
Floods
Lowflowsanddroughts
Max.andmin.precipitation
Max.andmin.temperature(heatwaves)
Maximumwind
Max.andmin.storages(e.g.reservoir,groundwater
levels,soilmoisture,snowpack,glaciers)
Max.sealevels
Max.erosionrates,sedimenttransport,andsediment
deposition
Maximumwaterqualitypollutantconcentrations
Manyothers
5
Drasticdamagesandchangesproducedbytheextraordinaryfloodoriginated
fromHurricaneMitchofOctober1998.(a)ariverandabridgeinGuatemala,
Source:TimeMagazine,Nov.1998,and(b)theCholutecaRiverinTegucigalpa,
Honduras,Source:NationalGeographicVol.196(5),p.111,Nov.1999.
ExtremeFloods
(a) (b)
Tous DamatRioJucar,Spain
breachedandcausedafloodof
about15,200m
3
/salongRiverJucar
inOctoberof1982. Source:Dr.J.
MarcoSegura,U.Valencia.
ExtremeFloods
FortCollinsflood,July1997
6
ExtremeFloods
ExtremeFloods
WashedoutroadoverMilfordDam
nearMilford,KansasinJuly1993
(source:coverofASCECivil
EngineeringMagazine,Vol.64(1),
January.)
Mississippifloodof1993,USA
(sourceunknown)
7
ExtremeFloods
DamageofextremefloodoccurredinVenezuela,Dec.15,1999
(source:CentralUniversityinCaracas,Prof.MarcoP.Rivero)
Venezuela
Venezuela
ExtremeFloods
Queensland,Australia,2011
Pakistan,2010
India,2010 China
8
ExtremeFloods
Thefrequency(ofextremeweathersituations)iswayup,Andrew
Cuomo,GovernorofNewYork,10/31/2012
ExtremeDroughts
Dustbowl,MidwestUS1930s
(sourceunknown)
HaymanfirenearCheesman Reservoir
(source:DenverWater)
9
ExtremeDroughts
Aphotoofawildfireinthewestin
2000duetodroughtconditions,Aug.
6,2000intheBitterrootValley
NationalForest,MT.
DroughtintheAmazonRiver
Source:G.Chamorro,SENAMHI
ExtremeDroughts
Cheesman Reservoir,Colorado
Drought2002
DillonReservoir,Colorado
Drought2002
10
19
TypesofExtremeValueData:
BlockMaxima
Blocks
12345
Blocklengthcouldbe:
OneYear
Season(wetanddry)
Example:DailyPrecipitation
TypesofExtremeValueData:
PeaksOverThreshold(POT)
In hydrology, this is
known as Partial
Duration Series
20
Threshold
Example:DailyPrecipitation
11
DefinitionsofLowFlows
Timeseriesofdailyflows,19512000
DefinitionofLowFlows(1)
UnittimeperiodT
u
d
d
d
v
1
v
j
v
m
12
Thedday lowflowforagivenunittimeperiodT
u
is:
Q=min(v
1
,v
2
,,v
m
)
wherem=numberofdday flowsinaunittimeperiod,
e.g.m=356ford=10andT
u
=1year
ForNunittimeperiods(e.g.forNyearsofrecord)we
willgetthesequence
Q
1
,Q
2
,...,Q
N
Then,frequencyanalysismustbeperformedtogetthe
Tyeardday lowflow.Notethatforlowflows
T=1/q(q=nonexceedance probability)
Lowflowforavariableduration
Inthiscasethedurationoflowflowsisarandomvariable
AthresholdflowQ
o
isselectedandthefollowinglowflow
variablescanbedefined(refertothefollowingfigure)
d*=max(d
1
,d
2
,...,d
m
)max.durationoflowflows
v*=max(v
1
,v
2
,...,v
m
)max.deficit
I*=max(I
1
,I
2
,...,I
m
)max.intensityoflowflows
Otherquantitiescanbedefinedsuchasaveragesor
maximums,etc.
Series:d*
1
,d*
2
,...,d*
N
todofrequencyanalysis.
13
d
1
d
2 d
3
v
1
v
2
v
3
Exampleoflowflowduration
frequencyanalysis
ThedailyflowsofEdisco River,SCforthe
period19512000
14
Exampleoflowflowduration
frequencyanalysis
Extrememultiyeardroughts
PoudreRiverannualflowsattheMouthoftheCanyon(18842002)
15
Droughtdefinitionandproperties
Criticaldroughtscanbeextractedfromthetimeseries
TypesofChanges(nonstationarity)
inHydrologicData
Trends(increasinganddecreasing)
Shifts(upwardordownward)
Mixed
Seasonality(periodicity,cycles)
Pseudo cycles
Clustering
Others
16
Trends(increasingfloods)
(a)Abjerjona Basin (b)LittleSugarCreekBasin.
t
=
0

t
=
0
+a t
Trendinthemeanandinthestandarddeviation
Trends(increasinganddecreasing)
insealevels
(a)KeyWest,Florida(b)Adak,Alaska
Example:c= 1.34mm/yr,b =2.71x10
5
,e =1079mm,
=111.3mm,and = 0.26fortheAdakgage.
17
ShiftsinHydrologicData
NigerRiverannualflows.Exampleofupwardsanddownwardsshifts
St.JohnsRiver(Florida)AnnualFloods
ShiftrelatedtotheAMO
ShiftsinFloodData
18
AnnualCycleinHydrologicData
Monthlystreamflow data
DailyCycleinHydrologicData
HourlyrainfallforthemonthofJulyforDenverAirport
19
CausesofNonstationarity
Humaninterventioninriverbasins(watersheds)
Constructionofhydraulicstructures
Dams
Diversions
Tunnels
Groundwaterpumping
Others
Landusechanges
Urbanization
Irrigationandfarmingactivities
Deforestation
Buildingtransportationsystems
Others
Industrialdevelopmentandoperations
Miningactivities
Others
Humaninterventioninriverbasins(watersheds)
Naturalclimatevariability
Effectsoflowfrequency(largescale)phenomenasuchas
ENSO(years)
PDO(decades),and
AMO(multidecades)
Climatechangeduetoincreaseofgreenhousegases
increaseinairtemperature
increaseinmoistureoftheair
hydrologiceffectsuncertainandstilldebatable
20
Effectofconstructionofhydraulicstructures
alongdeHanRiverinKorea(sourceDr.D.R.Lee)
Humanintervention:Constructionofhydraulicstructures
Effectsofconstructionofhydraulicstructures
alongdeHanRiverinKorea(sourceDr.D.R.Lee)
Humanintervention:Constructionofhydraulicstructures
21
(a) Aberjona River,Winchester
Massachusetts
(b)LittleSugarCreek,Charlotte
NorthCarolina
Humanintervention:effectsofurbanization
Naturalclimatevariability:Effectsoflowfrequency
33years
40years
26years
32years
warm
cold
cold
22
Effect of natural climate variability, PDO
(source: Akintug and Rasmussen, 2005)
Effect of natural climate variability, AMO
source: Dr. J . Obeysekera, SFWMD
Naturalclimatevariability:Effectsoflowfrequency
FinalRemarks
23
No-Estacionarias
Jose D. Salas
J. Obeysekera
LimaPer
ReviewofBasicConceptsofProbability,Statistics,
andTimeSeries
Outline
Randomevents
Probabilityofrandomevents
Randomvariable
Probabilitylaws
Independentanddependentrandomvariables
Populationmoments
Momentsoflinearfunctionsofrandomvariables
Momentsofnonlinear functionsofrandomvariables
Centrallimittheorem
ReviewofBasicConceptsofProbability
24
25
26
27
ReviewofBasicConceptsofStatistics
Outline
Randomsample
Samplemoments
Momentsofsamplemoments
Estimation(estimatesandestimators)
Methodsofestimation
Propertiesofestimators
Confidencelimitsonpopulationparameters
Confidencelimitsonpopulationquantiles
28
29
30
31
32
ReviewofBasicConceptsofTimeSeries
ReviewofBasicConceptsofTimeSeries
33
FinalRemarks
Riesgo Hidrlogico de Eventos
No-Estacionarias
Jose D. Salas
J. Obeysekera
LimaPer
ModelsofExtremeEventsforStationaryand
NonStationaryConditions
34
Outline
Typicalmodelsforextremeevents
(stationaryconditions)
Generalextremevalue(GEV)model
Generalextremevalue(GEV)model
(nonstationaryconditions)
ModelsofExtremeEventsforStationary
andNonStationaryConditions
Typicalmodelsforextremeevents(stationaryconditions)
Stationarydata
35
Lognormal
36
Gamma(Pearson)
LogPearsonTypeIII
37
Generalextremevalue
If
themodelis
calledLogistic
Generalizedlogistic
38
Exponentialif
Uniformif
GeneralizedPareto
Vilfredo Pareto
18481923
Kappa
Dr.PaulMielke
ProfessorEmeritus,CSU
39
Kappa
Mixturesandproducts
40
Example:Mixtureofthreedistrib.(Waylen &Caviedes,1986)
41
GeneralExtremeValue(GEV)Model
Assume block maxima coming from a series of
independent and identically distributed (i.i.d.)
observations:
Distribution of M
n
(assuming iids):
Then for large n, and some a
n
and b
n
:
42
G(z)belongstooneofthese:
EmilGumbel
18911966
Itseemsthattheriversknowthetheory. Itonlyremainsto
convincetheengineersofthevalidityofthisanalysis.
MauriceFrechet
18781973
Waloddi Weibull
18871979
43
44
TypeIII(<0)
TypeI(=0) TypeII(>0)
45
Nonstationarydata
Annualmaximumfloods
Builtout?
Nonstationarydata
MeanandmaximumsealevelsforsomesitesintheUSA
46
47
FinalRemarks
No-Estacionarias
Jose D. Salas
J. Obeysekera
LimaPer
ParameterEstimationandModelSelectionfor
StationaryGEVModels
48
Outline
Parameterestimationandquantile estimationforGEV
models
MOM,PWM,ML,Bayesian
ProfileLikelihood
Uncertaintyofparametersandquantiles
Approximatestandarderrors(deltamethod)
Betterapproximation(profilelikelihood)
Modelselection
Likelihoodratio(deviancestatistic)
AIC
Diagnosticplots
ParameterEstimation&ModelSelection
forStationaryGEVModels
Parameterestimationandquantile
estimationforGEVmodels
49
50
51
52
Example:Umpquaflooddata
53
PWMestimates
Likelihoodratiotest
MLestimatesand
standarderrors
CalculationsusingRsoftware
54
55
56
Standarderrorsofparameters
57
Variancecovariancematrix
Estimationofquantiles and
confidencelimits
58
59
Estimationofquantiles forT=5,10,25,50,100,250,500yearsandtheir
confidencelimits
60
61
*Keep
i
constantandmaximize
()withrespecttoallother
parameters,
I
*Repeatfordifferenti
p
(
i
)
Profilelikelihoodfori
62
Chisquaredistribution
i
Confidenceinterval
for
i
3.8415/2
fork=1
Profilelikelihoodconfidencelimitsfori
63
Confidence
Interval
Confidence
Interval
ShapeParameter 25yrquantile
Inferenceusingprofilelikelihood
218 390
267.6
0.11 0.24
0.0514
Deltamethodgives
:(0.24,0.73)
Deltamethodgives
Zq:(193.7341.57)
Resultsbasedonprofilelikelihood
64
65
ModelselectionbasedonDeviancestatisticandAIC
66
ResultsfortheUmpqua
FlooddatabasedonGEV
67
FinalRemarks
No-Estacionarias
Othermodelalternativesforextremeevents
Credit:Jana Sillmann
Jose D. Salas
J. Obeysekera (Obey)
LaboratorioNacional deHidrulica,UniversidadNacional deIngeniera,
LimaPer
68
Types of Extreme Value Data : Block
Maxima (review)
136
Blocks
1 2 3 4 5
Block length could be:
One Year
Season (wet and dry)
Example: Daily Precipitation
69
Models based on r-largest statistics
Daily Rainfall (Fort
Lauderdale, Florida)
5 largest values of
Rainfall every year
137
Models based on r-th largest statistics
H
n
(k)
= ktb lorgcst {X
1
, . , X
n
]
0
k
z = cxp -(z)
(z)
s
s!
k-1
s=0
where z = 1 +
z -p
o
-1{
M
n
r
= H
n
(1)
, , H
n
()
138
70
Modeling the r-th Largest Order
values
M
()
= z
1
, , z
i = 1,2, . . , m
Likelihood
I p, o, = _
_
cxp - 1 +
z
()
- p
o
-1{
x _o
-1 _
1
k=1
m
=1
+
z
()
- p
o
_
-
1
{
-1
_
Example: 10 largest sea-levels in Venice
Year r1 r2 r3 r4 r5 r6 r7 r8 r9 r10
1931 103 99 98 96 94 89 86 85 84 79
1932 78 78 74 73 73 72 71 70 70 69
1933 121 113 106 105 102 89 89 88 86 85
139
Example
r l o
1 -222.7 111.1(2.6) 17.2 (1.8) -0.077
(0.074)
5 -732.0 118.6(1.6) 13.7(0.8) -0.088
(0.033)
7 916.5 119.1(1.47) 13.25(0.7) -0.09
(0.029)
10 1139.1 120.5 (1.36) 12.8 (0.55) -0.113
(0.019)
r-largest order
statistics for
Venice sea levels
140
(*ignore trend for the moment)
71
141
Global Warming Protesters
Peaks Over Threshold (POT) Models
Motivation: Can we
use all extreme
values above a given
threshold?
142
Threshold Example: Daily Precipitation
72
Peaks Over Threshold - Theory
Stochastic model:
It can be shown that (Coles,2001):
then
143
y>0
Generalized Pareto Distribution
(GPD)
For large enough u and Y=X-u,
PDF
Case: ]=u
Threshold Selection is important
144
]=u
73
Can we get the equivalent GEV
parameters from GPD?
Neeu (,n,]) fiom ( , ])
] is the same for both GEV and GPD
Need to estimate n,
Need some new formulation to estimate the
third parameter
145
Poisson-GPD Model
The number of N exceedances of the
threshold level, u, has a Poisson distribution
with mean I
Assuming N 1, the excess values Y
1
,..,Y
n
are IIDs from GPD
For z > u, the probability that the annual
maximum of Y is,
146
74
Poisson-GPD model (cont.)
147
This will be identical to the form of the GEV if,
Knowing 2 (=# above u/total #) we can now
compute the equivalent GEV parameters
A more elegant approach: Point
Process Model (PP)
Interpretation of extreme value behavior
that unifies all models (GEV, GPD, Poisson-
GPD)
Leads to a likelihood (to be discussed) that
enables a more natural formulation of non-
stationarity in threshold excesses than from
GPD alone
148
75
Poisson Process (PP) Model (preview)
Domain, D=[0,T] x |u,)
X is viewed as two-
dimensional, non-
homogenous Poisson process
(t, xu )
149
0 T
u
A
t
1
t
2
x
The rate is,
The number of points in A, N(A) is a Poisson
Process with mean:
, also
Poisson Process (PP) Model (Cont.)
Advantages:
Direct estimation of GEV parameters
n is not a function of u (unlike in GPD)
Easier to implement in the non-stationary
setting (to be discussed later)
150
76
Summary of Models
Block Maxima
GEV
Gumbel
Others
Peaks Over Threshold (POT)
GPD
Poisson-GPD
PP
151
Generalize Pareto Distribution (GPD)
Maximum Likelihood Estimation
Recall:
Two parameters to estimate:
152
77
MLE for GPD - Spreadsheet
153
=-$K$5*LN($Q$5)-(1+1/$Q$6)*SUM(M7:M364)
File:Fortgpd.xlsx
Poisson Process (PP) Model (review)
Domain, D=[0,T] x |u,)
X is viewed as two-
dimensional, non-
homogenous Poisson process
(t, xu )
154
0 T
u
A
t
1
t
2
x
The rate is,
The number of points in A, N(A) is a Poisson
Process with mean:
, also
78
Poisson Process (PP) Model
Inference
Advantages:
Direct estimation of GEV parameters
n is not a function of u (unlike in GPD)
Easier to implement in the non-stationary
setting (to be discussed later)
Fort Collins Precipitation data again
155
Point Process (PP) Model - MLE
Likelihood function (Coles 2001, chapter 7)
I
i
is a small interval around x
i
, I is the full range
GEV parameters directly:
156
79
MLE for PP - Spreadsheet
157
=-$O$6*(1+$O$10*($O$7-
$O$8)/$O$9)^(-1/$O$10)-
$K$8*LN($O$9)-
(1+1/$O$10)*SUM(L10:L367)
File:Fortpp.xlsx
Dependent Sequences Declustering
158
Average Daily Temperature
U
U
Clusters for different r
r=1,2
r=3
80
Declustering Procedure
Use an empirical rule to define clusters of
exceedences
Identify maximum excess within each cluster
Assume cluster maxima are independent,
and that the conditional excess is distributed
as a generalized Pareto
Fit the GPD to the cluster maxima
159
Change to Return Level Formula
Rate at which clusters must be taken into
account. m observation return level:
0 is the extremal index
Let n
u
= number of exceedances above u
n
c
= number of clusters above u
160
81
Declustering of Fort Collins Rainfall
> load("Fort.RData")
> attach(a)
> dcRain1<-dclust(Rain,u=0.4,r=1)
> names(dcRain1)
[1] "xdat.dc" "ncluster" "clust"
>gpd<-gpd.fit(Rain,0.4,npy=214)
>gdc=gpd.fit(dcRain1$xdat.dc,0.4,npy=214)
gpd$nexc=358 dcRain1$ncluster=288
gpd$mle: 0.394(0.031) 0.164(0.058)
gdc$mle: 0.446(0.037) 0.130(0.060)
Quantile(100-yr)
gpd: 5.05 (3.84,7.89)
gdc: 4.79 (3.71,7.16)
161
Non-stationary GEV models
Parameters are function of time (or any
other variable-covariates)
Examples:
162
82
Non-stationary GEV: Examples
(cont.)
163
Change Point Models
Math Classic..
164
83
Final Remarks
165
No-Estacionarias
IntroductiontomodulesofthesoftwareRfor
modelingextremeevents
Jose D. Salas
J. Obeysekera
LimaPer
84
What is R
Most cost effective statistical package (why?)
R is a system for statistical computation and
graphics.
Also a programming language, high level
graphics, interfaces to other languages
The R language is a dialect of S which was
designed in the 1980s
167
What is R (Cont.)
an effective data handling and storage facility,
a suite of operators for calculations on arrays, in
particular matrices,
a large, coherent, integrated collection of intermediate
tools for data analysis,
graphical facilities for data analysis and display either
on-screen or on hardcopy, and
a well-developed, simple and effective programming
language which includes conditionals, loops, user-
defined recursive functions and input and output
facilities
168
85
Installation of R
http://www.r-project.org/
Select Comprehensive R Archive Network (CRAN) mirror
site
Download R for Windows (or for the particular operating
system)
Install base (install R for the first time)
Provides basic functions which let R function as a
language
Base package and contributed packages
For a complete list of functions, use library(help = "base")
169
Help for R
http://www.r-project.org/
http://www.rseek.org/
http://archive.org/details/TheRBook
170
86
Downloading R
171
Downloading R
172
87
Downloading R
173
Downloading R
174
88
Doing more..
Need to get contributed packages
http://cran.r-
project.org/web/packages/available_packages_by_name.ht
ml
> help("install.packages")
Eg. Installing package extRemes
> install.packages("extRemes")
Will ask for CRAN mirror site
>getwd() to find current working directory
>setwd(path) to set the working directory - or use
R.gui()
175
Data structures 1/6
Vector
A list of numbers, such as (1,2,3,4,5)
R: a<- c( 1, 2, 3, 4, 5) or a=c( 1, 2, 3, 4, 5)
Command c creates a vector that is assigned to object a
Factor
a = c(1,2,2,4,4,5)
A list of levels, either numeric or string
R: b<- as. f act or ( a)
Vector a is converted into a factor b:1 2 2 4 4 5
with Levels: 1 2 4 5
176
89
Data structures 2/6
Data frame
A table where columns can contain numeric and
string values
R: d<- dat a. f r ame( a, b)
Matrix
All columns must contain either numeric or string
values, but these can not be combined
R: e<- as. mat r i x( d)
Data frame d is converted into a matrix e
R: f <- as. dat a. f r ame( e)
Matrix e is converted into a dataframe f
177
Data structures 3/6
List
Contains a list of objects of possibly different types.
R: g<- as. l i st ( d)
Converts a data frame d into a list g
>g=gev. f i t ( x) g is now a list object
>names( g) check names of the objects in g
" t r ans" " model " " l i nk" " conv" " nl l h"
" dat a" " ml e" " cov" " se" " val s"
g$ml e gives the values of mle object:
301. 8081211 169. 9874853 0. 3544377
178
90
Data structures 4/6
Some command need to get, for example, a matrix, and do
not accept a data frame. Data frame would give an error
message.
To check the object type:
R: cl ass( d)
To check what fields there are in the object:
R: d
R: st r ( d)
To check the size of the table/matrix:
R: di m( d)
To check the length of a factor of vector:
R: l engt h( a)
179
Data structures 5/6
Some data frame related commands:
R: names( d)
Reports column names
R: r ow. names( d)
Reports row names
These can also be used for giving the names for the data
frame. For example:
R: r ow. names( d) <- c( " a", "b", " c" , " d", "e")
Letters from a to e are used as the row names for
data frame d
Note the quotes around the string values!
R: r ow. names( d)
180
91
Data structures 5/6
Naming objects:
Never use command names as object names!
If your unsure whether something is a command name,
type to the comman line first. If it gives an error message,
youre safe to use it.
Object names cant start with a number
Never use special characters, such as , , or in object
names.
Underscore (_) is not usable, use dot (.) instead:
Not acceptable: good_dat a
Better way: good. dat a
Object names are case sensitive, just like commands
181
Reading data 1/2
Command for reading in text files is:
r ead. t abl e( suomi . t xt , header =T, sep=\ t )
This examples has one command with three
arguments: file name (in quotes), header that
tells whether columns have titles, and sep
that tells that the file is tab-delimited.
182
92
Reading data 2/2
It is customary to save the data in an object in
R.
dat <- r ead. t abl e( suomi . t xt , header =T, sep=\ t )
Here, the data read from file suomi.txt is
saved in an object dat in R memory.
The name of the object is on the left and what
is assigned to the object is on the right.
Command r ead. t abl e( ) creates a data
frame.
183
Using data frames
Individual columns in the data frame can be accessed using
one of the following ways:
Use its name:
dat $year
dat is the data frame, and year is the header of one of its
columns. Dollar sign ($) is an opertaor that accesses that
column.
Split the data frame into variables, and use the names
directly:
at t ach( dat )
Use subscripts
184
93
Subscripts 1/2
Subscripts are given inside square brackets
after the objects name:
dat [ , 1]
Gets the first column from the object dat
dat [ , 1]
Gets the first row from the object dat
dat [ 1, 1]
Gets the first row and its first column from the
object dat
Note that dat is now an object, not a
command!
185
Subscripts 2/2
Subscripts can be used for, e.g., extracting a subset of the data:
dat [ whi ch( dat $year >1900) , ]
Now, this takes a bit of pondering to work out
First we have the object dat, and we are accessing a part of it,
because its name is followed by the square brackets
Then we have one command (whi ch) that makes an
evaluation whether the column year in the object dat has a
value higher than 1900.
Last the subscript ends with a comma, that tells us that we are
accessing rows.
So this command takes all the rows that have a year higher
1900 from the object dat that is a data frame.
186
94
Writing tables
To write a table:
wr i t e. t abl e( dat , dat . t xt , sep=\ t )
Here an object dat is written to a file called dat.txt.
This file should be tab-delimited (argument sep).
Wr i t e. csv( dat , dat . csv,
r ow. names=FALSE)
This file can be opened in Excel
187
Commonly used R commands in this
course
Command Whatitdoes
>R startR
>?plot Gethelpon'plot'command
>ls() Checktheobjectspace
>class(a) Checkthedatatypeofobjecta
>head(a) Whatthedataobjectlookslike(fewrowsprinted)
>names(a) namesofobjecta
>a=c(1,2,3,4,5) Createavectorobjectawithvalues1:5
>c=cbind(a,b) Combinevectorobjectsaandbintotheobjectc
>plot(x,y) plotyversusx
>is.na(a) Alogicalvectorshowingwhichvaluesinaaremissing
>sum(is.na(a)) Numberofmissingvaluesinobjecta
>b=a[20,] Remove20throwofaandplaceitinb
>a=read.csv("file.csv",header=T)
Readdatain"file.csv"whichhasaheaderandplaceintoobject
a
>write.csv(a,"file.csv") Writetheobjectaintocsv filecalled"file.csv"
>a=seq(5,100,5) Create asequenceofnumbers,5to100withincrementsof5
188
95
Creating a fancy plot
fout= myplot.png
#set the device for plotting
png(fout,pointsize=12,units="in",width=8,height=6,res=350)
par(mar=c(5.1,6.1,3.1,3.1))
par(font=2,font.lab=2,font.axis=2,cex=1.5,cex.axis=1.1,cex.lab=1.1)
#now plot
plot(yrs,x,pch=19,xlab="Year",ylab=Discharge, col="red")
title(myplot)
#set the device off
dev.off()
189
Reading and Manipulating Data
>dat=read.csv(sample.csv,header=TRUE)
#notice missing values denoted as NA
>class(dat)
>head(dat)
>attach(dat)
>amean=tapply(Rain,Year,mean,na.rm=T)
>momean = tapply(Rain,Month,sum,na.rm=T)
>sum(is.na(dat))
#setting missing values to zero
>dat[is.na(dat[,3]),3]=0
>plot(Rain)
>boxplot(Rain~Month)
190
96
Compiling & Running a function
myplot <- function(Year,Rain,outflag=0) {
if(outflag != 0 ) png("rainplt.png")
amean = tapply(Rain,Year,sum,na.rm=T)
yrs = as.numeric(names(amean))
plot(yrs,amean,pch=19,ylab="Rain",col="re
d",cex=1.5)
if(outflag !=0 ) dev.off()
return(list(yrs=yrs,amean=amean))
}
>v = myplot(Year,Rain)
191
> v$yrs
[1] 1895 1896 1897 1898 1899 1900 1901 1902 1903 1904 1905
> v$amean
1895 1896 1897 1898 1899 1900 1901 1902 1903 1904 1905
52.35 62.47 73.28 46.77 40.95 55.96 34.54 37.02 25.51 42.34 52.11
Quitting R
Use command q( ) or menu choise File->Exit.
R asks whether to save workspace image. If you
do, all the object currently in R memory are
written to a file .Rdata, and all command will be
written a file .Rhistory.
These can be loaded later, and you can continue
your work from where you left it.
Loading can be done after starting R using the
manu choises File->Load Workspace and File->
Load History.
192
97
193
A powerful IDE for R (www.rstudio.com)
Extremes Package Demo
library(extRemes)
extremes.gui()
194
98
Task ismev/extRemes evd evir fExtremes POT VGAM
Datageneration
gev gen.gev rgev rgev gevsim,gumbelsim rgev
gpd gen.gpd rgpd rgpd gpdSim rgpd rgpd
GEVfitting
MLE gev.fit fgev gev,gumbel gevFit,gumbelFit gev
ProfileLikelihood
gev.prof,gev.profxi,
gev.parameterCI profile.evd
Diagnostics gpd.giag
GPDfitting
Threshold mrl.plot mrlplot
meplot,
findthresh
mrplot,findThreshold,
mxfPlot mrlplot,lmomplot
gpd.fitrange tcplot tcplot
MLE gpd.fit fpot gpd gpdFit fitgpd gpd
ProfileLikelihood
gpd.prof,gpd.profxi,
gpd.parameterCI profile.evd gevrlevelPlot
gpd.pfrl,gpd.pfscale,
gpd.pfshape
Fisherbased
gpd.firl,gpd.fiscale,
gpd.fishape
bootstrapping
boot.matrix,
boot.sequence
PPfitting
MLE pp.fit fpot pot pointProcess fitpp
decluster declust, decluster deCluster
Covariates
GEV gev.fit fgev
GPD gpd.fit ?
PP pp.fit
195
Final Remarks
196
99
No-Estacionarias
Parameterestimationandmodelselectionfor
nonstationaryconditions
Jose D. Salas
J. Obeysekera
LimaPer
Climate change
198
100
Extreme Data Types
Stationary
Stochastic properties do not change with time
Non-Stationary
Stochastic properties vary with time
Trends (mean, frequency and intensity)
Cycles (diurnal, annual)
Because of co-variation with other variables (e.g.
Flood = f(El Nino phenomenon)
199
Non-stationary GEV models
Parameters are function of time (or any
other variable-covariates)
Examples:
200
101
Nonstationary GEV Models (cont.)
201
Change Point Model Seasons model
Parameter Estimation (Non-
stationary case)
Non-stationary GEV model:
Z
t
= GEV((t), o(t), (t))
Shape parameter is difficult to estimate and it is
unrealistic to model it as a smooth a function of
time
Log-likelihood:
202
0=(p(t) ,o(t),(t))
T
102
Parameter Estimation (Cont.)
Non-stationary Gumbel
Two parameters, but both functions of time
203
Parameter Estimation (Cont.)
Example 1
Parameters, 6 =(
0
,
1
, o, )
Example 2
Parameters, 6 =(
0
,
1
,
2 ,
o, )
Approximate Standard errors derived using the
same techniques that we discussed for stationary
case
204
103
205
Math Classic..
Example: Aberjona River,
Winchester, Massachusettes
Typical of
basins where
land use
changes have
caused
increasing
floods
Nonstationary
mean,
variability?
206
104
R and Excel Solutions
a=read.csv("Aberjona_1102
500.csv",header=T)
cov=yrs-mean(yrs)
cov=as.matrix(cov,ncol=1)
gev.fit(x,ydat=cov,mul=1)
Output:
$nllh 410.931
$mle 319.3587323 2.8814896
163.3669736 0.3040809
$se 25.3638274 1.0190117
21.1793573 0.1337115
207
Confidence Limits for parameters:
Approximate Normality of MLEs
For large n:
208
Expected
Information
Matrix
105
209
Response variable: Discharge
[1] "cov.selected = Cov"
Convergence successfull![1] "Convergence
successfull!"
[1] "Maximum Likelihood Estimates: MLE
Stand. Err.
MU: (identity) 319.35873 25.36373
Cov: (identity) 2.88149 1.01901
SIGMA: (identity) 163.36697 21.17922
Xi: (identity) 0.30408 0.13371
[1] "Negative log-likelihood: 410.931027606394"
Parameter covariance:
[,1] [,2] [,3] [,4]
[1,] 643.318930 7.19927261 358.945815 -
1.33843641
[2,] 7.199273 1.03838421 4.686030 -0.05198826
[3,] 358.945815 4.68603046 448.559343 -
0.37964699
[4,] -1.338436 -0.05198826 -0.379647 0.01787876
extRemespackage
Confidence Limits using
Bootstrapping
Recall that
Z ~ GEV(p(t),o(t),)
Then e = [Z-p(t)]o(t)
is GEV~(0,1,)
The basic approach
is to bootstrap the
residuals, e and fit
multiple models
Fit Z ~ GEV(p(t),o(t),)
Compute residuals, e
For k = 1, Nsamples
Resample residuals with
replacement = enew
Recreate z = Znew using (p(t),
o(t),
Fit Znew ~ GEV(p(t),o(t),)
Compute Z
T
for the desired T
Compute 5
th
and 95
th
interval
using Nsamples of Z
T
210
106
Model Selection (Likelihood Ratio
Test)
Let a model, M
1
=f(
(1)
,
(2)
) and
M
0
=f(
(1)
=0,
(2)
). M
0
is a subset of M
1.
The
question is, Is model M
1
any better than M
0
?
Deviance Statistic:
Reject M
0
in favor of M
1
if
where c
is the (1-) quantile of the

2
k
distribution dimension of
(1)
Alternatively:
211
Model Selection based on AIC & BIC

Akaikes Information Criteria (AIC)
AIC(k) = -2llh (k)+2 k, k=number of parameters
Select the model which has the minimum AIC
Bayesian Information Criterion (BIC)
BIC(k) = -2llh(k) + k ln T
k = number of parameter
T = sample size
Select the model which has the minimum BIC
212
107
Model Diagnostics (Non-stationary)
case
Z
t
~0EI(p t , o t ,
`
t )
Standardized variable
Z
t
=
1
`
(t)
log 1 +
`
(t)
Z
t
- p(t)
o(t)
Note: P Z
t
z = cxp -c
-z
Z
t
follows a Gumbel distribution
213
Model Diagnostics (Cont.)
Propbability plot:
|
m+1
, exp |-exp -z

] i = 1,2, , m
Quantile plot:
(z

, -log -log

m+1
; i = 1,2, , m
214
108
Modeling Non-stationarity -
Summary
Fit various modeling using MLE
Select an appropriate model using AIC, BIC
and Likelihood Ratio Test as criteria
Compute Return Level, and Risk for a given
Return Period
Compute confidence intervals for
parameters and return level
215
Example: Aberjona River,
Winchester, Massachusettes
Typical of
basins where
land use
changes have
caused
increasing
floods
Nonstationary
mean,
variability?
216
109
Aberjona River First Stationary
Case
>dat=read.csv("Aberjona_1102500.csv"
,header=TRUE)
>head(dat) (yrs x)
>attach(dat)
>gum<- gum.fit(x) #fit Gumbel
$gum$mle: {337.4(28.1) 209.3(23.0)}
$gum$nllh= 419.7192
> gum.diag(gum)
>AICgum=2*(gum$nllh+2) =843.4384
>gev<- gev.fit(x) #fit GEV
gev$mle {301.8(25.5) 170.0(22.3)
0.354(0.13)}
gev$nllh=414.9693
>gev.diag(gev)
217
Modeling Non-staionarity using
covariates
g = gev.fit(x, ydat = cov, mul=c(1,..),
sigl=c(1,..), shl=c(1,..), siglink = exp,
muinit=c(),)
Covariate matrix, cov
Year SOI AMO
1945 . .
1946 . .
R-Demo
218
110
Aberjona River - Model Checking &
Non-stationary Modeling
>D= -2*(gev$nllh-gum$nllh)
> chi=qchisq(0.95,1)
> p=pchisq(D,1,lower.tail=FALSE)
AICgev=2*(gev$nllh+3)
D = 9.499 ; chi =3.84; p=0.002;
AICgev= 835.9386
D > Chi, and AICgev < AICgum
GEV is chosen over Gumbel
#Probability Weighted Moments
> lmr<-lmom.ub(x)
> pwmgev<-pargev(lmr)
#nonstationarity in location parameter
>cov <- as.matrix(yrs - mean(yrs),ncol=1)
>gevmu <- gev.fit(x,ydat=cov,mul=1)
gevmu$nllh=410.931
gevmu$mle {319.4(25.4) 2.88(1.02)
163.4(21.2) 0.304 (0.134)}
> D= -2*(gevmu$nllh-gev$nllh)
> chi=qchisq(0.95,1)
> p=pchisq(D,1,lower.tail=FALSE)
> AICgevmu=2*(gevmu$nllh+4)
D=8.076 chi=3.841459 p=0.004
AICgevmu=829.8621
> Select GEV-MU(t) over GEV!
219
Aberjona River Model Comparison
Index
Model
Name Parameters nllh Comparison D chi p AIC
1 gum , o 419.72 843.4
2 gummu (t), o 414.01 2 vs. 1 11.42 3.84 0.001 834.0
3 gumsc , o(t) 412.21 3 vs. 2 15.01 3.84 0.000 830.4
4 gummusc(t), o(t) 406.23 4 vs. 2 15.56 3.84 0.000 820.5
5 gev , o, 414.97 5 vs. 1 9.50 3.84 0.002 835.9
6 gevmu (t), o, 410.93 6 vs. 5 8.08 3.84 0.004 829.9
7 gevsc , o (t), 411.99 7 vs. 6 5.96 3.84 0.015 832.0
8 gevmusc (t), o(t), 405.79 8 vs. 7 10.29 3.84 0.001 821.6
220
111
Non-stationary Model
GEV{ (t)n (t), ]]
221
Confidence Limits using
Bootstrapping
Recall that
Z ~ GEV(p(t),o(t),)
Then e = [Z-p(t)]o(t)
is GEV~(0,1,)
The basic approach
is to bootstrap the
residuals, e and fit
multiple models
Fit Z ~ GEV(p(t),o(t),)
Compute residuals, e
For k = 1, Nsamples
Resample residuals with
replacement = enew
Recreate z = Znew using (p(t),
o(t),
Fit Znew ~ GEV(p(t),o(t),)
Compute Z
T
for the desired T
Compute 5
th
and 95
th
interval
using Nsamples of Z
T
222
112
Snippets of code
#initial guess R0
library(BB)
p0 = 1/T0
yp0 = -log(1-p0)
R0 = mu0 - (sig/xi)*(1-yp0^(-xi))
#compute Return level for the desired
value of T finding root using dfsane
d =
dfsane(par=R0,fn=myfun,T=T,Nmax
=Nmax,mut=mut,sig=sig,xi=xi,quiet=
TRUE,
control=list(trace=FALSE))
Rm = d$par
conv <- d$convergence
myfun <- function(x,T,Nmax,mut,sig,xi) {
pt = 1- exp(-((1+(xi/sig)*(x-mut))^(-1/xi)))
pt[is.nan(pt)]<-1
ptc = cumprod(1-pt)
px =matrix(1,Nmax,1)
for(i in 1:Nmax) {
if(i == 1) px[i] = pt[1] else px[i] = ptc[i-
1]*pt[i]
}
t=1:Nmax
ex = sum(t*px)
y <- T - ex
#print(ex)
y
}
223
Rest of the code (for boostrapping)
for(k in 1:Nsample) {
enew <- sample(e,replace=TRUE)
znew <- enew * g$vals[,2]+g$vals[,1]
gnew<-gev.fit(znew,ydat=cov,mul=1)
muslope = gnew$mle[2]
mu0=gnew$vals[yrs==lastyr,1]+(cons
yr-lastyr)*muslope
sig = gnew$mle[3]
xi = gnew$mle[4]
mut = mu0 + (t-1)*muslope
R0new = mu0 - (sig/xi)*(1-yp0^(-xi))
.. Continued to the right
d =
dfsane(par=R0new,fn=myfun,T=T0,N
max=Nmax,mut=mut,sig=sig,xi=xi,qu
iet=TRUE,
control=list(trace=FALSE))
Rmnew = d$par
cvals[k] <- d$convergence
Rms[k]<-Rmnew
.
}
#5
th
and 95
th
confidence limits?
r <- quantile(Rms,probs=c(0.05,0.95))
224
113
Results - Parameters
225
Results Return Level (Design
Quantile)
226
114
Trend with a change point
Little Sugar Creek at Archdale
Drive in Charlotte, North
Carolina (Gumbel Distribtion)
>cov = yrs
>cov[yrs <= 1945] <- 1945
cov:
1945 1945 1945 1945 1945 1945 1945 1945 1945
1945 1945 1945 1945 1945 1945 1945 1945
1945 1945 1945 1945 1945 1947 1948 1949
1950 1951 1952 1953 1954 1955 1956 1957
1958 1959 1960 1961 1962 1963 1964 1965
1966 1967 1968 1969 1970 1971 1972 1973
1974 1975
>cov=matrix(cov-mean(cov),ncol=1)
>gum=gum.fit(x)
>gumu=gum.fit(x,ydat=cov,mul=1)
227
Two Change points
Mercer Creek,
Washington State
>yr1 = 1970
>yr2 = 1986
>cov=yrs
>cov[yrs <= yr1] <- yr1
>cov[yrs >= yr2] <- yr2
cov: 1970 1970 1970 1970 1970 1970
1970 1970 1970 1970 1970 1971
1972 1973 1975 1977 1978 1980
1981 1982 1983 1985 1986 1986
1986 1986 1986 1986 1986 1986
1986 1986 1986 1986 1986 1986
1986 1986 1986 1986
>gev = gev.fit(x)
>gevmu=gev.fit(x,ydat,mul=1)
228
115
Sea Level Rise Modeling extremes
Key West tide gauge
>a=read.csv("KeyWest.csv",
header=TRUE)
> head(a)
Year Mean Max
1913 1495.4 2042
1914 1481.2 2070
>attach(a)
>gevmu=gev.fit(Max,mul=1,
ydat=as.matrix(Year-mean(Year)))
>g=lm(Mean~Year) #regression
Non-linear change possible in the
future
Relation shop mean and max! Can
we use that property? later
229
Offset=552 mm
Sea Level Rise Projections (in US)
230
116
Modeling with Covariates
data(fremantle)
Year SeaLevel SOI
1897 1.58 -0.67
1898 1.71 0.57
attach(fremantle)
ydat=cbind(Year-mean(Year),SOI-
mean(SOI))
gev=gev.fit(SeaLevel)
gevmu=gev.fit(SeaLevel,ydat=ydat
,mul=1)
gevmusoi =gev.fit(SeaLevel,ydat=ydat,
mul=c(1,2))
So which model is better?
D<- -2*(gevmu$nllh-gev$nllh)
chi<-qchisq(0.95,1)
p<-pchisq(D,1,lower.tail=FALSE)
AIC=2*(gevmu$nllh+4)
231
Fremental Extreme Sea Levels
Model Comparison
Index
Model
Name Parameters nllh Compare D chi p AIC
1 gev , o, -43.567 NA NA NA NA -81.13
2 gevmu (t), o, -49.914 2 vs. 1 12.69 3.84 0.0004 -91.83
3 gevmusoi (t,soi), o , -53.898 3 vs. 2 7.97 3.84 0.0047 -97.79
3 gevmusoi (t,soi), o, -53.898 3 vs. 1 20.66 5.99 ~0 -97.79
gevmu is better than gev
Adding SOI appears to improve the model
further (compare 3 vs. 2)
232
117
Atlantic Multi-Decadal Oscillation
(AMO) index as a covariate
St. Johns River, Florida. Floods
influenced by the phase of AMO
>dat=read.csv(Stjohn.csv,header=T)
>attach(dat)
Year StjQ
1944 3300
1945 9230
>ydat=rep(1,1,length(Year))
>ydat[yrs <= 1969] = -1
>ydat<-matrix(ydat,ncol=1)
>gev=gev.fit(StjQ)
>gevmu=gev.fit(StjQ,ydat=ydat,mul=1)
gev$nllh=400.7574
gevmu$nllh=396.4719
D=8.571; Chi=3.841; p=0.003
AICgev=403.8; AICgevmu=400.5
233
Math Classic..
234
118
Final Remarks
235
No-Estacionarias
Jose D. Salas
J. Obeysekera
LimaPer
ReturnPeriodandRiskforStationaryand
NonstationaryExtremeConditions
119
120
ReturnPeriodandRiskUnder
StationaryConditions
ReturnPeriodUnderStationary
Conditions
121
Conditions
Designflood andconstantvaluesofexceeding(p)and
nonexceeding(q =1p)probabilitiesthroughoutyears1tot,
i.e.stationarycondition(SalasandObeysekera,2013)
DistributionoftheWaitingTime
forStationaryConditions
122
(alsoknownasFirstArrivalTime)
Floodoccurrence
Floodoccurrence
Conditions
123
Conditions
Conditions
124
HydrologicRiskUnderStationary
Conditions
HydrologicRiskUnderStationary
Conditions
125
Fig.5Exampleofnonstationaryannualflooddata
Developmentstodealwith
NonStationarity
126
127
forNonStationaryConditions
forNonStationaryConditions
128
ReturnPeriodfor
ReturnPeriodfor
129
HydrologicRiskfor
HydrologicRiskfor
130
HypotheticalExampleforDetermining
T andR forNonstationaryConditions
FinalRemarks
131
No-Estacionarias
Jose D. Salas
J. Obeysekera
LimaPer
ExamplesofAnalysisofExtremeEventsfor
NonstationaryConditions
ExamplesofAnalysisofExtremeEvents
forNonstationaryConditions
Outline
Examplesbasedontheexponentialdistribution
Examplesofincreasingfloods
Examplesofincreasinganddecreasingsealevels
Examplesofshiftingfloodregimes
132
ExampleUsinganExponential
Distribution
ExampleUsinganExponential
Distribution
133
134
ExamplesUsingtheGEVfor
IncreasingFloodEvents
135
136
137
Aberjona RiverBasin,Winchester,Massachusetts(Vogeletal,2011)
138
Aberjona RiverBasin,Winchester,Massachusetts
Aberjona RiverBasin,Winchester,Massachusetts
139
140
141
142
143
IncreasingandDecreasingSeaLevels
(a) (b)
(a)
(b)
144
(a) (b)
145
FurtherRemarks

Course Peru Salas Obey All

Загружено:

Сведения о документе

Исходное описание:

Авторское право

Доступные форматы

Поделиться этим документом

Поделиться или встроить документ

Параметры публикации

Этот документ был вам полезен?

Это неприемлемый материал?

Авторское право:

Доступные форматы

Course Peru Salas Obey All

Загружено:

Авторское право:

Доступные форматы

1

Riesgo Hidrolgico de Eventos

is the (1-) quantile of the

Model Selection based on AIC & BIC

Вам также может понравиться