|

 


 

  

Partini Pudjiastuti, Sudigdo Sastroasmoro

Child Health Department
Faculty of Medicine University of Indonesia
Population & Sample
Sudigdo Sastroasmoro
sudigdo_s@yahoo.com
Population is a large group of study subjects
(human, animals, tissues, blood specimens, medical
records, etc) with defined characteristics
[´Population is a group of study subjects defined
by the researcher as populationµ]
Sample is a subset of population which will be
directly investigated. Sample should be (or
assumed to be) representative to the population;
otherwise all statistical analyses will be invalid
All investigations are always performed in the
sample, and the results will be applied to the
population
Avoid using ambiguous terms

Sample population
Sampled population
Populasi sampel
Êarget population = domain = population in which the
results of the study will be applied. Usually character-
ized by demographic & clinical characteristics; e.g.
normal infants, teens with epilepsy, post-menopausal
women with osteoporosis.
Accessible population = subset of target population
which can be accessed by the investigator. Frame: time
& place. Example: teens with epilepsy in RSCM, 2000-
2005; women with osteoporosis, 2002 RSGS
Intended sample = subjects who meet eligibility criteria
and selected to be included in the study
Actual study subjects = subjects who actually
completed the participation in the study
   

  

  

!"#  

  


  



  

 





% 
$ 
$%
  $%


     

   
 
 External validity II:
Does AP represent ÊP?

 
 

Accessible
population

[External validity I:
Does IS represent AP?}


 
% $ 

[Internal validity: does

ASS represent IS?]
 $ 
  


& 

 


] Simple random sampling (r. table, computer)
] Stratified random sampling
] Systematic sampling
] Cluster sampling
] Others: two stage cluster sampling, etc

[
  

 


] Consecutive sampling
] Convenience sampling
] Judgmental sampling
  
All statistical analyses (inferences) are
based on random sampling
Whether or not a sample is representative
to the population depends on whether or not
it resembles the results if it were done by
random sampling
 
AN
—





 vs. 


 
'egligible clinical difference may be
statistically very significant if the number of
subjects >>>. e.g., difference in reduction of
cholesterol level of 3 mg/dl, n1=n2 = 10,000;
p = 0.00002
Large clinical difference may be statistically
non-significant if the no of subjects <<<, e.g.
30% difference in cure rate, if n1 = n2 = 10,
p = 0.74
|


 +







Cholesterol level, mg/dl

Standard x = 300  ''(

treatment mg/dl
n=10000
Ä Clinical
n=10000
'ew x = 300  ')*
treatment mg/dl

t= df = 9998 p = 0.00002
$


 Clinical importance vs. statistical significance

Cured Died

Standard Rx 0 10 (100%)

'ew Rx 3 7 (70%)

Absolute risk reduction = 30% |



Fischer exact test: p = 0.211 $



Correlation between abdominal circumference
and total cholesterol level in middle-aged men
' = 200
R = 0.22, p = 0.031
Conclusion: Êhere was a significant correlation
between abdominal circumference and total
cholesterol level in the subjects studied.
Measuring abdominal circumference may
predict the cholesterol level in middle-aged
healthy men.
 How important is important?
Êwo percent mortality reduction is probably not
important in your clinic
In a community prevention, a simple measure
that reduce 2% severe morbidity is probably
important.
² Low dose aspirin reduces 2% cardiac events
in 5 years (without aspirin 400 cardiac
events per 10,000, with aspirin 200 cardiac
events)
Requires judgment
 !  

 ,

Can the results of the study (in sample) be

applied in the accessible or target
population?
Hypothesis testing & confidence interval
 —

&  

An observed value drawn from the sample is

called a statistic (V
statistics, the science)
Êhe corresponding value in population is called
a parameter
We measure, analyze, etc statistics and
translate them as parameters
 þ  


-
Proportion OR
Percentage RR
Mean Sensitivity
Median Specificity
Mode Kappa
Difference in LR
proportion/mean ''Ê
ere are 2 ways in inferring statistic into
parameter:

Hypothesis testing ´ p value

Estimation: ´ confidence interval (CI)

P Value & CI tell the same concept in different

ways

value

Determines the probability that the

observed results are caused solely by
chance (probability to obtain the
observed results if Ho were true)
Group Success Failure Êotal

C 30 (60%) 20 (40%) 50

E 40 (80%) 10 (20%) 50

X2= ; df = 1; p = 0.0432
     

|

 





 



   

 

If drugs E and C were equally effective, we still can have
the above result (difference of success rate of 20%)
but the probability is small (4.32%)

If drugs E and C were equally effective, the probability

that the result is merely caused by chance is 4.32%

   V


 
 V




V


V V

Similar interpretation applies to ALL

hypothesis testing: t-test, Anova, non-
parametric tests, Pearson correlation,
multivariate tests, etc:

If null-hypothesis null were true, the

probability of obtaining the result was
««. (example 0,02 or 2%, etc)
 Confidence Intervals

Estimate the range of values (parameter)

in the population using a statistic in the
sample (as point estimate)
 If the observed
@ result in the
sample is X, what
is the figure in
— the population?

 
 |

A statistic (point estimate)

 #  | -

CI 90% corresponds to p 0.10

CI 95% corresponds to p 0.05
CI 99% corresponds to p 0.01

'ote:
p value ´ only for analytical studies
CI ´ for descriptive and analytical studies
 How to calculate CI

General Formula:

CI = p ÷ Z x SE

p = point of estimate, a value drawn from

sample (a statistic)
Z = standard normal deviate for , if =
0.05 ´ Z = 1.96 (~ 95% CI)
 Example 1

100 FKUI students ´ 60 females (p=0.6)

What is the proportion of females in Indonesian
FK students? (assuming FKUI represents FK in
Indonesia)
 þ


þ  

       
      


    
   

      
    
 þ  '-| 
 

)(( 
 [
 .(((—! /((

  
012| ,
012|   ÷ )
03 —þ#
 ‰ 

 
 ‰     

‰  
‰
 

Ú
 

‰  Ú  

 Examples 3: CI of difference between
proportions (p1-p2)

50 patients with drug A, 30 cured (p1=0.6)

50 patients with drug B, 40 cured (p2=0.8)

 
 4  ‰ 
 4   

 4 
 4

 4  ‰
4 4

  
 
 
4 

‰ ‰ ‰ 

  
 
 
 4  ‰ 
4

 
4 
  ‰ 
 
4
 þ
  !"#$%&

'

Mean systolic BP:

50 smokers = 146.4 (SD 18.5) mmHg
50 non-smokers = 140.4 (SD 16.8) mmHg
x1-x2 = 6.0 mmHg
95% CI(x1-x2) = (x1-x2) ÷ 1.96 x SE (x1-x2)
SE(x1-x2) = S x V(1/n1 + 1/n2)
þ
  !"#$%&

'

& &
 ± '  & ± ' &
'
  & ± &
â

)  &
' 
)
 
þ ± &     ( (
 
 
 ( ( ±   (
 "  
|

Relative risk (RR)

Odds ratio (OR)
Sensitivity, specificity (Se, Sp)
Likelihood ratio (LR)
Relative risk reduction (RRR)
'umber needed to treat (''Ê)
 —



95%CI: 1.5 to 4.5
95%CI: -2.5 to 4.3
95%CI: -12 to -6

'ot recommended: 3 +1.5

'ot recommended: -9+ -3
In contrast to CI for proportion, mean, diff.
between proportions/means, where the values
of CI are symmetrical around point estimate,
CI·s for RR, OR, LR, ''Ê are asymmetrical
because the calculations involve logarithm
 þ  

RR = 5.6 (95% CI 1.2 ; 23.7)

OR = 12.8 (95% CI 3.6 ; 44,2)
''Ê = 12 (95% CI 9 ; 26)
If p value <0.05, then 95% CI:
exclude 0 (for difference), because if A=B
then A-B = 0 à p>0.05
exclude 1 (for ratio), because if A=B then A/B
= 1, à p>0.05

For small number of subjects, computer

calculated CI may not meet this rule due to
correction for continuity automatically done by
the computer
 | 
   4

In every study sample should (assumed to) be

representative to the population. Otherwise all
statistical calculations are not valid
p values (hypothesis testing) gives you the
probability that the result in the sample is merely
caused by chance, it does not give the magnitude
and direction of the difference
Confidence interval (estimation) indicates estimate of
value in the population given one result in the
sample, it gives the magnitude and direction of the
difference
 | 
   4
p value alone tends to equate statistical
significance and clinical importance
CI avoids this confusion because it provides
estimate of clinical values and exclude
statistical significance
š whenever applicable, supply CI especially for
the main results of study
š in critical appraisal of study results, focus
should be on CI rather than on p value.
 1
Êhe ultimate goal of clinical research is the use
of evidence in source population
 2
Êhe best non-probabiity sampling is consecuitve
sampling
 3
P value refers to the probability of getting the
observed result when the Ho were false
 4
Êhe mean difference of 2 measurements is 20
mmHg, with 95% CI 15 to 25 mmHg. Êhe p
value should be ´statistically significantµ
 5
Confidence intervals give more information
than p value
 6
It is possible to have a study with good internal
validity but poor external validity
 7
If the odds ratio is 5, then the 95% CI may
have values from 3 to 11
 8
It is possible to have a significant difference
even when the clinical difference is not
important, but clinically important difference
always statistically significant
 9
Appropriate sampling method is mandatory to
ensure generalization
 10
Clinical epidemiology may include animal
studies
 11
Êhe more wide the confidence interval, the
more precise the result of any study
 12
Assessment of clinical importance requires
judgment
 13
Êhe confidence interval of any measure must
include the point estimate
 14
Selection of source population usually based on
practical reasons
 15
Diagnostic test, therapy, etiology, harm, are
examples of basic research
 Critical appraisal
(making reading more worthwhile)


|

 
,

1. Critical appraisal = quality assessment
2. «.process of weighing up evidence to see how
useful it is in decision making
3. .«a process of assessing the validity, importance,
and usefulness of evidence
4. Critical appraisal is about considering, evaluating
and interpreting information in a systematic and
objective way
 Critically appraise what you read

Separating the wheat from the chaff

Êime is limited ² you should aim to quickly stop
reading the dross
Others contain useful information mixed with
rubbish
Simple checklists enable the useful information
to be identified
 Critical appraisal ² Critical thinking

Appraising (evaluating/reviewing) the

available evidence to construct clinical
reasoning strategies and to make decisions
Finding strengths and limitations of written
¶evidence·
You need to decide what evidence to pay
attention to (what is ´worthyµ of your attention)
versus what to ignore
 Why critically appraise?

Supports sound decision making based on best

available evidence
Helps us determine (three R·s):
‡ How
  a piece of research is 5
,
‡ What the  are telling us   ,
‡ How + it is to our patient 
,
Why do we need evidence?

Resources should be allocated to things that

are EFFECÊIVE
Êhe only way of judging effectiveness is
EVIDE'CE
´In God we trust ² all others need evidenceµ
 Sources of Evidence

Primary sources
² Based on experiments and published
research
Secondary sources
² Systematic reviews
² Clinical guidelines
² Journals of secondary publication e.g.
Evidence Based Medicine
´5Sµ Pyramid of Evidence Resources
 Levels of evidence

1. Systematic reviews of RCÊs/high quality RCÊs

2. Systematic reviews of cohort studies, lower
quality RCÊs, outcomes research
3. Systematic reviews of case controls, case
control studies
4. Case Series
5. Expert opinion
—


V

   V
 
V
V

 Êypes of Evidence - Question Êypes

6
 [þ+

! 
 +
- 6

+-
treatment, prevention Systematic Review of RCÊs or RCÊ
!  þ
  6

+- Observational
Study - Cohort or Case Control
&  
 6

+- Observational
Study - Cohort, Case Control
!
 
   6

+-
Comparison to Gold Standard
þ 
 6

+-
Cost-effectiveness Study
#
 6

+-case study,
 Key quality parameters

Validity

Reliability

Importance
Validity: internal and external

Internal - Is the study designed in such a way that we

can trust the findings?
External - Is the study designed in such a way that I
can generalize the findings?
Studies with good internal validity may not have good
external validity if the study subjects do not represent
population
With poor internal validity, question about external
validity is not relevant
 Reliability

If the study was conducted again, would the

results be the same?
Usually interpreted as the accuracy of
measurement.
 Importance

What was the effect size or magnitude of

effect? (Would the evidence change your
practice?)
Clinical vs. statistical significance.
 Êools for Critical Appraisal

EBM ´simplifiedµ approach:

Are the results valid? â

What are the results?
Will the results help me in patient care?

 þ+
 


1
Formulate question

Evaluate
Efficiently track
performance
down best
available
evidence Implement
changes in clinical
Critically review the practice
validity and usefulness
of the evidence
 Check list for medical literature
(completeness)
1. Êitle
2. Authors
3. Abstract: structured? Informative? Abbreviation?
4. Introduction: length? Relevant references? Êarget population?
5. Methods:
‡ Design
‡ Eligibility (inclusion and exclusion) criteria
‡ Sample size, sampling method
‡ Randomization: technique, concealment
‡ Intervention: masking?
‡ Measurement: blinding? - Primary & secondary outcome
‡ Definitions
‡ Analysis
|4

 

   

6. Results
‡ Baseline characteristics
‡ Main outcome
‡ Secondary outcome
7. Discussion
‡ General
‡ Strength and weakness
‡ Conclusions
8. References
‡ Vancouver style
‡ Constant
9. Acknowledgments
10. Ethics approval
11. Conflict of interest
 What to assess?
(in study of cause-effect relationship)

A. General description
² Êype of design
² Êarget population, source population,
sample
² Sampling method
² Dependent and independent variables
² Main results?
 What to assess?
(in study of cause-effect relationship)

B. Internal validity, non-causal relationship

² Influence of bias
² Influence of chance
² Influence of confounders
 Bias
What is a bias? A process that tends to produce
results that depart systematically from the true
values existing in the study population

Êypes of bias
1. Sample (subject selection) biases, which may result in the
subjects in the sample being unrepresentative of the
population which you are interested in
2. Measurement (detection) biases, which include issues
related to how the outcome of interest was measured
3. Intervention (performance) biases, which involve how the
treatment itself was carried out.
 What to assess?
(in study of cause-effect relationship)
C. Internal validity, causal relationship
Êemporality (cause precedes effect)
Strength of association (large difference, RR, OR, etc) or
small p value or narrow confidence interval
Biological gradient (dose dependence)
Consistency among studies (diff. populations/designs)
Specificity (certain factor results in certain effect)
Coherence (does not conflict with current knowledge)
Biological plausibility: can be explained with current
knowledge (at least in part)
 What to assess?
(in study of cause-effect relationship)

D. External validity
‡ Applicable to study subjects
‡ Applicable to source population
‡ Applicable to target population
 11 items, each with 3 sections

1. Can you find this information in the

paper?
2. Is there any problem?
3. Does this problem threaten the validity?
 11 items
1. What is the research question?
2. What is the study type?
3. What are the outcome factors and how are they measured?
4. What are the study factors and how are they measured?
5. What important confounders are considered?
6. What are the sampling frame and sampling method?
7. In an exp., how were the subjects assigned to groups? In a
longitudinal study, how many reached final follow-up? In a
case control study, are the controls appropriate? (etc)
8. Are statistical tests considered?
9. Are the results clinically/socially significant (important)?
10. Is the study ethical?
11. What conclusions did the authors reach?
1. What is the research question?


² Is it concerned with the impact of an
intervention, causality or determining the
magnitude of a health problem?

 



 
!
² Is it a well stated research
question/hypothesis?
 2. What is the study type?
!
² Is the study type appropriate to the research
question?

 



 
!
² If not, how useful are the results produced
by this type of study?
3. What are the outcome factors and how
are they measured?
!
² a) are all relevant outcomes assessed
² b) is there measurement error?
 



 
!
² a) how important are omitted outcomes
² b) is measurement error an important source
of bias?
4. What are the study factors and how are
the measured?
!
² Is there measurement error?

 



 
!
² Is measurement error an important source of
bias?
 5. What important potential
confounders are considered?

!
² Are potential confounders examined and
controlled for?
 



 
!
² Is confounding an important source of bias?
6. What are the sampling frame and
sampling method?

!
² Is there selection bias?

 



 
!
² Does this threaten the   +

 of
the study?
 7. Questions of internal validity
!
² Experimental: how were the subjects
assigned to groups?
² Longitudinal study, how many reached
follow-up?
² Case control study, are the controls
appropriate?
‡ 'ote: other issues of relevance to internal
validity are considered under the other
headings in this critical appraisal system. You
can add your own questions, and also design
your own questions for other study types such as
cross sectional studies and systematic reviews
 



 
!
8. Are statistical tests considered?
!
² Were the tests appropriate for the data?
² Are confidence intervals given?
² Is the power given if a null result?
² In a trial, are results presented as absolute
risk reduction as well as relative risk
reduction?

 



 
!
² If not, how useful are the results?
9. Are the results clinically/socially
significant?
!
² Was the sample size adequate to detect a
clinically/socially significant result?
² Are the results presented in a way to help in
health policy decisions?

 



 
!
² Is the study useful?
10. Are ethical issues considered?

!
² Does the paper indicate ethics approval?
² Can you identify potential ethical issues?

 



 

² Are the results or their application
compromised?
11. What conclusions did the authors reach
about the study question?

!
² Do the results apply to the population in
which you are interested?
 



 

² Will you use the results of the study?
 Appraisal Êools
Êools from the Critical Appraisal Skills Programme
(CASP)
² Systematic Reviews
² Randomised Controlled Êrials
² Qualitative Research Studies
² Cohort Studies
² Case-Control Studies
² Diagnostic Êest Studies
² Economic Evaluation Studies
Available at:
http://www.phru.nhs.uk/casp/critical_apprai
sal_tools.htm
 Study Designs Recap
þ


+
 Randomised Controlled Êrial
 

Ä
47 8 Cohort Study

&  


!
 
 Survey using gold standard


9[

 Qualitative (Interviews,
Observations, etc)
 Critical appraisal

- Validity š Methods
- Importance š Results
- Applicability š Discussion
Ê