Loading...

PPT – Significance testing and confidence intervals PowerPoint presentation | free to download - id: 528789-NjkwM

The Adobe Flash plugin is needed to view this content

View by Category

Presentations

Products
Sold on our sister site CrystalGraphics.com

About This Presentation

Write a Comment

User Comments (0)

Transcript and Presenter's Notes

Significance testingand confidence intervals

- Ágnes Hajdu
- EPIET Introductory course
- 3.10.2011

The idea of statistical inference

Generalisation to the population

Conclusions based on the sample

Population

Hypotheses

Sample

Inferential statistics

- Uses patterns in the sample data to draw

inferences about the population represented,

accounting for randomness. - Two basic approaches
- Hypothesis testing
- Estimation
- Common goal conclude on the effect of an

independent variable (exposure) on a dependent

variable (outcome).

The aim of a statistical test

- To reach a scientific decision (yes or no) on

a difference (or effect), on a probabilistic

basis, on observed data.

Why significance testing?

- Botulism outbreak in Italy
- The risk of illness was higher among diners who

ate home preserved green olives (RR3.6). - Is the association due to chance?

The two hypothesis!

There is NO difference between the two groups (no effect) Null Hypothesis (H0) (e.g. RR1)

There is a difference between the two groups (there is an effect) Alternative Hypothesis (H1) (eg RR3.6)

When you perform a test of statistical

significance you usually reject or do not reject

the Null Hypothesis (H0)

Botulism outbreak in Italy

- Null hypothesis (H0) There is no association

between consumption of green olives and

Botulism. - Alternative hypothesis (H1) There is an

association between consumption of green olives

and Botulism.

Hypothesis, testing and null hypothesis

- Tests of statistical significance
- Data not consistent with H0
- H0 can be rejected in favour of some alternative

hypothesis H1 (the objective of our study). - Data are consistent with the H0
- H0 cannot be rejected

You cannot say that the H0 is true. You can

only decide to reject it or not reject it.

How to decide when to reject the null hypothesis?

- H0 rejected using reported p value
- p-value probability that our result (e.g. a

difference between proportions or a RR) or more

extreme values could be observed under the null

hypothesis

p values practicalities

Small p values low degree of compatibility

between H0 and the observed data ?you reject

H0, the test is significant

Large p values high degree of compatibility

between H0 and the observed data ?you dont

reject H0, the test is not significant

We can never reduce to zero the probability

that our result was not observed by chance alone

Levels of significance practicalities

- We need of a cut-off !
- 0.01 0.05 0.10

- p value gt 0.05 H0 non rejected (non

significant) - p value 0.05 H0 rejected (significant)

BUT Give always the exact p-value rather than

significant vs. non-significant.

Examples from the literature

- The limit for statistical significance was set

at p0.05. - There was a strong relationship (plt0.001).
- , but it did not reach statistical significance

(ns). - The relationship was statistically significant

(p0.0361)

- p0.05 ? Agreed convention
- Not an absolute truth
- Surely, God loves the 0.06 nearly as much
- as the 0.05 (Rosnow and Rosenthal, 1991)

p 0.05 and its errors

- Level of significance, usually p 0.05
- p value used for decision making
- But still 2 possible errors

- H0 should not be rejected, but it was rejected
- ?Type I or alpha error
- H0 should be rejected, but it was not rejected
- ?Type II or beta error

Types of errors

Truth

No diff

Diff

Decision based on the p value

No diff

Diff

- H0 is true but rejected Type I or ? error
- H0 is false but not rejected Type II or ?

error

More on errors

- Probability of Type I error
- Value of a is determined in advance of the test
- The significance level is the level of a error

that we would accept (usually 0.05) - Probability of Type II error
- Value of ß depends on the size of effect (e.g.

RR, OR) and sample size - 1-ß Statistical power of a study to detect an

effect on a specified size (e.g. 0.80) - Fix ß in advance choose an appropriate sample

size

Even more on errors

H1 is true

H0 is true

a

b

Test statistics T

Principles of significance testing

- Formulate the H0
- Test your sample data against H0
- The p value tells you whether your data are
- consistent with H0
- i.e, whether your sample data are consistent with

a chance finding (large p value), or whether

there is reason to believe that there is a true

difference (association) between the groups you

tested - You can only reject H0, or fail to reject it!

Quantifying the association

- Test of association of exposure and outcome
- E.g. Chi2 test or Fishers exact test
- Comparison of proportions
- Chi2-value quantifies the association
- The larger the Chi2-value, the smaller the p

value - the more the observed data deviate from the

assumption of independence (no effect).

Chi-square value

- sum of all cells for each cell, subtract the

expected number from the observed number, square

the difference, and divide by the expected number

Botulism outbreak in Italy2x2 table

Expected number of ill and not ill for each

cell

Ill

Non ill

9 43

4 79

Olives

x10 ill

52

x 90 non-ill

47

5

No olives

x10 ill

83

x 90 non-ill

8

75

13

122

135

Expected proportion of ill and not ill

10

90

Chi-square value

5.73

- Botulism outbreak in Italy

p 0.016

Ill

Non ill

Olives

No olives

Botulism outbreak in Italy

- The relative risk (RR) of illness among diners

who ate home preserved green olives was 3.6

(p0.016). - The p-value is smaller than the chosen

significance level of a 5. - ? Null hypothesis can be rejected.

There is a 0.016 probability (16/1000) that the

observed association could have occured by

chance, if there were no true association between

eating olives and illness.

Epidemiology and statistics

Criticism on significance testing

- Epidemiological application need more than a

decision as to whether chance alone could have

produced association. - (Rothman et al. 2008)
- ? Estimation of an effect measure (e.g. RR,
- OR) rather than significance testing.

Why estimation?

- Botulism outbreak in Italy
- The risk of illness was higher among diners who

ate home preserved green olives (RR3.6). - How confident can we be in the result?
- What is the precision of our point estimate?

The epidemiologist needs measurements rather than

probabilities

- ?2 is a test of association
- OR, RR are measures of association on a

continuous scale - ?infinite number of possible values
- The best estimate point estimate

- Range of values allowing for random variability

- Confidence interval ? precision of the point

estimate

Confidence interval (CI)

- Range of values, on the basis of the sample data,

in which the population value (or true value) may

lie. - Frequently used formulation
- If the data collection and analysis could be

replicated many times, the CI should include the

true value of the measure 95 of the time .

Confidence interval (CI)

e.g. CI for means 95 CI x 1.96 SE up to x

1.96 SE

a 5

1 - a

a/2

a/2

s

Lower limit upper limit of 95 CI of 95 CI

Indicates the amount of random error in the

estimate Can be calculated for any test

statistic, e.g. means, proportions, ORs, RRs

CI terminology

Confidence interval

Point estimate

RR 1.45 (0.99 2.1)

Lower confidence limit

Upper confidence limit

Width of confidence interval depends on

- The amount of variability in the data
- The size of the sample
- The arbitrary level of confidence you desire for

your study (usually 90, 95, 99)

A common way to use CI regarding OR/RR is If

1.0 is included in CI ? non significant If 1.0

is not included in CI ? significant

Looking the CI

Study A, large sample, precise results, narrow

CI SIGNIFICANT Study B, small size, large CI -

NON SIGNIFICANT

Study A, effect close to NO EFFECT Study B,

no information about absence of large effect

More studies are better or worse?

- Decision making based on results from a

collection of studies is not facilitated when

each study is classified as a YES or NO decision.

- Need to look at the point estimation and its CI
- But also consider its clinical or biological

significance

Botulism outbreak in Italy

- How confident can we be in the result?
- Relative risk 3.6 (point estimate)
- 95 CI for the relative risk
- (1.17 11.07)

The probability that the CI from 1.17 to 11.07

includes the true relative risk is 95.

Botulism outbreak in Italy

- The risk of illness was higher among diners who

ate home preserved green olives (RR3.6, 95 CI

1.17 to 11.07).

The p-value (or CI) function

- A graph showing the p value for all possible

values of the estimate (e.g. OR or RR). - Quantitative overview of the statistical relation

between exposure and disease for the set of data. - All confidence intervals can be read from the

curve. - The function can be constructed from the

confidence limits in Episheet.

Example Chlordiazopoxide use and congenital

heart disease

C use No C use

Cases 4 386

Controls 4 1250

OR (4 x 1250) / (4 x 386) 3.2 p0.08 95

CI0.8113

From Rothman K

3.2

p0.08

Odds ratio

0.81 - 13

Example Chlordiazopoxide use and congenital

heart disease large study

C use No C use

Cases 1090 14 910

Controls 1000 15 000

OR (1090 x 15000) / (1000 x 14910) 1.1 p0.04

95 CI1.05-1.2

From Rothman K

Precision and strength of association

Strength

Precision

Confidence interval provides more information

than p value

- Magnitude of the effect (strength of association)
- Direction of the effect (RR gt or lt 1)
- Precision of the point estimate of the effect

(variability)

p value can not provide them !

What we have to evaluate the study

- ?2 A test of association. It depends on

sample size. - p value Probability that equal (or more extreme)

results can be observed by chance alone - OR, RR Direction strength of

association if gt 1 risk factor if lt

1 protective factor (independently from

sample size) - CI Magnitude and precision of effect

Comments on p-values and CIs

- Presence of significance does not prove clinical

or biological relevance of an effect. - A lack of significance is not necessarily a lack

of an effect Absence of evidence is not

evidence of absence.

Comments on p values and CIs

- A huge effect in a small sample or a small effect

in a large sample can result in identical p

values. - A statistical test will always give a significant

result if the sample is big enough. - p values and CIs do not provide any information

on the possibility that the observed association

is due to bias or confounding.

?2 and Relative Risk

Cases Non cases Total ?2 1.3 E 9

51 60 p 0.13 NE 5 55 60 RR

1.8 Total 14 106 120 95 CI 0.6 - 4.9

Cases Non cases Total ?2 12 E 90

510 600 p 0.0002 NE 50 550

600 RR 1.8 Total 140 1060 1200 95 CI

1.3-2.5

Too large a difference and you are doomed to

statistical significance

Common source outbreak suspected

Exposure cases non cases AR Yes 15 20 42

.8 No 50 200 20.0 Total 65 220

?2 9.1 p 0.002 RR 2.1 95CI 1.4-3.4

23

REMEMBER These values do not provide any

information on the possibility that the observed

association is due to a bias or confounding.

Recommendations

- Always look at the raw data (2x2-table). How many

cases can be explained by the exposure? - Interpret with caution associations that achieve

statistical significance. - Double caution if this statistical significance

is not expected. - Use confidence intervals to describe your

results. - Report p values precisely.

Suggested reading

- KJ Rothman, S Greenland, TL Lash, Modern

Epidemiology, Lippincott Williams Wilkins,

Philadelphia, PA, 2008 - SN Goodman, R Royall, Evidence and Scientific

Research, AJPH 78, 1568, 1988 - SN Goodman, Toward Evidence-Based Medical

Statistics. 1 The P Value Fallacy, Ann Intern

Med. 130, 995, 1999 - C Poole, Low P-Values or Narrow Confidence

Intervals Which are more Durable? Epidemiology

12, 291, 2001

Previous lecturers

- Alain Moren
- Paolo DAncona
- Lisa King
- Preben Aavitsland
- Doris Radun
- Manuel Dehnert

About PowerShow.com

PowerShow.com is a leading presentation/slideshow sharing website. Whether your application is business, how-to, education, medicine, school, church, sales, marketing, online training or just for fun, PowerShow.com is a great resource. And, best of all, most of its cool features are free and easy to use.

You can use PowerShow.com to find and download example online PowerPoint ppt presentations on just about any topic you can imagine so you can learn how to improve your own slides and presentations for free. Or use it to find and download high-quality how-to PowerPoint ppt presentations with illustrated or animated slides that will teach you how to do something new, also for free. Or use it to upload your own PowerPoint slides so you can share them with your teachers, class, students, bosses, employees, customers, potential investors or the world. Or use it to create really cool photo slideshows - with 2D and 3D transitions, animation, and your choice of music - that you can share with your Facebook friends or Google+ circles. That's all free as well!

For a small fee you can get the industry's best online privacy or publicly promote your presentations and slide shows with top rankings. But aside from that it's free. We'll even convert your presentations and slide shows into the universal Flash format with all their original multimedia glory, including animation, 2D and 3D transition effects, embedded music or other audio, or even video embedded in slides. All for free. Most of the presentations and slideshows on PowerShow.com are free to view, many are even free to download. (You can choose whether to allow people to download your original PowerPoint presentations and photo slideshows for a fee or free or not at all.) Check out PowerShow.com today - for FREE. There is truly something for everyone!

You can use PowerShow.com to find and download example online PowerPoint ppt presentations on just about any topic you can imagine so you can learn how to improve your own slides and presentations for free. Or use it to find and download high-quality how-to PowerPoint ppt presentations with illustrated or animated slides that will teach you how to do something new, also for free. Or use it to upload your own PowerPoint slides so you can share them with your teachers, class, students, bosses, employees, customers, potential investors or the world. Or use it to create really cool photo slideshows - with 2D and 3D transitions, animation, and your choice of music - that you can share with your Facebook friends or Google+ circles. That's all free as well!

For a small fee you can get the industry's best online privacy or publicly promote your presentations and slide shows with top rankings. But aside from that it's free. We'll even convert your presentations and slide shows into the universal Flash format with all their original multimedia glory, including animation, 2D and 3D transition effects, embedded music or other audio, or even video embedded in slides. All for free. Most of the presentations and slideshows on PowerShow.com are free to view, many are even free to download. (You can choose whether to allow people to download your original PowerPoint presentations and photo slideshows for a fee or free or not at all.) Check out PowerShow.com today - for FREE. There is truly something for everyone!

presentations for free. Or use it to find and download high-quality how-to PowerPoint ppt presentations with illustrated or animated slides that will teach you how to do something new, also for free. Or use it to upload your own PowerPoint slides so you can share them with your teachers, class, students, bosses, employees, customers, potential investors or the world. Or use it to create really cool photo slideshows - with 2D and 3D transitions, animation, and your choice of music - that you can share with your Facebook friends or Google+ circles. That's all free as well!

For a small fee you can get the industry's best online privacy or publicly promote your presentations and slide shows with top rankings. But aside from that it's free. We'll even convert your presentations and slide shows into the universal Flash format with all their original multimedia glory, including animation, 2D and 3D transition effects, embedded music or other audio, or even video embedded in slides. All for free. Most of the presentations and slideshows on PowerShow.com are free to view, many are even free to download. (You can choose whether to allow people to download your original PowerPoint presentations and photo slideshows for a fee or free or not at all.) Check out PowerShow.com today - for FREE. There is truly something for everyone!

For a small fee you can get the industry's best online privacy or publicly promote your presentations and slide shows with top rankings. But aside from that it's free. We'll even convert your presentations and slide shows into the universal Flash format with all their original multimedia glory, including animation, 2D and 3D transition effects, embedded music or other audio, or even video embedded in slides. All for free. Most of the presentations and slideshows on PowerShow.com are free to view, many are even free to download. (You can choose whether to allow people to download your original PowerPoint presentations and photo slideshows for a fee or free or not at all.) Check out PowerShow.com today - for FREE. There is truly something for everyone!

Recommended

«

/ »

Page of

«

/ »

Promoted Presentations

Related Presentations

Page of

Home About Us Terms and Conditions Privacy Policy Presentation Removal Request Contact Us Send Us Feedback

Copyright 2018 CrystalGraphics, Inc. — All rights Reserved. PowerShow.com is a trademark of CrystalGraphics, Inc.

Copyright 2018 CrystalGraphics, Inc. — All rights Reserved. PowerShow.com is a trademark of CrystalGraphics, Inc.

The PowerPoint PPT presentation: "Significance testing and confidence intervals" is the property of its rightful owner.

Do you have PowerPoint slides to share? If so, share your PPT presentation slides online with PowerShow.com. It's FREE!