Rochefort-Maranda, Guillaume (2020) Inflated effect sizes and underpowered tests: how the severity measure of evidence is affected by the winner’s curse. Philosophical Studies. ISSN 0031-8116

Preview

Text inflated_sev.pdf Download (180kB) | Preview

Abstract

My aim in this paper is to show how the problem of inflated effect sizes (the Winner’s Curse) corrupts the severity measure of evidence. This has never been done. In fact, the Winner’s Curse is barely mentioned in the philosophical literature. Since the severity score is the predominant measure of evidence for frequentist tests in the philosophical literature, it is important to underscore its flaws. It is also crucial to bring the philosophical literature up to speed with the limits of classical testing. The Winner’s Curse is one of them. The problem is that when a significant result is obtained by using an underpowered test, the severity score becomes particularly high for large discrepancies from the null-hypothesis. This means that such discrepancies are very well supported by the evidence according to that measure. However, it is now well documented that significant tests with low power display inflated effect sizes. They systematically show departures from the null hypothesis H0 that are much greater than they really are. From an epistemological point of view this means that a significant result produced by an underpowered test does not provide evidence for large discrepancies from H0. Therefore, the severity score is an inadequate measure of evidence. Given that we are now aware of the phenomenon of inflated effect sizes, it would be irresponsible to rely on the severity score to measure the strength of the evidence against the null. Instead, one must take appropriate measures to try and avoid using underpowered tests by setting a threshold for the sample size or by replicating the results of the experiment.

---

Export/Citation:

EndNote | BibTeX | Dublin Core | ASCII/Text Citation (Chicago) | HTML Citation | OpenURL

Social Networking:

Share |

---

Item Type:	Published Article or Volume
Creators:	Creators Email ORCID Rochefort-Maranda, Guillaume guillaumemaranda@hotmail.com
Keywords:	Confirmation ;Evidence ;Statistics; Severity ;Error statistics ;Winner’s curse
Subjects:	Specific Sciences > Probability/Statistics
Depositing User:	Dr. Guillaume Rochefort-Maranda
Date Deposited:	04 Feb 2020 00:06
Last Modified:	04 Feb 2020 00:06
Item ID:	16876
Journal or Publication Title:	Philosophical Studies
Publisher:	Springer (Springer Science+Business Media B.V.)
DOI or Unique Handle:	https://doi.org/10.1007/s11098-020-01424-z
Subjects:	Specific Sciences > Probability/Statistics
Date:	2020
ISSN:	0031-8116
URI:	https://philsci-archive.pitt.edu/id/eprint/16876

Available Versions of this Item

• Inflated Effect Sizes and Underpowered Tests: How the Severity Measure of Evidence is Affected by the Winner's Curse. (deposited 22 Dec 2017 18:36)
  • Inflated effect sizes and underpowered tests: how the severity measure of evidence is affected by the winner’s curse. (deposited 04 Feb 2020 00:06) [Currently Displayed]

Monthly Views for the past 3 years

Monthly Downloads for the past 3 years

Plum Analytics

Altmetric.com

Actions (login required)

View Item