P-rep

This article needs additional citations for verification. Please help improve this article by adding citations to reliable sources. Unsourced material may be challenged and removed. Find sources: "P-rep" – news · newspapers · books · scholar · JSTOR (October 2011) (Learn how and when to remove this message)

In

For a while, the Association for Psychological Science recommended that articles submitted to Psychological Science and their other journals report p-rep rather than the classic p-value,^[2] but this is no longer the case.^[3]

Calculation

This section is missing information about basic definition from Cohen's d “effect size” and why the approximation works. Please expand the section to include this information. Further details may exist on the talk page. (July 2021)

Approximation from p

The value of the p-rep (p_rep) can be approximated based on the p-value (p) as follows:

${\displaystyle p_{\text{rep}}=\left[1+\left({\frac {p}{1-p}}\right)^{\frac {2}{3}}\right]^{-1}.}$

The above applies for one-tailed distributions.

Criticism

The fact that the p-rep has a

Among the criticisms of p-rep is the fact that while it attempts to estimate replicability, it ignores results from other studies which can accurately guide this estimate.^[4] For example, an experiment on some unlikely paranormal phenomenon may yield a p-rep of 0.75. Most people would still not conclude the probability of a replication was 75%. Rather, they would conclude it is much closer to 0: Extraordinary claims require extraordinary evidence, and p-rep ignores this. Because of this, p-rep may in fact be harder to interpret than a classical p-value. The fact that p-rep requires assumptions about prior probabilities for it to be valid makes its interpretation complex. Killeen argues that new results should be evaluated in their own right, without the "burden of history", with flat priors: that is what p-rep yields. A more pragmatic estimate of replicability would include prior knowledge, via, for instance, meta-analysis.

Critics have also underscored mathematical errors in the original Killeen paper. For example, the formula relating the