Notes on the Decline Effect

 Corollaries on the Probability that a Research Finding is...

Corollary 1: The smaller the studies conducted in a scientific field, the less likely the research findings are to be true. Corollary 2: The smaller the effect sizes in a scientific field, the less likely the research findings are to be true. Corollary 3: The greater the number and the lesser the selection of tested relationships in a scientific field, the less likely the research findings are to be true. Corollary 4: The greater the flexibility in designs, definitions, outcomes, and analy...

 A Classic Example of Selective Reporting

One of the classic examples of selective reporting concerns the testing of acupuncture in different countries. While acupuncture is widely accepted as a medical treatment in various Asian countries, its use is much more contested in the West. These cultural differences have profoundly influenced the results of clinical trials. Between 1966 and 1995, there were forty-seven studies of acupuncture in China, Taiwan, and Japan, and every single trial concluded that acupuncture was an effective tre...

 Publication Bias Produces a "Decline Effect"

Jennions, similarly, argues that the decline effect is largely a product of publication bias, or the tendency of scientists and scientific journals to prefer positive data over null results, which is what happens when no effect is found. The bias was first identified by the statistician Theodore Sterling, in 1959, after he noticed that ninety-seven per cent of all published psychological studies with statistically significant data found the effect they were looking for. A “significant” re...