Statistical Rethinking Chapter 1: The Golem of Prague

1.1. Statistical Golems

1.2. Statistical rethinking

Discussion on the place of falsification in (statistical) science.

“The greatest obstacle that I encounter among students and colleagues is the tacit belief that the proper objective of statistical inference is to test null hypotheses.”

Two main points here:

“Hypotheses are not models”

• Complex relations between research hypotheses, process models and actual statistical modeling of those processes. One statistical model can be appropriate for multiple, contradictory hypotheses, making strict falsification impossible.

“Measurement matters”

• Data cannot always be trusted, so even if the modeling seems to be right it’s not necessarily possible to conclude anything consensual.

1.2.1. Hypotheses are not models

An exemplified development on the general idea that there are no one-to-one relations between the three entities that are:

• Research hypotheses, which emerge from observations/data,
• Process models, conceptually proposed to explain the observed facts,
• Statistical models, concrete statistical implementations of the process models that can then be used to simulate behavior and compare to real data.

Evolutionary biology example

1.3. Measurement matters

Two examples to show that research hypothesis rarely lead to binary null hypotheses that can “easily” be falsified by providing a single counter-exemple.

First, the very notion of counter-examples is to be questioned when measurement errors can arise.

Second,

Most interesting research questions in science are not of the kind ‘all swans are white’ but rather of the kind: \[ \Hzero: \text{80\% of swans are white.} \] or \[ \Hzero: \text{Black swans are rare.} \]