The Whole Picture
(Trilogy: Part Three)
After class, Colum waited until most of the students had left before approaching Helen.
“Hi… it’s me again,” he said, half-smiling.
Helen laughed softly. “Hi, Colum. I had a feeling you’d be back.”
“Yeah,” Colum admitted. “I think I just need one last push to really see the whole picture.”
“Mmm,” Helen said, folding her arms and leaning against the desk. “Alright. What’s still bothering you?”
“Well,” Colum began, “I get now why we test the null hypothesis instead of the alternative. And I understand why 0.05 is an agreed threshold. But I’m still unsure what the p-value itself is really telling us.”
Helen nodded. “It makes sense, it is difficult to understand the whole thing within one shot, it needs time.”
“It’s about the data,” Colum said. “We collect them, run the analysis, and out comes this number. If it’s below 0.05, we reject the null. But it’s not comparing the null to the alternative… So what is it doing?”
Helen smiled. “You’re right — that’s where many people get lost. Let’s slow it down.”
She pulled a chair closer and sat. “In classical statistics, we don’t weigh the null against the alternative. We don’t ask which one is more likely.”
“So we’re not choosing between them,” Colum said.
“Exactly. Instead, we do something much narrower.” She paused. “We temporarily assume the null hypothesis is true.”
Colum blinked. “Assume?”
“Yes,” Helen said. “It is like we are pessimistic with our hypothesis, so we say: let’s imagine there really is no effect — nothing interesting going on at all.”
“And then?” Colum asked.
“And then we ask one question: if that were true, how unusual would our data be?”
Colum nodded slowly. “So the p-value is about the data — not the hypotheses.”
“Exactly. It tells us how the data behave in a world where the null hypothesis is true.”
She picked up a pen. “Imagine you compare two groups — coffee drinkers and non-drinkers — and you find a difference in memory scores.”
“But that difference could just be random,” Colum said.
“Right,” Helen replied. “So statistics builds a model of randomness: what kinds of differences we’d expect to see just by chance if coffee had no effect at all.”
She leaned forward. “Small differences are very common. Big ones are rare.”
“And we check where our result lands,” Colum said.
“Yes. We calculate a test statistic — like a t-value — which tells us how far our result is from what the null predicts, given variability and sample size.”
She continued, “The p-value then answers this: if the null hypothesis were true, how often would we see results this extreme, or more extreme, just by chance?”
“So a p-value of 0.04,” Colum said, “means only 4% of experiments would look like this if nothing real were going on.”
“Exactly. Nothing more than that.”
Colum frowned. “So it doesn’t mean there’s a 96% chance the alternative is true.”
“No,” Helen said gently. “That’s the classic mistake. The p-value doesn’t tell us what’s true. It tells us how compatible the data are with chance.”
“So a small p-value means the data don’t sit very comfortably in a ‘nothing-is-happening’ world.”
“Exactly,” Helen said. “And that’s why we use a reference point like 0.05 — a shared rule for deciding when the data are unusual enough to stop blaming chance.”
“And if the p-value is above 0.05,” Colum said, “we don’t reject the null.”
“Right. Not because we’ve proven it true — but because the data aren’t surprising enough to rule it out.”
Colum hesitated. “There’s still something that feels backwards. Why does smaller mean more unusual?”
Helen smiled. “Because the number isn’t measuring extremeness itself. It’s measuring how often chance would give you something at least that extreme.”
She gestured broadly. “Imagine all the results you could get if the null were true. Most cluster near zero. Only a small tail is extreme.”
“When the p-value is 0.20,” she continued, “you’re saying: this happens all the time. When it’s 0.05, it’s already rare. At 0.01, it’s very rare.”
Colum’s eyes lit up. “So the smaller the p-value, the less of the ‘chance world’ can explain my result.”
“Exactly.”
He exhaled. “So lower p-values mean randomness is struggling to explain what we saw.”
“Yes,” Helen said warmly.
Colum closed his notebook, then paused. “So… if I get a p-value like 0.5 or 0.7…”
Helen tilted her head. “What would you say it means now?”
Colum thought for a moment. “That my data look completely ordinary. Like something chance would produce all the time.”
Helen smiled. “That’s it.”
“So it’s not that the null hypothesis is ‘strong’,” Colum added. “It’s just… unchallenged.”
“Exactly,” she said. “Large p-values are quiet. They don’t argue for anything. They just shrug.”
Colum laughed softly. “Statistics really doesn’t like making bold claims.”
“No,” Helen agreed. “It prefers modest approaches. Statistics tend to find things passively rather than confronting them directly; instead, they speak in broad terms, examining whether the most probable option is sufficiently consistent.”
He stood up, slinging his bag over his shoulder. “I think I finally got it.”
Helen nodded. “Good. Because now you’re thinking like a statistician.”
Colum grimaced. “Well, I’m still in the process of internalizing it completely."
Helen laughed. “You’ll be fine.”
“Thanks Helen for your explanations, they have been very useful” and this time, he walked out without looking back.