Unit 3: Collecting Data

Experimental Design

Designing a randomized, controlled experiment and identifying confounding.

Advanced20 min lesson3 min readUpdated August 12, 2026Author not yet attributed

Prerequisites

Sampling Methods

What Actually Justifies a Causal Claim

A study observes that people who take vitamin supplements tend to have fewer colds than people who don't. Before reading on: does this observation, by itself, justify concluding that taking vitamins causes fewer colds?

Definition — Random Assignment, Confounding, and Causation

Random assignment places subjects into treatment groups by chance — distinct from random sampling, which selects who is studied in the first place. Random assignment supports causal conclusions because it tends to balance every other characteristic (known and unknown) between groups, isolating the treatment's effect. A confounding variable affects the outcome and differs between groups in a way that's tangled up with the treatment, making it impossible to attribute a result to the treatment alone. Without random assignment (an observational study), only association can be established — not causation.

Worked Example — Designing a Randomized, Controlled, Blinded Experiment

To test whether a new fertilizer increases tomato plant yield, 60 available plants are randomly assigned: 30 to the new fertilizer, 30 to a standard treatment (the control group) — random assignment, not sampling, is what makes a causal conclusion possible here. If whoever measures the final yield doesn't know which plants received which treatment, that's blinding, reducing the chance the evaluator's expectations bias the measurements. If the fertilizer group happened to also sit in a sunnier part of the greenhouse, sunlight would be a confounding variable — its effect on yield would be inseparable from the fertilizer's effect. Proper random assignment is specifically designed to spread lurking factors like sunlight evenly across both groups, on average, which is exactly why a well-designed experiment can support a causal claim that an observational study cannot.

Worked Example — Why the Vitamin Study Only Shows Association

The vitamin/colds study is observational — people chose whether to take vitamins themselves; researchers didn't randomly assign it. A plausible confounding variable is overall health consciousness: people who take vitamins might also exercise more, sleep more, and eat better, any of which could independently reduce colds. Because vitamin use wasn't randomly assigned, this study cannot separate the vitamins' own effect from these other, tangled-up habits. The honest conclusion is association — people who take vitamins tend to get fewer colds — not causation.

Tip

Ask a single diagnostic question before trusting any causal claim: was the treatment randomly assigned by the researchers, or did subjects end up in their groups some other way? Only the first supports causation.

Common Mistakes

Confusing random sampling (who gets studied) with random assignment (which group each subject lands in).
Random sampling supports generalizing results to a larger population; random assignment supports concluding the treatment caused the observed difference — a study can have either, both, or neither, and they justify different kinds of conclusions.
Concluding causation from a strong, statistically clear association in an observational study.
No matter how strong or statistically convincing an observational association is, a lurking confounding variable can never be fully ruled out without random assignment — strength of association and justification for causation are separate questions.

Key Takeaways

Random assignment (not random sampling) is what allows a causal conclusion, by balancing lurking variables between treatment groups.
A confounding variable is tangled up with the treatment's effect on the outcome, making the treatment's own effect impossible to isolate without random assignment.
Observational studies, however strong the association, generally support only association — not causation.

Summary

This closes Unit 3: how data is collected determines what conclusions are justified. Unit 4 turns to probability itself, the mathematical foundation everything in the rest of this course builds on.

Track your progress

← Sampling Methods Back to Unit 3: Collecting Data