A colleague and I recently had a conversation about “data” and its role in education. I maintained that advocated for using data have a fundamental misunderstanding of science and evidence. I further maintained that misapplication of the principles of science and inquiry makes the decisions made by “data-driven leaders” in schools dubious at best. This post contains my summary of the argument I made.
Let’s consider science for a moment. It is a process through which humans seek to understand the physical and natural world. Scientists seek to identify not only correlation but causation. Good scientific theory allows us to explain cause and effect relationships, and thus to make predictions: If we do A it will cause B, because of this mechanism.
To find these cause and effect relationships, scientists follow a recognized approach to making observations, testing hypotheses, generating laws, and elucidating theories. This is not “The Scientific Method” as presented in most introductory science textbooks; that mythical process is rarely used b practicing scientists. Science is a far richer endeavor that allowed in the simplistic method we present to students; it is connected by:
- The supremacy of observation. All disputes in science are resolved by evidence and observation, and evidence and observation must follow the rules of logic.
- Verification of observations. Before scientists accept an observation, it must be observed by others. This requires science be done in a public and open manner and scientists must report what they did so that others can verify. In addition, science requires at least triangulation; an idea must be confirmed by at least three different types of observation before it can be accepted.
- Avoidance of truth. Rather than demonstrating an idea is true, scientists demonstrate and idea is a false. In this way science keeps ideas that are not false (because they are more likely to be accurate) and discard those that are false.
- Accounting for variables. When designing experiments, scientists identify all relevant factors that will affect their measurements. The measurements are then done on two groups, the control group and the treatment group (in which one variable is changed). If variables cannot be controlled, then
- Peer review is necessary to evaluate one’s work and ensure that necessary data was collected and that it was properly and logically analyzed and that observation supports conclusions.