My undergraduate studies included many science courses, that’s what happens when one is studying to be a science teacher. As a graduate student in education, my mentors introduced me to qualitative research methods. For 20 years, I have been attending educational research conferences (and occasionally presenting at them). I’m certainly no expert, but I am sufficiently experienced so that I can be a critical consumer of data and evidence.
I’ve always been curious about social sciences and the “physics envy” that is attributed to its practitioners. According to this idea, social science researchers are envious of the objective and clear sources of data available to physicists. They design and conduct experiments—true experiments—in which variables are controlled and cause and effect can be established. Social science research (especially educational research) is messy—very messy—by comparison. We can infrequently conduct true experiments because we cannot control the relevant variables and we are dealing with humans, not physical systems, so the ethics are far more complicated.
Physics envy is the root cause (in my opinion) of the intense interest in being “data-driven” amongst educators in recent decades. While I do value data and evidence, I am confused about the insistence that all decisions be made based on data. My confusion arises from three realities.
First, the educators rarely evaluate the quality of the tests they administer. Internal validity, external validity, and reliability are never questioned.
Second, the tests are (presumably) objective. This also is unquestioned, despite the widely accepted conclusion that many of the tests that have been long used in schools have strong cultural biases.
Third, educators rarely interpret the results. Their assumptions are usually not identified, and changes in data are assumed to arise directly from their interventions, and the theory that might explain what is observed and how it can be affected is ignored.
Perhaps the most damning criticism that can be levied against a “data-driven” educator is subjectivity. We dare not have our own opinions cloud our data.
I must be clear that I value good data. We must be careful about what data we gather and how we interpret it, but we must accept that our data are subjective. Our subjectivity arises, in large part, from our philosophies. Educators’ ontological and epistemological and pedagogical beliefs (even those that are not recognized) affect the data we value and the way we make meaning of it. This subjectivity is a strength if we admit it. It is only through negotiation of the disparate interpretations of data that we can find the most accurate representation of data and what it means for the individuals and in the context of interest.
My fall has found me taking 5 mile walks most days (outside and socially distanced low impact physical activity is highly recommended!) and I have begun listening to audiobooks while walking. Most recently, I’ve been listening to Lawrence Kraus’ The Greatest Story Ever Told—So Far which recounts many of the greatest discoveries of physics. One of the observations Kraus makes is that measurements are observer dependent. What we measure depends on our tools and our perspectives.
Maybe it is time for educators to stop trying to be objective with our data. We will be more physicist-like and we will be more likely to make better decisions about our data and how we use it.