In my recent book Efficacious Technology Management, which I published under a Creative Commons license (you can find it here), I began a section with the paragraph:
“Data” has been widely, but imprecisely, used in education for most of the 21st century. Data-driven educators make decisions based on information they have gathered about their students’ performance. Ostensibly, this is done in an attempt to adopt the position of a researcher and to ground decisions in objective research, thus give more support for their decisions. Upon closer inspection, however, there is little resemblance between data collection, the data, and the analysis methods used by researchers and those used by most “data-driven educators.”
I went on to detail how the planning, collection, and analysis of data differed between “data-driven” educators and researchers. My position is that researchers are apply for more sophisticated methods to their work, and that sophistication leads to valid and reliable data that can be used to draw reasonable conclusions; the methods used by “data-driven educators” make their conclusion dubious in many cases.
Recently, I attended a presentation in which the speaker made some statements about teachers’ autonomy that ran contrary to my understanding of the phenomenon and its effects in classrooms. Basically, the speaker was making some claims that ran contrary to some observations I had made in another chapter of my book:
Autonomy is a complicated factor in many organizations and professions, including education. While autonomy is a factor that motivates individuals to engage with and adopt innovations, there is evidence that teachers may exert limited autonomy with regards to regarding instructional practices (Range, Pijanowski, Duncan, Scherz, & Hvidston, 2014). Blumenfeld, Kempler, and Krajcik (2006) suggested autonomy is grounded in authority to make decisions and the competence to identify and affect a solution. In many cases, teachers lack the authority to be autonomous and the technology that is the focus of the innovation is unfamiliar and outside their perceived are of expertise.
Following the speaker’s statements that ran against my understanding, I did what comes natural to one who attends research conferences: I asked questions that challenged what the speaker said. I quickly realized I had committed one of the most egregious offenses at a conference in which the keynote speaker was invited, I had dared to suggest the speaker might have been incorrect and his words were not a completely accurate representation of the realities.
When I was talking about this with a colleague who also frequents both conferences and presentations by and for researchers and those by and for practitioners, and she confirmed to me she had experienced negative reactions when she violated the taboo of challenging a presenter’s assumptions, methods, or conclusions.
We are both very progressive professionals and we are willing to take risks and support colleagues who are risk takers, so we do not challenge speakers in an attempt to discredit them. We seek deeper understanding of their positions and we are hoping to clarify what we heard.
It is unfortunate that educators allow experts and their data to go unchallenged. We spend way too much money and far too much time gathering “data” for any data collection practices to go unchallenged. If experts (or the leaders) who invited them are uncomfortable fielding questions about their assumptions, methods, and conclusions or if they deflect the questions because the “answers are propriety,” then whatever they are selling or advocating should be avoided until the questions are answered.