As “data” has becoming gate focusing mantra of school leaders, they have become fixed on changes in graphs. When the line illustrating data changes direction, and begins to show the desired changes, we commonly hear leaders say, “We are encouraged by the trends.”
Whatever the change, the leaders will then explain how recent interventions explain the changes and the new trend. There are two points for all in the audience to remember when hearing this from a leader:
- Trends require multiple data points. If there have been two data points showing (let’s say showing low test scores for two years), then a third point showing an increase, that third point is no a trend. If the increase shows up a second year, we may be looking at a trend; after three years, we might be able to identify a trend… but maybe not too.
- Complex systems vary. Any single change may be explained by regression to the mean. Any “really good” year is likely to be followed by a “not so good” year and any “not so good” year is likely to be followed by a “pretty good” one.
My message to those who follow data or make any wondrous claims is, “Wait.” Your improvements in math scores may be because of the Hawthorn Effect or natural variation, the “new math program” you recommended is unlikely the cause of any changes you see.