Evidence and Design

When I was studying biology as an undergraduate, a students asked the physiology about Daltons, the unit used to measure the size of large molecules such as proteins. The professor was so used to using the term, he had to look up the definition so he could explain it to the 250 or so students in the lecture hall.

Educators refer to “evidence” and “continuous improvement” so often that they appear to be suffering from the same lack of understanding my biology professor… “What exactly do we mean by these terms?” On first glance, it seems the definitions do not really matter… we can work with molecules without knowing the definition of the units of their weights and we can “use evidence to improve schools” without knowing how we define the terms. I maintain there is a fundamental differences between the “Dalton definition” (which measures a physical quantity)¬† and the “Improvement definition” (which measures a social quantity).

When educators and others attempt to create systems that are complex and that address equally complex problems, they must adopt a design approach. Design is an iterative process in which interventions are created, deployed, and then recreated or modified; with each iteration, the designers seek to improve the efficiency or effectiveness of the system.

For designers of educational systems in which they seek to use evidence for continuous improvement, the work is informed by three factors I have illustrated below:

design results from performance, observations, and theory

  • Performance represents the “things” we hope to accomplish with the system. Knowing exactly what this is can be difficult and the goals obtuse (just what is “career and college ready?”) or very explicit (passing a professional exam and demonstrating essential skills for nursing students).
  • Th performances that really matter are likely to be demonstrated in more than one observation. Standards-based or standardized test scores are one small indication of one’s college or career readiness, but there are many other performances we can observe that predict students success.¬†Designers of efficacious systems recognize and utilize multiple observations when making improvements.
  • Being using theory to design the interventions, we can both indicate the factors that are relevant in design decisions (we avoid red herrings which divert resources from what matters) and that help designers identify methods for measuring performance.

It appears this design process may be a tautology, but there is no escape from a tautology. In the iterations of design, each iteration ends at a different “place” from where it began.