A matter of life and death: evaluation as alpha discipline and the subordinate role of science

Science cannot solve ethical dilemmas. Science is wonderful. Science is useful for understanding the world. Actions that violate a scientific understanding of the way the world works are unlikely to end well. But science is not the only, or even the primary, means by which we make decisions that change the world. There may one day be an accepted science of ethics or a science of values but at this stage, we need other means of making decisions about how to act in a volatile, uncertain, complex and ambiguous world.

Evaluation is life. To evaluate is to survive. It may be pre-cognitive, but fight, flight or freeze; explore, exploit, or conserve are driving forces of survival. It is that part of natural selection over which we exert some control. In humans, our survival is largely a cognitive affair and evaluation is the motor of cognition. Cognition finds its highest expression in logic. Science is one of the things we do with logic, it is fun and interesting, and there is a logic of scientific discovery, but science does not have the criticality of evaluation. We already know enough science to address the major problems of the world – what we lack is leaders and citizens who will make the necessary trade-offs. We lack refinement of our natural evaluative abilities.  

When I question the value of treating evaluation as a science I am often misunderstood. People think I am arguing that evaluation should not be considered a science because it

cannot be reduced to RCTs or a narrow construction of ‘the scientific method’. This would indeed be a boring argument. There is far more to science than experiments. But there is also far more to life than science. I see little in the way of philosophical justification for treating evaluation as a form of science or means of accumulating knowledge about what things there are in the world and how they behave.

It is incumbent on those that claim evaluation as a form of science to say firstly what science is, and then why evaluation is part of it. If by applied science we mean we are applying science that is fine, but applying science is not doing science. They are different things. Engineers apply science but they are not doing science. Science is useful, but that does not mean everything that is useful, is science. I love science. But science is good at science, not life and it is not evaluation. To me claiming evaluation as a science denigrates both evaluation and science.

People of the world should not be lulled into a false sense that science will solve our problems, it is their agency that matters. Brett Walker SC once observed in his Royal Commission report that it was ridiculous to say science should determine how much water is taken out of the Murray Darling because there are many different uses for water and the ones we preference are questions of values and politics, not science. By rendering ethical questions as scientific ones, we invite passivity on the part of people awaiting a ‘scientific’ answer as to whether they should donate to charity or give money directly to a homeless person. In our discipline of program evaluation this is dangerous – looking for evidence-based solutions where only impossible trade-offs exist.

When we design programs or make plans, they will often fail to achieve their intended outcomes due to a lack of reasoned action – not a lack of science. We create programs with a veneer of scientific respectability offered by some kind of ‘theory of change’. A program may be hopelessly inadequate for the task at hand and so riddled with assumptions as to be an invalid proposition – but we can point to the ‘theory of change’, so all is well in the world.

Science does not generally point out the logical flaws in a plan of action to make change in the world. We need better tools to identify the logical flaws in propositions for action. We must evaluate the design of programs before we enrol the first participant. We cannot resort to an experimental mindset – ‘giving it a go’ is not good enough. Accountability means being able to question what is being done in our name and with our taxes – not hiding behind a theoretical model of dubious utility.

Evaluation is too important to be left to the evaluators, let alone the scientists. If we want to act in the world, we must empower decision-making. We are all evaluators. We all must make trade-offs and decisions in the face of great uncertainty. We must strengthen the evaluative muscles wherever we can and not defer to the mirage of scientific answers to evaluative questions, our survival and the habitability of the world depend upon it.