I was chatting with a colleague the other day about the difference between “evaluation questions” and “research questions”. She said my explanation was really helpful, so I thought I’d share it more widely.
Evaluation is sometimes called applied research, but it’s a bit more than that. The definition I like for evaluation is a process to determine the merit, worth or significance of something (Scriven, 1991). In other words, evaluation asks how good a thing is, or where it falls on a scale from bad to good – on whatever dimensions we decide are important for that thing.
Research is descriptive. It shows what is, or what happened. Evaluation is, well, evaluative – it identifies how good that thing was, or how appropriate, or how well it was done, etc. You need research questions to answer evaluative questions – you can’t answer “how good a thing is” without first answering “what is the thing”. But evaluation takes it that step further.
So when it comes to evaluation questions vs research questions, what’s the difference?
Evaluative questions position you to give an answer on a scale from bad to good. Questions like “how effective was the program” positions you to give an answer from “not effective” to “very effective”. Or “how appropriate was the program design” pushes you to respond somewhere between “not appropriate” and “very appropriate”.
Research questions are descriptive. They ask “what”, “who”, “how”, even “why”. You can’t answer a research question using a scale. Questions like “who did the program engage” need to be answered with a description or a list, not a position on a scale.
It’s common to see research questions framed or labelled as evaluation questions. They’re important, and you need research questions in order to come up with good answers to evaluative questions. But on their own, they do shy away from actually determining the merit, worth or significance of the thing, per Scriven’s definition of evaluation.
When we’re asking good evaluation questions, we need to be asking how good a thing is, not just what it is.
So some example evaluation questions could be:
- How effective was the program?
- How well was the program implemented?
- How engaged were the stakeholders?
- How aligned was the program to the overall policy goals?
And some research questions could be:
- Did the program achieve its intended outputs and outcomes?
- What were the unintended effects of the program?
- How many people did the program engage?
- What influenced the design of the program?
Research questions can be super useful. But when we’re doing evaluation, we do need to be thinking in terms of evaluative questions.
Do you need some support evaluating projects or programs? I specialise in strategic low-resource high-impact evaluations for small-to-medium organisations. Get in touch and let’s talk about what you need.