How to evaluate the quality of a process? By process, I mean any sequence of more or less complicated activities or tasks, which you (and others) are trying to do, to achieve some goals you agreed on. It can be something called a business process, a decision process, a problem-solving process, and so on – what I’m writing below is widely applicable.
In processes which involve uncertain outcomes (so, when you do the process, you are not 100% sure what you’ll get from it), the problem is that you can do it well, and get bad outcomes; at other times, you will do it well, and get good outcomes. And you see the point – there are four cases: well done but bad outcomes, well done with good outcomes, badly done with bad outcomes, and badly done with bad outcomes.
Because of the fact that, when you have an uncertain process, you cannot know the exact outcomes, we have two different kinds of quality to evaluate:
- Procedural quality is the evaluation of how you did the process; this usually involves you having some idea – before actually doing the process – of how it should be done. You then do the process, and the difference between how you should have done it, and how you actually did it, is your procedural quality. In requirements engineering, for example, you could have a method which tells you how you should elicit requirements, and then you will do it, and the differences tell you something about procedural quality.
- Outcome quality is the evaluation of what you got from the process, after you did it and was able to observe at least some of its outcomes. Outcome quality is going to depend on the difference between what you expected as outcomes, and what you actually got.
This difference between procedural and outcome quality makes it is important to understand – over time, as these processes are done and repeated – the factors which influence procedural quality and factors which influence outcome quality, and see which you can influence, how, and at what cost. In some cases, you can pay more and get higher confidence in outcome quality, while in others this cost will be too high. Procedural quality is slightly different, since you can influence to a considerable extent – through training, process improvements, and so on – how something is being done, and thereby influence procedural quality.
What remains critical, is not to confuse procedural and outcome quality, especially when evaluating people tasked with doing uncertain processes.