The subject is often called by the unhelpful name of “regression effects.” Here is how it works from a famous result by Kahneman and Tversky. Instructors of fighter pilots always gave their students a severe scolding if they did poorly. When the investigators asked why the instructors followed that practice, they were told the nearly all the time, a student who did poorly and was sharply scolded turned in a much better performance the next time in the air.
Anytime, you have an extreme group, either well above the mean or well below average, you have a group whose performance depends to some extent on luck. To do very poorly, I need to have the luck of having a bad day. Most days will not be extra bad nor extra good. So chances are that after a bad day, I will have a more average day and I will not perform quite so badly. However, my teacher may attribute the improvement to a scolding. Clearly, choosing a random half of the poor performers to scold but not scolding the other half will shed some light on the power of scolding or snacks or any other treatment.
Notice that the same treatment and the same regression effects (“we all tend to be average” so there is “regression” toward the mean from both high and low ends of a scale) can be used to help low performers but hurt high performers. If we scold the best performers after a very good job, we can expect them to do more poorly on the next attempt since their luck will be such that they can’t do quite as well the next time. If we confuse the treatment of scolding with the reason for doing more poorly, we will think we have uncovered the treatment of scolding that seems to help poor performers but hurt very good ones.
The truth will be that scolding has no effect and that after an extreme performance, the next performance will often be closer to average.
Deming saw this in the case of a bank manager. (Possibly there are school principals who fall into the same problem.) A bank manager watches the performance of his tellers and at the end of the month, rewards and celebrates the teller with the fewest errors. That person is given a bouquet of flowers and the Employee of the Month parking space. Trying to be fair, the same manager scolds the teller with the most errors, asking that person to be more like the shining example person. A month goes by and the manager goes through the same process. However, guess what? The employee of the month didn’t do as well this time and the worst person did better. (Exactly what we would expect from chance operations.). The manager doesn’t understand regression effects and gives the new Employee of the Month the same treatment. The new worst performer is also scolded, as before.
Now, what does the first Employee of the Month think? She or he can easily decide that they have gotten lazy and slipped. The first person on the bottom, who was scolded, will conclude that she or he has indeed improved.
If the manager continues this procedure for a year or two, every employee is likely to be told at some time that she or he is the best and at another time, that she or he is the worst teller. In truth, they are equal and differ only because of chance. Sickness, fatigue, mechanical accidents, and yes, instances of inattention can create different levels of performance but those differences are neither reliable nor dependable.
