"Try it!" "Test it!" These are cries of modernity. Today, we often have an impulse to check out things, ideas, hypotheses, beliefs, theories. The first idea is to run an experiment. Buy an X car and buy a y car and run them over the same routes with the same driver and see which one performs better. Getting that idea is very modern. We don't tend to think that it is an offense against God to test things.
If you do that and X performs better on all counts, there can be still be problems with concluding that X is the better car and buying that car from now on. What if you happened to get a flawed Y car, a lemon, when most of the y cars are actually better than the X kind? That is, suppose the two tested cars are not really "representative" of the mass of their type? The mass or 'run' of each type of car is envisioned as identical or very similar to each other but that vision is a concept and it might not reflect reality.
The theory of experiments figures that you can get a better handle on the performance of each type of car (or whatever you are researching) if you have several of each kind. The first more or less cut-and-dried procedure in experimental statistics to be invented is called Student's t-test and it was invented to compare the average of a group (of cars or bottles of beer in his case) to another figure, either the average of a different group or the desired figure. If you look at the typical t-test table, you can see the value of having 5 or 10 X cars and 5 or 10 Y cars instead of just one of each. You can also see why research is expensive.
The basic idea of such an experiment is to keep the performance and the driving of the two sets of cars as identical as possible. But time begins to rear its head here. We might have the same driver cover the same course with all ten cars, hoping the driver will drive "the same" each time. Besides the factors of boredom, we can see that the same driver cannot drive all ten cars at the same time. But doing them in succession, the last car will be driven AFTER SOME TIME lapse. The driver will be older, more experienced, etc. The road will be a little older. Most importantly, time might effect the representativeness of the cars. If they are all manufactured as identically as possible and no new model or run of products is introduced, that will not be a problem.
All of these problems arise in a corresponding way with any experimental comparison, even when we try to compare this year's harvest with an earlier year or this year's state of our marriage with last year's. We cannot get two years to be identical, no matter how hard we try. Put another way, we cannot live our lives "as they are" more than once. Many important parts of our lives are beyond experimentation. We cannot duplicate nor eliminate time.
To read a good article on current problems in medical research from the current issue of The Atlantic, click on this link.
If you do that and X performs better on all counts, there can be still be problems with concluding that X is the better car and buying that car from now on. What if you happened to get a flawed Y car, a lemon, when most of the y cars are actually better than the X kind? That is, suppose the two tested cars are not really "representative" of the mass of their type? The mass or 'run' of each type of car is envisioned as identical or very similar to each other but that vision is a concept and it might not reflect reality.
The theory of experiments figures that you can get a better handle on the performance of each type of car (or whatever you are researching) if you have several of each kind. The first more or less cut-and-dried procedure in experimental statistics to be invented is called Student's t-test and it was invented to compare the average of a group (of cars or bottles of beer in his case) to another figure, either the average of a different group or the desired figure. If you look at the typical t-test table, you can see the value of having 5 or 10 X cars and 5 or 10 Y cars instead of just one of each. You can also see why research is expensive.
All of these problems arise in a corresponding way with any experimental comparison, even when we try to compare this year's harvest with an earlier year or this year's state of our marriage with last year's. We cannot get two years to be identical, no matter how hard we try. Put another way, we cannot live our lives "as they are" more than once. Many important parts of our lives are beyond experimentation. We cannot duplicate nor eliminate time.
To read a good article on current problems in medical research from the current issue of The Atlantic, click on this link.
