|
Home
/ FAQ
/ News Classifieds / Events |
Audio Asylum Thread Printer |
Get a view of an entire thread on one page |
69.138.186.189
In Reply to: RE: "The Insignificance of Statistical Significance Testing" posted by rditmars on July 02, 2009 at 10:02:21
The author throws out the baby with the bath water here. While he is correct about the misuse of statistical significance, he infers that it is useless. In a properly designed experiment with a well-defined hypothesis, hypothesis testing is not only useful, but required.
Really, the misuse of significance testing comes into play when the investigator is more interested in the strength of an effect, which in itself has nothing to do with significance.
Bringing up Bayesian statistics is a tired old argument. This has even more problems than the hypothesis-testing approach.
Interesting also that the USGS distributes Blossom, a highly useful package for statistical testing.
In a well designed experiment you really need a random sample to use statistical significance to reject the null hypothesis. A good experiment will use a big enough random sample given the anticipated strength of the relationship to reject the null hypothesis.
Certainly statistical significance testing offers no insight into the strength of the relationship although it is not infrequently done. Statistical significance, of course, increases with the sample size as well as the strength of the relationship. I remember hearing a paper in international relations where rather than using 40 countries the author used their relations as diads. This, of course, greatly increase his "sample" size and won him many "significant" relationships. He was bombastic when I noted this. I said that it really didn't matter as although he had many "significant" relationships, he explain little of the variance and thus his research was trivial. He was livid, but it did not matter as I was doing the hiring for the position he applied for.
Statistical hypothesis testing is too often used as a substitute for the "goodness" of an effect. But this is dependent on sample size and other factors. It's just a lazy way to put a pseudo-scientific stamp of approval on whatever findings were developed. In an audio double-blind test, you might get a significant result without your ears registering enough of an effect to make it worthwhile. Now this is disregarding all the other problems that may make the test worthless or invalid.
a
When conducting any study based on acquiring data from the natural (or poluted) environment, you are conducting an undesigned experiment. The need to determine a meaningful effect size (subject-matter significance) to test for becomes a very important element of the sampling and analysis program.
*
"Whoever undertakes to set himself up as a judge of truth and knowledge is shipwrecked by the laughter of the gods." - Albert Einstein
Post a Followup:
Top of Page ]
[ Contact Us ]
[ Support/Wish List ]
[ Copyright © 2009, Audio Asylum, all rights reserved ]
[ Go Green ]
[ Take5 ]