|
Home
/ FAQ
/ News Classifieds / Events |
Audio Asylum Thread Printer |
Get a view of an entire thread on one page |
24.245.46.177
In Reply to: RE: statistics question posted by mike1127 on June 25, 2009 at 11:17:03
It is not logical to "accept" a hypothesis, only to tentatively reject the null hypothesis. There is a good body of literature on the notion of statistical significance when you do not test with the entire population but rather a random sample of sufficient size. You take a chance with random samples that your sample is not representative of the population and thus falsely reject the true null hypothesis. When this is sufficiently improbable, typically only 5 samples in a hundred, most reject the null hypothesis.
You are really just talking about probabilities. If something is quite improbable, most would say that random guessing is the best explanation. The problem is that you merely are saying that you could be guessing. You cannot say that the cables sound alike.
You may find the link below interesting. It has a good discussion on the difference between subject-matter and statistical significance.
*
"Whoever undertakes to set himself up as a judge of truth and knowledge is shipwrecked by the laughter of the gods." - Albert Einstein
The author throws out the baby with the bath water here. While he is correct about the misuse of statistical significance, he infers that it is useless. In a properly designed experiment with a well-defined hypothesis, hypothesis testing is not only useful, but required.
Really, the misuse of significance testing comes into play when the investigator is more interested in the strength of an effect, which in itself has nothing to do with significance.
Bringing up Bayesian statistics is a tired old argument. This has even more problems than the hypothesis-testing approach.
Interesting also that the USGS distributes Blossom, a highly useful package for statistical testing.
In a well designed experiment you really need a random sample to use statistical significance to reject the null hypothesis. A good experiment will use a big enough random sample given the anticipated strength of the relationship to reject the null hypothesis.
Certainly statistical significance testing offers no insight into the strength of the relationship although it is not infrequently done. Statistical significance, of course, increases with the sample size as well as the strength of the relationship. I remember hearing a paper in international relations where rather than using 40 countries the author used their relations as diads. This, of course, greatly increase his "sample" size and won him many "significant" relationships. He was bombastic when I noted this. I said that it really didn't matter as although he had many "significant" relationships, he explain little of the variance and thus his research was trivial. He was livid, but it did not matter as I was doing the hiring for the position he applied for.
Statistical hypothesis testing is too often used as a substitute for the "goodness" of an effect. But this is dependent on sample size and other factors. It's just a lazy way to put a pseudo-scientific stamp of approval on whatever findings were developed. In an audio double-blind test, you might get a significant result without your ears registering enough of an effect to make it worthwhile. Now this is disregarding all the other problems that may make the test worthless or invalid.
a
When conducting any study based on acquiring data from the natural (or poluted) environment, you are conducting an undesigned experiment. The need to determine a meaningful effect size (subject-matter significance) to test for becomes a very important element of the sampling and analysis program.
*
"Whoever undertakes to set himself up as a judge of truth and knowledge is shipwrecked by the laughter of the gods." - Albert Einstein
Post a Followup:
Top of Page ]
[ Contact Us ]
[ Support/Wish List ]
[ Copyright © 2009, Audio Asylum, all rights reserved ]
[ Go Green ]
[ Take5 ]