|
Critic's Corner Discuss a review. Provide constructive feedback. Talk to the industry. |
|
|
|
Critic's Corner Discuss a review. Provide constructive feedback. Talk to the industry. |
|
In Reply to: Excellent post... posted by mkuller on July 11, 2012 at 11:53:55:
it's been a long time - 1995 when I was conducting blind tests and writing papers at university so there are a few bits I may be fuzzy on as all my textbooks and notes are back in Canada.But as I recall more trials reduced type 2 errors. More trials for subtle difference = more "reliable" results = lower likely hood of fluke. Which is why certain tests run millions of times in simulations to be more assured of the results. Significance level is a statistic based percentage but lower trials have higher error rates of predictability. Which is why people dislike low trial tests in audio.
If the person scores 6/10 in a 10 trial test they would not meet .05 significance and would be deemed a failure (could not tell a from B any better than chance. But if they run that person through ten times and he/she scored 6/10 ten times with one miss for 59/100 they would indeed meet the .05% significance level to satisfy them not having made the selections via pure chance. One test they're said to be failures but had the test continued they may indeed "pass." And more trials would be more "reliable." This is a problem with stats - it does not take into account the listener's making a mistake nor does it take into account that for possibly significant portions of music playback A is in fact sounding the same as B.
It's seems to me all the people who run all of these tests spend so much of their time on "reliability" and very little time focusing on "validity" which is arguably the 95% most important part of the test. If the test was perfectly valid and all accounted for it would only require one trial.
But it's been a very long time and I don't have my materials. And these days I'm starting to forget where I put my keys and umbrella. So...
Edits: 07/11/12
This post is made possible by the generous support of people like you and our sponsors:
Follow Ups
- RE: Excellent post... - RGA 17:07:45 07/11/12 (0)
Top of Page ]
[ Contact Us ]
[ Support/Wish List ]
[ Copyright © 2024, Audio Asylum, all rights reserved ]
