|
Computer Audio Asylum Music servers and other computer based digital audio technologies. |
|
|
|
Computer Audio Asylum Music servers and other computer based digital audio technologies. |
|
In Reply to: RE: Let me get this out of my system... posted by Phelonious Ponk on March 1, 2010 at 06:17:16:
Believe if you will that sighted listening tests present no more opportunity for bias than unsighted ones, no matter how informal.
I argued quite the opposite. Making a fetish of the experimenter being "blind" is to clutch at methodological straws. No one is suggesting that the subject in an experiment can safely be "sighted". However, the notion that a competent experiment calls for an "unsighted" experimenter is, to me, risible. The more serious debate is about whether sub-standard experiments are more reliable than informal, subjective listening sessions. I'm far from convinced they are.
The results of a poorly-designed "blind" test are usually useless and have the added drawback of a veneer of objectivity that deceives those who haven't thought about the issue much. I don't know what an "informal" test is: I know only of good tests, bad ones and those that, neither one nor the other, could do with being run again.
Simply because a test is "unsighted" does not mean that it is not subject to a wide range of possibly fatal biases. For example, what order do you play your samples in - does it matter if the "better" sample is played first (so the perception of the "poorer" one is coloured by information present in the better one)? It might do, it might not - you need to test for it and design accordingly. How long do you play the samples for (allowing for what, if memory serves, psychologists used to call "interference" - maybe they still do)? How loud? What may be missed at one volume may be apparent at another. How closely do you need to match volume levels? How to ensure that your kit is even good enough to reveal the differences (pace the Meyer & Moran fiasco)? How important is subject selection?
Does it matter how often you repeat the same sample? In other words, how random is "random"? Truly random sample selection will lead to sessions where one alternative is played far more often than the other and even to those where one only is played. The pressure on a subject in such sessions to identify some instances as different would be intense - the test scenario introduces a psychological bias of which the experimenter is blissfully unaware by design. It may be "common sense" to be unaware of this possibility but it seems more like poor technique to me.
I wonder if it's even crossed your mind that failing to control any of these variables can render the results useless. Sloppy standards may be quite the thing when deciding between brands of salad dressing, I don't know - but they will let you badly down in this field. No amount of bluster will conceal that.
So, in short, I have considerable sympathy with those who say that it is too difficult to design reliable experimental tests and that, absent the resources, it is better to eschew sloppy experiments and look to subjective rankings instead. They have serious flaws as well as benefits - but at least we know what the flaws are.
You have only revealed that you don't understand the subject very well.
That's the second or third time you've said that in our exchanges. With respect, it's insolent. I might well be poor on experimental technique - but how would you know?
Tell me that you believe, as Gordon said, that the effect of excess digital processing power on analog audio has "well been shown," or whatever that was, and you will have pushed beyond misunderstanding into foolishness.
Why do you always change the subject? I never suggested anything of the sort, have already dismissed GR's tests and have argued often enough the opposite point.
This post is made possible by the generous support of people like you and our sponsors:
Follow Ups
- RE: Let me get this out of my system... - Ryelands 07:57:42 03/01/10 (1)
- RE: Let me get this out of my system... - Phelonious Ponk 16:12:04 03/01/10 (0)
Top of Page ]
[ Contact Us ]
[ Support/Wish List ]
[ Copyright © 2024, Audio Asylum, all rights reserved ]
