|
Computer Audio Asylum Music servers and other computer based digital audio technologies. |
|
|
|
Computer Audio Asylum Music servers and other computer based digital audio technologies. |
|
In Reply to: RE: Thanks posted by Christine Tham on March 2, 2010 at 15:30:44:
It's not really about losing control, it's about ensuring that your personal biases and knowledge do not influence the results.An experimenter who cannot control "personal bias" in tests of this sort is out of his/her depth - it's undergraduate stuff. The fact that some who perform ever-so-scientific tests in the audio field are out of their depth is sad but true.
A classic example of this is "psychic" experiments where they found out that subjects were getting the correct answer simply by reading the body language of the experimenter.
The story is often told the other way round - it's the "psychic" who reads the body language of the subject :>) Whatever, even if serious "psychic" research (so help me, there's a Chair of Parapsychology at Edinburgh University) is, for most of us, an oxymoron, competent researchers first addressed the issue 100-odd years ago when there was a craze for things "spiritual" in Britain and loads of sensible folk keen to debunk it.
But it is also possible that you can do everything right (use all the right buzzwords and even apply them correctly) and still get it wrong. An example of this is that infamous study which "proved" by exhaustive double blind testing that people can't differentiate between SA-CD and DVD-Audio.
The Meyer and Moran paper (M&M - the one that "proved" we can't differentiate between RBDC and SACD) did almost nothing right: it was deeply flawed from the off. See e.g. these posts:
see: http://db.audioasylum.com/cgi/m.mpl?forum=pcaudio&n=57852
and:
http://db.audioasylum.com/cgi/m.mpl?forum=pcaudio&n=57904
from July last year where I try to explain why. (Sorry about all those ’ thingies - they turned up after a recent AA archive revamp.)
(Ironically, I was also debating with PP who resorted as readily then as now to ad hominem abuse and a mix of sarcasm and Google.)
Problem was, the researchers didn't take into account that it takes time for someone to get used to the differences. And hearing acuity can be
improved with training.M&M was flawed on several fronts, not just on "training" issues (though I'm sure you're right to raise them). The authors' competence in other fields didn't protect them from schoolboy howlers in this one.
An average person would find it difficult to distinguish between MP3 and CD, and yet people can train themselves to hear the differences. The study ignored the fact that there were at least two subjects they tested that were able to spot the differences consistently - they just brushed those subjects aside rather than investigate the reasons why these two can tell the difference and others couldn't.
Assuming we're talking of the same paper, M&M didn't get five per cent significance for any subjects so I'd hesitate to accuse them of brushing aside awkward results.
On a personal note, I find that mp3 quality varies from execrable to acceptable. I have some mp3-sourced recordings that sound really quite good but others, made from what I know to be a good source (as I have the CD), I simply cannot listen to. Whether that's inherent in the process, down to bad engineering, a bit of both or something else altogether, I can't say.
I suspect that the mp3 v CD thing would actually prove tricky to test properly because you have two variables at play - quality differences within and between formats, not to mention vested interests.
This shows how powerful experimenter bias is in clouding thinking. The experimenters wanted to believe there was *no* difference, so they deliberately interpreted the results to fit their prejudice.
Experimenter bias is powerful in clouding the thinking of incompetent experimenters only. Any half-way decent design addresses it early doors. I read M&M several times before writing the above critique and have just re-read it. I find no grounds for suggesting that the authors "brushed aside" any results and I don't think they fairly can be accused of "deliberately interpreting" them to suit preconceptions. They genuinely got them - but they were always going to get them because of flaws in the experiment's design.
Inevitably, those who wanted to believe the results did so without reading the paper with much care. Now those folk were biased . . .
Edits: 03/03/10
This post is made possible by the generous support of people like you and our sponsors:
Follow Ups
- RE: Thanks - Ryelands 04:14:22 03/03/10 (4)
- RE: Thanks - Christine Tham 10:16:33 03/03/10 (3)
- RE: Thanks - Tony Lauck 12:36:53 03/03/10 (2)
- Thanks Tony for the link to the paper (nt) - Christine Tham 14:26:50 03/03/10 (0)
- RE: Thanks - Ryelands 13:03:56 03/03/10 (0)
Top of Page ]
[ Contact Us ]
[ Support/Wish List ]
[ Copyright © 2024, Audio Asylum, all rights reserved ]
