[
Top of Page ]
[ Contact Us ]
[ Support/Wish List ]
[ Copyright © 2007, Audio Asylum, all rights reserved ]
[ General ] [ Speakers ] [ Tubes ] [ Vinyl ] [ Digital ] [ Hi-Rez ] [ Video Asylum ] [ Cables ] [ Tweaks/DIY ] [ Music ] [ Films ]
|
Home
/ FAQ
/ News Classifieds / Events |
Audio Asylum Thread Printer |
Get a view of an entire thread on one page |
24.63.6.232
In Reply to: RE: Complettely OT - ABXing absolute polarity posted by Axon on September 24, 2007 at 22:32:57
Yeah, by the time you got 10/10 your results were pretty solid, since the chances of that happening randomly are less than 1 in 1000. The advantage of using headphones like those, which seal in the ear canal, is that any very low-frequency stuff in the mix, which might tip you off if it's asymmetrical, will get through.
Did you really do 140 trials with the same source material, or did you use a variety of songs? I hope the latter, unless you enjoy, say, banging on your head with a board (or, as Jerry Lewis once said, "because it feels so good when I stop!"). -- E. Brad
I did everything with "Hamburger Train". Once I found that one sample I was too scared of failure to hunt down any other reproducible sample. I figured that just getting a positive result out of a single popular release was significant enough.
"I did everything with "Hamburger Train". Once I found that one sample I was too scared of failure to hunt down any other reproducible sample. I figured that just getting a positive result out of a single popular release was significant enough."
Aiee! cried Mowgli softly.
You deserve some kind of prize for that; I'm just not sure what it should be.
It's entirely reasonable to do a large number of trials when you discover a source that works for you, of course. That just means you're searching diligently for a way to quantify and establish what you claim to be hearing. We gave our subjects chances to do that when there were few enough in a trial, which was most of the time -- they got to pick the material they thought was best.
But still... after 25 trials, when you've already got good data... it's time to stop, for god's sake. Unless of course you *like* hearing it that many times, which is something I don't care to think about. -- E. Brad
The second test I attempted was at home instead of at work, and it failed miserably (7/16 or something like that). That bugged the hell out of me, so several of the tests following that were simply attempts to isolate the system change that triggered the change in result. That chase alone resulted in an extra five tests.
The chase was not for naught: I couldn't get a good statistical result at all with one pair of headphones, and getting good results for another sound card was a hell of a lot harder. That could be the start of an extremely interesting test: One might be able to correlate the difference in results to some objective THD criteria.
After that, people commented that I shouldn't have run the tests without fixing the number of trials beforehand, because I could have cherry-picked where the test stopped. That alone necessitated redoing the tests at N=32.
It's worth noting that I've done a lot fewer ABX tests since I did that. :)
Axon:
Well, you were following the evidence, like a good experimenter. Of course that way is often tiresome and annoying. If your blind test method gives you the answer after each trial, then there is a possibility that you could affect the overall result by deciding to stop at a certain point, but there's a relatively easy way around that: Decide beforehand how many trials you're going to do, and stick to it. There's a mathematical rescue of a sort if your results are positive too. A score of 10/10 is quite conclusive, and you can stop there.
Yes, you could get 8/10, be disappointed, and choose to go on to try for a better score, which is a bit slippery. But if you miss any at all, you know the effect is at least not obvious. If you do another ten, and again get 7 or 8, you're in that gray area, where effects probably make themselves heard, but not all the time. Remember too that in the case of polarity, asymmetrical distortion in your playback transducer aids audibility, so the device on which you can't hear it may be a better reproducer, not a less revealing one.
Finally, about that hypothetical 8/10 score: The usual confidence level that is asked for is 95%, and 8/10 is 94.5%, so that one is close enough unless you're writing it up for publication. After getting 8/10, I'd also call it fair to do another ten and total them -- but not to do another 10 and throw it out if you do less well. It's pretty easy to grasp what is cheating here and what isn't. -- E. Brad
Top of Page ]
[ Contact Us ]
[ Support/Wish List ]
[ Copyright © 2007, Audio Asylum, all rights reserved ]