A flaw in A/B testing...(very long)

General Asylum

General audio topics that don't fit into specific categories.

207.2.10.250

Well, first the disclaimer: This is just what happened to *ME* in *MY* system. I hope you can learn from my experience, and maybe one or two of you will gain some useful insight. YMMV. Void where prohibited by law. Certain restrictions apply.
Ok, let me start with a little background. I was a cable objectivist and am a card-carrying skeptic (CSICOP). But I needed a little more "oomph" and "zing" from my system. It just wasn't very engaging. The system in my office is the one I listen to most, but it's space-limited and cost-constrained compared to my main system, which I am very happy with.
My main system (which does double duty as my HT system) consists of Martin-Logan SL3's, assisted by a Velodyne HGS-12, driven by B&K Components AVR-202, with Sony DVD and CD-carousel transports (both are using the B&K's onboard DAC). Cables by DH Labs. For both two-channel music and DD HT, it sounds wonderful.
My office system is still in flux, to a degree, but is currently a NAD 314 integrated, with Vienna Acoustics Haydn speakers and an Adcom GCD-700 cd carousel. My interconnects were RatShack gold.
I didn't at first believe cables would make a difference, but experimenting in my main system lead me to the inescapable fact that I (and my Wife, and two other friends) could call the DH Labs speaker cables vs. Monster XPHP in blind testing in the neighborhood of 16-18/20. After that experience, I came to believe that some of the "zing" missing from my office system may have been due to the RatShack cable. Granted, the NAD is not the last word in microdynamics, but it's respectable, and I wasn't getting the level I expected from it.
So, after reading a number of posts here on the board, I picked up some Harmonic Tech interconnects, and proceeded to the test bench.
The Adcom is equipped with dual outputs: One fixed and one variable. Perfect! I thought, and hooked the Harmonic cables to one output and the Ratshack to the other. Each of these was in turn hooked to an input of the NAD, and the connections were covered so as to be unknown to others. The test subjects were allowed to listen to whatever music they liked, for as long as they liked, and switch inputs whenever they wanted. Scores were kept, and the subjects tried to identify the cable attached to each input.
The three subjects (my wife (dragged into testing), a friend of mine (budding audiophile himself), and I (audiophilia nervosa in beginning terminal stages) scored as follows:
Correct/Trials
9/20, 11/20, 11/20.
Hmm. I thought. Totally random results. Well, that about wraps it up for the Harmonic cables, I'll send 'em back.
Then, quite by accident, while in the process of unhooking everything and doing some casual listening, I noticed that after disconnecting the Harmonic cables (so I was now running RatShack only), the sound stage seemed deeper, with more resolution. I was shocked; there was an immediate, palpable difference. So, I reattached the Harmonic cables instead.
WOW! This was much much better! Microdynamics and detail shot through the roof. Imaging snapped into a rock-solid focus directly between the speakers, with soundstaging beyond the outside of the speakers and much more depth to the sound.
AHA! I thought. Further experimented showed that the NAD, like many inexpensive preamps, suffers from cross-talk on other inputs. In the case of a very strong CDP signal, the cross-talk is VERY pronounced, and my hypothesis is that the two separate signals into the amp at the same time blurred the signal through excessive cross talk and distortion.
The blind test was repeated, this time by having a judge change the cables when the subject wanted to change. This time, the results were a little different:
19/20, 20/20, 18/20
While I know this isn't a perfect test protocol, I'd say it's fairly solid, and there is an absolutely undeniable correlation there. There *WAS* an audible difference between those two cables.
The moral of this story is not that the objectionists are always wrong. Many cables at the high-end are frauds, and are built (like my wife says) with "Hooey, bullsh!t, and unobtainium." But I can't help but wonder how many ABX tests that showed no cable difference were flawed by this same subtle bug in the protocol.
Comments welcome.

This post is made possible by the generous support of people like you and our sponsors:

Schiit Audio

Topic - A flaw in A/B testing...(very long) - James Mancini 09:22:30 05/29/99 (5)

Re: A flaw in A/B testing...(very long) - Jon Risch 08:29:17 05/30/99 (3)

Related question...splitting outputs from DVD/HDCD player... - Lee M. 08:58:30 06/01/99 (2)

Re: Related question...splitting outputs from DVD/HDCD player... - Jon Risch 18:41:54 06/02/99 (1)

Jon...thanks for the guidance! (nt) - Lee M. 06:40:01 06/03/99 (0)

Re: A flaw in A/B testing...(very long) - mikem 12:22:15 05/29/99 (0)

You can not post to an archived thread.