General Asylum General audio topics that don't fit into specific categories.

---

[ Follow Ups ] Thread: [ Display  All  Email ] [ General Asylum ]

---

[ Alert Moderator ]

---

Some Comments on Method/Procedure

Posted by Jon Risch (B) on September 29, 2003 at 21:04:25

In Reply to: Wanted : participants for long-term ABX test posted by Klaus on September 29, 2003 at 03:46:23:

What you seem to be proposing is a VERY limited and low reliability test scenario.

A single cut on a CD-R, sent to only 15 people?

This would only provide a set of 15 single trial test runs, which is niot even enough for warm-up on a well-designed test.

Aside from the very low number of test trials (15), and the extremely small number of trials per subject (1), there are a large number of significant problems with the proposed method.

1. How will the source material be deemed to be suitable for showing up cable differences, especially between two particular cables, to a wide range of different people, all with different systems?

If you select the music to be used, and you master the CD's, how will anyone else know that the CD-R's would have the inherent capability to reveal any specific set of cable pairing differences?

Since you yourself have not been able to detect said differences, and have even gone so far as insist that you can not hear absolute polarity, there is strong evidence that your capacity to distinguish such subtle sonic differences is not too good.

Hanging the test results on just one musical selection of your choosing, and depending on you for the final quality of the CD-R, and it's inherent resolving power, will not suffice. There MUST be independant confirmation of the CD-R's quality, and in the very least, that experienced and trained listeners can actually seem to hear a difference between A and B under sighted conditions on high resolution systems or their own system.

I would suggest that any subject that was sent a CD-R, and could not initially tell a differnce in sound between A and B, stop right there, and go no further, as there is in essence, no initial "claim" to hear a difference in the firstr place. This could easily be due to one of several things, all unrelated to whether or not folks can actually hear the difference between cable A and cable B. It could be due to a lack of inherent resolving power of the subjects system, lack of sufficient recording quality, or a lack of experience on the part of the subject with listening tests in general, and specificaly, an ABX scenario.

2. What kind of ABX/DBT training will the subjects be given? What instructions on how to listen and what to do?

Experienced DBT administrators will tell you that training is a large part of any blind listening test, and without it, the test results are very likely to end up a worthless null result.

Again, at the least, a complete and proper set of instructions would be needed, and they would have to be made available for public inspection and critique, or common mistakes might not be avoided, providing yet another opportunity for a completely worthlerss null.

3. One trial for each listening subject is an absurd procedure. Even if the very best compromise "universal" music were somehow determined, and each listening subject is an experienced listener, and has been exposed to ABX procedures, etc., having only one trial per person is a waste of their time, and of anyone who would care to follow the 'results' of the test.

I would say that each subject should get at least 9 different musical selections, and the A, B and X version of these tracks present, for a total of 27 individual tracks.

I cover more of the details of what I feel are the best length's for the tracks in posts I made in Prop Heads some time ago, see the URLs at the bottom of this post for details.

4. I would include as many subjects as possible (to the limits of the number of people volunteering), and only limit the numbers to below say 35 or 50 people, in order to get as much data as possible. I realize that this means that the "umpire" will now have to correlate 35 or 50 times 9 sets of data, BUT, if you get more than two volunteer umpires, split up how many folks they each have to process, and let them get together after they have documented their data in order to present it.
Unless you, and the subjects and umpires, are willing to do this kind of a test well enough to have any sort of significant scientifc meaning, it just becomes another half-hearted null result with absolutely no meaning at all, and certainly not the meaning that some might tend toward if an overly simplistic test consisting of 15 single trial that came up with null results, and was promptly accepted as yet another 'scientific' negative result by you and others.

If you are not going to do it well, don't bother to do it at all, as it is just misleading and bad science.

For more on DBT/ABX testing flaws, and common mistakes, see:
http://www.audioasylum.com/forums/prophead/messages/2190.html
and at:
http://www.audioasylum.com/forums/prophead/messages/2579.html
and
http://www.audioasylum.com/forums/prophead/messages/2580.html

Jon Risch

This post is made possible by the generous support of people like you and our sponsors:

Schiit Audio

---

[ Top of Page ] [ Contact Us ] [ Support/Wish List ] [ Copyright © 2025, Audio Asylum, all rights reserved ]

---

[ General ] [ Speakers ] [ Tubes ] [ Vinyl ] [ Digital ] [ Hi-Rez ] [ Video Asylum ] [ Cables ] [ Tweaks/DIY ] [ Music ] [ Films ]

---