http://www.revelspeakers.com/i/listening_lab.pdf

I would like to stick to if testing is possible and not get into issues on the audibility of diferences between components.

Thanks

 It CAN be done. It's NOT, repeat NOT easy. - jj 21:39:30 11/13/02 (0)
 Issues - audioengr 21:12:48 11/11/02 (1)
 Re: Issues - eico1 18:55:05 11/12/02 (0)

...

Show full thread outline!

 It CAN be done. It's NOT, repeat NOT easy. - jj 21:39:30 11/13/02 (0)
 Issues - audioengr 21:12:48 11/11/02 (1)
 Re: Issues - eico1 18:55:05 11/12/02 (0)
 Some DBT are not needed.The change is apparent - NEAR SOTA 09:37:16 11/11/02 (0)
 Yes, but - Jitter_by_Coffee 07:03:29 11/11/02 (32)
 Also, just for further reference on the issue of "training" and importance of DBTs... - Phil Tower 11:14:39 11/11/02 (0)
 Good point on the training - Robert Hamel 10:36:08 11/11/02 (0)
 Re: Yes, but - Rod M 08:13:13 11/11/02 (29)
 Re: Yes, but - Jitter_by_Coffee 18:47:15 11/11/02 (28)
 No training? - E-Stat 14:57:58 11/13/02 (10)
 Re: No training? - Jitter_by_Coffee 20:46:04 11/13/02 (9)
 Re: No training? - Rod M 08:18:18 11/14/02 (5)
 Re: No training? - Jitter_by_Coffee 09:18:18 11/14/02 (4)
 My apologies for the choice of generic name - E-Stat 13:52:22 11/14/02 (3)
 Re: My apologies for the choice of generic name - Jitter_by_Coffee 09:24:38 11/15/02 (0)
 Re: My apologies for the choice of generic name - Rod M 14:14:38 11/14/02 (1)
 Re: My apologies for the choice of generic name - Jitter_by_Coffee 09:23:50 11/15/02 (0)
 Simple - E-Stat 03:43:21 11/14/02 (2)
 Re: Simple - Phil Tower 12:34:25 11/16/02 (1)
 I'm amused by... - E-Stat 17:27:33 11/16/02 (0)
 Your tests are meaningless.... - Rod M 08:03:23 11/12/02 (16)
 Where was I. - Jitter_by_Coffee 21:08:13 11/13/02 (2)
 Re: Where was I. - Rod M 07:59:14 11/14/02 (1)
 Re: Where was I. - Jitter_by_Coffee 09:04:05 11/14/02 (0)
 Re: Your tests are meaningless.... - eico1 20:11:04 11/12/02 (11)
 That's what training is for, to learn how to use the equipment. - jj 18:02:47 11/13/02 (10)
 Sorry, that's wrong. That's PART of what training is for. - jj 18:03:59 11/13/02 (9)
 Re: Sorry, that's wrong. That's PART of what training is for. - Jitter_by_Coffee 20:49:18 11/13/02 (8)
 LISTENER TRAINING is what I'm talking about. - jj 21:32:35 11/13/02 (7)
 Re: LISTENER TRAINING is what I'm talking about. - Jitter_by_Coffee 09:20:05 11/14/02 (6)
 Doggonit, I hate computers sometimes..... - Jitter_by_Coffee 09:24:17 11/14/02 (5)
 Err, to be clear... - jj 17:56:59 11/14/02 (4)
 Re: Err, to be clear... - Jitter_by_Coffee 09:26:35 11/15/02 (3)
 Err. No. If we're talking about the most sensitive tests... - jj 18:16:07 11/15/02 (2)
 Re: Err. No. If we're talking about the most sensitive tests... - Jitter_by_Coffee 07:03:03 11/18/02 (1)
 Really? What's your controls and their result? - jj 09:36:38 11/18/02 (0)
 Uhm, sorry, but yes, they are. - Jitter_by_Coffee 15:51:42 11/12/02 (0)
 Re: DBT Testing, Can it be done at home?? - Tom N. 06:36:46 11/11/02 (0)
 Re: DBT Testing, Can it be done at home?? - Monstrous Mike 20:43:39 11/10/02 (2)
 Re: DBT Testing, Can it be done at home?? - Phil Tower 05:45:15 11/15/02 (0)
 Glad to see I am not alone. - Robert Hamel 10:30:38 11/11/02 (0)
 Re: DBT Testing, Can it be done at home?? - Timbo in Oz 19:49:33 11/10/02 (0)

;-)

timbo

To get a meaningful result from an audio DBT is probably beyond the scope of anybody here. So we have to stick with what is likely, what is probable and what is reasonable, based on what we know. That's why we have such disagreements.

Now on the other hand, if you are conducting DBT to aquire information to be used to answer that age old question... Then yes, the room acustics may be an important factor.

Just my opinion at the moment, subject to change without notice.

Tom N

Anyway, in my opinion, in order to properly conduct a DBT, the devices to be compared need to be switched around rapidly and quietly during a musical passage(or even pink noise). This, to me, excludes the unplug/plug-in methodology most would tend to use at home.

Rapid switching helps to eliminate the problems with our short aural memory and during rapid switching if there is an audible difference, it would stand out immediately.

Rapid switching is useless unless you have the proper stimuli and have determined what to listen for via long term training. In a nutshell, this is the critical error in the methodology in nearly every 'casual' DBT. Complicating this issue are components that do not lend themselves easily to rapid switching such as a power cord. To test this, you'd need identical components and cabling and generally speaking this is far beyond the capabilities of a 'home' test.

• Importance of training

I usally use recordings that I am very familar with that will give me the best idea of what to look for when I switch out cables.

If you know what things should sound like then you are better able to have a point of reference to go by.

A week ago somebody was kind enough to send me some Silver Plated wire to try.The top end was great and the mid's ok ,but when I tried Roger Water's Amused to Death CD everything fell apart.The characters that were suppose to be at the corner's of the room were now coming from the speaker's and the Slay Bells that suppose to sweep through the room were not.

Other things I look for are the "Shhh!" or vocals that have alot of "S" sounds in them.My favorite for this test is Stan Ridgeways 16 Tons which has numerous words that start with "S" St.Peter,Soul,Sold etc..If they drag and don't sound real then I know there is something wrong someplace.

Cymbals are another thing to look for.In the 1812 Overture,can you hear the scale of cymbals ,chymes and Bells at the climax.Can you distinguish them apart from one another.

Then there are the Female Vocal's.There are many to choose from,but for me Karen Carpenter is the one I mostly use because I had seen her sing when I was young and my ears were awhole lot better.Does she sound real.

Many factor's can make a system not right and just because things are not right in one system does not mean they will not be right in another.Synergy is the key ,but once you have it the magic has been caught.

I am not a believer in the A/B testing done in controlled atmosphere's only because you are out of your normal element and you do not always have particular recordings that you are use to.

It makes more sense to me to have them done in your home setting with recordings you know.If you could do a DBT in that setting you stand a better chance then having it done anywhere else. Plus having them in the system over time will give you a better idea of what the product sounds like to your satisfaction.

JMO

http://emusician.com/ar/emusic_objective_subjectivity/

and

http://www.harman.com/wp/pdf/AudioScience.pdf

I do agree that the listeners must be carefully selected and that having trained listeners would certainly make the tests more efficient and the results more credible. I believe that for any DBT's, trained listeners are beneficial, but carefully selected listeners can be just as effective.

However, I dont agree with their music list. I have some of these recordings and the ones that I have are in no stretch of the imagination good recordings. I believe that good recordings are critical to getting meaningful results.

Also, I am a strong believer in "Blind-folded" DBT tests. I feel that visible stimulus interferes with effective listening, particularly in a such a contrived performance venue as a living-room etc..

Finally, since the object of such studies is to compare and determine "more accurate" components, how can this be done if the starting set of components is not reference quality? This is sort of like cart before the horse. You need the reference components to determine the best musical pieces and to evaluate a single component in a more complex system, yet the system itself is not optimized until after the optimum components are identified and included in the system..... This is what makes this a difficult multi-dimensional problem that typically takes reviewers and designers decades to figure-out. This is probably the most pervasive reason why it is difficult to derive meaningful conclusions from home DBT's.

I find this to be analagous to saying that drug trials do not require a control group. Training involves many things, one of them is testing the listener for the ability to discern known JNDs. If, for example, a listener can not realibly hear known, scientifically proven differences about say 10Khz, then using that individual to test for suspected differences in that area would be pointless.

>>The whole purpose of rapid switching is to create that "proper stimulus"

No, the purpose of rapid switching is to avoid short term memory issues and has nothing to do with the stimulus. The test tone or material would be the stimulus and must be designed to test where suspected differences lie. Again, if that is above 10Khz as an example, then playing tones in the 1Khz area or bass instruments would be pointless.

What you're describing is surely fine as a casual test to help you determine gross differences, but I wouldn't call it scientific or in line with accepted methodologies.

Damn...lightning....more later........

steve

Great. What then are you trying to prove? Conclusively demonstrating that Joe Sixpack can't hear the difference between a Sony HT receiver and Krell amplifier doesn't say anything as to the actual differences between the two pieces of electronics. All is succeeds in doing is exploring the limitations of Joe's expertise.

rw

What are YOU trying to prove? All you seem to want to do is make useless arguments.

Yes, but it isn't the people that are the control element. And it wouldn't be people that are the control element in audio testing either.

"Training involves many things, one of them is testing the listener for the ability to discern known JNDs."

Testing isn't training. I agree with a limited number of participants you would want to know what their hearing apability is - as well as with more targeted testing.

"If, for example, a listener can not realibly hear known, scientifically proven differences about say 10Khz, then using that individual to test for suspected differences in that area would be pointless."

I agree.

"No, the purpose of rapid switching is to avoid short term memory issues and has nothing to do with the stimulus."

I disagree, in part, the rapid switching in itself creates a stimulus. If it didn't we wouldn't react to it. Which is why such great lengths must used to level match, among other things, to avoid erroneous results.

"The test tone or material would be the stimulus and must be designed to test where suspected differences lie. Again, if that is above 10Khz as an example, then playing tones in the 1Khz area or bass instruments would be pointless."

Well, okay, I can agree there. Now I'm really curious, you seem to be around 10kHz all the time, any particular reason?

" in line with accepted methodologies."

With all the arguing about methodologies, I wonder if thre truely are any, at least pertaining to cable DBTs.

I'm getting the impression that we are on different pages regarding this very broad subject and the discussion will continue in a fragmented mode unless we agree on what we're specifically talking about. So perhaps hitting the reset button and starting over might be in order.

I'll say it again. ALL of the reliable evidence I've seen suggests very very strongly that when a listener has learned to hear a difference, they can do so in a short test.

Learning HOW to hear that difference, however, is a learning process. There is no replacement for that process, and it is not fast or simple.
JJ - Philalethist and Annoyer of Bullies

I very strongly suspect that this is not anything like a proof of feasibility for the general public.

The problem is not running the test, per se, it's the experience, etc, involved in the setup, design, and so on. The devil here is most definately in the details, which is why I won't give people an "exhaustive" list of what they need to do in a DBT, as any list can be circumvened, most likely by pure accident.
JJ - Philalethist and Annoyer of Bullies

Likewise, if you were trying to evaluate the ultimate performance of a Ferrari, then you need trained drivers. Having a school bus driver and your Uncle Buck to do the testing will reveal little of a high performance car's envelope.

rw

>>With all the arguing about methodologies, I wonder if thre truely are any, at least pertaining to cable DBTs.

I think this is the key point and it has absolutely nothing to do with cables. Yes, there are accepted methodologies if one intends to publish the results of an audio DBT. These methodologies are every bit as rigorous as those applied to medical DBTs and any other scientific experiment. Without them, the experiments are not repeatable by others. If the experiment can not be repeated, then obviously, it's neither scientific nor it is scientifically accepted.

On the topic of listener training, let me try this another way. In many respects, the training is somewhat analogous to calibrating your equipment or measuring device. From this perspective, using an untrained listener and inappropiate material is akin to performing a test with uncalibrated equipment. You can do the experiement, but your results will be suspect and quite likely will prove unrepeatable.

That's my problem when people point to cable DBTs. I don't know of any that meet the standards of accepted science in terms of documentation and use of accepted methods.

OK, thanks.

"I think this is the key point and it has absolutely nothing to do with cables. Yes, there are accepted methodologies if one intends to publish the results of an audio DBT. These methodologies are every bit as rigorous as those applied to medical DBTs and any other scientific experiment. Without them, the experiments are not repeatable by others. If the experiment can not be repeated, then obviously, it's neither scientific nor it is scientifically accepted."

Agreed.

....lemme go kick the server or something.

rw

Anyway, I thought this quote from a link that Jitter posted was applicable and clearly points out the value of training.

We measured discrimination thresholds of 13 to 32 naive listeners in a variety of conditions during a pretest and again, 2 weeks later, during a posttest. Between those two tests, we trained a subset of listeners 1 h per day for 9 days on a single ILD or ITD condition. Listeners improved on both ILD and ITD discrimination. Improvement was initially rapid for both cue types and appeared to generalize broadly across conditions, indicating conceptual or procedural learning.....

• Different patterns of human discrimination learning .....

Switching should be solely under the LISTENER's control, not under the experimenters. All the experimenter should do is observe, quietly, and help if the subject (that's the listener) is having trouble with the equipment.

The TEST CONDITIONS should be arranged from outside so that the experimenter and the listener are both ignorant of the exact test conditions, that's what DBT means, of course, so the experimenter should not be setting up the test conditions in the listener's presence.
JJ - Philalethist and Annoyer of Bullies

While the uncertainty over this issue affords some of us the opportunity to see ourselves making, what we believe, are inteligent and articulate arguments for our particular point of view, this issue of cables has been an extremely divisive issue in our hobby for over 20 years, and it would be nice to see some level of certainty enter the picture.

You have said the companies should do the testing and I have argued that individual companies probably don't have the resources. But I do think it fair to ask why the industry has not pooled resources to support such testing.

Assuming that will not be happening any time soon, it would be nice if some extremely wealth audiophile gadfly, with money to burn, would appear on the scene as a white knight and fund such testing. Any candidates out there?

I suspect if those of us who spend so much time arguing over this subject would just spend half the time we do arguing holding neighborhood car washes instead, we'd already have far more than enough to fund such testing.

Switching should be solely under the LISTENER's control, not under the experimenters. All the experimenter should do is observe, quietly, and help if the subject (that's the listener) is having trouble with the equipment."

Well, that works for a single listiner test, but when you have 10 at a time.....

In general, ever running more than one subject at a time is a very bad thing. It can lead to all sorts of problems, related to the listener's reactions as well as interfering in the listening process.
JJ - Philalethist and Annoyer of Bullies

Yet it may be no more practical for an audio reviewer to test a particular piece of equipment under review with similar units the reviewer has heard in the past. Yet, audio reviewers are often criticized for writing reviews of sighted auditions.

I'm currious as to where people see possible distinctions between the two.

I have yet to see any credible test where the tester:

A) Identified the components in the reproduction chain

AND

B) Used anything beyond mediore

I would be pleasantly surprised to be proven wrong.

But then again, the unwritten rule against such disclosure may well prevent that from occurring.

rw