|
Audio Asylum Thread Printer Get a view of an entire thread on one page |
For Sale Ads |
In Reply to: DBT Testing, Can it be done at home?? posted by Robert Hamel on November 10, 2002 at 18:03:11:
I've seen that site before, it's too bad all the fill information is mising as those are obviously highlight slides for a presentation.....looks like it was a grat prresentation.Anyway, in my opinion, in order to properly conduct a DBT, the devices to be compared need to be switched around rapidly and quietly during a musical passage(or even pink noise). This, to me, excludes the unplug/plug-in methodology most would tend to use at home.
Rapid switching helps to eliminate the problems with our short aural memory and during rapid switching if there is an audible difference, it would stand out immediately.
Follow Ups:
See:http://emusician.com/ar/emusic_objective_subjectivity/
and
http://www.harman.com/wp/pdf/AudioScience.pdf
Yes that is a very important issue as well as a standard hearing test to know if you would be a reliable testor.
> > rapid switching if there is an audible difference, it would stand out immediately.Rapid switching is useless unless you have the proper stimuli and have determined what to listen for via long term training. In a nutshell, this is the critical error in the methodology in nearly every 'casual' DBT. Complicating this issue are components that do not lend themselves easily to rapid switching such as a power cord. To test this, you'd need identical components and cabling and generally speaking this is far beyond the capabilities of a 'home' test.
All the tests I've participated in did not require training. The whole purpose of rapid switching is to create that "proper stimulus" - the brain seems to respond well to the transient differential, if one exists that is audible. And I've found this to be the case everytime, if there is an audible difference, it really stands out at the moment of switchover.
All the tests I've participated in did not require training.Great. What then are you trying to prove? Conclusively demonstrating that Joe Sixpack can't hear the difference between a Sony HT receiver and Krell amplifier doesn't say anything as to the actual differences between the two pieces of electronics. All is succeeds in doing is exploring the limitations of Joe's expertise.
"Great. What then are you trying to prove?"What are YOU trying to prove? All you seem to want to do is make useless arguments.
Please stay on topic. His point is reasonable. If the test is invalid, then the conclusions drawn from it are equally erroneous.
I would if it weren't for the assinine Joe Sixpack comments. Go harrang someone else.
Please substitute "John Doe" for Joe Sixpack. Is that OK? The point I was trying to make is that by using the average untrained ear you will do little other than to prove that most folks cannot hear (or care to, for that matter) the kinds of audible differences we are discussing.
Thanks.
Thanks for the clarification E, let's all keep it respectful.Anyway, I thought this quote from a link that Jitter posted was applicable and clearly points out the value of training.
We measured discrimination thresholds of 13 to 32 naive listeners in a variety of conditions during a pretest and again, 2 weeks later, during a posttest. Between those two tests, we trained a subset of listeners 1 h per day for 9 days on a single ILD or ITD condition. Listeners improved on both ILD and ITD discrimination. Improvement was initially rapid for both cue types and appeared to generalize broadly across conditions, indicating conceptual or procedural learning.....
Yeah, but if you actually look at the data set provided, it doesn't really bear this out.
If the goal of such a test is to evaluate the ultimate performance of a high resolution component, then any listeners used in such a test need to have an educated ear.Likewise, if you were trying to evaluate the ultimate performance of a Ferrari, then you need trained drivers. Having a school bus driver and your Uncle Buck to do the testing will reveal little of a high performance car's envelope.
Seems to me you raise an interesting point. Auto magazines regularly print comments of professional drivers on handling capabilities of automobiles. No one ever thinks to suggest that there is no value to such comments because they weren't performed under "blind" conditions (for obvious reasons).Yet it may be no more practical for an audio reviewer to test a particular piece of equipment under review with similar units the reviewer has heard in the past. Yet, audio reviewers are often criticized for writing reviews of sighted auditions.
I'm currious as to where people see possible distinctions between the two.
an apparently unwritten rule by which the DBT proponents always fail to state the equipment (amplifiers/speakers/etc) used in the test. One such inmate comes to mind who chooses a characteristic "mysterious" style wherein he continually fails to state his point of reference and refuses direct questions as to what he has heard. He prefers instead to hide behind "I didn't say yes or no to that" sort of posts. I was rather suprised, however, when I followed a DBT link posted by him that did break the "components used" silence. What I discovered is that the tester used patently mediocre "pro" gear for testing where he stated that "it is assumed that the speakers are phase coherent". Yeah, let's (attempt to) draw conclusive tire performance determinations of the Pirelli P Zero on a Hyundai!I have yet to see any credible test where the tester:
A) Identified the components in the reproduction chain
AND
B) Used anything beyond mediore
I would be pleasantly surprised to be proven wrong.
But then again, the unwritten rule against such disclosure may well prevent that from occurring.
> > All the tests I've participated in did not require trainingI find this to be analagous to saying that drug trials do not require a control group. Training involves many things, one of them is testing the listener for the ability to discern known JNDs. If, for example, a listener can not realibly hear known, scientifically proven differences about say 10Khz, then using that individual to test for suspected differences in that area would be pointless.
> > The whole purpose of rapid switching is to create that "proper stimulus"
No, the purpose of rapid switching is to avoid short term memory issues and has nothing to do with the stimulus. The test tone or material would be the stimulus and must be designed to test where suspected differences lie. Again, if that is above 10Khz as an example, then playing tones in the 1Khz area or bass instruments would be pointless.
What you're describing is surely fine as a casual test to help you determine gross differences, but I wouldn't call it scientific or in line with accepted methodologies.
" find this to be analagous to saying that drug trials do not require a control group."Yes, but it isn't the people that are the control element. And it wouldn't be people that are the control element in audio testing either.
"Training involves many things, one of them is testing the listener for the ability to discern known JNDs."
Testing isn't training. I agree with a limited number of participants you would want to know what their hearing apability is - as well as with more targeted testing.
"If, for example, a listener can not realibly hear known, scientifically proven differences about say 10Khz, then using that individual to test for suspected differences in that area would be pointless."
I agree.
"No, the purpose of rapid switching is to avoid short term memory issues and has nothing to do with the stimulus."
I disagree, in part, the rapid switching in itself creates a stimulus. If it didn't we wouldn't react to it. Which is why such great lengths must used to level match, among other things, to avoid erroneous results.
"The test tone or material would be the stimulus and must be designed to test where suspected differences lie. Again, if that is above 10Khz as an example, then playing tones in the 1Khz area or bass instruments would be pointless."
Well, okay, I can agree there. Now I'm really curious, you seem to be around 10kHz all the time, any particular reason?
" in line with accepted methodologies."
With all the arguing about methodologies, I wonder if thre truely are any, at least pertaining to cable DBTs.
I'm getting the impression that we are on different pages regarding this very broad subject and the discussion will continue in a fragmented mode unless we agree on what we're specifically talking about. So perhaps hitting the reset button and starting over might be in order.
No, there's nothing significant related to the example using 10Khz samples, it was simply to illustrate the point.> > With all the arguing about methodologies, I wonder if thre truely are any, at least pertaining to cable DBTs.
I think this is the key point and it has absolutely nothing to do with cables. Yes, there are accepted methodologies if one intends to publish the results of an audio DBT. These methodologies are every bit as rigorous as those applied to medical DBTs and any other scientific experiment. Without them, the experiments are not repeatable by others. If the experiment can not be repeated, then obviously, it's neither scientific nor it is scientifically accepted.
On the topic of listener training, let me try this another way. In many respects, the training is somewhat analogous to calibrating your equipment or measuring device. From this perspective, using an untrained listener and inappropiate material is akin to performing a test with uncalibrated equipment. You can do the experiement, but your results will be suspect and quite likely will prove unrepeatable.
That's my problem when people point to cable DBTs. I don't know of any that meet the standards of accepted science in terms of documentation and use of accepted methods.
"No, there's nothing significant related to the example using 10Khz samples, it was simply to illustrate the point."OK, thanks.
"I think this is the key point and it has absolutely nothing to do with cables. Yes, there are accepted methodologies if one intends to publish the results of an audio DBT. These methodologies are every bit as rigorous as those applied to medical DBTs and any other scientific experiment. Without them, the experiments are not repeatable by others. If the experiment can not be repeated, then obviously, it's neither scientific nor it is scientifically accepted."
Just a note, one problem with quick switching is if it occurs just before a large change in dynamics, one segment will appear louder than the other.
And training is ABSOLUTELY required.
JJ - Philalethist and Annoyer of Bullies
A test without training is like a bicycle without tires.
JJ - Philalethist and Annoyer of Bullies
I think we've now gone down a different training path here. One of test facilitator vs one aimed at the listening subject.....
Not sure which you think I was talking about.I'll say it again. ALL of the reliable evidence I've seen suggests very very strongly that when a listener has learned to hear a difference, they can do so in a short test.
Learning HOW to hear that difference, however, is a learning process. There is no replacement for that process, and it is not fast or simple.
JJ - Philalethist and Annoyer of Bullies
Well, a couple lines up you were talking about learning how to use the equipment. Is there an assumption that the listener is also the facillitator??
For whatever reason, a later post where you corrected yourself didn't properly come up on refresh until after my last posting. Sorry about that chief.
Part of the listener's learning should be for when the LISTENER switches between stimulii at the LISTENER's will.Switching should be solely under the LISTENER's control, not under the experimenters. All the experimenter should do is observe, quietly, and help if the subject (that's the listener) is having trouble with the equipment.
The TEST CONDITIONS should be arranged from outside so that the experimenter and the listener are both ignorant of the exact test conditions, that's what DBT means, of course, so the experimenter should not be setting up the test conditions in the listener's presence.
JJ - Philalethist and Annoyer of Bullies
"Part of the listener's learning should be for when the LISTENER switches between stimulii at the LISTENER's will.Switching should be solely under the LISTENER's control, not under the experimenters. All the experimenter should do is observe, quietly, and help if the subject (that's the listener) is having trouble with the equipment."
Well, that works for a single listiner test, but when you have 10 at a time.....
One subject at a time.In general, ever running more than one subject at a time is a very bad thing. It can lead to all sorts of problems, related to the listener's reactions as well as interfering in the listening process.
JJ - Philalethist and Annoyer of Bullies
That won't work too well when there are 50 to 150 people to run through the test......done bunches, never been a problem.
Your experience very strongly contradicts mine.
JJ - Philalethist and Annoyer of Bullies
Sorry OM, they aren't my tests and they most certainly are meaningful.
However, I understand what you are driving at. But then, we don't screw around with only a handful of people, casual testing uses no less than 50 people, 150 minimum when serious. There is more to it than that, but I can't divulge more without getting into what some would consider proprietary information.
This post is made possible by the generous support of people like you and our sponsors: