 
|
Audio Asylum Thread Printer Get a view of an entire thread on one page |
For Sale Ads |
|
|
Audio Asylum Thread Printer Get a view of an entire thread on one page |
For Sale Ads |
61.209.162.69
In Reply to: Reliability for determining what? posted by E-Stat on April 23, 2006 at 19:14:08:
Reliability for determining if there is a difference, I would presume. Given two cables that differ in price by an order of magnitude, one would expect to hear a difference either real or imagined. The purpose of the DBT would be to establish that the difference heard is real. If you cannot, then Occam's razor applies ...Good science is about repeatability and independent observation. Neither of these are in prominence in the audio (High end) world. Hence, the skepticism and well-founded accusations of "bad science".
Follow Ups:
I get the concept, just not the assumptions behind the presumed accuracy behind any DBT I've seen for cables. Null outcomes prove nothing.rw
nt
Drug trials do not involve perception. A drug is not considered effective if it "makes the patient feel better." A drug is considered effective if reduces or eliminates measureable symptoms of a disease; kills the disease, prevents the disease from recurring etc.When you're speaking of audio equipment, you're simply comparing perceptions, i.e., do people perceive an aural difference between component A and B? It is widely reported that in many cases, these tests show that people don't perceive a difference. What people forget is that this kind of a DBT is merely a test of perception, and the nature of the "thing" suppossedly under test is inferred from the perception. But the logical assumption that people want to make from this datum -- that there is no difference between the two things -- does not, fact, follow.
Consider some well-known art forgeries. If they're good, many, many people do not see a difference between the original and the forgery -- even though they may have a much longer time to study the forgery and the original than the typical tester has to compare two devices in an aural DBT. Nevertheless, the fact that many -- even most -- people cannot distinguish the fake from the real painting does not mean that the "fake" is "real."
On the more mundane level, consider Coca-Cola and Pepsi-Cola. If you read the fine print in the "preference survey ads" ("people who tried them both, preferred the taste of Pepsi"), you'll see that the "fine print" in the ad contains the disclaimer "among those who expressed a preference." That is there because the majority of the participants in these surveys do not express a preference between the two cola drinks -- probably because they can't tell them apart -- and the FTC said it was deceptive to run an ad without the disclaimer, because it implied than a majority of the tasters preferred (Pepsi) when, in fact, they didn't.
Does that mean that Coca-Cola is the same as Pepsi-Cola? I don't think so.
Well, I know that this is going to fall on deaf ears (pun intended) but here goes ---Scientific methodology demands that experiments are repeatable by independent researchers at different times.
Wherever human perceptions are involved, to remove the risk of perceptions being affected by (for example) the brand or price of the thing under test, a double-blind test is conducted. Tests have proved that a single-blind test is not good enough (the experimenter provides clues to the tester).
The above is well known and is used successfully in many diverse fields.
In high-end audio, there is a serious lack of scientific rigour and this makes some people very angry. Others (subjectivists) couldn't care a hoot for this and are happy discussing the virtues of thousand dollar cables.
Things would be fine if there were no appeals to science or engineering to justify why one component is better than another or justifies its high price. If subjectivists simply said that this was an art and not a science, they would be on much stronger ground, i.e., there would be nothing to argue about. This is the situation that actually exists today except that subjectivists will readily ignore materials science and elementary knowledge of electrical engineering while at the same time invoking molecular level and radio-frequency level effects in an effort at "explaining" why they think a particular cable is magical.
Well, now, that's a fool's errand, isn't it when it comes to human perceptions!You don't want to acknowledge the truth of my basic point -- a DBT of audio components is, first and foremost, a test of perception not of the components themselves.
The question that is answered is: Do the testers perceive a difference between test subject A and test subject B?
How reasonable is it to expect that fine tests of human perception are going to be repeatable to any degree of precision? Sure, people can reliably and repeatedly tell the difference between red and blue. But can they tell the difference between shades of violet, when the two color samples are not held side-by-side?
I guess you don't want to talk about a DBT of Coke vs. Pepsi or of a good art forgery vs. the real thing, do you?
Well, here's a blindingly obvious point: if the testers are deaf, a DBT of two audio components will never reveal any difference. Are you going to argue that such a DBT does, in fact, establish that the two components under test are (or sound) the same?
I thought not.
Tests have proved that a single-blind test is not good enough (the experimenter provides clues to the tester).You mean like 3 out of 4 Dentists prefer Dentyne?
This is the situation that actually exists today except that subjectivists will readily ignore materials science and elementary knowledge of electrical engineering ...
Materials science? Elementary electrical engineering ?(childlike actually) You're kidding, right? You can't possibly believe that current metrics used for audio have any relevancy to perceived audio quality WHATSOEVER. If you really do, then I sentence you to life listening to a Crown IC-150 preamp! Measures great...sounds absolutely dreadful.
We observationalists make observations in the current dearth of any useful measured performance criteria. History has proven time after time that there have been many a clueless pocket protector engineer that hasn't a foggiest notion as to what really matters. The first generation SS designers were all over negative feedback. Wrong. The first generation digital guys touted "perfect sound forever". What a joke!
rw
You snipped what I said to change the basic thrust of the sentence.The point was that they (subjectivists) ignore basic science while providing "explanations" based on esoteric branches of science that are not really applicable in the case in question.
I did not state (anywhere) that I thought that the current specifications have (or not) any relevance to perceived audio quality. So I'll pass on the Crown amp if you don't mind ...
The "perfect sound forever" was a joke because if any one had actually measured a waveform through a A/D converter, pulled it back from a D/A converter and subtracted the original signal, they would have noticed that the difference was non-zero. There were studies showing this by the way and these were not by "subjectivists" but by the same engineere that you are denigrating.
The current state of high-end audio is so confused that there are people who swear by directional cables for speakers. If that does not fly in the face of all science, what does?
The solution to the problem is (by careful testing) to find the variables that can be measured and correlate reliably with perceived sound quality. It is not a solution to abandon measurements altogether or to rely on subjective testing which often depends on the level of Scotch in the bottle next to the listening chair.
Let's start with the answer:The solution to the problem is (by careful testing) to find the variables that can be measured and correlate reliably with perceived sound quality.
We don't yet know what we don't know. Who said anything about abadoning measurements? The Absolute Sound was founded on the observation (that still holds true today over thirty years later) that they STILL DO NOT CORRELATE to qualitative differences.
Then the question:
If that does not fly in the face of all science, what does?
Let me see if I understand your point. Despite the fact that we really cannot draw any meaningful conclusions over the sound quality of a component using present scientific measurements, we do know all the sonic effects of cables? I think we know the least about their ultimate effects.
That reminds me of a funny story when I was a kid. My cousin worked part time at a gas station and had a conversation with one of the veteran grease monkeys about condensation on Coke bottles. My cousin told him the source. His response: "I know damn well it doesn't come out of the air!"
rw
The "basic thrust" it now appears to me is that you're just another "angry" anti-cable guy.You should really try to get over that. You see many audiophiles, myself included, are fully cognizant that much of the 'chatter' surrounding cables and cable marketing is a pile of bunk, and moreover there's currently little to separate the wheat from the chaff.
But that doesn't change the fact that, for whatever the reasons, cables *do* make a difference. So we simply live with it and go about choosing our cables the way we choose everything else ... with out ears!
Will the explanations, the science, finally catch up? Well it would be nice, yet we in the hobby has lived without the science to explain much more then just why cables matter, we've adjusted to it ... it's like breathing ... better than choaking anger.
interpretation on the situation it seems to me. For one thing you paint those who are "very angry" in a rather passive light. But just study Richard BassNut Greene's post above and we see a demonstration, a rather extreme demonstration I'll admit, of much of narrative of such "angry" folks.Essentially they paint a picture of a hobby as a collection of individuals who's common experience, a experience spanning decades, is mostly if not completely an exercise in mass self-delusion. They commonly bemoan the behavior of said hobbyists, call their activities an embarrassment, and on and on.
And their beloved, but unfortunately ill practiced, blind testing methodologies are the magic pills that this population refused to partake of to cure itself.
But how about the facts? Well Stereophile itself has done investigations into blind testing. They have discovered that much care and effort is required to design such tests. They have documented results, results that *do* demonstrate differences between components.
Yet this work has demonstrated that such is a non-trivial exercise, for example great care and effort is required to design and conduct a test that demonstrates differences between power amplifiers that are widely accepted to be easy to discern under casual listening circumstances.
Hence is it a surprise that such a methodology hasn't become a popular tool for evaluating components? Of course to the ardent objectivists, those that would deny much of the experience of audiophiles, those that believe that well designed electronics are virtually indistinguishable, etc., etc. would likely say that the since the effort is great the differences must be mostly insignificant. But this is a self serving exercise.
Much the same can be said of objective measurements. Audiophiles generally accept that objective tools are lacking when it comes to providing measurements that correlate well with common experience. For the experience denying objectivists this is just another case of the comical behavior of the self deluded masses.
But how about the facts? Well despite the current limits of objective measurements it is still true that meaningful work and results have been accomplished. As just one trivial example, it has been demonstrated that odd order harmonic distortion products that are generated in certain circuit topologies correlate with subjective observation of 'harsh' sound, and the studies have show that even relatively small amounts of such can be detrimental, especially when compared to relatively massive levels of even order harmonic distortion that are found to be subjectively benign. This study has lead to improved circuit topologies, ones that are careful in the utilization of feedback for example. The discovery of the importance of jitter in digital audio another example.
But nontheless the simple set of specifications that fill even today 'spec. sheet' are generally found to be inadequate for determining the sonic performance of an electronic component. Yet this is also disputed by the staunch objectivists who seem to feel otherwise.
So it goes. Who then are the deniers of reality, who self-deluded? I guess each individual must decide based upon their own personal experience. And on that point I would suggest that when the objectivists, the nayasayers, deny the 'common experience' of audiophiles as a group they should first ask why that experience ... e.g. that not all electronic components sounds the same, that all power amplifiers do, that CDPs don't, and not just in cases of gross aberrations ... is 'common' in the first place; although that's not to say there is unaminity of experience, of preferences, etc., the lack of which seems to be a popular "proof" of the self delusion amongst objectivists who merely underestimate the complex world of human perceptions, values, and judgements, etc. But anyway, such reflection doesn't seem to feature large in their discourse, rather they spent consider effort coming up with imaginative explanations for the mass self-delusion, explanations that easily qualify as conspiracy theories and much other things in the way of fanciful narratives.
"Well Stereophile itself has done investigations into blind testing. They have discovered that much care and effort is required to design such tests."This very statement flies in the face of your argument. So basically not that much care and effort is required or taken to properly document the claimed differences in any review?
Richard, old3randr and myself are not disagreeing that there are differences between components and cables. What we are saying is that if the differences are so great as to come up with the superlatives that are always claimed, then the people who make them should also be able to take the same two components and use the same superlatives to determine which is which. Otherwise their initial claims are suspect.
If two components sound differences are so minute that you can't objectively tell them apart, then the frequency response differences inherent in most recordings will more than offset the differences. Only a fool would upgrade. Or an audiophile.
I am not sure if I should reply to such a deeply felt emotional post. I have no desire to cause any offence.However, for what it's worth ---
1. You called Ben Goldacre's article "sensationalistic trash". It is no such thing. The entire series is about bad science and there is no attempt at sensationalism.
2. If, as you state above, it is true that "simple set of specifications that fill even today 'spec. sheet' are generally found to be inadequate for determining the sonic performance", then the answer is to find the specifications that are adequate and not by abandoning measurements (as in the case of modern day turntables).
3. I was originally responding to a post about cables and not about power amps. However, the point remains that even if you have valid reasons to distrust a particular DBT ( an implementation ), there is no reason to doubt the validity of DBTs in general (i.e., the methodology ).If I understand you right, you are mainly complaining as in (3) and the insensitivity and rudeness of the "objectivists" and I do not want to argue about that.
1. "Bad science", "sensationalistic trash"I'd be willing to drop the second, but not the first. The reasons have now been articulated repeatedly. Frankly, however, I must admit I don't see much distance between the two.
2. No one is suggesting dropping specifications, if you read that suggestion in my post then I was unclear on that issue. I do hope the state of the art of correlating measurements with subjective observation improves.
However, my personal perspective as an audiophile, as a consumer if you like, is that they don't do much for me at this time. I've used my ears to select my equipment, and I have generally, and especially of late, been enormously pleased with the results.
I trust you can understand there is no fundamental contradiction in those statements.
3. Cables, amplifiers, CDPs ... I see no fundamental difference.
Please understand that I do not 'doubt' DBTs in general. However, it is basic commonsense that one must apply the proper tools for the particular job, in this sense 'science' is no different that 'fixing ones' sink'. I've seen nothing to suggest that the rigorous controlled blind testing methology provided a sensitive and practical tool for judging questions of interest to audiophiles.
Moreover in light of the fact that the methodology has seen so little practice in this field, currently almost none, makes the whole issue rather moot. In that sense the great interest of the objectivists in the methodology rather makes them appear to be grasping at straws.
Finally, you should probably refect on the fact that most audiophiles are fully comfortable with their reality, we don't place our audio hobby purchasing decisions on the same level as things that require scientific validation. It is after all a hobby, the goal of which is to enhance enjoyment of music by means of reproduction in the home. Those suffering "anger" from such activity had perhaps determine if there is some other hobby which might better provide them with satisfaction and a pleasant experience.
---
"deeply felt emotional post"? Hmmmm ...? Thanks for pointing that out, obviously something that needs work as "cool and rational" is what I was aiming for.
:)
"will readily ignore materials science and elementary knowledge of electrical engineering while at the same time invoking molecular level and radio-frequency level effects in an effort at "explaining" why they think a particular cable is magical."Only too true!
____________________________________________________________
"Nature loves to hide."
---Heraclitus of Ephesus (trans. Wheelwright)
You state "A drug is not considered effective if it "makes the patient feel better." A drug is considered effective if reduces or eliminates measureable symptoms of a disease; kills the disease, prevents the disease from recurring etc."I respectfuly disagree; certainly in the case of analgesics, anxiolytics and antidepressants, efficacy is indeed determined by how the patient feels.
Moreover, it is worth noting that with the possible exception of some infections diseases, there are very few drugs that "kill" a disease, or "prevent it from recurring." Despite the best efforts of those in the bio-medical community, the best we can do is usually to treat symptoms, not the underlying disease.
Larry Borden
As if "infectious diseases" were a relatively insignificant portion of the universe of diseases.As if vaccines against viruses (can you say "polio"?) are "usually treat[ing] the symptoms."
As if various cancer therapies do not shrink tumors or have a statistically significant record of preventing recurrences, e.g. tamoxifen and relaxofene for breast cancer?
I accept your point about analgesics, anti-depressants and so on, but suggest that they are a small subset of the class of substances we call "medicinal drugs." Not surprisingly, their "effectiveness" in patients is also the most variable.
to the question at hand, namely practical considerations.The drug companies not only have vast resources to conduct such tests but in fact their motivation to do so is guaranteed by the requirement to meet relevant legislation.
After all, JA articulates in his editorial (and in the material referenced [links] in same) that it is possible to design and conduct sensitive controlled blind conditions tests, even if they do have shortcomings in replicating casual listening conditions, yet in any event such tests are non-trivial and require great care, effort, and resources.
Thus to simply say do as the drug companies is a rather extreme example of pie-in-the-sky thinking.
There is an utter lack of such quality studies.rw
The lack of quality studies is what this argument is about if I am not mistaken.
of the merits of a methodology in lieu of the practice that demonstrates such is suspect to say the very least.The fact that the other 'blind' testing methodologies are well established in other fields, e.g. drug testing, does not in any sense validate it's usage in home audio. Borrowing directly from JA's editorial without unduly abusing his intent I hope ... many of the vocal proponents appear "... to be making the naïve assumption that the mere fact that a test is blind inherently ... confers legitimacy on the test and its results. That assumption, I [JA] suggest, is 'bad science'—even voodoo."
That is the point.
Basically, nothing John Atkinson says invalidates the methodology of double-blind testing. Sure, he casts oblique aspersions on the technique as in your quote above. However, this is an opinion and nothing more.You may choose to believe it. I don't.
This post is made possible by the generous support of people like you and our sponsors:
Top of Page ]
[ Contact Us ]
[ Support/Wish List ]
[ Copyright © 2025, Audio Asylum, all rights reserved ]
