Critic's Corner

Discuss a review. Provide constructive feedback. Talk to the industry.

Return to Critic's Corner

A Paucity of Certitude Within the Community of Critics.

205.188.116.16
Posted on October 19, 2009 at 11:45:39
regmac
Audiophile

Posts: 1843
Joined: April 7, 2002
 Admittedly, I have some sympathy for the position that reviewers are being intellectually dishonest (not to mention lazy) by refusing to adopt a credible methodology that confirms or disproves their sighted impressions. Should a reviewer refuse to adopt such a methodology he is doing a disservice to himself and his readers. Adopting such measures will control for any preconceived bias that may distort the review process (sunshine being a wonderful disinfectant).

Suppose reviewer Smith spends two months evaluating a power amp, only to discover at the end of that time that he is unable to distinguish the review amp -- Blind -- from his reference? It seems there are but two conclusions one can draw from this scenario: 1) the two amps sound the same; 2) Smith’s powers of discernment aren’t what they used to be.

Should Smith find himself facing this quandary more and more, readers could be forgiven for insisting that poor Smith be turned out to pasture and a more astute reviewer take his place. Meanwhile, objectivists would merely shrug and say, “We told you so. What’s more, there’s not a damn thing wrong with Smith’s hearing.”

Of course objectivists suffer from their own rigid dogma. Commonsense would suggest it is absurd to insist all components sound alike. The sheer odds of that circumstance occurring strain credibility. Moreover, Messrs. Fremer and Atkinson have demonstrated (on at least one occasion) that they can distinguish -- Blind -- cheap cables from expensive models. Of course, just because *they* were successful doesn’t mean every critic at Stereophile would be successful. (More on this aspect later.)

Again, adopting a scientific approach would do much good insofar as confirming a critic’s impressions and ending this tedious debate. And that is one reason such standards will never be adopted. Both sides prefer a moronic status quo to the sobering alternative of adopting measures that would allow critics to validate their sighted listening impressions.

What is lacking in both camps is a healthy and rigorous philosophical skepticism vis-à-vis one’s own beliefs. In seeking knowledge the philosopher does not begin with a favorite idea or belief system. Instead, he brings a certain disinterest to the whole process and is content to let the best idea win in his epistemological quest. The question for the subjectivist is, “How do I *know* what I hear is accurate and devoid of any preconceived bias?”

The intellectually honest response is to admit one doesn’t *know* until one confirms his listening impressions (blind). To insist otherwise is to forsake the scientific method for a quasi-religious approach. Now I have nothing against religion, in fact I think it a fine thing. But isn’t the goal of the audiophile to strive for the best sound based on accurate impressions, rather than seeking refuge in a comfortable belief system that reinforces his prejudices?

The problem with critics and audiophiles is that they never get to the bottom of things. When it comes to examining their own bias, most of them exhibit about as much intellectual curiosity as an anemic guppy. Such blind adherence to one’s own deeply ingrained dogma is the sort of stance that a suicide bomber would find admirable. But those of us interested in factual certitude have no interest in joining either cult.

Please. No infantile excuses about “forced choices” or “null results” or, my personal favorite: “But it takes months to detect subtle differences…” Childish excuses such as these are unconvincing and not germane to my example above. Again, reviewer Smith has months to evaluate the review sample against his reference.

At the end of that time Smith could enlist a colleague’s help in confirming his listening impressions by having his colleague connect one of the amps (without Smith knowing which one). At that point Smith would be free to take as long as needed in validating his sighted impressions.

Moreover, for the purposes of this exercise Smith would listen to his favorite recordings. Those he is most familiar with. Smith’s room. Smith’s recordings. Smith’s reference. Smith’s review sample. And all the time Smith needs. No one is pushing Smith for the dreaded quick response. Either Smith can identify his familiar reference when he hears it or he can‘t. It’s just that simple.

Let’s assume Smith’s powers of discernment are superb. If the subjectivists are correct, then Smith should have little trouble differentiating between the two amps. If not, then something is quite wrong with the subjectivist position. Perhaps the amps are so similar as to evade human detection. Which is not to say that *all* amps sound the same, just these two.

Perhaps it will take but 15 minutes for Smith to render a judgment as to which unit he’s hearing. Perhaps it will take all afternoon or several afternoons. If the latter, then any perceived “differences” are probably so slight as to interest only the most neurotic among us. Not to worry, commonsense has never been this hobby’s strong suit and most audiophiles will understand should Smith think it appropriate to pen 500 words separating the pepper from the fly dung.

(It would be imperative that another critic, reviewing for the same publication, undertake the same task for obvious reasons.) Yes, such measures would be time consuming. So what? It’s not uncommon for two or more reviewers to evaluate the same piece of gear. And I’m told it can take a full year from the time a reviewer takes delivery of a product until the time his evaluation is published.

Sensitive to such criticism, it’s at this juncture that critics will throw up their hands and resort to their fallback position by insisting that they are only offering their opinion and that their journal exists as much for its entertainment value as anything else. Pity. I suspect audio journals are capable of more. Nor do I think it too demanding for readers to insist that audio journals adopt a methodology that is at least as credible as that of the Wine Spectator.

I suspect the chief reason audio critics resist such a credible methodology is due to the fact that they would -- gasp! -- get it wrong on occasion. Moreover, it wouldn’t take long for readers to catch on to the fact that certain critic’s possess greater powers of discernment than others.

Currently, no reviewer enjoys any one-upsmanship over another in this area. That would change if a methodology were implemented that could easily confirm or falsify a reviewer’s sighted impressions. And because some critics would boast a better batting average than others is yet another reason said methodology will never be embraced by audio journals.

I disagree with you on this statement, posted on October 20, 2009 at 20:53:42
JustJoe
Audiophile

Posts: 128
Location: Essex
Joined: July 8, 2009
 "Of course objectivists suffer from their own rigid dogma. Commonsense would suggest it is absurd to insist all components sound alike. The sheer odds of that circumstance occurring strain credibility."

Really? Then prove it with scientific methodology. Do a double blind listening test. If you see no differences, then does that not imply that differences are unmeasurable. Then it;s up to the overactive imaginations of the reviewers to come up with "differences". Old cliches suffice here (SETs with single driver crossover less speakers, etc etc).

Just Joe
Blind testing of gear works: If it doesn't sound better with your eyes closed, then it doesn't sound better!
"scientific methodology" - audio DBT is such only in crippled minds,..., posted on October 21, 2009 at 08:14:18
carcass93
Audiophile

Posts: 2634
Location: NJ
Joined: September 20, 2006
 ... screwed up by lack of common sense, excessive cheapness, and worshipping moronic "accountability in audio" idols.

While I agree with you, one should keep in mind, posted on October 20, 2009 at 20:48:40
JustJoe
Audiophile

Posts: 128
Location: Essex
Joined: July 8, 2009
 These things are obvious to anyone with a decent high school science education (which rules out most audiophiles I guess),

BUT

the way the audio industry is structured, critics simply cannot be critical. They CANNOT write bad reviews. OR they won't be critics much longer. And then what will the poor losers do, if they can't play with free gear?

But don't worry, none of tis will matter in a year or two. The "audiophile industry" is dying. Maybe it needs a "stimulus".
Or maybe not, it's doing a fine job of dying without any help from the government.

Just Joe





Blind testing of gear works: If it doesn't sound better with your eyes closed, then it doesn't sound better!
"I guess" - guess what, you guessed wrong. But judging ..., posted on October 21, 2009 at 08:10:19
carcass93
Audiophile

Posts: 2634
Location: NJ
Joined: September 20, 2006
 ... by the nonsense you believe in, and the nonsense you're spreading here, it certainly rules out you.

What is it with know-nothings, that they must always appeal to science in various forms, to support they pathetic blathering?

RE: While I agree with you, one should keep in mind, posted on October 21, 2009 at 05:22:11
andy19191
Audiophile

Posts: 158
Location: Europe
Joined: June 26, 2008
 > But don't worry, none of tis will matter in a year or two. The "audiophile industry" is dying.

If it is dying why are devices like valve driven ipod docks, fancy usb cables, turntables, etc... growing in number in department stores? Or is your definition of "audiophile industry" different from mine?
RE: While I agree with you, one should keep in mind, posted on October 21, 2009 at 11:12:47
JustJoe
Audiophile

Posts: 128
Location: Essex
Joined: July 8, 2009
 Maybe you just live in a different planet than I do.

It's pretty clear to me it's dying. No one really lsitens to music anyomre, it's just not "cool" to sit down and "spin a few records on your hi fi".

No, it's cool to "download" or "rip" a few tracks on your "ipod" and listen with your cheap, plastic "ear buds" or on your cheap plastic "bose IPOD soundbox".

Only uber nerds with no children and lots of income in really big cities are indulging in the hobby, and they are doing it more for show than a real love for listening to music.

So...it's dying.

Just Joe
Blind testing of gear works: If it doesn't sound better with your eyes closed, then it doesn't sound better!
RE: While I agree with you, one should keep in mind, posted on October 21, 2009 at 15:00:59
andy19191
Audiophile

Posts: 158
Location: Europe
Joined: June 26, 2008
 > Maybe you just live in a different planet than I do.

Yes I think that may be true.

> It's pretty clear to me it's dying.

It depends what it is. For you, what audio hardware is audiophile and what is not?

> No one really lsitens to music anyomre,

So dancing to music, playing music, socializing in a musical environment, etc... doesn't count. What matters is sitting in a chair listening to commercially produced music coming from an overly expensive audiophile shrine?

> it's just not "cool" to sit down and "spin a few records on your hi fi".

I confess to spinning a few of the records that I have not yet got around to transferring to the hard disk because the process is too bloody time consuming. If I was richer I would pay someone to transfer the remaining ones (apart from the dross which seems to have grown bafflingly large) because getting up and spinning records gets in the way of listening to music.

> No, it's cool to "download" or "rip" a few tracks on your "ipod" and
> listen with your cheap, plastic "ear buds"

This is what I do when exercising and I can assure it is an enormous improvement on listening to my puffing and panting.

> or on your cheap plastic "bose IPOD soundbox".

What about the expensive(ish) valve ipod docks?

> Only uber nerds with no children and lots of income in really big
> cities are indulging in the hobby, and they are doing it more for show
> than a real love for listening to music.

I do not understand the relevance of the cities?

> So...it's dying.

I see no evidence that audiophile belief is dying. My point was that it now has a presence in the consumer sector (and the proaudio sector) albeit still a fairly modest one.

What probably is reducing substantially is the ultra expensive, increasingly silly, specialist hi-fi shop audiophile hardware that grew after the stereo boom. Developing more extreme beliefs within a closed community may keep the juices flowing for the existing believers but it makes it harder for the mainstream to jump the widening credibility gap and replenish or grow the community. Which makes the apparently growing presence in the consumer and pro sectors interesting. It may be related to where much of the hardware now comes from?
I had a hard time following this, posted on October 20, 2009 at 12:44:16
Bruce from DC
Bored Member

Posts: 18019
Location: U.S. Capital
Joined: October 13, 1999
Contributor
  Since: March 1, 1999
 but I read some of your follow-ups to others' comments, so I think I understand your point.

1. It's a lot easier for Wine Spectator to have multiple critics sampling and reviewing a bottle of wine, than it is for an audio magazine to have multiple critics reviewing the same piece of audio equipment. I don't think I have to explain why. Positive Feedback and, to a lesser extent, The Absolute Sound used to do that, some. But they don't any more. I suspect the unwieldiness of it all overwhelmed them.

2. The audio mags are in the business of selling criticism. People buy the magazines (at least the print ones) because they want to read what so-and-so has to say about thus-and-such loudspeaker. That's the foundation of their business. If people don't read the magazines, advertiser's don't' want to spend money buying ads that no one will see. Ultimately, therefore, they succeed or fail depending upon how credible their audience finds the criticism.

3. Back in the day, I was a regular reader (and sometime subscriber) to Stereo Review. The late Julian Hirsch was a big tester and measurements guy. He rarely said much subjective about any of the stuff he was testing, and even then his comments were bland. In fairness, back in his heyday, lots of stuff that went on Mr. Hirsch's test bench flunked the elementary tests that he subjected them too. That does not seem to be true today. And, I think everyone can agree that, while a piece of electronics that exhibits 1% THD sounds better than on that exhibits 10% THD; it's not the case that a piece of electronics that exhibits 0.1% THD reliably sounds better than a piece of electronics that exhibits 1% THD. In other words, the traditional distortion and amplitude response measurements that Mr. Hirsch used are no longer useful discriminators among pieces of electronics . . . because the overall level of performance is much better.

3. The big "audio" mags of the past -- Stereo Review, High Fidelity, Audio -- were much less focused on equipment than today's big audio magazines. A lot of people probably bought those magazines for the articles about music and the record reviews. So, implying that the current subjectivist audio magazines ought to go back to the models of the 1960s and 1970s involves a false comparison.

4. Stereophile at least, does produce a suite of objective measurements of most of the stuff it reviews (except some things reviewed exclusively in some of the columns) and JA often comments on the apparent relationship (or lack thereof) of his measurements and the reviewer's comments.

Why this quixotic effort to eliminate subjectivity (for "certainty") in audio? Let's face it, for a million reasons, the playback of the recorded performance is not -- and can not be -- the same as the recorded performance itself and, of course, for many recordings, the "recorded performance itself" never existed. I'm not trying to make the case for complete subjectivity, but, it seems to me that the quest for absolute objectivity ultimately will not result in a tool that anyone finds to be useful.

It's worth noting that folks in the business of selling audio equipment who have scientifically tried to correlate measurements with customer preferences, still use listening panels as the final arbiters of what should be manufactured. I'm thinking of Revel here. Not coincidentally, Revel speakers usually measure pretty well in Mr. Atkinson's tests; and the subjective reviews of them are pretty positive also. And, I suspect, even for electronics manufacturers, ultimately what you hear is what the designer believes "sounds right," even though just about all electronics have pretty respectable traditional measurements.

In short, the fallacy of "objectivity" is the belief (and I say "belief" because there is no proof of this proposition) that measurements capture all of the audible performance of a piece of audio equipment. This postulate leads objectivists to argue that people who claim to hear differences between two pieces of equipment that measure the same are delusional.

Blind testing (either double or single) doesn't offer a way out, either, because it requires the tester to operate at a high level of sophistication and subtlety. Consider a problem that comes up in the art world regularly: two paintings -- a genuine (e.g. Rubens) masterpiece and a very good fake. In some cases, these paintings can not be distinguished, visually. Only an analysis of the paint, close examination of brushstrokes, etc. distinguishes between the two. In a DBT of these two paintings, does a null result (i.e. that the test panel can not tell that they are different) mean that the two are products of the same artist? No. It means that the test panel lacks the level of subtlety and discrimination to tell the two apart.

A DBT of audio components has the same problem: the limiting factor is the subtlety of discrimination of the testers. So, a null result does not necessarily mean that they sound the same. It simply means that the panel of listeners couldn't hear differences between them.

So, DBT leads you right back to the place you were trying to avoid -- the quirks and limitations of the testers.

Now, if you're saying that a particular reviewer is biased, that's something that is pretty easily discoverable by the reader. Just go listen to a product that reviewer X has reviewed. Does Joe Reader hear the same things that reviewer X heard? If, yes, then Joe Reader will trust reviewer X. If, no, then he will not trust reviewer X. If that happens too many times, reviewer X will lose credibility, as will his magazine, etc.

But, for the life of me, I don't see how measurements, DBTs, and such offer a way out of this box.


Well, counselor, that would help to explain why your remarks are not germane to my posts. :), posted on October 21, 2009 at 07:39:42
regmac
Audiophile

Posts: 1843
Joined: April 7, 2002
 Exhibit A: “So, implying that the current subjectivist audio magazines ought to go back to the models of the 1960s and 1970s involves a false comparison.”

I imply no such thing. In my original post I state: “Commonsense would suggest it’s absurd to insist all components sound identical. The sheer odds of that circumstance occurring strain credibility.”

Exhibit B: “But, for the life of me, I don't see how measurements, DBTs, and such offer a way out of this box.”

I make no mention of measurements and my hypothetical example (reviewer Smith) does not amount to a DBT. There’s no need to muddy the water, counselor. Why is it every time I have this discussion (on here) the knee-jerk reaction is to center the discussion around DBTs? I’m not advocating for DBTs and I’m well aware of their limitations. But when a critic claims that the preamp he’s evaluating casts a wider soundstage than his reference, “extending a couple of feet beyond the outer edges” of his speakers, he’s offering more than mere opinion. He is making what for him amounts to a factual assertion. This significant difference should be observable in a relaxed setting without his knowledge of which preamp he’s hearing. Is this so unreasonable?

Exhibit C: “Consider a problem that comes up in the art world regularly: two paintings -- a genuine (e.g. Rubens) masterpiece and a very good fake. In some cases, these paintings can not be distinguished, visually. Only an analysis of the paint, close examination of brushstrokes, etc. distinguishes between the two. In a DBT of these two paintings, does a null result (i.e. that the test panel can not tell that they are different) mean that the two are products of the same artist? No. It means that the test panel lacks the level of subtlety and discrimination to tell the two apart.”

I take your point, but fail to see how this is germane to my posts. In my reply to Peter I go out of my way to emphasize the fact that I’m not after “subtlety,” but rather *significant* claims on the part of the critic. Here’s the quote: “When a critic waxes euphoric about the sonic attributes of (xyz) product, insisting that the component in question offers a significant improvement over his reference, then he should have little trouble identifying those improvements (blind). To plead that these significant improvements are not distinguishable (blind) doesn’t pass the smell test.”

I’m not overly concerned with claims of *nuance* as anyone can readily see. That said, I do insist that perceived improvements of major significance should be identifiable (blind). Do you disagree?

Exhibit D: “Stereophile at least, does produce a suite of objective measurements of most of the stuff it reviews (except some things reviewed exclusively in some of the columns) and JA often comments on the apparent relationship (or lack thereof) of his measurements and the reviewer's comments.”

I’m aware of that fact. Again, what has that to do with my point, to wit, that a relaxed (Blind) evaluation in reviewer Smith’s listening room -- after months of evaluating a component Smith feels offers a quantum leap forward in terms of quality, compared to his reference -- should pose little trouble for Smith insofar as identifying either unit.

Why is this simple control check anathema for audio critics but not for other critics? Wouldn’t such a practice bring some much needed credibility to Smith’s anecdotal observations? Socrates tells us the unexamined life is not worth living, but subjectivists aren’t having any when it comes to confirming their sighted impressions. :)

Exhibit E: “I'm not trying to make the case for complete subjectivity, but, it seems to me that the quest for absolute objectivity ultimately will not result in a tool that anyone finds to be useful.”

Agreed. Nor do I propose such a rigid objectivist approach. And I’m puzzled as to why you were left with that impression. Perhaps you should reread my original post. But let’s become hardcore subjectivists for a moment. ANY sort of blind testing is anathema because it presents huge obstacles for guys like us. It’s just sooo unfair!

How far down this road shall we walk? Does our zeitgeist preclude the possibility of the venerable JA distinguishing (Blind) a $400 amp from a $40,000 Krell? “Nonsense,” you say? “JA can do THAT with one ear covered!” So critics *can* confirm some of their sighted impressions (blind). It’s merely a matter of degree.

One area of my remarks that you do touch on is the notion of more than one critic evaluating a component. Your response: “I suspect the unwieldiness of it all overwhelmed them.”

So evaluating a component and then shipping it UPS to a colleague, for his evaluation, presents a logistical challenge that is insurmountable for audio critics? Not a terribly ambitious bunch, are they?

Three questions: Audio critics purport to hear myriad differences without a methodology in place to control for observer bias. Convenient, no? Can you think of another example (in the adult world) where a group of individuals are given such latitude?

Subjectivists place great emphasis on visual clues. Please explain what visual aids are forfeited by being able to see everything in the room save the brand name of the component under consideration. How does not having a priori knowledge of the brand name hinder a critic’s sighted listening evaluation?

Concerning credibility: No, I don’t view it as a panacea for what ails high end audio, but it does seem terribly important to many critics. And rightly so. A few years back, there was a high profile reviewer (on here) who was having a heated back and forth with a couple of posters who were getting under his skin by impugning his credibility.

Things were becoming a bit tedious so I emailed the critic asking why he bothered replying to such people. After all, what was the point? They had his number and they were enjoying it. He responded to me with a peevish email, the gist of which was: “How would YOU like having your integrity attacked all the time. I don’t appreciate being labeled a fraud!”

When I tactfully suggested that he take steps to bolster his credibility with such types by adopting a more demanding methodology, he responded with the usual reasons as to why that was a poor idea. Fair enough. Unsolicited advice is rarely taken.

My point is that he had no interest in *earning* respect and credibility from others. Rather, he thought it should be automatically conferred upon him because of the Masthead his name appeared under. This sense of entitlement has become ubiquitous in America, hasn’t it?

Great fun.

Blind test vs DBTs..., posted on October 21, 2009 at 12:42:31
mkuller
Audiophile

Posts: 15929
Location: SF Bay Area
Joined: April 22, 2003
Contributor
  Since: December 28, 2003
 >I make no mention of measurements and my hypothetical example (reviewer Smith) does not amount to a DBT. There’s no need to muddy the water, counselor. Why is it every time I have this discussion (on here) the knee-jerk reaction is to center the discussion around DBTs? I’m not advocating for DBTs and I’m well aware of their limitations. But when a critic claims that the preamp he’s evaluating casts a wider soundstage than his reference, “extending a couple of feet beyond the outer edges” of his speakers, he’s offering more than mere opinion. He is making what for him amounts to a factual assertion. This significant difference should be observable in a relaxed setting without his knowledge of which preamp he’s hearing. Is this so unreasonable?>

Yes, it's unreasonable.

It's the same arguement Mr. Nut used over and over which finally got him booted out.

Listen without seeing the brand name, I beleive he said ad nauseum.

So you're now carrying the Nut mantle forward?

If DBTs can't detect a difference between amplifiers, leading pro-DBTers to say they all sound the same as long as..., why would a single listener, listening blind, produce different results?

Blind tests don't appear effective or sensitive enough to detect small audible differences.

If they were an effective mechanism for determining small audible differences, reviewers would use them.
Poor Michael. I see the “Bass Nut” continues to hold a firm grip on your psyche., posted on October 22, 2009 at 07:53:22
regmac
Audiophile

Posts: 1843
Joined: April 7, 2002
 "It's the same arguement (sic) Mr. Nut used over and over which finally got him booted out."

Did you have a hand in Richard's banishment? Why is it so terribly important to you that everyone reflect your fundamentalist dogma? Your intolerance for differing views has taken on jihadic proportions. Odd that. Given the fact that you pride yourself on being a good progressive. Where’s that wonderful liberal tolerance one hears so much about these days? Do you have a favorite prayer rug you use each morning as you chant the subjectivist creed? :)

You think my example “unreasonable” because: “Blind tests don't appear effective or sensitive enough to detect small audible differences.”

Fair enough. But my example involves a critic purporting to hear a *large* difference. Is it your position that even large (perceived) differences are undetectable under blind conditions?



I feel yur pain man, so allow me to, posted on October 22, 2009 at 10:59:43
bjh
Audiophile

Posts: 11876
Location: Toronto, Canada
Joined: November 22, 2003
 ROTFLMFAO
Everything matters, don't forget to tweak your placebos!
How come you're so defensive..., posted on October 22, 2009 at 10:09:45
mkuller
Audiophile

Posts: 15929
Location: SF Bay Area
Joined: April 22, 2003
Contributor
  Since: December 28, 2003
 ...and never address the issues I raise?

As was asked to a poster above - why are you even here and what's your point?

Well I don't know that that's quite correct., posted on October 21, 2009 at 12:55:38
bjh
Audiophile

Posts: 11876
Location: Toronto, Canada
Joined: November 22, 2003
 I don't have a problem with blind tests, e.g. "casual" blind tests as they are sometimes called, yet such tests are routinely discounted by the more vociferous DBT proponents as having no greater weight than sighted comparisons ... they aren't scientific, lack sufficient controls, etc., etc.

These "casual" blind test resemble more the type of conditions common with "normal" listening yet apparently a test with a rigerous methodology, typically conducted listenting to an unfamiliar system, more often than not using skeptics as listening subjects, etc., etc. is deemed more valid... Hmmmmm?
Everything matters, don't forget to tweak your placebos!
The biggest problem with blind tests..., posted on October 21, 2009 at 13:18:42
mkuller
Audiophile

Posts: 15929
Location: SF Bay Area
Joined: April 22, 2003
Contributor
  Since: December 28, 2003
 ...of any kind, is that you are listening in a "critical listening" mode and that isn't how you listen to music.

You are forcing yourself to pay attention for differences, which interferes with your ablitly to hear and identify them, unless they are large enough.

The only kinds of differences identified with blind tests are gross frequency response (like with speakers), loudness and added distortion or noise.
RE: The biggest problem with blind tests..., posted on October 21, 2009 at 13:50:23
bjh
Audiophile

Posts: 11876
Location: Toronto, Canada
Joined: November 22, 2003
 It doesn't present any great effort to me to be honest.

I have done many blind comparisons for a cable designer friend. Simple setup, listen to a few tunes, switch (not knowing if the *new* is in fact the "new" iteration being sampled), listen again... perhaps repeat for a specific tune or section of a tune, etc.

What's interesting is that we hear differently, he may have focused on a specific sonic trait during development, I may place more emphasis on other things. And beyond that what I would call an extended high end seems to never match his idea of same ... we just hear different!

Generally I can make him cognizant of things that stick out for me, just as he can do for me. That's not to say we always agree, that's a different story. However the "blind" aspect doesn't present any special challenges (mind you I'm very familiar with the system).

---

We do avoid trickery as a rule, however one time he "tested" me for something that he had always been rather dubious about. I was told only that there would be a standard comparison but that I wouldn't be told what it was.

In the end, after I have sufficiently explained the differences I was hearing and expressed a preference he said, "OK, now I believe it."

Turns out he was testing me on Shakti Stones. I had consistently claimed that their presence (on top of components) sucked the life out ot the system, the sound for me became more bland, less colorful.... the test was comparing the sound with just one Stone in/out of the system, and when it was in it was seated on top of the (dedicated) power supply of his SimAudio Moon Andromeda CD Player.

True story, but like most (if not all) reports of "casual" blind results I don't suffer any illusion that (DBT) skeptics would believe it.

:)

Everything matters, don't forget to tweak your placebos!
Sounds like you learned how to overcome the test's problems...(nt), posted on October 21, 2009 at 14:28:26
mkuller
Audiophile

Posts: 15929
Location: SF Bay Area
Joined: April 22, 2003
Contributor
  Since: December 28, 2003
 (nt)

I think what you're saying is, posted on October 21, 2009 at 08:28:49
Bruce from DC
Bored Member

Posts: 18019
Location: U.S. Capital
Joined: October 13, 1999
Contributor
  Since: March 1, 1999
 that you have a problem with hyperbole. That is, that reviewers' language overstates the differences between component X and component Y.

This point has been made before, more concisely. I agree.

Audio equipment reviewers should watch their language. To wit: the anonymous Consumers Union reviewer who described the sound of the original Bose 901 "direct/reflecting" loudspeaker said the sound from the speakers "tended to wander about the room." Dr. Bose sued, saying the statement was false and defamatory. Leaving aside the (to me) significant question of how such an obvious (to me) statement of opinion could be actionable, the publisher of Consumer Reports had to go all the way to the Supreme Court and suffer the expense of probably $750K on lawyers' fees to get that one turned around.

What the reviewer meant to say was something like "tended to wander between the speakers," which would have been a less exaggerated and, based on my hearing of the 901s, accurate statement.

So, we agree that hyperbole is a bad thing. However, even the question of whether a statement like the ones you complain about (e.g. "dramatic improvement") is hyperbolic is inherently subjective. An untrained listener might find the difference mild, or even non-existent. A trained listener might find the same difference to be "substantial."

There's just no certainty in audio reviewing, Reg! ;-)

And depending on the numbers to tell you something is hardly better than auditioning with earplugs in place.


RE: I think what you're saying is, posted on October 21, 2009 at 11:54:08
bjh
Audiophile

Posts: 11876
Location: Toronto, Canada
Joined: November 22, 2003
 Hyperbole aside, and granting that there's no certainty in audio reviewing, it seems to me that perhaps the main reason that many (perhaps most) hobbyists *don't* suffer pangs at the lack of certitude is because to a greater or lesser extent much of what described in audio reviews is common experience!

Who with sufficient exposure to comparisons of components ... in one's home, in audio shops, at audio get-togethers, etc. ... *hasn't* noticed things such as differences in soundstage (e.g. width), differences in clarity, or what have you...?

Sure one might feel that some reviewer is stretching it, waxing poetic, or whatever, and for that matter the reader may be especially dubious if the DUT is one which he/she has firsthand experience and with a distinctly differing POV, but it's not often the case that what is described is beyond experience in general.


Everything matters, don't forget to tweak your placebos!
RE: I think what you're saying is, posted on October 21, 2009 at 10:13:59
regmac
Audiophile

Posts: 1843
Joined: April 7, 2002
 "There's just no certainty in audio reviewing, Reg! ;-)"

Good advice. I was unaware that subjectivists view your statement as being axiomatic. I will keep that fact uppermost in mind the next time I’m tempted to enter one of these discussions. ;-)

RE: I had a hard time following this, posted on October 20, 2009 at 14:31:20
jamesgarvin
Audiophile

Posts: 4074
Location: southern ohio
Joined: July 9, 2004
 Bruce, I agree with everything you write. But I think the following paragraph misses the mark relative to advocates of DBT:

"Only an analysis of the paint, close examination of brushstrokes, etc. distinguishes between the two. In a DBT of these two paintings, does a null result (i.e. that the test panel can not tell that they are different) mean that the two are products of the same artist? No. It means that the test panel lacks the level of subtlety and discrimination to tell the two apart."

The problem with that analogy is that the DBT, I think, is not designed to test the products, but rather the listener. The advocates of DBT already have their conclusion. Rather, they use the test not to investigate the differences between components, but rather to test whether the listener who says he or she can hear differences really can hear those differences. Their goal is to prove that the listener who says he or she can hear differences really can hear those differences.

Proof? Some years ago, when Fremer wrote for TAS, JA reported on his results on a DBT test, administered by DBT adherents. As I recall, he successfully distinguished 5 of 7 tests, and Fremer scored 7 of 7 tests. They were called lucky. In other words, the DBT adherents would not accept their own test results, because they believed the components indistinguishable, and no test was going to change their conclusions. They were more interested in proving that Fremer and Atkinson were full of shit, and when the test indicated otherwise, the DBT folks minimized their own test. When Fremer and Atkinson's results shot a hole in the alleged theory, they were called 'lucky coins.'

Back to you painting analogy. If someone says they can tell the difference between a fake and a real painting, there are objective tests which can prove the validity of the so-called expert viewer which are beyond dispute. On the other hand, I am not aware of any objective tests which can prove that Fremer and Atkinson can hear differences, or at leat objective tests which everyone will accept. Hell, there are no objective tests which can prove I can hear differences I hear.

I simply hear them, and, frankly, could not give a shit whether someone says I cannot hear them. I am still mystified why we care what the DBT crowd has to say on this subject, and so much time arguing about this.
Some flaws in your post, posted on October 20, 2009 at 20:56:26
JustJoe
Audiophile

Posts: 128
Location: Essex
Joined: July 8, 2009
 "Proof? Some years ago, when Fremer wrote for TAS, JA reported on his results on a DBT test, administered by DBT adherents. As I recall, he successfully distinguished 5 of 7 tests, and Fremer scored 7 of 7 tests. They were called lucky. In other words, the DBT adherents would not accept their own test results.."

Ahem. One third-hand anecdote does not a proof make.

Or do you also believe that 2 stories of corporate greed prove the free market system is evil?

Just Joe

Blind testing of gear works: If it doesn't sound better with your eyes closed, then it doesn't sound better!
RE: Some flaws in your post, posted on October 21, 2009 at 04:17:57
John Atkinson
Reviewer

Posts: 1900
Location: New York
Joined: November 24, 2003
 >One third-hand anecdote does not a proof make.

This was reported in Stereophile (see link). The actual results were Michael, 5 out of 5
identifications; me, 4 out of 5 identifications. Neither score reaches a 95% confidence
limit on its own, but the combined score is close to the 99% limit. Was this due to
chance or was there an audible difference between the amplifiers? In an ideal world,
we would have retaken the test, but the venue was an AES Convention and this was
not possible.


John Atkinson
Editor, Stereophile
YOu can't combine a time series of scores across 2 subjects, posted on October 21, 2009 at 11:23:07
JustJoe
Audiophile

Posts: 128
Location: Essex
Joined: July 8, 2009
 "The actual results were Michael, 5 out of 5
identifications; me, 4 out of 5 identifications. Neither score reaches a 95% confidence
limit on its own, but the combined score is close to the 99% limit."

You can't combine a time series of scores from 2 subjects. The scores for one subject across time are autocorrelated.
They are not the same as asking 9 independent subjects to do a DBT comparison.

THe confidence interval analysis does not hold here.

Just Joe
Blind testing of gear works: If it doesn't sound better with your eyes closed, then it doesn't sound better!
RE: YOu can't combine a time series of scores across 2 subjects, posted on October 21, 2009 at 11:32:15
bjh
Audiophile

Posts: 11876
Location: Toronto, Canada
Joined: November 22, 2003
 Well holy crappers! ... just about the entire history of DBT in audio (well what little there is of it anyway) is out the window!

Poof! She's gone!
Everything matters, don't forget to tweak your placebos!
Huh?, posted on October 21, 2009 at 12:13:46
JustJoe
Audiophile

Posts: 128
Location: Essex
Joined: July 8, 2009
 How did you reach that conclusion?

huh?

Just Joe
Blind testing of gear works: If it doesn't sound better with your eyes closed, then it doesn't sound better!
RE: Huh?, posted on October 21, 2009 at 16:57:52
John Atkinson
Reviewer

Posts: 1900
Location: New York
Joined: November 24, 2003
 >>>You can't combine a time series of scores from 2 subjects. The scores for one
>>>subject across time are autocorrelated. They are not the same as asking 9
>>>independent subjects to do a DBT comparison.
>>
>>Well holy crappers! ... just about the entire history of DBT in audio (well what little
>>there is of it anyway) is out the window!
>
>How did you reach that conclusion?

I hope I am not putting words in bjh's mouth, but I believe he is referring to the fact
that in all the blind tests using multiple test subjects in the 1980s, such as the one at
the 1988 AES Convention to which Michael Fremer and I were referring, the
published analysis of the data combined the results from multiple listeners. If you
now say that that wasn't justifiable, then all those blind tests that "proved" amps
sounded the same were bogus! Your gun, your foot, as it were.

So let me get this straight: reading this thread, it appears that the objection of
various "objectivists" to the results of the 1988 AES amplifier tests were a) that the
listeners couldn't hear a difference to a statistically significant degree and b) of
course they could hear the difference to a statistically significant degree but
that was because the amplifiers had different frequency responses into the test
loudspeakers. It would be a lot easier discussing this subject of you guys could first
agree on your position. :-)

More seriously, you are all demonstrating that if someone submits to your demands
to perform blind tests, you still feel free to discard the results of those tests if they
don't conform with your beliefs. So why should anyone pay you any attention?


John Atkinson
Editor, Stereophile
RE: Huh?, posted on October 21, 2009 at 21:33:48
JustJoe
Audiophile

Posts: 128
Location: Essex
Joined: July 8, 2009
 "in all the blind tests using multiple test subjects in the 1980s, such as the one at
the 1988 AES Convention to which Michael Fremer and I were referring, the
published analysis of the data combined the results from multiple listeners. If you
now say that that wasn't justifiable, then all those blind tests that "proved" amps
sounded the same were bogus! Your gun, your foot, as it were."

I'm not trying to prove all amps sound alike. If this test you are referring to, used time series data from multiple subjects and trated each observation as independent, then it was a fatally flawed data analysis to begin with.
I fail to see why it's "my gun, my foot", I am NOT trying to prove anything.

Just Joe
Blind testing of gear works: If it doesn't sound better with your eyes closed, then it doesn't sound better!
RE: Huh?, posted on October 21, 2009 at 12:18:42
bjh
Audiophile

Posts: 11876
Location: Toronto, Canada
Joined: November 22, 2003
 I don't have the compulsion to explain it to you, but it's the case.

End of discussion.
Just Go Away

LOL

Everything matters, don't forget to tweak your placebos!
even if 5/5 is 100%..., posted on October 21, 2009 at 05:19:54
Tom Schuman
Audiophile

Posts: 1861
Location: Hamburg
Joined: October 22, 2003
 ...statistically, it's well within the realm of chance (i.e., 50/50 odds suffice) that one gets 5 in a row 'right'. Now 30 in a row would be interesting.

'the combined score is closer to the 95% confidence limit"...

ask a statistician. I believe he would say, you can't talk about confidence limits unless you are prepared to also show much large sample sizes.
RE: even if 5/5 is 100%..., posted on October 21, 2009 at 07:17:13
John Atkinson
Reviewer

Posts: 1900
Location: New York
Joined: November 24, 2003
 >>The actual results were Michael, 5 out of 5 identifications; me, 4 out
>>of 5 identifications. Neither score reaches a 95% confidence
>>limit on its own...
>
> ...statistically, it's well within the realm of chance (i.e., 50/50
> odds suffice) that one gets 5 in a row 'right'.

I didn't say different.

>> but the combined score is close to the 99% limit.
>
>ask a statistician. I believe he would say, you can't talk about
>confidence limits unless you are prepared to also show much large
>sample sizes.

Not according to the textbooks on experimental design I used as an
undegraduate too many years ago. Yes, greater sample size gives the
statistics more analytical power, but 9 out of 10 correct does give a
likelihood of the results being due to chance at about 1.1% (assuming
a binomial distribution of the data, and no, I don't have time to argue
whether or not it does).

More importantly, you illustrate the point that was being made: that
"objectivists" dismiss the possibility of there being a real difference
even when a blind test organized according to their own rules suggests
the possibility that there might be.






John Atkinson
Editor, Stereophile
Your satisitical argument is flawed, posted on October 21, 2009 at 12:16:11
JustJoe
Audiophile

Posts: 128
Location: Essex
Joined: July 8, 2009
 "Not according to the textbooks on experimental design I used as an
undegraduate too many years ago. Yes, greater sample size gives the
statistics more analytical power, but 9 out of 10 correct does give a
likelihood of the results being due to chance at about 1.1% (assuming
a binomial distribution of the data, and no, I don't have time to argue
whether or not it does)."

All of the above assumes INDEPENDENT sample points (independent subjects). NOT one subject giving you a few data points, then another giving you a few more. Each person can ONLY give you ONE data point.

Just Joe

Blind testing of gear works: If it doesn't sound better with your eyes closed, then it doesn't sound better!
RE: even if 5/5 is 100%..., posted on October 21, 2009 at 12:11:17
Tom Schuman
Audiophile

Posts: 1861
Location: Hamburg
Joined: October 22, 2003
 "More importantly, you illustrate the point that was being made: that
"objectivists" dismiss the possibility of there being a real difference
even when a blind test organized according to their own rules suggests
the possibility that there might be. "

I illustrated nothing of the kind. Must be responding to someone else. I don't think that sets of measurements are so comprehensive that they fully describe/explain any possible subjective experience. Does anybody think that? And I think anyone would admit that experienced listeners are more likely to do better than chance than inexperienced listeners at identifying subtle differences.

Stereotyping positions gets the debate nowhere. No audiophile is only objectivist or only subjectivist; even audioholics need engineers to build their equipment for them. Engineers, in turn, need marketing specialists to sell it.


"but 9 out of 10 correct does give a
likelihood of the results being due to chance at about 1.1%"

Hate to nitpick, but who got 9 out of 10 correct? There was 5/5 for one listener, 4/5 for another.

O come now, John., posted on October 21, 2009 at 08:30:45
Pat D
Audiophile

Posts: 7558
Joined: June 20, 2000
 A comparison of the FRs the Adcom and VTL amps into the speaker load shows that they were audibly different. Your own Figure 3 on page 6 shows the difference in the FR at the speaker terminals:

http://www.stereophile.com/asweseeit/113/index5.html

This is above known thresholds established in DBTs, so the results are not surprising:

http://www.provide.net/~djcarlst/abx_crit.htm

JA
"More importantly, you illustrate the point that was being made: that
"objectivists" dismiss the possibility of there being a real difference
even when a blind test organized according to their own rules suggests
the possibility that there might be."

Nonsense. Indeed, the two amps were compared using a different speaker in a smaller but better designed DBT run by Banks and Krajicek. They achieved much clearer results.

http://www.stereophile.com/features/587/index.html

There's not much doubt about the results of that test.










"Probability is the very guide to life."---Cicero
The end of objectivism in audio--at the hands of the objectivists, posted on October 21, 2009 at 19:23:17
Jim Austin
Reviewer

Posts: 491
Location: Northern New England
Joined: November 8, 2007
 I had read the first article (about the listening test) previously, but not the second. Before I say anymore, let me point out that I am trained as a physicist, with a PhD, and though I've forgotten a lot of what I learned, I have routinely used statistics in my work, including chi-2 tests. I understand a fair amount about statistics.

The key point about a listening test is that it's a matter of skill. Some small number have that skill; most don't. So at best, a few individuals in a typical population will be able to pass a difficult test under less than ideal conditions.

What's extraordinary about this follow-on paper is that it makes it impossible, in practical terms, EVER to determine whether a particular individual is capable of distinguishing among amplifiers, or cables, or anything. With this argument, you and the other audio objectivists can feel totally safe: No one will ever violate your smug certainty, because the rules of statistics won't allow it!

One of the basic premises of science is that a statement that is not testable is not scientific. This argument renders what might be a scientific statement--"John Atkinson/Michael Fremer can/cannot distinguish between two amplifiers in a blind test"--and renders it unverifiable, therefore unscientific. Objectivism triumphs by making science impossible, or irrelevant! Talk about irony!

The beauty of it is that by rendering your own claims ultimately safe, you render those of your opponent equally safe. If you, or they, remove the very possibility of testing that which needs to be tested--that a particular individual can or cannot pass a test--then you are rendering science irrelevant on this question. You're removing objectivism from the game. "There is no way to test this scientifically because a non-null result is unattainable."

Fine. So now you can go away and leave those who don't care about your chi-2 tests and your scientific standards in peace so that they can happily know what they know they know, making their aesthetic judgments without having to meet some objective standard--which, after all, cannot be done.

In other words, they can continue to do what they've been doing all along, in peace, to the apparent great annoyance of people like you.

Jim
http://www.jazz-etc.com
interesting observations, posted on October 23, 2009 at 02:28:14
Tom Schuman
Audiophile

Posts: 1861
Location: Hamburg
Joined: October 22, 2003
 there's no reason though that tests have to be triple blind, i.e., the identity of the test subjects is unknown or unimportant. there's nothing unscientific per se about testing variations in individual perceptual ability.
therefore your stereotyped 'objectivists' are a bit like paper tigers.
Spare me., posted on October 22, 2009 at 09:33:46
Pat D
Audiophile

Posts: 7558
Joined: June 20, 2000
 You didn't read the report by Banks and Krajicek of the DBTs they administered? Pity---the results were positive, much less ambiguous than the highly criticized results from the poorly run test Stereophile did.

On the first page, there is mention of a test done by AES--which JA (now) points out did not have a sufficient number of trials

Of course, as I said, the measurements done by JA indicated that the differences between the frequency responses of SS and tube amps into the speaker load exceeded known thresholds of audibility established long ago by David L. Clark.

Indeed, JA uses a simulated speaker load in his tests of amplifiers to give some idea of the kinds of changes in FR that an amp will have driving a typical speaker load and sometimes comments on the effects. Bascom King does the same thing when he measures amplifiers for Soundstage and sometimes comments on audibility.

Unless there is any real reason to believe that reviewers can tell the difference between the sound of the equipment they are reviewing and other similar types of equipment, there is no reason to pay any attention to their assertions that they sound different and certainly no reason to pay attention to their aesthetic judgments about the sound. Of course, we do know that various speakers sound different, so subjective reviews can be of some value in establishing an audition list.

Some reviewers have used DBTs fairly regularly, so sorry, it is not impossible or impractical to do them with electronics.

Aesthetic judgments of the sound of equipment seem to me to be misplaced. I make aesthetic judgments about the music I hear over my system (my CDP, preamp, and amp and speakers measure very well, BTW, and I have no reason to suspect my interconnects and speaker cables do not), but it never occurs to me to attribute the beauty of the music to my equipment. I have just been listening to lovely Naxos recordings called Dances from the Terpsichore and Chominciamento di gioia.

You seem to be under the impression that I am telling people how to choose equipment (and some even seem to think I am telling them how to listen to music!). As far as I am concerned, how people choose their equipment and what they want to buy are matters of preference.

I am amazed that any writer for Stereophile would have the colossal gall to accuse anyone else of smugness! You must have read the explanatory material before the Recommended Components sections--how smug can you get? I am only toooooooo familiar with the argument that you guys are pros, highly trained listeners, and we should rely on you to tell us what sounds different and what sounds good--but you can't or won't actually show you can detect what you claim.





"Probability is the very guide to life."---Cicero
Then don't! (in answer to ...., posted on October 22, 2009 at 09:59:07
bjh
Audiophile

Posts: 11876
Location: Toronto, Canada
Joined: November 22, 2003
 "Unless there is any real reason to believe that reviewers can tell the difference between the sound of the equipment they are reviewing and other similar types of equipment, there is no reason to pay any attention to their assertions that they sound different and certainly no reason to pay attention to their aesthetic judgments about the sound."

Just don't! Problem solved!

Now run along...
Everything matters, don't forget to tweak your placebos!
RE: The end of objectivism in audio--at the hands of the objectivists, posted on October 21, 2009 at 21:58:40
JustJoe
Audiophile

Posts: 128
Location: Essex
Joined: July 8, 2009
 "John Atkinson/Michael Fremer can/cannot distinguish between two amplifiers in a blind test"

This is easily verifiable, scientifically with a DBT.
Give each one repeated trials, and see if they can achieve a number higher than chance.
use matched levels, plenty of breaks.

use at least 30 trials for JA and 30 trials for MF.

This is simple. HIgh school level stuff.
Maybe not for an almost-was scientist with a correspondence school degree.
Just Joe
Blind testing of gear works: If it doesn't sound better with your eyes closed, then it doesn't sound better!
Did you read the article?, posted on October 22, 2009 at 09:45:21
Jim Austin
Reviewer

Posts: 491
Location: Northern New England
Joined: November 8, 2007
 That is what I was referring to. They were suggesting that any methodology that involved more than one trial per individual was unacceptable.

>>Maybe not for an almost-was scientist with a correspondence school degree.
Just Joe<<

Maybe, but who were you referring to? Presumably not me; my degree is from the University of North Carolina, and I earned it the old-fashioned way, with 6 years of full-time post-graduate research and study.

Jim
http://www.jazz-etc.com
You are correct..., posted on October 22, 2009 at 10:29:28
bjh
Audiophile

Posts: 11876
Location: Toronto, Canada
Joined: November 22, 2003
 > That is what I was referring to. They were suggesting that any methodology that involved more than one trial per individual was unacceptable.

Correct, I wrote about this in Prophead last year ...

from this post...
The authors provide an interesting statistical analysis that challenges some of the the assumption used in the test procedure and hence the statistic that were derived. The primary assumption challenged is the assumed independent of observations which they claim was violated by taking multiple observations from each test subject; in the test each subject provided 7 observations, i.e. participated in 7 trials. They claim the multiple observation from a single subject are not truly independent "Because each observer's seven responses are influenced by that observer's biases, accuracy, and any other inherent characteristics he or she may have."

While noting that the that authors provide convincing scholarly references to buttress their case it is also an inescapable fact that the criticism equally applies to virtually all audio DBTs conducted/documented since utilizing a subject for participation in multiple trials and counting each trial as an independent observation would appear to be exceedingly common. Hence we must conclude that the non-independence "error" seems to be pretty much universal for Audio DBTs, a rather interesting observation! [emphasis added (this time) - bjh]







Everything matters, don't forget to tweak your placebos!
RE: Did you read the article?, posted on October 22, 2009 at 10:21:07
John Atkinson
Reviewer

Posts: 1900
Location: New York
Joined: November 24, 2003
 >>Maybe not for an almost-was scientist with a correspondence school degree.
>
>Maybe, but who were you referring to? Presumably not me; my degree is from the
>University of North Carolina, and I earned it the old-fashioned way, with 6 years of
>full-time post-graduate research and study.

Perhaps he was referring to me? I odn't have a PhD, Jim, but I do have a BSc (honors)
in physics and chemistry from the University of London and a post-graduate
qualification in the teaching of science at the high-school level.

BTW, you are one of 4 PhDs among Stereophile's reviewers. I don't think the others
got their qualifications from a correspondence school, either. :-)


John Atkinson
Editor, Stereophile
Then what was the point of the DBT test?, posted on October 21, 2009 at 09:05:23
jamesgarvin
Audiophile

Posts: 4074
Location: southern ohio
Joined: July 9, 2004
 If, as you conjecture, measurements between the two amplifiers in that test with that speaker load predicted that the two amplifiers sounded different, then why did the DBT advocates conduct the test? To prove that two subjects should hear a difference that should be present? And why two subjective reviewers as the subjects? Would it not have made more sense to test two objective reviewers to determine whether they can actually hear differences that measurements predict they should hear? How embarassing would that be? An objective reviewer cannot hear a difference their own measurements suggest they should hear.

Let's face it. The DBT folks conducted the test because they did not believe that Atkinson and Fremer could hear a difference, regardless of what the measurements 'showed.' They used those particular very different amplifiers because, if they could demonstrate that Fremer and Atkinson could not differentiate between two very different amplifiers, with different measurements, they could then make the argument that, if they cannot hear the difference between two very different amplifiers, they certainly could not hear the differences between two very similar amplifiers which they often describe in print.

The reverse would not necessarily be true. If they could not reliably detect the difference between two very similar amplifiers, they certainly may be able to distinguish between two very different amplifiers, and so lack of differentiating between the former would not be mean lack of ability to differentiate between the latter, which Fremer and Atkinson could claim as their escape clause. Nope. The DBT advocates were attempting to corner their subjects.

Like the folks who performed the DBT test, you seem intent on explaining away the results, and raise issues that were curiously not raised until after the results.

Why do you believe that DBT proponents want to "prove" all components sound the same?, posted on October 21, 2009 at 11:19:48
JustJoe
Audiophile

Posts: 128
Location: Essex
Joined: July 8, 2009
 "Let's face it. The DBT folks conducted the test because they did not believe that Atkinson and Fremer could hear a difference, regardless of what the measurements 'showed.'"

I'm curious. Just because DBT is a widely accepted scientific methodology in ALL areas of subjective comparisons, why should proposnents of DBT always want a null result?
Maybe they just want comparison studies to USE DBTs, regardless of the results?
So if 2 components sound different in a DBT, then they sound different, and that;s great. If one is subjectively preferred in a DBT, then that is great. A scientifically valid comparison has been made.

Why should someone who supports the use of DBT ALWAYS want a null result?
I am not sure why you think that supporters of DBTs want to somehow "prove" that all components sound alike.
They don't.
If many components sound alike in a DBT, then unfortunately, they DO sound alike. IF they don't sound alike in a DBT, then they are different sounding.

End of discussion.
Just Joe


Blind testing of gear works: If it doesn't sound better with your eyes closed, then it doesn't sound better!
"End of discussion." but but... Oh poo! ... darn you! nt, posted on October 21, 2009 at 11:27:40
bjh
Audiophile

Posts: 11876
Location: Toronto, Canada
Joined: November 22, 2003
 .
Everything matters, don't forget to tweak your placebos!
???????????????, posted on October 21, 2009 at 09:17:49
Pat D
Audiophile

Posts: 7558
Joined: June 20, 2000
 What am I explaining away? Did you actually read the article by Banks and Krajicek? They got positive results.

As to why Stereophile bothered to perform the DBT, you will have to ask Stereophile. Though flawed, it was an learning exercise and they did find that identification was easier with some types of signal than others.

"Probability is the very guide to life."---Cicero
I think you are missing the point..., posted on October 21, 2009 at 09:40:53
jamesgarvin
Audiophile

Posts: 4074
Location: southern ohio
Joined: July 9, 2004
 I am not asking you to explain anything away. But had Fremer and Atkinson failed the test, do you think for a moment you would call that test flawed? Would the organizers of the test call it flawed? No. It was later called flawed because the results did not confirm what the organizers wanted to confirm. Had Fremer and Atkinson failed the test, every supporter of DBT tests would scream from the high heavens that Fremer and Atkinson cannot reliably distinguish between two different amplifiers.

This sort of reminds me of police officers and lie detector tests. They swear by them and their accuracy. If a cop wants a criminal to take a lie detector test 'to prove their innocence,' and then you suggest a deal whereby if the subject passes the test, they promise not to prosecute, all of a sudden, that infallible test is not infallible.

Fail the DBT test, that proves all amplifiers sound the same. Pass the test, and well, the test is flawed, and proves nothing. The DBT test showed some signals were easier to distinguish than others? Hardly. There was only one 'signal,' as I assume you mean that term, and no comparisons to more than two different or similar amplifiers were made.

You need to look to the methodology to learn the motives. They picked two different amplifiers for a reason. They asked two subjective reviewers to take part for a reason. Everything was done for a reason. The organizers purpose was not prove that certain signals could be reliably differentiated. It to prove that they could not be reliably differentiated. They, like you, attempt to change the purpose after the results, and change was being measured.

The fact is that those organizers could not tell a difference, and did not believe Fremer and Atkinson could hear a difference. Rather than try to come up with half baked excuses why Fremer and Atkinson could hear differences, perhaps it would be an interesting investigation to find out why the organizers could not hear differences that are now so plain to exist.

As Captain Quint would say, "What it proves is that all you fancy city boys with all your education don't have the education to admit when you are wrong."

RE: I think you are missing the point..., posted on October 21, 2009 at 14:44:59
Pat D
Audiophile

Posts: 7558
Joined: June 20, 2000
 Again, what is the test you are talking about?

"Probability is the very guide to life."---Cicero
Why are the tester's motives necessarily relevant to the results?, posted on October 21, 2009 at 12:41:36
Tom Schuman
Audiophile

Posts: 1861
Location: Hamburg
Joined: October 22, 2003
 What IS relevant is the number of trials and the experience of the participants. As well as the design of the test.

The test showed what it showed, that is, that for the chosen parameter, two experienced reviewers could make the distinction correctly in x/x number of trials. Why is that a bad result for anybody? Now we could argue about whether what the subjects heard is somehow explainable with measurements x y or z, I think that is the more interesting question. and whether the number of trials were sufficient, etc.

RE: Why are the tester's motives necessarily relevant to the results?, posted on October 21, 2009 at 14:05:34
jamesgarvin
Audiophile

Posts: 4074
Location: southern ohio
Joined: July 9, 2004
 They are not relevant to the results. They are relevant to the interpretation of the results. They conduct a test which they purportedly believe to be a valid test. The result of the test is not to their liking, so they question the validity of the test, and what it is they were testing for.

We know that the organizers were of the opinion that all amplifiers sound the same, and to prove their hypothesis, they organize a test, and ask two subjective reviewers to undergo the test. The results come in which demontrate that the subjective reviewers heard a difference.

Did the organizers conclude that they could hear a difference? After all, the test was designed to determine whether they could hear a difference. No. The organizers concluded that the reviewers were lucky. Well, if a subject could simply lucky, then why do the test? Were they conducting the test to determine whether a reviewer could get lucky? No.

So, if a reviewer can get lucky, and that explains the result, the test proved nothing, and was a waste of time. On the other hand, had the reviewers been wrong, do you think the organizers would have concluded they were merely unlucky? Um. How I put this. HELL NO.

Consequently, the organizers conclusions were based upon the results. And that is what passes for scientific in some quarters.

I think you nailed the point squarely, posted on October 21, 2009 at 11:49:09
kerr
Audiophile

Posts: 3695
Location: Central Indiana
Joined: November 10, 2003
 >Fail the DBT test, that proves all amplifiers sound the same. Pass the test, and well, the test is flawed, and proves nothing.<

And So It Has Been Spewed For Decades... And So Shall It Be Spewed Forever!

DBT's don't prove anything. Consequently, they are pointless.

Actually, there is never 'proof' even in science., posted on October 21, 2009 at 12:50:34
Tom Schuman
Audiophile

Posts: 1861
Location: Hamburg
Joined: October 22, 2003
 'Proof' is a logical, mathematical concept.

For science, you need evidence, the reliability and amount of which may add to adoption of theory x or y. You really can't prove a scientific theory right (or wrong) with any amount of evidence. In fact, every theory is 'wrong' to the extent that it is an idealization of reality, not the reality itself.

I would think that if we're going to do science, as opposed to just entertaining ourselves with expensive audio equipment, good quality evidence would be necessary, and I don't believe the marketing hype that appears in many reviews should be counted among that evidence ('I can't think of another speaker in the price range that gets as many things right...liquid midrange...soaring highs', etc).
Now you talking about something different..., posted on October 21, 2009 at 14:11:20
jamesgarvin
Audiophile

Posts: 4074
Location: southern ohio
Joined: July 9, 2004
 Now you are talking about the language of subjective reviewing, and describing what one hears. A midrange may be liquid to one person, and muffled to another. One tweeter may be bright to one, detailed to another. That is a function of value and perception, and the cure to that is, one, find a reviewer who shares your values and perceptions, and, two, to listen to it yourself, and decide for yourself if you agree with the description, and value the speaker enough to purchase it.

I am not aware of anyone who has claimed that those descriptions amount to science.

Save your breath..., posted on October 21, 2009 at 10:32:51
bjh
Audiophile

Posts: 11876
Location: Toronto, Canada
Joined: November 22, 2003
 he's a total and complete idiot!
Everything matters, don't forget to tweak your placebos!
RE: I think you are missing the point..., posted on October 21, 2009 at 09:56:15
Pat D
Audiophile

Posts: 7558
Joined: June 20, 2000
 I think Fremer's letter refers to a quite different test, not the on JA linked.

They didn't fail the test. I believe it was a mass test at an AES convention with only five trials. That a few might get all of them right is not unexpected just considering pure chance. On the other hand, they may have heard the differences--but that test is not sufficient to show it, and they didn't replicate it as JA said. I wonder if that was the test the administrators were not permitted to level match.

"Probability is the very guide to life."---Cicero
Anything 'might' happen.., posted on October 21, 2009 at 17:41:38
jamesgarvin
Audiophile

Posts: 4074
Location: southern ohio
Joined: July 9, 2004
 but is it more probable that they might get lucky and get 9/10 correct, or they would be unlucky, and get less than 9/10 correct. If I were a betting man, I'd say it is more likely than not that, if luck were involved, they would have hit 5/10.

RE: Anything 'might' happen.., posted on October 21, 2009 at 22:01:20
JustJoe
Audiophile

Posts: 128
Location: Essex
Joined: July 8, 2009
 Ummm 5 trials each is too little.
They need 30 trials each. IF they get 20 or more right, then they are probably hearing differences.
Test has to be propoerly conducted ofcourse. No AA denizens need apply to design such a test, it would be a Cluster.

Blind testing of gear works: If it doesn't sound better with your eyes closed, then it doesn't sound better!
apparently those particular clowns at the AES convention, posted on October 22, 2009 at 06:21:01
Analog Scott
Audiophile

Posts: 4001
Joined: January 8, 2002
 had no business designing DBTs either? What was their point?

RE: Anything 'might' happen.., posted on October 22, 2009 at 05:13:39
John Atkinson
Reviewer

Posts: 1900
Location: New York
Joined: November 24, 2003
 >5 trials each is too little.

Yes, because even 5 correct our of 5 doesn't meet the 95% confidence limit. I made
this point to the test organizers at the time. They limited the number of trials to 5 in
order to test the maximum number of subjects in order to be able to combine
everyone's scoring in the statistical analysis.

>They need 30 trials each. IF they get 20 or more right, then they are probably
>hearing differences.

Increasing the number of trials increases the number of incorrect identifications that
can be tolerated within a specific confidence limit, For example, for a 95% confidence
limit, the following scores are all approximately equivalent: 9/10, 9/11, 10/12, 10/13,
11/14, 12/15, 12/16, 13/17, 13/18, 14/19 etc.

However, as an AES paper by Soren Bech a decade or so ago demonstrated, you
can't arbitrarily increase the umber of trials without having to consider the fatigue of
the listeners. It appears from the literature that a listener in a blind test only
produces reliable results for 40-45 minutes.


John Atkinson
Editor, Stereophile
Designing a good DBt test, posted on October 22, 2009 at 10:02:38
JustJoe
Audiophile

Posts: 128
Location: Essex
Joined: July 8, 2009
 Apart from the obvious mechanisms of a DBT,
YOU would need:
1. matched volumes
2. 30 trials per listener if you want to see if the listener actually can discern differences, and if they actually have a preference
3. at least 30 subjects to see if the equipment being tested is favored by whatever population he sample of listeners represents.

If the folks at this AES convention did it with 2 listeners and 5 trials, it was a mickey mouse test, done more for fun than anything else.

Its results are meaningless.

Just Joe
Blind testing of gear works: If it doesn't sound better with your eyes closed, then it doesn't sound better!
RE: Designing a good DBt test, posted on October 22, 2009 at 11:04:15
John Atkinson
Reviewer

Posts: 1900
Location: New York
Joined: November 24, 2003
 >Apart from the obvious mechanisms of a DBT, YOU would need:
>1. matched volumes

Of course. No-one has said otherwise, although it is fair to point out that matching
loudness between components that have non-flat frequency responses is not trivial.

>2. 30 trials per listener if you want to see if the listener actually can discern
>differences, and if they actually have a preference

You keep saying "30"? Why? You need as many trials as required by your
requirement for a confidence limit. Let's say that you would be happy with a 99%
limit, ie, there is 1% or lower possibility than the result would be achieved by chance
alone. In which case, 11 trials would be sufficient as there would be just an 0.6%
chance that a listener would get 10 identifications correct out of 11.

>3. at least 30 subjects to see if the equipment being tested is favored by whatever
>population he sample of listeners represents.

But you have already stated that the results for different listeners cannot be
combined. So you cannot examine the results for a single population, surely? Only
for each individual listener.


John Atkinson
Editor, Stereophile
Let's declare DBTs unsuitable for audio, thereby removing (what for me) is a red herring., posted on October 23, 2009 at 08:08:33
regmac
Audiophile

Posts: 1843
Joined: April 7, 2002
 Hello John:

Help me out here. Audiophiles place great emphasis on the importance of visual cues when evaluating a component. Thus the need for sighted listening. Fair enough. But I fail to see why one needs to know the brand name of the component under consideration. Please explain what visual aids are forfeited by being able to see everything in the room save the brand name of the component under consideration. How does not having knowledge of the brand name hinder a critic’s listening abilities and his ability to evaluate the component?

Grow up, would you ... can you? LOL nt, posted on October 23, 2009 at 12:14:29
bjh
Audiophile

Posts: 11876
Location: Toronto, Canada
Joined: November 22, 2003
 .
Everything matters, don't forget to tweak your placebos!
You just can't seem to make up your mind., posted on October 22, 2009 at 10:49:57
bjh
Audiophile

Posts: 11876
Location: Toronto, Canada
Joined: November 22, 2003
 One minute it's complaints about combining scores across two subjects (because the results for a single subject are autocorrelated), now it's
30 subjects with 30 trials for each subject!

LOL

Everything matters, don't forget to tweak your placebos!
It's making me ill..., posted on October 22, 2009 at 11:17:01
Jim Austin
Reviewer

Posts: 491
Location: Northern New England
Joined: November 8, 2007
 ...no, not you BJH, but this attempt by these objectivist morons to raise the bar ridiculously high in order to protect their turf. It's the Randi Challenge all over again.

Pat D, in an earlier post, wrote that he doesn't believe in judging audio equipment aesthetically. He judges only his recorded music aesthetically. One has to wonder how he avoids the intervention of the equipment in the listening experience--how he manages to separate the recorded music from the equipment that's making the actual sound. I can think of two possible answers, either He merely chooses to ignore the equipment, or he listens to music abstractly, ignoring its existence as actual sound.

I think I'd be an objectivist myself if those who claim that title for themselves didn't insist on messing it up. I really wish I could be on a side here too bad they're both wrong.

Jim
http://www.jazz-etc.com
RE: It's making me ill..., posted on October 23, 2009 at 11:45:57
Pat D
Audiophile

Posts: 7558
Joined: June 20, 2000
 Interesting that you chose to bring my earlier post up in an answer to bjh rather than answering my post directly. Do you expect bjh to be able to give an intelligible comment on what I say? The last I remember him saying to me was "Shut up."

Now, I did not say that I excluded aesthetics from judging equipment. I said I thought aesthetic judgments of the "sound" of equipment are misplaced. The system is a tool and there is nothing to prevent a tool from being aesthetically pleasing, such as my needle nose pliers. I like the styling and operation of some of my equipment, notably the Quad preamp and amp, and the styling and performance of the speakers. That kind of judgment had some influence on my purchase decision and pleasure in using the equipment. I expect them to be neutral unless I use the tone controls or insert an EQ or signal processor into the signal path. I expect my amplifier to be able to drive any speakers I am likely to buy.

As I said, my main speakers measure very well, and the sound that comes out of them is affected by their characteristics and varies with the levels, the speaker placement (and my placement), the recordings used, and any signal processing used. Which "sound" that comes out of my system is the "sound of the system?" The sound changes from moment to moment.

"Probability is the very guide to life."---Cicero
I think I get it, posted on October 23, 2009 at 12:02:00
Jim Austin
Reviewer

Posts: 491
Location: Northern New England
Joined: November 8, 2007
 Your system is a completely neutral tool that doesn't affect the delivery of the music. Everything that matters is easy to measure. Good for you. But it makes me wonder why you bother to engage in discussions about subjectivist audio.

I think I understand that, too. You're convinced that you're right and that everyone whose experience varies from yours is delusional--or at least deluded.

Fine. Have fun.

Jim
http://www.jazz-etc.com
I can't stop you from making up opinions for me . . ., posted on October 23, 2009 at 12:53:14
Pat D
Audiophile

Posts: 7558
Joined: June 20, 2000
 What makes you think subjective reviewers are immune from criticism?

"Probability is the very guide to life."---Cicero
RE: I can't stop you from making up opinions for me . . ., posted on October 23, 2009 at 12:56:13
Jim Austin
Reviewer

Posts: 491
Location: Northern New England
Joined: November 8, 2007
 That's not my intention. I'm just trying to understand.

>>What makes you think subjective reviewers are immune from criticism?<<

Nothing. I'm just trying to wonder why it matters to you.
http://www.jazz-etc.com
Well, it's not complicated . . ., posted on October 23, 2009 at 14:11:48
Pat D
Audiophile

Posts: 7558
Joined: June 20, 2000
 It's a matter of wanting to know if people can detect the differences they claim they can.



"Probability is the very guide to life."---Cicero
RE: Well, it's not complicated . . ., posted on October 24, 2009 at 12:17:15
kerr
Audiophile

Posts: 3695
Location: Central Indiana
Joined: November 10, 2003
 >It's a matter of wanting to know if people can detect the differences they claim they can.<

Why does it matter to you what other people can and can't hear? How is what someone else can hear going to be relevant to you, your system and your room?
RE: Well, it's not complicated . . ., posted on October 25, 2009 at 18:53:36
Pat D
Audiophile

Posts: 7558
Joined: June 20, 2000
 It also matters when people ask for advice.

"Probability is the very guide to life."---Cicero
It's a case of the Gimmies..., posted on October 23, 2009 at 19:01:06
bjh
Audiophile

Posts: 11876
Location: Toronto, Canada
Joined: November 22, 2003
 you got it bad!

LOL
Everything matters, don't forget to tweak your placebos!
I think there's another way of looking at it., posted on October 23, 2009 at 14:24:36
Jim Austin
Reviewer

Posts: 491
Location: Northern New England
Joined: November 8, 2007
 You want to see if they can prove it to your satisfaction. And as some of your own posts have demonstrated, that's a VERY difficult thing to do in terms of logistics, focus, etc.

I have to say I'm intrigued by that tagline. It's actually wise, but I probably interpret it quite differently than you do. It makes me think of inference, whereas with you, I'm guessing, it's more about deduction. To me it says that there are degrees of knowing and partial truths, and that sometimes it's OK to settle for less than absolute certainty (or anyway, 95% confidence levels). That there are things you can know without feeling a compulsion to prove them.

Jim
http://www.jazz-etc.com
There are always different ways of looking., posted on October 25, 2009 at 20:40:17
Pat D
Audiophile

Posts: 7558
Joined: June 20, 2000
 One of the things people ask for on this site is advice and recommendations. I try to base mine on things that I know will make a significant difference to the sound.

I don't know where you get the inference/deduction thing.

Jim
"To me it says that there are degrees of knowing and partial truths, and that sometimes it's OK to settle for less than absolute certainty (or anyway, 95% confidence levels)."

I don't disagree with that at all. We seldom have anything close to absolutely certainty in practical matters.

Jim continues:
"That there are things you can know without feeling a compulsion to prove them."

I have to wonder just what sort of things you mean. I have a good idea what I prefer, and I feel no need to prove that I prefer something. The notion makes no sense.



"Probability is the very guide to life."---Cicero
RE: It's making me ill..., posted on October 22, 2009 at 12:04:23
bjh
Audiophile

Posts: 11876
Location: Toronto, Canada
Joined: November 22, 2003
 The answer in Pat's case is ... Rigid Philosophical Stance + Zip Cord

LOL (sorry, but he really shouldn't be taken seriously ... at any level!)
Everything matters, don't forget to tweak your placebos!
You got it..., posted on October 20, 2009 at 14:46:46
mkuller
Audiophile

Posts: 15929
Location: SF Bay Area
Joined: April 22, 2003
Contributor
  Since: December 28, 2003
 >The problem with that analogy is that the DBT, I think, is not designed to test the products, but rather the listener.>

One of the problems, anyway.

The test is designed to test only the listener, because the results change with experience, training, critical listening and test-taking abilities of the listeners, unlike medical DBTs.

>Their goal is to prove that the listener who says he or she can hear differences really can hear those differences.>

No, the pro-DBTers use the test to discredit and hammer the listeners who claim they hear a difference.

As audiophiles, we only have to convince ourselves we hear a difference.

RE: I had a hard time following this, posted on October 20, 2009 at 14:20:10
andy19191
Audiophile

Posts: 158
Location: Europe
Joined: June 26, 2008
 A clear and informative post for those of us (possibly only me) interested in the basis of audiophile beliefs.
Get a room..., posted on October 20, 2009 at 17:40:25
bjh
Audiophile

Posts: 11876
Location: Toronto, Canada
Joined: November 22, 2003
 for just yourself!

LOL
Everything matters, don't forget to tweak your placebos!
You mean experiences, - that's a better word than beliefs, posted on October 20, 2009 at 14:27:28
Sordidman
Dealer

Posts: 9245
Location: San Francisco
Joined: May 14, 2001
 as "beliefs" cannotates a lack of experience, and something outside reason.

Given your lack of experiences with both audiophiles and high end equipment, - your wording, - and corresponding implications, ruins your credibility.



Surrendered to self preservation,
From others who care for themselves.
A blindness that touches perfection,
But hurts just like anything else.
RE: You mean experiences, - that's a better word than beliefs, posted on October 21, 2009 at 05:09:09
andy19191
Audiophile

Posts: 158
Location: Europe
Joined: June 26, 2008
 > You mean experiences, - that's a better word than beliefs

No I mean beliefs. Bruce has provided a reasoned response which is fairly rare around here. It is built on what he believes to be true (his beliefs). These differ in number of respects from what I believe to be true.

One does not usually refer to "flat earth experiences" even though the experience of seeing a flat earth almost certainly contributed towards building the belief that the earth is flat. A belief is more fundamental than an experience.
I'm afraid that you'll need to look up the definitions of the terms:, posted on October 21, 2009 at 07:27:52
Sordidman
Dealer

Posts: 9245
Location: San Francisco
Joined: May 14, 2001
 as you're certainly using them incorrectly.

""which is fairly rare around here""

Not really true, but then, your unqualified opinion isn't the issue.



Surrendered to self preservation,
From others who care for themselves.
A blindness that touches perfection,
But hurts just like anything else.
LOL! And Regmac's reinforced your pro-DBT objectivist religious beliefs? (nt), posted on October 20, 2009 at 14:23:05
mkuller
Audiophile

Posts: 15929
Location: SF Bay Area
Joined: April 22, 2003
Contributor
  Since: December 28, 2003
 (nt)

That was a good read. Thanks (nt), posted on October 20, 2009 at 13:46:16
robert young
Audiophile

Posts: 2226
Location: new york
Joined: October 19, 2003
Contributor
  Since: July 13, 2004
 !

Excellent response...., posted on October 20, 2009 at 13:30:41
mkuller
Audiophile

Posts: 15929
Location: SF Bay Area
Joined: April 22, 2003
Contributor
  Since: December 28, 2003
 ...probably much more thoughtful and well organized than the OP deserved.

A couple of comments:

Wine Spectator only has one critic evaluating each of the different types of wine, like TAS these days.

I suspect a 90 point rating is similar to an A in Stereophile's RCL, in terms of the effect on sales.

The pro-DBT objectivist position is it's own box, with no way out.

But while trapped in there, they feel superior to us audiophiles.

One question (this time to the right person..my duh...), posted on October 20, 2009 at 12:28:42
Analog Scott
Audiophile

Posts: 4001
Joined: January 8, 2002
 Do you listen to your system under sighted conditions?

I listen to my system thus...., posted on October 21, 2009 at 22:06:10
JustJoe
Audiophile

Posts: 128
Location: Essex
Joined: July 8, 2009
 I listen to mine with my eyes closed.
And usually in the nude.

Are you suggesting I should test gear while I am in the nude, or it will not be a valid test?
:)
Just Joe
Blind testing of gear works: If it doesn't sound better with your eyes closed, then it doesn't sound better!
RE: I listen to my system thus...., posted on October 22, 2009 at 06:26:59
Analog Scott
Audiophile

Posts: 4001
Joined: January 8, 2002
 "I listen to mine with my eyes closed.
And usually in the nude."

OK I didn't really need to know that.

"Are you suggesting I should test gear while I am in the nude, or it will not be a valid test?"

Now I have to carry that image with me all day. Thanks.


What I am asking ultimately is whether Regmac or anyone else for that matter makes the same demands of proof of their own perceptions of their own system that Regmac is suggesting we should demand of audio reviewers.

So Reg, you don't want to answer that question. I can understand why., posted on October 21, 2009 at 07:43:14
Analog Scott
Audiophile

Posts: 4001
Joined: January 8, 2002
 The answer points to the utter absurdity of demands on others to provide proof of their impressions via DBTs.

The pro-DBT objectivists say..., posted on October 20, 2009 at 10:42:26
mkuller
Audiophile

Posts: 15929
Location: SF Bay Area
Joined: April 22, 2003
Contributor
  Since: December 28, 2003
 ...essentially that all well-designed amplifiers (implying solid state, of course) which are not clipping sound the same - as evidenced by DBTs with null results.

This is part of the rigid objectivist dogma you refer to.

>Commonsense would suggest it is absurd to insist all components sound alike.>

Commonsense would suggest it is absurd to think that any components - with different parts, different layouts, different topologies/designs, at different costs with different measurements - would sound the same.

You are asking why audio equipment reviwers don't perform DBTs to tell if they can identify the amplifier they are reviewing.

The obvious answer is DBTs don't usually work for telling amplifiers apart or with other components having small audible differences between them.

The blind methodology is flawed - refer to Harley's or Risch's AES papers on the subject.

Reviewers use their own methodolgy, usually long term A/B/A/B type comparisons with careful level matching in between switches, which have a greater sensitivity for small differences.

At least that was J. Gordon Holt's conclusion after he initially endorsed the ABX box, but found the methodology lacking in sensitivity.

Blind testing is not a "credible methodology" nor is it "scientific" for determining audible differences between components.

If you don't like the audio reviewers methodology, don't read the reviews.

Obfuscation, equivocation and affectation., posted on October 20, 2009 at 15:13:01
regmac
Audiophile

Posts: 1843
Joined: April 7, 2002
 “The pro-DBT objectivists say ...essentially that all well-designed amplifiers (implying solid state, of course) which are not clipping sound the same - as evidenced by DBTs with null results.”

Why raise this red herring…again? I’ve not advocated for DBTs. Nor are they germane to my original example.

“Commonsense would suggest it is absurd to think that any components - with different parts, different layouts, different topologies/designs, at different costs with different measurements - would sound the same.”

And commonsense would argue against the nonsensical notion that critics can always be sure of the subtle differences they purport to hear. As things stand, critics offer no evidence to support their claims (merely unproven assertions and anecdotal observations).

“The blind methodology is flawed - refer to Harley's or Risch's AES papers on the subject.”

Oh come on. That’s akin to putting the fox in charge of the henhouse. Mr. Harley has a vested interest in protecting the subjectivist position. One might as well cite the "research" of tobacco companies while arguing that cigarettes don’t cause cancer. You're not at your best today, Michael.

Common sense or lack of it?, posted on October 20, 2009 at 15:19:37
mkuller
Audiophile

Posts: 15929
Location: SF Bay Area
Joined: April 22, 2003
Contributor
  Since: December 28, 2003
 >And commonsense would argue against the nonsensical notion that critics can always be sure of the subtle differences they purport to hear. As things stand, critics offer no evidence to support their claims (merely unproven assertions and anecdotal observations).>

Critics are certain or they wouldn't put it in print.

Can they be mistaken? Are they human?

Nobody asks you to believe every word of it, and for people like you there is no evidence or proof that will satisfy you.

You merely want to criticize the critics.

What we ask is that you go listen for yourself if the product sounds interesting.

One question, posted on October 20, 2009 at 11:48:34
Analog Scott
Audiophile

Posts: 4001
Joined: January 8, 2002
 When you listen to your system do you know what is in it?

Yes, if I didn't listen blind..., posted on October 20, 2009 at 12:05:57
mkuller
Audiophile

Posts: 15929
Location: SF Bay Area
Joined: April 22, 2003
Contributor
  Since: December 28, 2003
 ...of course I knew what was in my system.

And I frequently used other listeners to confirm my impressions or identify aspects of a component's performance I might have missed.

Many times I found my first impressions were wrong, long term listening is the only way to get the measure of a component and new equipment never ceased to surprise and amaze me.

Some of the biggest names - like a top-of-the-line Krell amp I borrowed form a neighbor - were disappointing.

I don't think reviewers are affected by brand names.

Every reviewer would like to be the one who revealed that the highly touted component of the month actually sucked.

Oh crap! I meant to ask regmac not you!, posted on October 20, 2009 at 12:26:04
Analog Scott
Audiophile

Posts: 4001
Joined: January 8, 2002
 never mind.

RE: The pro-DBT objectivists say..., posted on October 20, 2009 at 11:19:28
kerr
Audiophile

Posts: 3695
Location: Central Indiana
Joined: November 10, 2003
 >Blind testing is not a "credible methodology" nor is it "scientific" for determining audible differences between components.<

Furthermore, if the reader doesn't believe the reviewers sonic impressions of the component, why should he believe the reviewer listened blind or not? Where does disbelief start and end? And why should we perceive a reviewers blind test as scientific without untold numbers of trials and, more importantly, peer review? If you want scientific results, you'll have to go all the way.

>If you don't like the audio reviewers methodology, don't read the reviews. <

Precisely! I hate liver so I don't eat it. Simple.
Hating liver? That's OK.... but you owe it to yourself to try tongue., posted on October 20, 2009 at 12:20:33
carcass93
Audiophile

Posts: 2634
Location: NJ
Joined: September 20, 2006

Jellied beef tongue, for instance (as pictured), is extremely good, consumed with horseradish and rye bread, and 50-100 ml of vodka is essential, too.

In order of preference, from best to worst: beef, veal, pork, ordinary objectivist, active HA member, HA member with at least 1000 posts (this one is barely edible, better avoided).

When the goal is beauty, accuracy and detached questioning, posted on October 20, 2009 at 09:05:39
Sordidman
Dealer

Posts: 9245
Location: San Francisco
Joined: May 14, 2001
 become secondary at best....

Surrendered to self preservation,
From others who care for themselves.
A blindness that touches perfection,
But hurts just like anything else.
Boooo, posted on October 20, 2009 at 05:54:32
Steyr
Manufacturer

Posts: 79
Joined: September 11, 2009



This is 1 lame post. So I spice it up 4u.

RE: A Paucity of Certitude Within the Community of Critics., posted on October 20, 2009 at 04:25:52
Peter Breuninger
Reviewer

Posts: 292
Joined: August 28, 2002
 This whole DBT thing has piqued my interest. My next "at bat" is the Phase Linear 700 Series one. For the listening sessions I'm going to ask the panel (Bob Grossman-Phila Orchestra, Chuck Digiorgio-Vintage Expert, Jack Speers- Circuit Expert) to wear blind folds or cover their eyes as I switch the speaker cables. I'll lay a cloth over the cables when I've completed the change.

I try to be very clear when I describe the sound of vintage and classic audio equipment. I do not beat around the bush. There are no hidden agendas. Also, I use a state-of-the-art static reference system for substitutions based upon MBL 101Es.

The Phase Linear 700 was ground breaking in its day. It's amazing how well it compares to classic highend designs such as the Jadis JA-500 and some of the more famous uber big guns from Threshold, Rowland and Krell.

GO PHILLIES!
Peter Breuninger

Ah, Phase Linear amps!, posted on October 20, 2009 at 13:38:32
John Atkinson
Reviewer

Posts: 1900
Location: New York
Joined: November 24, 2003
 >This whole DBT thing has piqued my interest. My next "at bat" is the
>Phase Linear 700 Series one.

I think you'll find it easy to tell the Phase Linear blind, Peter. it
will be the one not making any sound! :-)

The band I toured with in the mid-1970s had three Phase Linear 400s: one
for the left PA rig, one for the right PA rig, and one broken. We
were constantly rotating them as it was rare for an amp to last more than
2 nights without breaking. Went very loud, however, driving Electrovoice
horn speakers.


John Atkinson
Editor, Stereophile
RE: A Paucity of Certitude Within the Community of Critics., posted on October 20, 2009 at 10:36:40
regmac
Audiophile

Posts: 1843
Joined: April 7, 2002
 "This whole DBT thing has piqued my interest. My next "at bat" is the Phase Linear 700 Series one. For the listening sessions I'm going to ask the panel (Bob Grossman-Phila Orchestra, Chuck Digiorgio-Vintage Expert, Jack Speers- Circuit Expert) to wear blind folds or cover their eyes as I switch the speaker cables. I'll lay a cloth over the cables when I've completed the change."

Careful! Such talk can get one banned from this venue. :)


“I try to be very clear when I describe the sound of vintage and classic audio equipment. I do not beat around the bush.”

That’s what I’m after here, Peter. When a critic waxes euphoric about the sonic attributes of (xyz) product, insisting that the component in question offers a significant improvement over his reference, then he should have little trouble identifying those improvements (blind). To plead that these significant improvements are not distinguishable (blind) doesn’t pass the smell test.

“There are no hidden agendas.”

That is another thing I’m after. Confirming one’s sonic impressions without benefit of a sighted advantage eliminates much mischief. A casual, relaxed blind methodology works for the Wine Spectator and I’m convinced it can be turned to Stereophile’s advantage as well. So what if a reviewer gets it wrong occasionally? There’s no shame or crime in getting it wrong! Check your ego at the door, do the best evaluation you’re capable of and let the chips fall where they may. A critic who is willing to put his expertise on the line is much more interesting, not to mention credible, than one who resorts to a rigged game.

(I suspect what readers will find substantive and interesting is the degree to which the critic’s sighted impressions dovetail with his blind impressions.) That’s the sort of analysis that can capture an audience. Any journalist can talk a good game, but can he back up his claims (blind)? Why the reluctance on the part of audio critics to get to the bottom of things? James Laube has garnered enormous respect over the years because he’s willing to put his expertise where his palate is. Reader’s may write in and argue vociferously over his impressions of this or that wine, but his integrity remains intact. That's the sort of bulletproof ethics that a blind methodology can bring to audio journals.

Ironically, it's the manufacturer who has more at risk (with a blind methodology) than the audio journal.

"The Phase Linear 700 was ground breaking in its day. It's amazing how well it compares to classic highend designs such as the Jadis JA-500 and some of the more famous uber big guns from Threshold, Rowland and Krell."

Bob Carver is something of a maverick in the audio community, isn’t he? He seems to delight in building “giant-killers,“ products that compete with ultra-expensive units but selling for a fraction of the price.

I recall the famous incident, chronicled in the pages of Stereophile, in which Carver was confined to a motel room for several hours with his solid state amp and a pricey tube unit. After a few hours of tinkering Carver was able to make his amp sound identical to the tube unit that cost thousands more. This fact was confirmed by the critics at Stereophile. And the tube manufacturer was sweating bullets that Stereophile would go public with the name of his company.

A friend of mine has been using Carver’s large planars for years. Best of luck to you with your revolutionary testing approach. I'll be curious to learn the results.

Sweating bullets? nah!, posted on October 20, 2009 at 12:23:02
DustyC
Audiophile

Posts: 502
Joined: November 4, 2000
 If I was that tube manufacturer I'd be proud that out of all the amps on the market, Carver chose his (theirs), to mimic.
It must be, posted on October 20, 2009 at 02:59:06
Frihed89
Audiophile

Posts: 8435
Location: Copenhagen
Joined: March 21, 2005
 All amplifiers sound the same. Therefore a $150 amp is just as good as a $40,000 amp. Therefore, there is no need to listen to critics who can't hear this.

Well, that's one way to save money.
"What did the Romans ever do for us?"
RE: A Paucity of Certitude Within the Community of Critics., posted on October 19, 2009 at 17:07:37
gofish
Audiophile

Posts: 20
Location: Brooklyn
Joined: November 28, 2004
 I don't understand why someone would spend so much time writing something that really doesn't matter. Basically what you are saying is that reviewers are inept and/or corrupted by their need to keep all these manufacturers in business to get their hands on long term loaners or heavy discounts or are they just so devoid of morality that they want to lead us on and what? bankrupt us? .

If you truly believe that or anything similar then why are you here in the first place? Trying to save the admittedly dwindling number of us that buy hi end gear. Minus the odd person who will dump a wad of cash sight unseen, most of us know the game. Read the mags for information and entertainment then try and find a place to listen to what we read about.

Did you buy all of your equipment under blind conditions? Do you listen to a table radio and are just here to stir things up?

Just for transparency I tend to believe in system synergy, have kept most of my system together for over a decade, and that power cords and wire don't make much of a difference in the price bracket I shop in. I bought all of my stuff sighted except when I covered my eyes as the credit card was being run through.

so what you are saying is..., posted on October 20, 2009 at 11:59:12
Analog Scott
Audiophile

Posts: 4001
Joined: January 8, 2002
 Regmac is saying that reviewers are lying about the virtue of products to keep the makers of such virtueless products in business so the reviewers can get these virtueless products for less cash and lie about them again.

That isn't just devoid of morality. It just don't make sense. Why would reviewers want products that they have to lie about to get others to buy?

Reviewer says under his breath.

"Yeah this product really doesn't offer anything special but I will say it does because I gotta keep em in business so I can keep getting stuff from them I don't really truly like!"

RE: so what you are saying is..., posted on October 21, 2009 at 05:49:14
gofish
Audiophile

Posts: 20
Location: Brooklyn
Joined: November 28, 2004
 Not quite. He is asserting that sighted reviews bias a reviewer to the point they cannot properly evaluate the product. he is also asserting that they know that and still continue to do it (i.e. corrupted). That doesn't mean the products themselves are worthless just the reviews of them. I don't believe he thinks the reviewers are lying about the products just being deluded.

I was also asking if he followed his own advice and auditioned/purchased his own equipment using the DBT methodology

RE: so what you are saying is..., posted on October 22, 2009 at 06:29:57
Analog Scott
Audiophile

Posts: 4001
Joined: January 8, 2002
 You asked the million dollar question there. I see no one on the side in favor of insisiting reviewers prove themselves with DBTs stepping up and asnswering that question for themselves. go figure

RE: A Paucity of Certitude Within the Community of Critics., posted on October 19, 2009 at 17:34:45
Spendor Harbeth
Audiophile

Posts: 601
Location: Left Coast
Joined: May 15, 2009
 Dude, thanks for this post. It puts it all nicely into perspective. Well done.

RE: A Paucity of Certitude Within the Community of Critics., posted on October 19, 2009 at 16:19:39
Spendor Harbeth
Audiophile

Posts: 601
Location: Left Coast
Joined: May 15, 2009
 There is no doubt in my mind that many other cues come in to play when evaluating hifi components and speakers.

Brand name and face plate, as mentioned here, play a huge role. When a reviewer mentions Nordost Valhallas, dcs digital rigs,or Wilson speakers you can almost hear there knees knocking like teenage girls at a John Mayer concert. I bet they get sweaty palms too. They are like Pavlov's dogs, all drooley before they even hear a note.

Reputation, mostly created by the hyperbolic audio press precedes many items that are reviewed.

But what's in it for you (us)?, posted on October 19, 2009 at 14:20:05
carcass93
Audiophile

Posts: 2634
Location: NJ
Joined: September 20, 2006
 Suppose Smith can - or cannot - discern between two amps in blind test. What does that tell you about YOUR ability to discern between them with your ear/brain, in your audio system?

What value would this add to the review, especially in "cannot" case? Would you require magazine to print hearing evaluations of all of its reviewers, to add some credibility to these results? Even if that was the case, I personally wouldn't give a damn - hearing test surely doesn't tell the whole story about how a particular individual would percept MUSIC.

The link below is to a 6moons comparison (sighted) between some digital cables, where reviewer CAN'T hear any difference between some of them, including toslink vs. coax. In your opinion, what conclusion could be drawn from this, besides possible inadequacies in his system and/or hearing?

Yeah, I read that., posted on October 19, 2009 at 14:10:10
keith_d
Audiophile

Posts: 931
Joined: June 15, 2002
 Oops, I lied.

“But it takes months to detect subtle differences…”, posted on October 19, 2009 at 13:45:57
Enophile
Audiophile

Posts: 7139
Location: Las Vegas
Joined: October 15, 2005
 I find that even if it does take a while to identify a difference, once I have done so, it becomes instantaneoulsy identifiable therafter.

Once identified, differences become much more obvious.

If you were to want to include blind listening, I think it would be coolest to do it at the end of a review.

So, yes, it may take a while to identify, but thereafter should be readily apparent - even when faced with the daunting task of blind listening.

_

As to visual bias, I think this diminishes over time and with the sum of different items that have been used in the past - I give credit for reviewers being able to listen past a product's visuals over time. Many have literrally 'seen it all,' so to speak. I would think they get a little desensitized to this aspect.

___

I recall a reviewer discussing this topic mentioning that if reviewing were to be made more rigorous, he would stop, as that sort of listening is not 'fun.' Given the current shortage of potential reviewers, we have to be careful not to alienate them.
RE: A Paucity of Certitude Within the Community of Critics., posted on October 19, 2009 at 13:19:25
andy19191
Audiophile

Posts: 158
Location: Europe
Joined: June 26, 2008
 > It seems there are but two conclusions one can draw from this scenario: 1) the two
> amps sound the same; 2) Smith’s powers of discernment aren’t what they used to be.

3) visual and others cues play a significant part in sound perception?

> The intellectually honest response is to admit one doesn’t *know* until one confirms
> his listening impressions (blind).

Why? Audiophiles do not listen their stereos under blind conditions but sighted conditions. What they perceive blind and sighted is not generally the same.

What is Smith trying to achieve when he/she writes a review? What does the publisher of the review want from Smith's review? What would happen to the expensive end of home audio if reviewers did perform conventional hardware reviews using controls, references, blinding and the like?

I would suggest that it might be wise to consider a few possible failings in your own interpretation of what is going on before being too critical of others.
I am shocked that you would even get half of it right., posted on October 20, 2009 at 11:53:29
Analog Scott
Audiophile

Posts: 4001
Joined: January 8, 2002
 "> It seems there are but two conclusions one can draw from this scenario: 1) the two
> amps sound the same; 2) Smith’s powers of discernment aren’t what they used to be.

3) visual and others cues play a significant part in sound perception?

> The intellectually honest response is to admit one doesn’t *know* until one confirms
> his listening impressions (blind).

Why? Audiophiles do not listen their stereos under blind conditions but sighted conditions. What they perceive blind and sighted is not generally the same."

Where did this sudden burst of insight come from?

"What would happen to the expensive end of home audio if reviewers did perform conventional hardware reviews using controls, references, blinding and the like?"

Why did it go away as suddenly as it came?
RE: I am shocked that you would even get half of it right., posted on October 20, 2009 at 14:05:36
andy19191
Audiophile

Posts: 158
Location: Europe
Joined: June 26, 2008
 > Where did this sudden burst of insight come from?

Like most, I have always known since a small child that my perception of sound did not always line up with what had happened. As a teenager this was reinforced by messing about with sound kit and thinking something was in circuit when it was not. I have a recollection of being embarrassed in front of others at a school play. Formal knowledge first came from sound perception lectures many decades ago and has been added to occasionally since. It is not a subject that holds much interest for me.

> Why did it go away as suddenly as it came?

Assuming you are referring to conventional reviews, changing market conditions.
RE: A Paucity of Certitude Within the Community of Critics., posted on October 19, 2009 at 14:23:12
regmac
Audiophile

Posts: 1843
Joined: April 7, 2002
 “visual and others cues play a significant part in sound perception?”

I’ll bet so. Up to and including the name on the faceplate.

“What would happen to the expensive end of home audio if reviewers did perform conventional hardware reviews using controls, references, blinding and the like?”

That’s a great question and I’m not sure what sort of consequences would arise. I do know that the Wine Spectator adopted such procedures decades ago and appears no worse for it (with a circulation of nearly 350,000). Nor has the demand for expensive wines lessened due to its blind methodology, quite the opposite. (The price of ultra-premium wines has skyrocketed over the past two decades.)

I’m also cognizant of what has happened to hi-end audio over the same time span: it has suffered a steady decline. If memory serves, the bestselling audio journal has a circulation of about 80,000. And I’m quite sure a certain someone will correct me if my number is wrong. :)

Time for dinner and a concert.

RE: A Paucity of Certitude Within the Community of Critics., posted on October 20, 2009 at 13:44:54
andy19191
Audiophile

Posts: 158
Location: Europe
Joined: June 26, 2008
 > “What would happen to the expensive end of home audio if reviewers did perform
> conventional hardware reviews using controls, references, blinding and the like?”
>
> That’s a great question and I’m not sure what sort of consequences would arise.

Consider a more professional version of the Audio Critic, cleaned up by removing the audiophile bashing and some of the more irritating editorial comment and published regularly every month.

What proportion of the posters to AA would subscribe to such a publication?

How many times can one publish a conventional review of the technical capabilities of competent amplifiers, DACs, CD/DVD players, hard disks, cables, etc... before readers lose interest in purchasing home audio as a hobby?

How many manufacturers of amplifiers, DACs, CD/DVD players, hard disks, cables, etc... would want to pay to have adverts of their products next to reviews stating or implying that a modestly priced product is functionally good enough?

There could be enthusiastic reviews of speakers and room treatment but what about record players? How would such a magazine handle them?

I am confident that such a magazine is not commercially viable as a mainstream paper publication. I suspect it might be as a niche web publication although neither Audio Critic or Audioholics makes much of a case.

> I do know that the Wine Spectator adopted such procedures decades ago and appears
> no worse for it (with a circulation of nearly 350,000). Nor has the demand for
> expensive wines lessened due to its blind methodology, quite the opposite.

There is a rather big difference between blind tasting of wine and blind listening to competent non-electromechanical audio components: people perceive differences between the wines and they don't between the audio components. When it comes to sighted testing of wine and audio components then a comparison is much more reasonable as others cues kick in to change the perception of both taste and sound.

> I’m also cognizant of what has happened to hi-end audio over the same time span: it
> has suffered a steady decline.

After the stereo boom it was less a question of decline but more a question of what to do in order to survive for the home audio sector in the developed world. What was done was to move towards the audiophile sector as we know it today albeit in a less extreme form initially. Changing away from conventional reviews of technical equipment in the mainstream press was one of the requirements for the audiophile view to move from the fringes where I suspect it had probably always been around to the mainstream and grow.

RE: A Paucity of Certitude Within the Community of Critics., posted on October 22, 2009 at 04:31:46
kerr
Audiophile

Posts: 3695
Location: Central Indiana
Joined: November 10, 2003
 >Consider a more professional version of the Audio Critic, cleaned up by removing the audiophile bashing and some of the more irritating editorial comment and published regularly every month.

What proportion of the posters to AA would subscribe to such a publication?

How many times can one publish a conventional review of the technical capabilities of competent amplifiers, DACs, CD/DVD players, hard disks, cables, etc... before readers lose interest in purchasing home audio as a hobby?

How many manufacturers of amplifiers, DACs, CD/DVD players, hard disks, cables, etc... would want to pay to have adverts of their products next to reviews stating or implying that a modestly priced product is functionally good enough?

There could be enthusiastic reviews of speakers and room treatment but what about record players? How would such a magazine handle them?

I am confident that such a magazine is not commercially viable as a mainstream paper publication. I suspect it might be as a niche web publication although neither Audio Critic or Audioholics makes much of a case.<

Such a publication would be of very limited use to those interested in high performance audio. "Good enough" just isn't good enough to the average audiophile. The interest in such a magazine would be very low indeed. To use the long worn out car analogy, it would be like a car reviewer writing about a Yugo - "Well, it gets me from Point A to Point B just as well as a Porsche, so it's good enough".

People don't perceive differences between audio components? Who told you that?, posted on October 20, 2009 at 14:52:49
carcass93
Audiophile

Posts: 2634
Location: NJ
Joined: September 20, 2006
 Yes, there are people who don't, apparently yourself included - but why would anybody be interested in such a fact? Especially here, on audiophile forum, where ability to perceive them is significant part of the hobby?

Oh, that's right - you actually made it up. Nevermind...

RE: People don't perceive differences between audio components? Who told you that?, posted on October 20, 2009 at 15:00:31
andy19191
Audiophile

Posts: 158
Location: Europe
Joined: June 26, 2008
 > People don't perceive differences between audio components?

Please quote in context what you responding to and reference something that shows it to be false.
I'm sure you're capable of finding in your own post what I'm responding to., posted on October 20, 2009 at 15:16:41
carcass93
Audiophile

Posts: 2634
Location: NJ
Joined: September 20, 2006
 But, just in case I'm overestimating you, here goes:

"people perceive differences between the wines and they don't between the audio components."

As for reference - let's see you demonstrating first that people don't perceive differences between audio components. Like, demonstrate to me that I don't, for instance. Otherwise, your statement above is just a BS - or, in other words, is made up, as I said before.

RE: I'm sure you're capable of finding in your own post what I'm responding to., posted on October 20, 2009 at 15:56:00
andy19191
Audiophile

Posts: 158
Location: Europe
Joined: June 26, 2008
 > I'm sure you're capable of finding in your own post what I'm responding to.

The point of the request was, of course, to indicate your dishonesty since anyone interested can look at the context in the original post. Your choosing to dig a deeper hole is not wholly unexpected given your past posts but, on the other hand, the name calling is still absent.
Do you really think high end audio is in decline because magazines don't do blind testing?, posted on October 20, 2009 at 08:17:15
Rob Doorack
Audiophile

Posts: 4706
Joined: May 26, 2000
 Do you actually believe that consumers are saying to themselves "I'm going to Best Buy and get a Bose one - box HT rig rather than a high end component system from an audio salon because audio magazines don't do rigorous blind tests?" Are you actually arguing that the explosion of other home entertainment options - the internet, game consoles, home theaters, iPods, cell phones, and so on - is less significant than the absence of blind tests in some hobbyist magazines?

You are a pyromaniac in a field of straw men., posted on October 20, 2009 at 09:54:26
regmac
Audiophile

Posts: 1843
Joined: April 7, 2002
 “Do you actually believe that consumers are saying to themselves "I'm going to Best Buy and get a Bose one - box HT rig rather than a high end component system from an audio salon because audio magazines don't do rigorous blind tests?" Are you actually arguing that the explosion of other home entertainment options - the internet, game consoles, home theaters, iPods, cell phones, and so on - is less significant than the absence of blind tests in some hobbyist magazines?”

I argue no such thing. A poster was concerned about adopting a blind methodology and asked me to consider the consequences that might arise if audio journals were to adopt a methodology that would control for personal bias. I merely made the observation that adopting such a methodology has done the Wine Spectator no harm (if its robust circulation, pricey subscription fee and booming wine sales are any indication).

I would agree that modern gadgetry has contributed to the decline of expensive two-channel audio. A point I’ve made (on here) before.

That's exactly what you implied...(nt), posted on October 20, 2009 at 10:50:34
mkuller
Audiophile

Posts: 15929
Location: SF Bay Area
Joined: April 22, 2003
Contributor
  Since: December 28, 2003
 (nt)

RE: A Paucity of Certitude Within the Community of Critics., posted on October 19, 2009 at 12:03:12
robert.gaboury@videotron.ca
Manufacturer

Posts: 132
Location: Canada
Joined: October 16, 2007
 What if Smith is an auto proclaimed reviewer.
What if Smith listened to only 2 or three amps before deciding this or that is the best (or worst) he ever listened to.
What if Smith thinks he's Mr Big and wants to bully his way up the ladder of influence.

Does it really matter if Smith used any "scientific" method to reach such and such conclusion?

I'll trust any experienced reviewer over any inexperienced one, and make up my own mind to whether or not I trust the opinion expressed. Because that's what its all about: Opinion. What's dangerous is biaised opinion on accounts other than sound.

You mean "adopting a pseudoscientific approach"...(nt), posted on October 19, 2009 at 11:59:35
mkuller
Audiophile

Posts: 15929
Location: SF Bay Area
Joined: April 22, 2003
Contributor
  Since: December 28, 2003
 (nt)

RE: A Paucity of Certitude Within the Community of Critics., posted on October 19, 2009 at 11:54:07
bjh
Audiophile

Posts: 11876
Location: Toronto, Canada
Joined: November 22, 2003
 >It seems there are but two conclusions one can draw from this scenario: 1) the two amps sound the same; 2) Smith’s powers of discernment aren’t what they used to be.

Nothing but those two eh? Can't think of any others? LOL
Everything matters, don't forget to tweak your placebos!

Page processed in 0.238 seconds.

[ Top Top of Page ] [ Contact Us ] [ Support/Wish List ] [ Copyright © 2009, Audio Asylum, all rights reserved ] [ Go Green ] [ Take5 ]
[ General ] [ Speakers ] [ Tubes ] [ Vinyl ] [ Digital ] [ Hi-Rez ] [ Video Asylum ] [ Cables ] [ Tweaks/DIY ] [ Music ] [ Films ]