|
Audio Asylum Thread Printer Get a view of an entire thread on one page |
For Sale Ads |
209.104.141.158
In Reply to: Stereophile - Some Data posted by John Atkinson on January 24, 2006 at 07:05:01:
The primary problem with Stereophile is the strong bias toward favorable reviews which seems to lead to an enormous recommended components list.The list would provide much more value if it was 90% shorter ... and also included a NOT recommended list of components reviewed that year (a very short list!).
The odds of a favorable Stereophile review are so high that if you merely assume the component received a favorable review, you will almost always be right, and won't have to read the review!
In fact, I recommend reading the list of components reviewed in the table of contents and then going to an audio store to audition any component that interests you.
After the audition you may want to read the entire component review, although it would be redundant at that time.
Reading a positive review first is likely to bias an audition that follows.
Positive reviews are what the advertisers want, IMHO.
Doesn't matter if it's their products or others -- a high percentage of positive reviews keeps the advertisers happy. IMHO.
Stereophile readers, however, deserve more than "happy face" reviews.
The audiophiles who used to criticize Julian Hirsh reviews in Stereo Review and Sound & Vision are getting the same positive-review bias in Stereophile.
I challenge Stereophile to publish a list of reviewed components that are NOT recommended, and why they are not recommended (one sentence is fine) -- adding real value to the ridiculously large list of recommended components we see every year.
Stereophile offers a perfect example of how NOT to do a recommended components list!
But the list is not the core problem -- the root cause of the huge list is the positive review bias.
If that bias has absolutely no link to the fact that Stereophile is supported mainly by advertising, then I'll eat my hat.
Follow Ups:
> Red Herring data -- perhaps correct, but not important
Ah, Mr. Greene -- or may I call you "Bass Nut"? -- it must feel
good to be so righteous. Yet the fact is that you don't appear to
have comprehended my post and its data. I was clearly responding to
the statement made by Gregg Straley of Reality Cables on his
website that "Magazine [sic] like Stereophile and Absolute Sound
require you to advertise with them for about 6 months before they
review your products." I thought the fact that in 7 consecutive
issues of Stereophile, 48 out of 90 brands reviewed do _not_
advertise definitively proved Mr. Straley's statement false. Why is
that a "red herring"?
> The odds of a favorable Stereophile review are so high that if you
> merely assume the component received a favorable review, you will
> almost always be right, and won't have to read the review!
This is an old strawman argument that has been addressed both here
and in my magazine, Richard. As I have patiently but apparently
fruitlessly explained before, Mr. Nut, we attempt to cherry-pick
products for review on the grounds that they are worth writing about,
ie, will probably sound good. If we do that job perfectly, 100%
of the products we review will subsequently appear in "Recommended
Components." The fact that the percentage is less than 100% suggests
we should try harder at that goal. :-)
> a high percentage of positive reviews keeps the advertisers happy.
But no, we don't do this to please advertisers. Otherwise, how do
you explain why so many products from _non_-advertisers also appear
on the list? Why would we want to please those people? Or are you
going to retreat, as one inmate did a while back, into spluttering
that _that_ merely proves how tricky we are?You can get back in your box, now, Mr. Bass.
John Atkinson
Editor, Stereophile
"Ah, Mr. Greene -- or may I call you "Bass Nut"? -- it must feel
good to be so righteous."RG: Meaningless character attack #1
.
.
.
"Yet the fact is that you don't appear to have comprehended my post and its data."RG: Meaningless character attack #2
.
.
.
"I was clearly responding to the statement made by Gregg Straley of Reality Cables on his website that "Magazine [sic] like Stereophile and Absolute Sound require you to advertise with them for about 6 months before they review your products." I thought the fact that in 7 consecutive issues of Stereophile, 48 out of 90 brands reviewed do _not_ advertise definitively proved Mr. Straley's statement false. Why is that a "red herring"?RG:
Someone slings mud at Stereophile.
Stereophile is bad!
You counter with statistics to prove him wrong.
Now Stereophile is good!
You win the "battle".However this was a meaningless "battle" over a false charge which served only as a distraction (red herring) from the real problem with Stereophile: -- Too high a percentage of positive reviews and far too many recommended components.
.
.
.
> The odds of a favorable Stereophile review are so high that if you
> merely assume the component received a favorable review, you will
> almost always be right, and won't have to read the review!YOUR REPLY:
"This is an old strawman argument that has been addressed both here
and in my magazine, Richard. As I have patiently but apparently
fruitlessly explained before, Mr. Nut, we attempt to cherry-pick
products for review on the grounds that they are worth writing about,
ie, will probably sound good. If we do that job perfectly, 100%
of the products we review will subsequently appear in "Recommended
Components." The fact that the percentage is less than 100% suggests
we should try harder at that goal. :-)"RG: The argument may be old but it is correct and can't be refuted by the infamous and lame "Julian Hirsch defense". Stereo Review readers also complained there were too many positive reviews. Hirsch defended himself by claiming his editors selected only very good components for his reviews. It was suggested that if a reader was very careful reading each word of a Hirsch review, he might be able to differentiate between good products and very good products.
In StereoReviewLand, just as in StereophileLand, virtually all products are above average!If Stereophile editors are so good at "cherry picking" above average products, then there's little need for publishing detailed reviews. Just present a "cherry-picked list" of products every month and readers can assume they are all good products. The probability of that assumption being wrong seems very low. The magazine could have one dozen components on each cherry-picked list -- versus only 4 or 5 full reviews every month. Much more useful for readers!
.
.
.
> a high percentage of positive reviews keeps the advertisers happy.
"But no, we don't do this to please advertisers. Otherwise, how do
you explain why so many products from _non_-advertisers also appear
on the list? Why would we want to please those people? Or are you
going to retreat, as one inmate did a while back, into spluttering
that _that_ merely proves how tricky we are?"RG:
Your analysis does not support the conclusion (I'm assuming the questions you write imply conclusions).
You review products from non advertisers for three reasons:
(1) You would be accused of favoritism if you did not
(2) You would have too limited coverage of all available products (many, perhaps most, manufacturers do not advertise in Stereophile)
(3) Non-advertisers today are potential advertisers tomorrow.Advertisers want to be confident the money they invest convincing Stereophile readers to buy their products over many months or years is not likely to be offset instantly by a negative review. They can be confident their investment will not be undermined by a negative review if the magazine rarely publishes negative reviews! This is true of Stereophile, Absolute Sound, Sound & Vision and possibly every magazine supported by advertising that wants to stay in business!
.
.
.
"You can get back in your box, now, Mr. Bass."
John Atkinson
Editor, StereophileRG: Meaningless character attack #3
PS: My local Birmingham Michigan library is one of the few that has Stereophile in its magazine collection -- maybe the only one!. I can read your magazine for free ... but I don't bother.Reason: Any magazine editor who thinks a 500+ Recommended Components list is useful, and writes: "If we do that job perfectly, 100% of the products we review will subsequently appear in "Recommended Components." " ... is the king of "Happy Face Reviews".
Move over Mr. Hirsch, Mr. Atkinson wants your title!
Mr. John Atkinson
Future Title: World Champion Audio Happy Face Review Editor
Richard,I don't see Stereophile (or any audio magazine) as a Consumer Reports. I don't want to read about components that sound bad. I do want to read about things that sound good or great, that have something special about them - something that might make me want to find out more - because I'm a connoisseur. By the way - you are, too.
And connoisseur magazines don't waste their readers' time by devoting column inches to bad cigars (Cohiba Esplendido = good, Swisher Sweets = bad), nor do Haut Horologie magazines cover ugly watches (Chopard, very elegant ... G-Shock, not so much). Same for magazines about fine wines, classic cars, etc.
Many people seem to be making the mistake that Stereophile somehow has the core responsibility of a consumer advocate magazine such as Consumer Reports. It does not. It is a journal for the audio connoisseur, regardless of the expense level the connoisseur is participating at, and it is with this in mind that they publish their reviews ... in order to inform the connoisseur of something they might like. Not of something they might dislike.
However, there may be room in the marketplace for an audio consumer-advocate magazine ala Ralph Nader. I don't know for sure ... I don't think I'd be interested in it, but I'm sure others might. How many others is hard to determine ... it'll take a brave soul such as yourself to try it out.
Cheers!
Chris
"In theory there is no difference between theory and practice. In practice there is." - Yogi Berra
Were I running the magazine, I'd categorize them as "advertisers" and "potential advertises" and treat them both with equal deference.
But that's just me. ;-)
__________________________________________________
Boo!
This post is made possible by the generous support of people like you and our sponsors: