After finishing the article, however, it is clear that all of his points are really circular, "chicken and the egg" type of arguements.

He first points out that he "has never been approached by a manufacturer offering equipment, long-term loans of equipment, or any other compensation for favorable coverage." Well, so far, it doesnt seem that any manufacturer would actually need to, does it? The fact that manufacturers are aware of probable favorable reviews from their loan of products doesnt provide much evidence for pure motives of reviewers, in my opinion.

Next, Harley says that he is pretty sure there is no quid pro quo between manufacturers and reviewers by offering the following evidence. "The reality is that I could spend an afternoon on the phone and assemble a reference-quality system of components on long-term loan, components of my choosing - before a word has been written, and with no promise of a favorable review."

Yeah, no kidding. If TAS or others never blast a coponent for being an over-prices piece of crap, manufacturers will always be willing to send whatever Mr. Harley wants. What do you suppose would happen if, let's say, one-quarter of reviewed products were miserably reviewed? How many reference-quality components do you suppose you would get then, Mr. Harley?

He goes on to then justify long-term loans of equipment to reviewers by explaining that reviewers benefit from having a reference standard by which to judge other reference components. I can understand that, I suppose. He says this has value to the reader. But, the value must not be too great, because even those long-term loans are limited by TAS policy to 6 months. I would think that if the value to readers is so significant, that maybe the reviewer should be allowed to keep the component indefinitely, after TAS actually purchases it.

Anyway, sorry to rant, but his arguement just seems weak to me.

 Re: TAS Editor's comments defending the review process - hexenboden 18:40:07 05/23/04 (2)
 Re: TAS Editor's comments defending the review process - Sean 01:02:15 05/25/04 (1)
 Harley's sentence... - LarryB 07:35:37 05/25/04 (0)

...

Show full thread outline!

 Re: TAS Editor's comments defending the review process - hexenboden 18:40:07 05/23/04 (2)
 Re: TAS Editor's comments defending the review process - Sean 01:02:15 05/25/04 (1)
 Harley's sentence... - LarryB 07:35:37 05/25/04 (0)
 I don't get your rant. - bwkendall 11:26:12 05/23/04 (2)
 No, you certainly don't! - uw312@aol.com 14:16:45 05/23/04 (1)
 Just a couple of comments..... - bwkendall 14:48:59 05/23/04 (0)
 Re: Well then, how about this arguement. for favorable reviews... - mkuller 14:44:23 05/22/04 (8)
 Re: Well then, how about this arguement. for favorable reviews... - TomLarson 18:06:48 05/26/04 (0)
 For the most part, I agree... - uw312@aol.com 14:54:22 05/22/04 (2)
 Re: The reviewer's job.... - mkuller 10:48:23 05/26/04 (0)
 Re: For the most part, I agree... - elmuncy@yahoo.com 15:55:33 05/22/04 (0)
 Re: Well then, how about this arguement. for favorable reviews... - Fisherdude 17:10:51 05/22/04 (0)
 If products standards have gone up in 20yrs, maybe reviewer standards need to go up too! (nt) - Steve Cortez 19:39:15 05/22/04 (0)
 Re: Well then, how about this arguement. for favorable reviews... - jrb15 20:17:55 05/22/04 (1)
 Re: I always thought it was the reviewer's job to compare... - mkuller 11:39:12 05/24/04 (0)
 Re: TAS Editor's comments defending the review process - Sean 15:32:40 05/22/04 (10)
 Re: TAS Editor's comments defending the review process - uw312@aol.com 09:51:25 05/23/04 (0)
 well said sean. - nigelbetts 00:58:29 05/23/04 (8)
 Then you and Sean must be test-tube people. - bwkendall 16:19:30 05/23/04 (2)
 Re: Then you and Sean must be test-tube people. - nigelbetts 18:03:36 05/23/04 (1)
 I don't know Sam and have never read his stuff. - bwkendall 16:00:46 05/24/04 (0)
 Can I say something? - Sue Kraft 06:18:55 05/23/04 (4)
 Re: So Sue, have you blasted any of the products you've reviewed? (nt) - mkuller 11:57:08 05/23/04 (2)
 no - Sue Kraft 12:46:49 05/23/04 (1)
 Sue, some people don't 'get' that a bad component... - mcbrion@aol.com 11:18:58 06/01/04 (0)
 Re: Can I say something? - nigelbetts 09:15:10 05/23/04 (0)
 Something I would like to see. . . - unrulyjulie 22:20:00 05/22/04 (3)
 What nonsense...a review doesn't take months?! - mcbrion@aol.com 11:10:52 06/01/04 (1)
 Typo correction on the last five lines of my post... - mcbrion@aol.com 10:40:45 06/02/04 (0)
 Re: Something I would like to see. . . - mkuller 12:14:15 05/23/04 (0)

I would like to see a list of all the items submitted for review in each issue.

My guess is that many pieces are submitted, but few are chosen for a written review (for whatever reason). If the reviewer has a sub-optimal piece of gear to review, it's much easier (and less likely for lawsuits) to decline to review it rather than write a bad review.

Hence the mostly positive reviews we see. The exception may be equipment for which review is demanded by its popularity on other forums and the review just doesn't like.

I would also like to see a "second opinion" on each piece of gear. I find it hard to believe that these magazines don't have the resources to let someone else listen to something for a week or two.

And no, I really don't buy that it takes months to review a piece, unless the reviewer has another full-time job/family/etc.

I don't know how many review for a 'living.' or as a primary pasttime

PS... I wonder what he considers "long term" in terms of how long someone keeps a piece of gear for review and / or "temporary use"???

Most manufacturers do whatever they can to accomodate a magazine and a reviewer within reasonable guidelines. Many times they will even arrange for the reviewer to be sent some associated equipment from other manufacturers that they think brings out the best in their products. Most magazines and reviewers are concerned primarily with maintaining their independence, credibility and delivering a thorough, fair, and impartial review. Manufacturers that make the process easier and have great and interesting products to write about (Musical Fidelity, for example) tend to get more reviews.

Sorry, conspiracy buffs, but that's reality.
Regards,
Mike

Mike:

A couple of observations:

First: audio products--including many loudspeakers--are not, to my ears, necessarily better than the very good, and fairly priced, high end audio products of twenty years ago. I've needed to augment some Linn LK-280/Sparks Power Supply amps for the last few years. Only one amplifier I've had in my home (a previous generation Chord)--among a group of expensive high end players (Rowland, BAT, Audio Research, YBA, ATC, etc)was I thought even marginally better into some four or so different pairs of loudpseakers.

Second: perhaps a few honestly critical reviews might have dampened the reckless price inflation that has contributed to the demise of the high end. I remember some years ago a series of articles in, I believe, Positive Feedback that compared--to the point of preferring--the sanely priced Von Schweikert to what was then the current generation of the WATT/Puppies. Even then, the point taken was, I believe, that the VS's were a steal. The point should have been that the Wilson's were embarrassingly overpriced--as have been, I suspect, the vast majority of high ticket items that have dragged the cost of high end audio into its death spiral with dwindling demand. Pearson started it with what I think was his basic lack of common sense, but reviewers have followed him like lemmings.

If you tiptoe through the posts here, you'll repeatedly find folks questioning the worth of, after listening, critic's darlings. I don't think they are all simply cheap or all wrong. Critics should have a determined interest in sifting the junk from the good stuff, not telling their readers that it is all good. It isn't.

Good grief! That's EXACTLY what readers want to find out, whether or not it's possible, or easy, or difficult, to get the best out of a piece of equipment, or combination of equipment, or whether or not the reviewer liked or didn't like what it did!

You've just made the argument that reviewers only review what they already like, or what other reviewers already said they should like, or what they're afraid not to like.

I think you are right. But, I have been reading audio mags on and off for more than 25 years, and I have yet to see a reviewer say "You know, this is just a bad product that really needs to be re-thought." or "This product is adequate, but the truth is, you can do a lot better for the money." But, when I go listen to components, particularly speakers, I find that there is only maybe 20 percent that I would even want to consider taking home for any length of time. You mean to tell me reviewers NEVER stumble onto those types of products?

2.) most reviewers don't want to spend the time and work necessary to review a product if they don't like it - it's very difficult to be inspired to write about a product that is inferior - unless it is highly regarded or has been favorably reviewed elsewhere. Otherwise there are too many good to great products the reviewers would rather spend their time with, and

You mean, reviewers only review products that they already know they like, or that everyone else says they like. Jeepers!

3.) as a former reviewer, I was reluctant to write a negative review about a product because I was afraid that just because I was not able to get its best out of it, or personally didn't like what it did, others may not feel the same way about it - and a negative review could do serious damage to a small manufacturer

If you were reluctant to write a negative review (or, conversely, predisposed to writing a positive review) then you simply weren't doing your job. A reviewer in my opinion shouldnt base his review on how one person out of a thousand potential users MIGHT feel about it, nor do I think a reviewer should worry about what might happen to a company for fear of telling the truth.

I'm not suggesting there is some big conspiracy, I just think that as long as a magazine is largely funded by the companies whose products get reviewed, reviewers are going to cop out, the way you mentioned you did. Consumer Reports doesnt seem to be offended when they rate a product as inferior and their testers / reviewers are free to be truthful.

Maybe other reviewers are more fortunate than I am, but I don't get to pick what I review. I have made suggestions from time to time of things I'd like to review, but rarely do I get my wish. I simply get review assignments, which I suppose I could turn down if I wanted to. But to date, I have not turned any of them down. I just figure I'll review whatever they send me, and hope I like it when it shows up. Maybe others can "cherry pick" what they review, but that has not been my experience.

Sue

With the likes of our writer buddies here--it's like they are in the skyboxes watching the game, whereas the forums are not even the seats--they are standing room only, i do believe that the likes of "some" writers find it very hard to be down in the stands with the regulars fans --& cannot adjust themselves accordingly.In the stands your toes will be stepped on as such.
They might have control over their editorials, but in no way do they have control of the posters here.(as far as we can see at least).

when the posters smell a rat as such(even if they are wrong), then they are going to say, so the result is that theres not gonna be any editorial control.This is live here(well at least it used to be--i think the taped delay has recently gone from 3 seconds to 10 seconds).

Even the most antagonistic posters in here i believe do have some element of respect for the writers(some more than others).
Sam T is a perfect example of how to alienate yourself--yes he has a base of diehard fans, but he just recently succeded in pushing a lotta wouldbes over the other side of the fence.
IMHO.
thanks again for your even keeled explaination . nigel.

Re-read my original post and you may find the answer!

What is the big deal about manufacturers and long-term loans? Do you think reviewers have nothing but time, and are able to immediately focus on a component as soon as it arrives? I suspect there are times when there two or three sets of speakers a couple of amps and maybe a turntable all competing for review time. If the review is going to be thorough, it must be done of time and through extended listening sessions. First impressions are not always accurate. BTW, most reviewers also have day jobs and families that consume most of their time.

I don't know what you do for a living, but I know I have access to financial and legal services at pricing most people would love to get because of my profession. If a reviewer or employee of an audio magazine gets a slightly discounted rate on some demo speakers or CDP, who cares?

I think most people who are critical of the review process are actually just jealous because they don't get a stream of top-notch components delivered to their door to listen to for awhile. Personally, I think it would be a pain in the ass to have to tear my system apart, put it back together again with the review component and then devote the time necessary to get the best performance out of the item being reviewed. Maybe I only know how to do these things in slow motion, but it would take me more time than I would be willing to give up.

Unfortunately, it doesn't work quite that way. Products are offered or requested, then a reviewer is usually assigned, and the component is sent to him. He takes a couple of months working with it, is either given a deadline for the review or just submits it - it may or may not be included in a particular issue.

"I would also like to see a "second opinion" on each piece of gear. I find it hard to believe that these magazines don't have the resources to let someone else listen to something for a week or two."

TAS used to do this and it was an important part of the magazine until it changed ownership and was discontinued, most likely for logistical and time reasons.

"And no, I really don't buy that it takes months to review a piece, unless the reviewer has another full-time job/family/etc. I don't know how many review for a 'living.' or as a primary pasttime"

Very few reviewers do this as a full-time job. My guess would be two or three people at TAS and Stereophile are full-time (including the editors who read and edit all the copy) and the rest have other full-time jobs, families, friends and lives.

Here's an exercise - try taking your newest audio component and write a review of it. Be sure to use it with numerous other components and cables to insure you're getting the most performance out of it. Be sure to use musical examples anytime you describe a positive or negative aspect of it's musical reproduction. Write, say, 1500 words and let me know how long it takes.
Regards,
Mike

It's like me saying. You know what, I have a lot of friends even though I serve everyone filet mignon and shots of King Louie. How do I know I have a lot of friends? Because they come over all the time and it has nothing to do with what I do for them. Harley is saying that "I know my reviewes must be honest because I can make a phone call and get manufacturers to send me any component that I want. Getting any component I want with a phone call must prove that my always positive reviewers must be honest."

I was simply wondering how many manufacturers would be eager to send products if a significant percentage of them got bad reviews. I wonder how many of my "friends" would continue to visit if I served BigMacs and Pepsis?

And I said that I didnt have a problem with long-term loans. I understand how that could help set a benchmark for the reviewing of other components. I wondered why, if this is so valuable to reviewers providing accurate reviews (which I have no doubt that it is), why the loans are not extended past six months. Why wouldn't the magazine buy the product for the reviewer if it is going to be his reference for 6, 12 or 18 months.

Some mags, like Home Theater, have invested a fair amount of money buying different types of gear to create a pretty good listening room "lab" for comparisons and testing under varying conditions. I don't think most do this. You might want to find a way to develop a dialogue with some reviewers to discuss this further, but I think there is an inherent requirement that to be a reviewer you must be serious about the hobby, which means you have to have some good equipment to begin with. Your opportunities to make comparisons exist when you have a couple of different speakers and other components on-hand for review. Otherwise, you have to use your own stuff for comparison (I would like to go through the homes of some reviewers, like RR who usually writes for Stereophile, because he must have a lot of stuff........I'm not a reviewer, and even I have a room that is full of idle equipment).

Before I close, I'll describe something from past experience. In the 80's and 90's I worked for a large PC manufacturer. We used to send computers out for review. Generally, they were alpha or beta units, but they were never the same thing as would be sold off the shelf. The reviewers would want the stuff picked up as soon as the review process was done, because they didn't have the real estate to accomodate and keep all of the stuff they received for review. We loved it when someone wanted to actually buy and keep a "evaluation" unit, because we couldn't sell that unit as new anyway. Otherwise, the computer submitted for review would go on someone's desk, or be torn apart and used to maintain other PC's, or maybe even sold at one of our annual "manufacturer's parking lot sale" where great stuff was sold at a fraction of MSLP only because we could not sell it as new.

The last really critical (rather than negative) review in TAS was written on the Thiel 7.2 by Robert E. Greene several years ago, before Harley's time. I have never read a negative review of a major manufacturer since. I concur with your view that manufacturers wouldn't seem to need to "bribe" TAS reviewers since they get favorable reviews anyway. And no measurements are provided (as in Stereophile) to at least expose the engineering prowes of each manufacturer. IMHO most reviewers write bosh anyway.

As to price inflation, you probably paid more for your new car this year than the one you bought 10 years ago, too. I believe the real improvement in sound for dollar value is in the medium priced High End stuff , say $2000 to $5000 components and speakers where "trickle down" is a reality.
Regards,
Mike

Since Harley had commented on the original model while working for another magazine, TAS brought him in as a third wheel for this particular review / article. What did he have to say about all of the negatives that the other two TAS reviewers had brought up about the original? Harley stated that he didn't notice any of those problems with the unit that he reviewed. In the article, Harley surmised that his review sample was probably later in production and actually closer to the Mk II revision that TAS was currently reviewing. Hmmm.... sounds kind of convenient to me, but i guess it is a plausable situation.

The funny thing here is that, even though the earlier TAS commentary disagreed with Harley's findings, everybody was able to agree on the statements that the newer unit was better. In effect, Harley's integrity wasn't challenged even with conflicting responses on the original units. One would guess that ARC was able to sell more of the "new & improved" units due to universal acceptance from all three reviewers. As such, everybody is happy, there's no confrontations / questions asked, nobody has their toes stepped on and the revenue keeps rolling in.

Based on this situation and a few others, in MY opinion, i would take anything that Harley says with a king-sized grain of salt. In my not-so humble opinion, this is especially true when it comes to ARC products. I am of the personal belief there are "alliances" that have been formed in various audio circles and some players in the game of audio aren't letting other players know what is going on. Then again, there are a lot of conspiracy theories floating around and these are only MY personal opinions. Obviously, you are entitled to share your point of view / opinions just as i did here. Sean
>

PS.... The last sentence in Harley's rant states "Anyone who says other-wise simply doesn't know what HE'S talking about". Shouldn't this read "Anyone who says other-wise simply doesn't know what THEY are talking about"??? "He's" implies that there is one male that is being overly critical and that one male doesn't know what he is talking about. This statement / wording is completely contradictory in nature to the genderless plurality of "minority of audio magazine readers" that he mentions in the opening of the article. Given Harley's ability to write in a manner that doesn't contradict itself as evidenced by his own published works, and that he is the current Editor in Chief for one of the best known audio magazines in the world, one would think that this could have been worded in a more concise and cohesive manner. While this is open to interpretation, it is my opinion that this could have been some type of "Freudian slip". Then again, i'm not a psychologist or analyst, so what do i know??? I'm just sharing my personal opinions and observations : )

Regards,
Larry

Right. If I didn't like the product or find it does something new or worthwhile, I had a hard time writing about it. If the product is highly regarded or has been positively reviewed, and I find problems or negative qualities that were overlooked, then I would want to write up the negative stuff.

"If you were reluctant to write a negative review (or, conversely, predisposed to writing a positive review) then you simply weren't doing your job. A reviewer in my opinion shouldnt base his review on how one person out of a thousand potential users MIGHT feel about it, nor do I think a reviewer should worry about what might happen to a company for fear of telling the truth."

My job was to write reviews that are truthful and objective as possible about audio products. I've never written a review where I didn't criticise some aspect of a product's performance - after all there is no such thing as audio perfection - at least not yet. So while an overall review may be positive, there are always negative parts. It's just that I wasn't usually interested in writing about products that I felt 80 or 90% negative about. I didn't think the readers would be interested either.

"I'm not suggesting there is some big conspiracy, I just think that as long as a magazine is largely funded by the companies whose products get reviewed, reviewers are going to cop out, the way you mentioned you did. Consumer Reports doesnt seem to be offended when they rate a product as inferior and their testers / reviewers are free to be truthful."

So, do you find Consumer Reports' reviews of, say, loudspeakers, more useful than those in the audio review publications? Seldom do the audio publications go to the trouble of reporting on a large survey of say, 10 different $5000 loudspeakers. If they did, some would be rated as inferior. Of those 10 loudspeakers, which would you rather read extensive reviews of - the top 2 or the bottom 2? Iknow I would much rather write about the best ones, and if you consider that a cop out, so be it.
Regards,
Mike

It's fine if you have an opinion, but how can you go on about a process if you have nothing concrete to go by?
As for the baloney stuff about how components are selected, I think I'd know about that better than most of the people on this site. Components are sometimes selected by: manufacturer contacting magazine; magazine hearing thru grapevine or other manufacturers about a product; company having great track record (i.e. Audio Research or CJ, etc) or just dumb luck. Sometimes reviewers hearing something they want to review becaue they're struck by some trait it evinces.
However, in some cases, if a component is so flawed that to review it would be suicide for the company, the loan is declined (or isn't requested by the magazine). What, you think it would be fun to destroy someone's livelihood by taking the component, writing a scathing review and holding that up as some sort of integrity??? Only a fool would suggest that that be done (and I'm not saying you did). It would be cruel and irresponsible. Especially if someone is just starting out. Didn't Lincoln get rejected numerous times when he ran for office? Not every new product is successful, especially those by small designers who have a vision. Sometimes a designer makes a mistake and he then goes back to the drawing board. It's not unusual for a company to try something and then realize it just won't work. That's how life works. And so, components get sent back. That's irrelevant, because nothing made by man is ever going to be perfect, anyway. So, the "good" reviews are a matter of finding the small imperfections and pointing them out. But the rest of the component's virtues might be -- and especially for a particular price point -- worth bringing to the audiophile community's attention.
I will say that TAS' older writers were much more astute in their assessments of components than the stuff out now. No wonder people think the reviews are 'nice.' I think the same thing. Somebody suggested in another thread that the intercommentary system was great . It was. Look at the interconnect survey in TAS last year. One reviewer loved Nordost Bue Heaven. Then, the other reviewer stepped in and said, yes, it sounds like he said, but it is VERY lean sounding. Now, if there hadn't beeen an intercommentary, that would have gone unnoticed by a great many (potential) buyers. So, now, people with lean systems know not to insert this into their system.
The reviews seems nice because they're short and not very involving and you still don't get much a feel for how it sounds. Nobody has a common language anymore, so 'smooth' can mean anything. It can also mean "smoothed over' as in, the upper midrange is smooth, but it's also dull (and it shouldn't be!).
This is why the whole review process needs a facelift.
I have criticisms , just as many of you do, but I see stuff written here that is nothing more then someone's projection of their own dark side onto the mags and reviewers. I've been associated with a couple of the publications and most of the "guesses" on this forum are completely off the mark, as to how reviews are written. The stuff about Harley is just plain drivel, and I've never met Harley. From what I've heard about Harley from people in the business, he's an upright type. Goes 'by the book' as it were. I did know reviewers who got equipment when they were supposed to go thru me to get it (and I remember the names clearly). And they'd get something and have it for months before I even knew they had it. They were in the majority, however.
The business might have changed since I was in it, but when I was in it, it was pretty honest and direct. Although I do remember some instances...