Just seeking to survey opinion here. Many people at AA have complained about the lack of sufficiently critical reviews in the mainstream audio press. Yet, at Stereophile, JA performs and analyzes a suite of measurements on reviewed equipment (excluding equipment discussed in columns). These "measurements" sections are often critical, even for equipment that wins raves in the subjective review.

Here are my questions:
1. Do you read the measurements section?
2. If the measurements section is critical, and the subjective review is highly complementary, how do you respond? Do you dismiss the measurements? Dismiss the subjective review? Weigh them both? Dismiss them both?

Disclaimer: Though I occassionally write for Stereophile on a freelance basis, I ask these questions merely to satisfy my own curiosity. I don't, in this, represent the magazine.

Thanks very much,
Jim Austin

 It's too early to begin an analytical process or evaluation after reading a review. - Don T 09:44:22 06/16/04 (0)
 Re: Measurements/Subjective Reviews - Pat D 11:32:10 06/02/04 (0)
 I go to the measurments section FIRST..... - rp1@surfnetusa.com 13:45:56 05/31/04 (0)

...

Show full thread outline!

 It's too early to begin an analytical process or evaluation after reading a review. - Don T 09:44:22 06/16/04 (0)
 Re: Measurements/Subjective Reviews - Pat D 11:32:10 06/02/04 (0)
 I go to the measurments section FIRST..... - rp1@surfnetusa.com 13:45:56 05/31/04 (0)
 Re: Measurements/Subjective Reviews - theaudiohobby 17:00:55 05/29/04 (3)
 Re: Measurements/Subjective Reviews - John Atkinson 07:05:26 05/31/04 (2)
 Dynaudio? Did you mean Confidence? - John Atkinson 08:48:27 05/31/04 (1)
 Oops - John Atkinson 08:50:58 05/31/04 (0)
 Re: Measurements/Subjective Reviews - Arvind Kohli 10:04:12 05/27/04 (0)
 Re: Measurements/Subjective Reviews - Mike Currie 06:46:53 05/27/04 (0)
 Re: Measurements/Subjective Reviews - Robert Hamel 10:04:23 05/26/04 (1)
 Re: Measurements/Subjective Reviews - abajaj11 10:57:18 05/26/04 (0)
 Have you noticed? - UK 16:02:17 05/25/04 (0)
 Re: Measurements/Subjective Reviews - Sean 19:46:20 05/24/04 (1)
 Digression: am I "R", "I", or "A"? - Jim Austin 05:38:05 05/25/04 (0)
 Re: Measurements/Subjective Reviews - Ohlala 19:27:14 05/24/04 (0)
 answers - Bruce from DC 13:27:41 05/24/04 (0)
 Re: Measurements/Subjective Reviews - hexenboden 13:05:48 05/24/04 (0)
 Re: Measurements/Subjective Reviews - abajaj11 11:15:12 05/24/04 (3)
 Re: Measurements/Subjective Reviews - hexenboden 13:07:29 05/24/04 (2)
 Re: Measurements/Subjective Reviews - abajaj11 06:46:22 05/25/04 (1)
 Re: Measurements/Subjective Reviews - hexenboden 16:11:34 05/26/04 (0)

So, let us accept that audiophile mags cater to what i call "haute audio" (see my post a few posts down on this) and it's more about appearance & "feeling good" than about real sonic excellence. This is where the "subjective reviews" and the superficially clever & flippant writing (Jim Austin excepted of course!) come in. I enjoy reading these as well, but i would NEVER give them any credence whatsoever.
-akhilesh

"1. Do you read the measurements section?"

Yes

"2. If the measurements section is critical, and the subjective review is highly complementary, how do you respond? Do you dismiss the measurements? Dismiss the subjective review? Weigh them both? Dismiss them both?"

It depends on what the measurements indicate on the one hand and who the reviewer is and what he has written on the other. I'm afraid that with a very small number of exceptions (eg. John Marks, Michael Fremer, John Atkinson, maybe a few others) I have litle respect for what the Stereophile reviewers write. So usually when in doubt the mesurements will carry the day for me. But not always.

Question 2: (oops, doesn't look like a "yes" or "no" question).

A big disparity between measured performance and subjective comments disturbs me least when it comes to loudspeakers, especially when the speaker is not a moderately small forward-firing box speaker with a couple of cone drivers inside. JA himself is quite candid about the limitations of measuring planar speakers, horn speakers and very large box speakers with many drivers.

With respect to digital players, there seems to be very little correlation between measured differences (which, in all events, seem to be very slight) and subjective reactions. So, I'm not troubled by a disparity here, because the only differences that show up in the measurements seem mostly to be jitter levels, low-level square wave reproduction, very low-level linearity and little glitches in the noise level 80 or more dB down.

With respect to differences between measured performance and subjective reactions to preamplifiers and amplifiers, I am more skeptical of an amp that measures poorly and is said to sound great. Very commonly, some reviewers seem to express a preference for the sound of an amp or preamp that has a fairly high level of measured distortion, declining evenly as the harmonic multiple of the fundamental increases. I'm aware of some papers that claim that this kind of distortion somehow produces a less distorted sound at the ear than a product with a truly low level of measured distortion; but I have a hard time accepting that. That said, I believe, at some low level (maybe .1%), differences between conventionally measured levels of amplifier/preamp distortion are meaningless.

Net-net, if an amp measures really bad, but the reviewer says it sounds great, I begin to wonder if the reviewer just doesn't have a taste for a certain kind of distortion. With other equipment, most of the time, I'll accept the reviewer's conclusion independent of the measurements, although I find the measurements interesting.

Hell yeah. It is the most important part of the review. That's because i can interpret the results for myself and don't have to rely on someone else's "subjective" or "biased" interpretation of what's going on.

2. If the measurements section is critical, and the subjective review is highly complementary, how do you respond? Do you dismiss the measurements? Dismiss the subjective review? Weigh them both? Dismiss them both?

If it came down to it, i would dismiss the subjective reviewer and trust the specs. That is, so long as the specs were derived using both consistent and repeatable procedures consistent with industry standards. As it is, some of the reviewers provide enough "between the lines" info to figure out / confirm my thoughts about how the units perform in comparison to my deciphering of said test results.

3. Disclaimer: Though I occassionally write for Stereophile on a freelance basis, I ask these questions merely to satisfy my own curiosity. I don't, in this, represent the magazine.

Why aren't you listed as a "R" ( Reviewer ) or a "I" ( Industry Pro )? If you write about audio related subjects, but don't review gear, that would make you an Industry Professional as you comment on the industry as part of your profession. Sean
>

Good question, Sean. The answer is that I guess I'm not used to thinking of myself as either a reviewer (since I'm not, so far) or an industry pro. I'm still mainly a hobbiest...just a guy who enjoys audio, takes it seriously, tries hard to understand it on a technical level (and has a background in science, which, I like to think, makes that at least possible, though I certainly haven't managed it yet)...who also happens to write (and edit) for a living. The Stereophile gig, much as I value it, is something I do (very seriously) in my free time, once my other professional (and parenting) responsibilities have been met.

That doesn't mean you're wrong, though. Could be that writing about audio part-time (but not reviewing) makes me, for the purposes of this forum, an industry pro. Anyone else care to chime in?

Jim

Also, if you could share any examples of how you used a combination of parameters to make a decision, and how it turned out, would be very interesting to learn, at least from my perspective.
thanx
-akhilesh

1. Yes

2. Weigh them both

Jeff

1. Yes I do read the measurements section.

2. If the disparity is large I am left scratching my head. Not really sure what to make of it. I tend to favor the measurements. But that said I wouldn't make a major purchase based on a review. I have to go and hear for myself and use them for references. If a reviewer said OK but look out for X and X is something I worry about. Then I would still be willing to listen but I would just bring music that the short comming would be audible on.

1. Do you read the measurements section?

Yes, always.

2. If the measurements section is critical, and the subjective review is highly complementary, how do you respond? Do you dismiss the measurements? Dismiss the subjective review? Weigh them both? Dismiss them both?

Depends if I have personal knowledge of the subject. For instance, tube gear is rarely going to measure well by most conventional standards, yet I know damn good and well that well designed tubes sound more like music. More recently, I was extremely disturbed by a particular comment made by JA regarding the Innersound Eros, a speaker that I know and love. Quote: "But the woofer's use of transmission-line loading is nothing I could be enthusiastic about; it introduces the possibility of upper-bass coloration and as LG found, it does not significantly extend the speaker's low-frequency limit".

First, I have heard the Eros in 4 different rooms. I am 100% sure that the 'upper-bass coloration' is a room mode. That, however, is outside of this particular discussion. The second part of JA's statement regarding the low-frequency extension - or rather lack thereof - is probably true, but it misses the point entirely. It is the QUALITY that is significant here. Transmission line loading provides far more natural pitch definition and proper overhang (not too tight OR loose), and is the primary reason that the transition to the 'stat panels has been described over and over again as 'nearly seamless' - which is absolutely true. Such a characteristic, of course, does not show up in any conventional measurement, yet is integral to the notion of accuracy in my opinion.

It's a cliche, but.....measurements are often useful but are absolutely no substitute for critical listening.

1. Do you read the measurements section?

Absolutely. Dont yet understand them all, but the learning continues. Personally, I would find articles on reading/interpreting them useful, similar to the ones done for speaker and preamp measurements.

2. If the measurements section is critical, and the subjective review is highly complementary, how do you respond? Do you dismiss the measurements? Dismiss the subjective review? Weigh them both? Dismiss them both?

I think JA has done a bang-up job of measurements. A contradictory listening indicates a few possibilities.

a. We yet do not have meaningful measures that relate to all aspects of human hearing. Someday we shall have more/better tests that will better corelate measurements to listening. I feel what JA is doing today is state of the art, and I consider him a pioneer. But we have to accept the fact that this 'science' is yet in it's infancy.

b. The ancillary gear/setup used for the review was so 'eccentric' that the listening results were substantially different from the measurements. Different results would have been yielded with different setup/gear. I somehow do not think this is the case very often. Especially since most reviewers have enough gear to play around with.

c. The reviewer interjected personal values when evaluating the equipment. Ie. the reviewer may prefer a large bass hump, rolled off highs, second-order harmonics and gave the gear a high grade for having it or a low grade for not. I have to admit it is especially difficult not to do this as a reviewer.

d. The reviewer is churning out yet another review, and spews the standard superlatives when describing the sound. I cannot say how often this is the case, but I have my suspicions.

I thank that you raised this topic, because the objective reviews in the Stereophile are becoming a source of angst for me after a couple of incidents that have occurred in recently.

Firstly, for a magazine for whom most it’s reviewers regard turntables as sonically superior to most digital equipment, it is no small irony that this category of equipment is not subject to any form of objective measurements. All claims to its sonic superiority are arrived at based solely on objective listening. This policy is justified on the basis of costs and time constraints and that is very unfortunate.

More importantly, recently I was at the Speaker asylum following a thread where the Dynaudio Contour came up for discussion and some of the replies made some comments based on the speaker’s measurements in Stereophile. Doug Schneider must have felt duty bound to post the actual anechoic measurements of the Contour because of the margin of difference between Stereophile’s pseudo-anechoic results and the actual anechoic measurements. The actual measurements gave a totally different picture of the performance of the speakers in question. Now if measurements are not reasonably correct then they are of no value at all.

Finally, what is the value of published 1/3-octave analyser measurement for digital systems, I fail to see why JA persists in using this method when the industry relies on FFT analyser measurements. For me, I think this was probably brought to head recently when Keith Howard published a series of measurements in Stereophile using an FFT analyser, what can be gained from the departing from the industry standard especially when the alternate method is not peer reviewed nor any credible industry support that I am aware of. I note that not a single AES member critic of SACD/DSD has yet even referred to Stereophile’s 1/3-octave analyser measurements and I suspect it is because they are to put it bluntly of no value whatever, because the results produced simply do not stand up to public scrutiny.

Measurements are a source of objective reference, a standard by which all products can be compared with reference to any bias. Therefore if they are wrong as in the case of the Dynaudio Contour or as in SACD/DSD at odds with the industry standard with no valid justification, what value do they really provide? I rarely see a restatement of measurements, so does that suggest that they are always right?

In the light of the comments above, a response to your questions

<< Do you read the measurements section? >>

Yes, sometimes with much more attention than the subjective review.

<< If the measurements section is critical, and the subjective review is highly complementary, how do you respond? Do you dismiss the measurements? Dismiss the subjective review? Weigh them both? Dismiss them both? >>

I look for a second opinion from one of the European magazines that also does measurements, if the measurements to do not agree, then it is dismissed. That said all 1/3-octave analyser measurements are dismissed outright because of their dubious value, to the best of my knowledge no other magazine uses a 1/3-octave analyser, therefore it cannot be verified, so it fails on the first count.

It a'int all about the music I also want excellent sonics!

The measurements section has saved me a lot of money, and time, by keeping me away from badly engineered and built equipment.
Give Me Ambiguity or Give Me Something Else!

With electronics, the measurements very often do not indicate an audible difference (which I regard as a significant issue) and so I don't worry about the subjective reactions of the reviewers. Sighted listening is subject to bias, whether done by someone else or me. I do take note of ergonomics and any difficulties or reliability issues with the equipment.

With speakers, the measurements will indicate some audible differences and also are useful for compatibility issues. It is interesting to see if there is much correlation between the measured results and the reviewers subjective responses. For example, Art Dudley's taste seems to be very different from mine, though he does like the Quad electrostats. I find John Atkinson makes very useful remarks, and this is true of some others, notably Andrew Marshall of AIG.

I would not use measurements to choose speakers, but I do use them as a screening tool. That is, I wouldn't put a speaker that measures badly on my audition list and I wouldn't seek it out. However, if I came across one, I might very well listen to it for the interest if reviewers thought it was good. Mostly, the speakers I audition turn out to be very good ones.

____________________________________________________________
"Nature loves to hide."
---Heraclitus of Ephesus (trans. Wheelwright)

It's always nice to read a review after we've heard a component to see how well what we heard correlates with both the measurements and subjective opinion of the reviewer.

Give me rhythm or give me death!