Home Critic's Corner

Discuss a review. Provide constructive feedback. Talk to the industry.

RE: Well, no

Actually the tests you speak of are medical tests adapted by engineers (engineers are NOT scientists nor I think do they understand scientific methodology in any real credible sense because if they did they would not be adapting them this way). The fact that it relies upon statistical modelling is problematic for a number of reasons.

The first is that stats lie to get results the tester desires at the outset. They typically use a 10 to 16 trial test and wish to satisfy the .05% statistical level of significance - this means a subject/listener needs to choose correctly 9/10 times.

But if a listener score 6/10 they are deemed to NOT meed statistical significance and therefore some putz will say that A sounds the same as B or people can't hear the difference between A and B.

Unfortunately what these hacks never mention is that if you score 6/10 ten times to 59/100 correct "guesses" you ALSO MEET the .05% statistical significance level meaning that if you are correct 59/100 that is EXACTLY THE SAME as 9/10. In fact it is more "credible" since more trials/guesses increased the level of reliability than less trials.

And this is just the statistics they fail miserably on and we have not even started on the "Validity of the test."

Validity means the test PERFECTLY resembles EXACTLY the way a person NORMALLY listens to music at home. That is not A/B/X methodology and does not incorporate a forced choice or switch box into the thing. They make no mention of the psychological FACT that music listening uses one hemisphere of the brain - decision making uses the other - any test asks the brain to use the decision making hemisphere which is now ALSO doing the listening because it knows it has to make a decision.

This has been illustrated countless times in the education field and is called test stress (no matter how benign you try to make it - it is invasive and decreases the level of validity of the test).

Having said that I am not at all against basic blind tests because they can are helpful - but it does not need to be a test like the medical industry. Those tests were for drugs vs sugar pills. The guy either gets better or he doesn't and it's a physical response - not a "preference" response requiring interpretation of a listener.

I proposed a blind test methodology that would satisfy "practically" this hobby previously. It doesn't take long to set-up has no contact from the listeners and setters of the tests. This takes out price and sight bias and the listeners don't even know the brands they will be listening to that day. Since the players are set to repeat the person conducting the test can be removed from the building. Zero contact with any listener.

1) two rooms of identical size - each room has a stereo covered via black light - no one can see the stereo.

2) level is matched

3) both rooms play the same disc on repeat at the same volume levels.

4) a selection of classically/jazz trained musicians who have an ear for correct pitch and notes (30 listeners)

5) each listener gets a card - they listen to both rooms as much as they like.

6) drop their card in the box outside the room they think sounds better.

7) add up cards for each room. Determine if preconceived notions of what sounds better is deemed so in the real world.

So if one major corporation puts out white papers that says SS amps with narrow multi-driver speakers in a deeper than wider cabinet will sound better than anything that is NOT designed in this way then room A will be the best of this design that said major corporation supports. In room B a Soundlab with LAM tube amps - the antithesis of said major corporation - or something like an Audio Note system or something like a big ole Horn SET.

If major corporation is correct then professionally trained musicians in such a listening session should choose it 27-3 or some easily blow room B out of the water scenario. If however it is closer to 50-50 split or room B mops the floor with room A then you have to consider the marketing force of a billion dollar company inventing pseudo science in order to generate sales.

Any Double blind test of STEREO speakers that places only ONE loudspeaker (not both of them) on a shifter machine in the middle of a room (not taking into account specific set-up recommended by the manufacturer or amplifiers/cables etc) for listeners is complete BS. You'd be surprised at the sheeple who eat that stuff up as being "scientific." LOL. It seems so because they spent so much money on the machines and space etc. If they spent large it must be right - right?

They dismiss any dissent as "nuisance variables" (thanks E-stat for providing the link). No real science considers deeply all variables that could influence a given test - but this is science run by people who want to make a profit selling their vinegar as fine wine.



Edits: 07/11/12

This post is made possible by the generous support of people like you and our sponsors:
  Herbie's Audio Lab  


Follow Ups Full Thread
Follow Ups

FAQ

Post a Message!

Forgot Password?
Moniker (Username):
Password (Optional):
  Remember my Moniker & Password  (What's this?)    Eat Me
E-Mail (Optional):
Subject:
Message:   (Posts are subject to Content Rules)
Optional Link URL:
Optional Link Title:
Optional Image URL:
Upload Image:
E-mail Replies:  Automagically notify you when someone responds.