Home Critic's Corner

Discuss a review. Provide constructive feedback. Talk to the industry.

[ Follow Ups ] [ Post Followup ] Thread: [ Display  All  Email ] [ Critic's Corner ]
[ Alert Moderator ]

Blind testing is a hypothesis test and has the same mathematical rules

If you set the confidence limit too low / don't have enough trials /subjects aka 'n', your probability of type 2 error (beta) can exceed 50%. Which is into bull-shit territory.

The double blindness in ABX DB testing is NOT comparable to the DB nature of medical hypothesis testing. It is ignorant to suggest that it is. And ignorant to suggest that blindness is the scientific aspect of testing. Proper and open use of statistics is where 'proofs' lie.

Far too many ABX DB tests that get published don't publish their alpha OR beta figures. It ought to be essential.

I wouldn't bother publishing or promoting any ABX DBT results where beta was significantly larger than alpha. None of us should tolerate it from the hard line objectivists.

For me, any blind home-audio testing should involve a single seated listener in the sweet spot, in a treated room. Not a room-full of people.

Thus, getting enough n is a problem, and yet it is the only way to get alpha and beta down while having a reasonable confidence limit. 10% / 90% is a LOT more appropriate most of the time that 5%/95 was used.

Bear in mind that unwillingness to post the two salient numbers - of alpha and beta - is rife in published hypothesis test science, of all kinds.

Where the science is an estimate of the value of a variable, the question of what sort of estimator they chose to use is very rarely discussed. This is not surprising because everyone seems to think that an unbiassed estimator is the best in all cases.

But, it rarely is appropriate. It is only appropriate if errors above and below the mean are of equal concern to the people likely to be affected by the use of the estimated value.

Science? Schmience!

Having an abiding respect for real, solid science it worries me that the prevailing 'scient-ism' about so much of 'science' might not be justifiable, because of these, to me, manifest flaws. You know, those "fuck I LOVE Science" threads on social media. ? yes?

Value judgements just do have to be made, when we are using statistical techniques, and using the usual settings isn't 'objective' it is simply ignorant, perhaps even arrogant.



Warmest

Tim Bailey

Skeptical Measurer & Audio Scrounger


Edits: 11/08/16
This post is made possible by the generous support of people like you and our sponsors:
  Signature Sound   [ Signature Sound Lounge ]

Follow Ups Full Thread
Follow Ups
FAQ

Post a Message!

 Forgot Password?
Moniker (Username):
Password (Optional):
  Remember my Moniker & Password  (What's this?)    Eat Me
E-Mail (Optional):
Subject:
Message:   (Posts are subject to Content Rules)
Optional Link URL:
Optional Link Title:
Optional Image URL:
Upload Image:
E-mail Replies:  Automagically notify you when someone responds.

[ Top Top of Page ] [ Contact Us ] [ Support/Wish List ] [ Copyright © 2024, Audio Asylum, all rights reserved ]
[ General ] [ Speakers ] [ Tubes ] [ Vinyl ] [ Digital ] [ Hi-Rez ] [ Video Asylum ] [ Cables ] [ Tweaks/DIY ] [ Music ] [ Films ]