Home
AudioAsylum Trader
Critic's Corner

Discuss a review. Provide constructive feedback. Talk to the industry.

For Sale Ads

FAQ / News / Events

 

Use this form to submit comments directly to the Asylum moderators for this forum. We're particularly interested in truly outstanding posts that might be added to our FAQs.

You may also use this form to provide feedback or to call attention to messages that may be in violation of our content rules.

You must login to use this feature.

Inmate Login


Login to access features only available to registered Asylum Inmates.
    By default, logging in will set a session cookie that disappears when you close your browser. Clicking on the 'Remember my Moniker & Password' below will cause a permanent 'Login Cookie' to be set.

Moniker/Username:

The Name that you picked or by default, your email.
Forgot Moniker?

 
 

Examples "Rapper", "Bob W", "joe@aol.com".

Password:    

Forgot Password?

 Remember my Moniker & Password ( What's this?)

If you don't have an Asylum Account, you can create one by clicking Here.

Our privacy policy can be reviewed by clicking Here.

Inmate Comments

From:  
Your Email:  
Subject:  

Message Comments

   

Original Message

Blind testing is a hypothesis test and has the same mathematical rules

Posted by Timbo in Oz on November 8, 2016 at 19:02:17:

If you set the confidence limit too low / don't have enough trials /subjects aka 'n', your probability of type 2 error (beta) can exceed 50%. Which is into bull-shit territory.

The double blindness in ABX DB testing is NOT comparable to the DB nature of medical hypothesis testing. It is ignorant to suggest that it is. And ignorant to suggest that blindness is the scientific aspect of testing. Proper and open use of statistics is where 'proofs' lie.

Far too many ABX DB tests that get published don't publish their alpha OR beta figures. It ought to be essential.

I wouldn't bother publishing or promoting any ABX DBT results where beta was significantly larger than alpha. None of us should tolerate it from the hard line objectivists.

For me, any blind home-audio testing should involve a single seated listener in the sweet spot, in a treated room. Not a room-full of people.

Thus, getting enough n is a problem, and yet it is the only way to get alpha and beta down while having a reasonable confidence limit. 10% / 90% is a LOT more appropriate most of the time that 5%/95 was used.

Bear in mind that unwillingness to post the two salient numbers - of alpha and beta - is rife in published hypothesis test science, of all kinds.

Where the science is an estimate of the value of a variable, the question of what sort of estimator they chose to use is very rarely discussed. This is not surprising because everyone seems to think that an unbiassed estimator is the best in all cases.

But, it rarely is appropriate. It is only appropriate if errors above and below the mean are of equal concern to the people likely to be affected by the use of the estimated value.

Science? Schmience!

Having an abiding respect for real, solid science it worries me that the prevailing 'scient-ism' about so much of 'science' might not be justifiable, because of these, to me, manifest flaws. You know, those "fuck I LOVE Science" threads on social media. ? yes?

Value judgements just do have to be made, when we are using statistical techniques, and using the usual settings isn't 'objective' it is simply ignorant, perhaps even arrogant.