Critic's Corner

Discuss a review. Provide constructive feedback. Talk to the industry.

Return to Critic's Corner

Message Sort: Post Order or Asylum Reverse Threaded

Blind testing assumes that one steps in the same river twice

75.39.16.143
Posted on November 7, 2016 at 14:46:05
Jay Buridan
Audiophile

Posts: 10275
Location: Michigan
Joined: January 21, 2004



My ears and mental states change in the course of a single listening session. It's a matter of biochemical fluidity that affects or comprises the living subject. While relative to the listener, e.g., once it warms up, the preamp is relatively stable; in a way, listener and preamp are in different rivers.

But non-blind testing adds a variable. And so is less reliable... or more reliable. What do you think?

"Horse sense is the thing a horse has which keeps it from betting on people. "
― W.C. Fields

 

Hide full thread outline!
    ...
Complete nonsense, posted on November 19, 2016 at 13:31:31
Analog Scott
Audiophile

Posts: 9933
Joined: January 8, 2002
 DBT as a means of eliminating bias effects is the gold standard in science. There has been a massive amount of research in the field of human hearing. DBTs work just fine in that field when done by actual scientists doing actual peer reviewed published research. IOW when it's done well, blind listening tests work just fine.

 

So, DBT audio judgement is more reliable than sighted? , posted on November 19, 2016 at 19:38:40
Jay Buridan
Audiophile

Posts: 10275
Location: Michigan
Joined: January 21, 2004
 That was my original question.

Or are you saying that Heraclitus' point about the stream of consciousness is nonsense? Or that it's not applicable to DBT? Don't mean to be dense, but I don't understand the wig out.

"Horse sense is the thing a horse has which keeps it from betting on people. "
― W.C. Fields

 

Heraclitus didn't have powerful statistical analytics. (NT), posted on November 20, 2016 at 14:01:57
Kal Rubinson
Reviewer

Posts: 12436
Location: New York
Joined: June 5, 2002

 

RE: So, DBT audio judgement is more reliable than sighted? , posted on November 19, 2016 at 20:13:20
Analog Scott
Audiophile

Posts: 9933
Joined: January 8, 2002
 It's too simple of a question for such a complictated subject. I would say that well designed comparisons done under blind conditions are more reliable than the same comparisons under sighted conditions IF all else is equal and the blind protocols are implimented well. When done well blind protocols do remove biases. But eliminating all relevant bias can be a bit more complicated than just putting blinders on. Biases can run both ways and there has to be controls for both positive and negative biases.

I don't buy the "stream of consciousness" argument. Blind protocols do not have to affect that. If the blind protocols are dulling one's perceptions then they are poorly designed and/or poorly implimented.

Now let me explain the "wig out." (and you are right, it was a wig out) It's the utter abuse that science as a dicipline and as a superior means of discovery recieves in the world of audiophilia. IMO both the objectivist and subjectivist camps in their most extreme are anti-science. One side rejects it because they feel that it runs contrary to their experiences and the other misapplies it because they are are uncomfortable with subjectivism in something as technical in nature as audio. At the risk of getting political, I will point out that we just elected a president who is a climate change denier. Denial of science as the best means of discovering the nature of reality is a serious problem. So when either side of the audio debates abuse science and/or misreperesnt it I get pissed off.

As for the use of DBTs or blind protocols in audio for us, the consumer I say do what works for you. Were not curing cancer, were not dealing with serious stuff like climate change. In the end if an audiophile likes what he or she has and is enjoying it that is great. It is the point of audio. I personally like using blind protocols in my audition process. Unlike a lot of these objectivist idiots I don't confuse my blind auditions with real science. I also do sighted comparisons along side the blind comparisons because that is how we listen when we normally listen. I understand that when I do this my opinions may be swayed by biases. But so what? My biases are in full play when I listen to my stereo normally. That's life. I would never tell anyone their opinions on audio are any more or less valid based on whether or not they use blind protocols. That's what ego driven audiophile assholes do. This big question about what is the right way and what is the wrong way and what is and is not valid is so easily answered. If you like it, it's good. I think audiophiles are better served if they look at the different approaches to auditioning and broaden their perspective. Then just find what works for you. The only validation we really need is our own joy of audio.

 

RE: So, DBT audio judgement is more reliable than sighted? , posted on January 6, 2017 at 08:47:31
Inmate51
 There is much interesting back-and-forth in this thread. And, given that the same arguments have been made ad nauseam for decades, it's unlikely that the status quo will change any time soon. This is partly due to successful marketing on the part of manufacturers who have a vested interest in selling products which bear little resemblance to accurate sound reproduction, and their willing surrogates in the hi-fi promotion media.

Anyway, to your other point, which you may have thought you could slip past some of us, "At the risk of getting political, I will point out that we just elected a president who is a climate change denier. Denial of science as the best means of discovering the nature of reality is a serious problem." No such thing happened. The man acknowledges that the climate is changing, but he simply doesn't believe the pseudo-science and models of the promoters who have a vested stake in being right. Secondly, the often-used and ongoing tactic of labelling a person a "denier" isn't very scientific, but rather goes directly into the political name-calling realm.

Carry on.

And, you might want to look into getting a better spell-checker.

:)


 

"The man acknowledges climate change" really? , posted on January 21, 2017 at 06:18:16
Analog Scott
Audiophile

Posts: 9933
Joined: January 8, 2002
 Donald J. TrumpVerified account
‏@realDonaldTrump


"The concept of global warming was created by and for the Chinese in order to make U.S. manufacturing non-competitive."

His tweet. His words.

 

RE: So, DBT audio judgement is more reliable than sighted? , posted on November 26, 2016 at 15:21:57
John Atkinson
Reviewer

Posts: 4045
Location: New York
Joined: November 24, 2003
 >I would say that well designed comparisons done under blind conditions
>are more reliable than the same comparisons under sighted conditions IF
>all else is equal and the blind protocols are implemented well.

Blind testing as commonly practiced tends to produce false negatives (ie,
not finding a difference when one exists), whereas sighted testing tends
to produce false positives (identifying a difference when there isn't one).
The choice between these two imperfect scenarios is subjective.

>I think audiophiles are better served if they look at the different
>approaches to auditioning and broaden their perspective. Then just find
>what works for you. The only validation we really need is our own joy
>of audio.

Amen.

John Atkinson
Technical Editor, Stereophile

 

RE: So, DBT audio judgement is more reliable than sighted? , posted on November 26, 2016 at 18:05:18
Analog Scott
Audiophile

Posts: 9933
Joined: January 8, 2002
 "Blind testing as commonly practiced tends to produce false negatives (ie,
not finding a difference when one exists), whereas sighted testing tends
to produce false positives (identifying a difference when there isn't one).
The choice between these two imperfect scenarios is subjective."

I suppose this very much depends on what "commonly practiced" is refering to. I wouldn't have the data needed to even guess at what is "commonly practiced." If we are talking about what is commonly practiced by the fundamentalist objectivists in the form of ABX DBTs it would not surprise me at all that this is the case. I have yet to see any of these tests show that they did anything to either control same sound biases or calibrate the sensitivity of the test using known audible differences. But that just shows that these dorks are actually anti-science posers pretending to be all about science when they are really all about their ego driven agenda. OTOH if we are talking about real scientific research in human auditory perception that leads to peer reviewed published studies in actual medical and scientific journals I would suspect that the incidence of false nulls would be fairly low. And of course in the world of real science researchers know that this is a potential pitfall in any given test and treat the results and draw their conclusions with that in mind. Now if we are talking about the sort of blind auditions I personally do I don't think there would be many false nulls but they are a possibility. The thing is I don't bother testing for differences. I go straight to testing for preferences and most of the things I compare are not particularly contraversial as to whether or not they sound the same or different.

 

RE: Blind testing assumes that one steps in the same river twice, posted on November 17, 2016 at 19:17:55
Pat D
Audiophile

Posts: 12506
Location: Fredericton NB
Joined: June 20, 2000
 Actually, the same objection applies to sighted listening, in spades, not just to blind auditions. So I don't think your post makes much sense.

What is the purpose of an audition? If you want to use equipment to see what you like, sighted auditions can work just fine. But will you prefer the same equipment at some other time?


-----
"A fool and his money are soon parted." --- Thomas Tusser

 

Rather than claiming, I'm asking a question, posted on November 18, 2016 at 10:19:35
Jay Buridan
Audiophile

Posts: 10275
Location: Michigan
Joined: January 21, 2004
 About adding the variable of sight. And your question is exactly to the point: "But will you prefer the same equipment at some other time?"

"Horse sense is the thing a horse has which keeps it from betting on people. "
― W.C. Fields

 

RE: Blind LISTENING is where it is at... earplugs and muffs. plus humming the whole time.., posted on November 17, 2016 at 19:16:00
Smelly_Socks
 Then one can be totally objective. You cannot hear any of it.

 

My friend, you should found an audio journal :), posted on November 18, 2016 at 10:29:47
Jay Buridan
Audiophile

Posts: 10275
Location: Michigan
Joined: January 21, 2004



Also, may want to try some oxygen-free listening :)

"Horse sense is the thing a horse has which keeps it from betting on people. "
― W.C. Fields

 

Blind dogs don't bark at shadows, posted on November 12, 2016 at 11:03:30
Jay Buridan
Audiophile

Posts: 10275
Location: Michigan
Joined: January 21, 2004
 Blind dogs don't bark at sculptures. I'll put that on my grave stone.

"Horse sense is the thing a horse has which keeps it from betting on people. "
― W.C. Fields

 

Hifitommy..., posted on November 11, 2016 at 07:11:46
kerr
Audiophile

Posts: 4376
Location: Central Indiana
Joined: November 10, 2003
 ...would have had a field day with the Heraclitus quote back in the "Chihuahua" days on the PropHead forum. There were certainly a lot of objectivist "dogs" barking...lol

If I understand your point, I would say that double blind testing is less reliable due to the listener being in "different rivers" during the listening session.

That said, blind testing (double blind is too problematic to achieve in one's home IMHO) can be useful at times if one is unsure of what they're hearing. It's no substitute for long term listening, however.

 

Dan, it was audioreview.com, posted on November 13, 2016 at 13:38:50
hifitommy
Audiophile

Posts: 15387
Location: canyon country califiornia, orig from buffalo ny
Joined: June 9, 2000
 in the old days w/me vs eyespy and mtry. ARF! that was actually a lot of fun, eyespy being more reasonable that mtry.

they maintained that wire is wire and amps all sound the same. aaah the memories....
...regards...tr

 

RE: Dan, it was audioreview.com, posted on November 14, 2016 at 14:10:44
kerr
Audiophile

Posts: 4376
Location: Central Indiana
Joined: November 10, 2003
 Yep, Audioreview was full of those l'il boogers. But I think you stomped a few on PropHead, did you not? For awhile there, it rated almost as high as AR for objectivist non-experiential nonsense.

 

Now, posted on November 14, 2016 at 15:06:00
E-Stat
Audiophile

Posts: 37609
Joined: May 12, 2000
Contributor
  Since: April 5, 2002
 the militant Mediocritists are found on AK. :)

 

RE: Now, posted on November 15, 2016 at 04:09:49
kerr
Audiophile

Posts: 4376
Location: Central Indiana
Joined: November 10, 2003
 Whole slew of 'em on hydrogenaudio

 

H2, posted on November 17, 2016 at 08:52:03
hifitommy
Audiophile

Posts: 15387
Location: canyon country califiornia, orig from buffalo ny
Joined: June 9, 2000
 omiGOD! those "people" there are just intractable. they pose as the elite educated and impede any and all audio improvement trails.

ignorami all.
...regards...tr

 

H2. Home for a small band of idiots posing as weekend scientists, posted on November 19, 2016 at 14:05:06
Analog Scott
Audiophile

Posts: 9933
Joined: January 8, 2002
 A group a malcontent pseudo scientific/intelectual assholes who can't stand subjectivists enjoying their hobby. It kills them that some audiophiles spend more money on audio than they will ever have in their miserable lives so they have to try to find some way to take a piss on the joy those audiophiles get from their hobby.

When one calss them on their bullshit they get banned. Don't burst that bubble!

 

H2. Home for a small band of idiots , posted on November 19, 2016 at 21:52:34
hifitommy
Audiophile

Posts: 15387
Location: canyon country califiornia, orig from buffalo ny
Joined: June 9, 2000
 they are likely the jerks who have been ousted from AA and AR (i had a ball making the lives of chihuahuas miserable.

a couple of the ARFed and the others bobbed their heads like the plaster pups with their nose pointing to the rear of the cars who back windows they were in.
...regards...tr

 

I'm just happy..., posted on November 17, 2016 at 12:06:32
kerr
Audiophile

Posts: 4376
Location: Central Indiana
Joined: November 10, 2003
 ...that they have their own playground so they don't infect the rest of us with their crap. Interesting to see them fight amongst themselves though. But sheesh, bring them here with us- they fight, put them in a padded cell together, they fight. I guess you just can't please the deranged.

 

You are frequently reminded, posted on November 18, 2016 at 09:49:30
E-Stat
Audiophile

Posts: 37609
Joined: May 12, 2000
Contributor
  Since: April 5, 2002
 of the TOS (Terms of Service) rules when any notion of *opinion* is stated.

Violating TOS#8 is punishable by death. :)

 

Oh, my, posted on November 15, 2016 at 06:31:01
E-Stat
Audiophile

Posts: 37609
Joined: May 12, 2000
Contributor
  Since: April 5, 2002
 that's where banned members like Mr. Voodoo, aka AJinFLA (Ammar Jadusingh) ended up along with guys like Arny Krueger.

I read a couple of threads including selecting amplifiers and bass traps. Naturally, all "well designed" amps sound the same. :)

Apparently, Arny is not one of Mr Voodoo's fans either. Even I've measured the benefits of using bass traps in my room.

 

Gotta love objectivist on objectivist forum violence, posted on November 19, 2016 at 14:08:14
Analog Scott
Audiophile

Posts: 9933
Joined: January 8, 2002
 Hillarious. Imagine these guys recomending their favorite burger joints. "Did you do a double blind taste comparison?" Then you are dellusional about your favorite burger!!!!!

 

LMAO!!!!!, posted on November 15, 2016 at 12:19:45
kerr
Audiophile

Posts: 4376
Location: Central Indiana
Joined: November 10, 2003
 Once again, you prove to be invaluable at finding (and re-finding) the most humorous posts.

Did you read the thread? Basically AJ is arguing with Krueger because the latter is claiming he heard differences in rooms with bass traps and AJ wants proof (can't hear it).

That is funny on so many levels, I don't know where to begin! Arny hearing something, for starters....

You owe me for one keyboard with spilled Coke all over it plus a $90 doctor visit for cracked ribs! TOOOOOOO funny!!!!! If you haven't read the whole thing, you gotta! :)

Anyway... sorry mods, for the thread jack. Now back to DBT's....

 

Isn't that link a gem?, posted on November 15, 2016 at 13:56:25
E-Stat
Audiophile

Posts: 37609
Joined: May 12, 2000
Contributor
  Since: April 5, 2002
 Arny:

"Room treatments can effect such dramatic improvements that no DBTs are necessary...

That may explain the surprising recent skeptical comments about room acoustics - the commentators are stone deaf and don't hear the natural differences in the acoustics of various rooms that the rest of us hear."

Priceless. :)

edit: I almost forgot to mention that jj is over there, too!

 

A quitessential example of the disconnect between objectivists and their pet peeve..., posted on November 19, 2016 at 14:15:52
Analog Scott
Audiophile

Posts: 9933
Joined: January 8, 2002
 bias effects. ""Room treatments can effect such dramatic improvements that no DBTs are necessary..." Psychoacoustics 101. Bias effects are in play regardless of how dramatic actual objective changes to the sound. And if one really thinks about it and is concerned about bias effects it when there are actual audible differences that we should be most concerned about them. It's really simple. If two components really do sound alike and someone forms a preference based on sighted bias they are not getting an objectively audibly inferior component. OTOH if bias effects are the persuasive factor in a choice between two components that are objectively audibly different one could end up with an objectively inferior product. It's funny that Arny, who has seemned to have dedicated his life to ABX DBTs would get such a basic aspect of bias effects so plainly wrong.

 

RE: Isn't that link a gem?, posted on November 16, 2016 at 07:03:59
SpotcheckBilly12345
 Arny doesn't miss many opportunities to pimp room treatments since his buddy Ethan (Winer) is in the room treatments business.

Cheers,
SB

 

RE: Isn't that link a gem?, posted on November 16, 2016 at 13:21:28
kerr
Audiophile

Posts: 4376
Location: Central Indiana
Joined: November 10, 2003
 >Arny doesn't miss many opportunities to pimp room treatments since his buddy Ethan (Winer) is in the room treatments business.<

And yet, the Godfather of Audio DBT, a guy who doesn't trust whether he heard a fart or his mommy calling him without at least 30 trials, has determined that DBT's are not necessary for room treatments.

I find that hilarious! But maybe there's hope for him.


 

This militant mediocrist-on-militant mediocrist violence must stop!!! (nt), posted on November 15, 2016 at 14:37:32
kerr
Audiophile

Posts: 4376
Location: Central Indiana
Joined: November 10, 2003
 nt

 

LOL!!! -nt, posted on November 15, 2016 at 15:46:39
E-Stat
Audiophile

Posts: 37609
Joined: May 12, 2000
Contributor
  Since: April 5, 2002
 .

 

My point is that blind and sighted testing are unreliable in different ways, posted on November 11, 2016 at 18:18:05
Jay Buridan
Audiophile

Posts: 10275
Location: Michigan
Joined: January 21, 2004
 We don't know whether the added variable is helpful or harmful. All we know is that adding the visual changes the result.

Here's a true anecdote from my college days:

There was a life size sculpture of a human male figure in the area where I used to walk my Malamute. My dog would bark and growl at the sculpture, thinking that it was a threat. Though I never tried it, I'm sure that my dog would not have barked had he been blindfolded. Was he a better watch dog without the blindfold?

"Horse sense is the thing a horse has which keeps it from betting on people. "
― W.C. Fields

 

RE: My point is that blind and sighted testing are unreliable in different ways, posted on November 14, 2016 at 13:56:33
kerr
Audiophile

Posts: 4376
Location: Central Indiana
Joined: November 10, 2003
 >There was a life size sculpture of a human male figure in the area where I used to walk my Malamute. My dog would bark and growl at the sculpture, thinking that it was a threat. Though I never tried it, I'm sure that my dog would not have barked had he been blindfolded. Was he a better watch dog without the blindfold?<

Excellent story - and I do now understand... I hope!

The answer, I think, is that sighted testing overinflates differences, while blind testing deflates them. Neither is terribly reliable, but unless one wants to blindfold oneself repeatedly, we go with sighted. And if the statue were actually a human intent on doing harm, sans blindfold would be best. Your dog took a "better safe than sorry" approach which in my mind made him an excellent watchdog! :)

 

blind and sighted, posted on November 13, 2016 at 13:34:07
hifitommy
Audiophile

Posts: 15387
Location: canyon country califiornia, orig from buffalo ny
Joined: June 9, 2000
 obviously, animals and humans react to visuals vide the cat videos where someone places a cucumber behind the cat that is eating and they go frantic upon seeing it. same for the dog and the statue.
...regards...tr

 

RE: My point is that blind and sighted testing are unreliable in different ways, posted on November 11, 2016 at 19:01:56
hahax@verizon.net
Audiophile

Posts: 4308
Location: New Jersey
Joined: March 22, 2006
 But if you do it with extensive tests and with correlation to objective measurements as Floyd Toole did over decades it becomes a very useful tool to go along with other tests. In fact Toole claims that doing both types of tests he's come up with over 95% certainty of predicting an excellent speaker.

 

RE: My point is that blind and sighted testing are unreliable in different ways, posted on November 24, 2016 at 23:36:52
andy_c
Audiophile

Posts: 1470
Joined: June 2, 2007
"But if you do it with extensive tests and with correlation to objective measurements as Floyd Toole did over decades it becomes a very useful tool to go along with other tests. In fact Toole claims that doing both types of tests he's come up with over 95% certainty of predicting an excellent speaker. "


Here's a video where Toole discusses this idea. The relevant discussion starts at 21 minutes 35 seconds. I can't seem to make that start time work with the youtube video embedding feature, so you'll need to fast-forward it manually to that point. Or you can use this link to go directly to the video on youtube at the right start time.

Floyd worked out which loudspeaker measurements to perform (the so-called "spinorama"), and I believe it was Todd Welti who figured out how to process the data in such a way as to give a figure of merit having a high correlation with listener preference on double-blind listening tests. There's some more discussion and links in this post.

 

I like the speakers that Toole designed for Infinity, posted on November 12, 2016 at 11:00:13
Jay Buridan
Audiophile

Posts: 10275
Location: Michigan
Joined: January 21, 2004
 But Infinity failed to provide parts support and left owners holding the bag (their bags)... Ouch!

"Horse sense is the thing a horse has which keeps it from betting on people. "
― W.C. Fields

 

Toole didn't design any Infinity speakers, or any Harman speakers for that matter (nt), posted on November 24, 2016 at 23:56:28
andy_c
Audiophile

Posts: 1470
Joined: June 2, 2007
 nt

 

You're right. Designer was Allan Devantier, posted on November 25, 2016 at 04:58:44
Jay Buridan
Audiophile

Posts: 10275
Location: Michigan
Joined: January 21, 2004
 Toole was VP of engineering.

"Horse sense is the thing a horse has which keeps it from betting on people. "
― W.C. Fields

 

holding the bag (their bags)... Ouch!, posted on November 13, 2016 at 13:35:25
hifitommy
Audiophile

Posts: 15387
Location: canyon country califiornia, orig from buffalo ny
Joined: June 9, 2000
 now THERE is a visual.
...regards...tr

 

I tried double blind testing but , posted on November 10, 2016 at 11:50:50
Pjay
Audiophile

Posts: 2415
Location: Northern VA
Joined: June 29, 2000
 I stubbed my toe while fumbling around trying to find the second blindfold.

:)

As I slowly slip into the dark cesspool of audiophalia neurosis. . . .

My speaker building site

 

Ouch!, posted on November 11, 2016 at 17:59:14
Jay Buridan
Audiophile

Posts: 10275
Location: Michigan
Joined: January 21, 2004
 That Hertz :)

"Horse sense is the thing a horse has which keeps it from betting on people. "
― W.C. Fields

 

Ironically, the medical "gold standard" shares little in common with audio trials, posted on November 8, 2016 at 06:59:33
E-Stat
Audiophile

Posts: 37609
Joined: May 12, 2000
Contributor
  Since: April 5, 2002
 1. Nothing is compared. Let me say that again - nothing is compared. Participants don't judge two different choices.
2. One group gets control drug while other gets placebo.
3. There is no "skill" involved and training has no effect on results.
4. The test procedure involves no change whatsoever to the way in which the results are evaluated. As opposed to the *assumption* that extraneous gear introduced into the process (aka controls) has no effect.

 

It isn't just the "nedical gold standard. It's the scientific gold standard, posted on November 19, 2016 at 13:37:48
Analog Scott
Audiophile

Posts: 9933
Joined: January 8, 2002
 And in that regard it serves exactly the same purpuse in all of it's scientific uses. To eliminate human bias effects.

 

RE: It isn't just the "nedical gold standard. It's the scientific gold standard, posted on November 24, 2016 at 17:38:14
geoffkait
 The main problem with blind tests is this: negative results mean nothing. You cannot generalize and say this device doesn't work. In that regard blind test is jus lie any test..

 

RE: It isn't just the "nedical gold standard. It's the scientific gold standard, posted on November 25, 2016 at 17:25:45
Analog Scott
Audiophile

Posts: 9933
Joined: January 8, 2002
 "The main problem with blind tests is this: negative results mean nothing."

First of all, there is no such thing as negative or positive results. There are simply results. The we can apply a statistical analysis to those results to get probabilities of what was and was not being detected by the testee in that particular test.


"You cannot generalize and say this device doesn't work. In that regard blind test is jus lie any test.."

A propper analysis of a single dbt abx test will not offer such a conclusion. Were someone to draw such a conclusion from a dbt they would be making a poor analysis of the results. But making a poor analysis of the results does not make a dbt just like any test.

 

So, hi-fi is a science?, posted on November 19, 2016 at 19:41:27
Jay Buridan
Audiophile

Posts: 10275
Location: Michigan
Joined: January 21, 2004
 Or a matter of preferences?

"Horse sense is the thing a horse has which keeps it from betting on people. "
― W.C. Fields

 

neither. It's marketing slogan, posted on November 19, 2016 at 20:25:52
Analog Scott
Audiophile

Posts: 9933
Joined: January 8, 2002
 Audiophilia is a consumer hobby. It's all about an aesthetic experience. So I say whatever floats your boat is right for you. This idea that there is a "right way" or a "wrong way" of enjoying music is kind of absurd. Science, real science is an amazing tool. If the designers and manufacturers of audio equipment can use it to make a better product I am all for it. If they don't use it and still make a kick ass product that is great too. But consumers need only please themselves and how they do it is not for anyone else to judge.

Now if that isn't good enough for an audiophile. If a "subjectivist" feels that it isn't enough to enjoy audio but feels they have to deny the massive body of scientific research on the subjects of audio and human hearing and deny that they are subject to bias effects I am going to call bullshit on them. Likewise if any "objectivist" thinks they are doing elgitimate science in their livingroom and that anyone who doesn't share their opinions on audio is an "audiophool" I will call bullshit on them too. The great audio debate isn't about science. Science is the victim of the great audio debate. The great audio debate is about ego. It's a stupid pissing contest that nobody ever wins.

 

Blind testing, like any testing, is not foolproof. Nt, posted on November 8, 2016 at 03:24:11
geoffkait
 Nt

 

RE: Blind testing, like any testing, is not foolproof. Nt, posted on November 8, 2016 at 05:22:46
hahax@verizon.net
Audiophile

Posts: 4308
Location: New Jersey
Joined: March 22, 2006
 Does that make it useless?

 

RE: Blind testing, like any testing, is not foolproof. Nt, posted on November 8, 2016 at 05:27:54
geoffkait
 It doesn't make it useless. You just can't draw conclusions from a blind test that gave negative results.

 

RE: Blind testing, like any testing, is not foolproof. Nt, posted on November 8, 2016 at 12:14:43
hahax@verizon.net
Audiophile

Posts: 4308
Location: New Jersey
Joined: March 22, 2006
 Really the main difference double blind can reveal is whether a difference is perceived or not. Then you can do many blind tests and ask many observers simple questions related to what you are trying to get at. That involves taste to a degree. It helps then if those results can be correlated to measured tests. That's one of the positive facets of Floyd Toole's tests. They are both double blind tests and correlated to measurements and for what it's worth he believes he can be over 95% sure of a good speaker's performance from the measurements he does that were correlated to double blind tests.

 

Blind testing is a hypothesis test and has the same mathematical rules, posted on November 8, 2016 at 19:02:17
Timbo in Oz
Audiophile

Posts: 23221
Location: Canberra - in the ACT - SE Australia
Joined: January 30, 2002
 If you set the confidence limit too low / don't have enough trials /subjects aka 'n', your probability of type 2 error (beta) can exceed 50%. Which is into bull-shit territory.

The double blindness in ABX DB testing is NOT comparable to the DB nature of medical hypothesis testing. It is ignorant to suggest that it is. And ignorant to suggest that blindness is the scientific aspect of testing. Proper and open use of statistics is where 'proofs' lie.

Far too many ABX DB tests that get published don't publish their alpha OR beta figures. It ought to be essential.

I wouldn't bother publishing or promoting any ABX DBT results where beta was significantly larger than alpha. None of us should tolerate it from the hard line objectivists.

For me, any blind home-audio testing should involve a single seated listener in the sweet spot, in a treated room. Not a room-full of people.

Thus, getting enough n is a problem, and yet it is the only way to get alpha and beta down while having a reasonable confidence limit. 10% / 90% is a LOT more appropriate most of the time that 5%/95 was used.

Bear in mind that unwillingness to post the two salient numbers - of alpha and beta - is rife in published hypothesis test science, of all kinds.

Where the science is an estimate of the value of a variable, the question of what sort of estimator they chose to use is very rarely discussed. This is not surprising because everyone seems to think that an unbiassed estimator is the best in all cases.

But, it rarely is appropriate. It is only appropriate if errors above and below the mean are of equal concern to the people likely to be affected by the use of the estimated value.

Science? Schmience!

Having an abiding respect for real, solid science it worries me that the prevailing 'scient-ism' about so much of 'science' might not be justifiable, because of these, to me, manifest flaws. You know, those "fuck I LOVE Science" threads on social media. ? yes?

Value judgements just do have to be made, when we are using statistical techniques, and using the usual settings isn't 'objective' it is simply ignorant, perhaps even arrogant.



Warmest

Tim Bailey

Skeptical Measurer & Audio Scrounger


 

RE: Blind testing is a hypothesis test and has the same mathematical rules, posted on November 9, 2016 at 05:41:44
geoffkait
 w2hen you say a single listening sitting in the sweet spot id like to just say that many listeners who are comfy and content sitting in the sweet spot of their system are actually sitting right smack in the middle of a standing wave. And as we all know standing waves are not good for the sound.

 

Go jump in the lake, Geoff., posted on November 10, 2016 at 00:29:11
Timbo in Oz
Audiophile

Posts: 23221
Location: Canberra - in the ACT - SE Australia
Joined: January 30, 2002
 Please.

:-)


Warmest

Tim Bailey

Skeptical Measurer & Audio Scrounger


 

Is that your best argument?, posted on November 10, 2016 at 04:22:13
geoffkait
 Nt

 

It wasn't an argument it was a suggestion, posted on November 19, 2016 at 14:19:20
Analog Scott
Audiophile

Posts: 9933
Joined: January 8, 2002
 That's how it looked to me. Besides it should be pretty obvious that if the "sweet spot" is suffering from standing wave issues it isn't really the "sweet spot."

 

RE: It wasn't an argument it was a suggestion, posted on November 23, 2016 at 16:58:45
geoffkait
 Of course my point was most folks who happened to be sitting in the middle of a standing wave wave would be blissfully unaware if it. Duh!

A suggestion to go jump in the lake is what we call argumentum ad hominem. Which is a logical fallacy. I.e., a poor argument. It's always best to stick to the topic at hand. 😛

 

RE: It wasn't an argument it was a suggestion, posted on November 24, 2016 at 14:03:38
Analog Scott
Audiophile

Posts: 9933
Joined: January 8, 2002
 "Of course my point was most folks who happened to be sitting in the middle of a standing wave wave would be blissfully unaware if it. Duh!"

Your point was neither supported nor even on topic if that was your point.

"A suggestion to go jump in the lake is what we call argumentum ad hominem. Which is a logical fallacy. I.e., a poor argument."

No, it's an insult for the sake of insulting someone. In this case you. The ad hominem fallacy is an insult for the sake of countering the argument by discrediting the person one is arguing with. Anyone with even a modest level of intelegence should be able to understand that telling one to go jump in a lake or take a long walk off a short peer etc. etc. is not an ad hominem argument but merely a garden variety insult strictly for the purpose of insulting someone.


"It's always best to stick to the topic at hand."

No doubt. You might consider taking your own advice in this case.

 

RE: It wasn't an argument it was a suggestion, posted on November 25, 2016 at 03:57:57
geoffkait
 Go jump in the lake.

 

Is that your best argument?, posted on November 25, 2016 at 10:04:32
Analog Scott
Audiophile

Posts: 9933
Joined: January 8, 2002
 Given your inability to tell the difference between an ad hominem argument and a garden variety insult. Or....did you actually learn something?

 

you got it in one, and most music isn't made up of sustained notes, , posted on November 19, 2016 at 22:25:13
Timbo in Oz
Audiophile

Posts: 23221
Location: Canberra - in the ACT - SE Australia
Joined: January 30, 2002
 even in the bass, and even on organ music.


Warmest

Tim Bailey

Skeptical Measurer & Audio Scrounger


 

I'm curious, posted on November 8, 2016 at 02:43:44
4everyoung
 How many here have ever participated in a blind listening test? In my opinion, blind listening tests are simply a test of your acoustic memory and little else. I would never make a component purchase based on being able to correctly identify a 30 second snippet of music. Sounds ridiculous, doesn't it? Long term listening is still the only way to go.

That said, blind listening tests can still be fun if you are able to check your emotions at the door and simply rely on your acoustic memory. I have participated in a few, the most memorable being the time a salesman at the local hifi store challenged me to a blind listening test of four different CD tweaks. Piece of cake - I got them all correct and then was promptly accused of cheating. Still makes me laugh.

 

My wishes are modest, posted on November 12, 2016 at 07:21:30
DAP
Audiophile

Posts: 668
Location: Toronto
Joined: January 1, 2010
 Don't all comparisons involve a test of acoustic memory, blind or otherwise?

As a reader, I'd kind of like to know if when Jacob Heilbrunn et al write stuff like "The improvement rendered by [TRANSPARENT REFERENCE XL DIGITAL LINK] ... proved to be one of the most flabbergasting experiences I have ever had in the high end ... the Reference line did not improve the sound; it took it into another realm", that such "flabbergasting experiences" are as apparent when testing blind.

Daniel

 

RE: My wishes are modest, posted on November 14, 2016 at 06:01:11
Pat D
Audiophile

Posts: 12506
Location: Fredericton NB
Joined: June 20, 2000
 Of course, audio comparisons involve acoustic memory, whether blind or sighted.

Sighted and blind auditions also are equally subject to the objection that you can't step in the same river twice.


-----
"A fool and his money are soon parted." --- Thomas Tusser

 

RE: Blind testing assumes that one steps in the same river twice, posted on November 7, 2016 at 21:31:42
MackRetep
Audiophile

Posts: 9
Location: Holstein
Joined: October 31, 2016
 You hear what you think.
If you buy a pair of cable which you like or do a trick that you go in the hifi store and let your wife decide what pair of cable looks good then you are in a good mood cause you think positive
do some silly thinks to have a good feeling
drink what you like
eat what you like
go in the zoo or in the park for a walk, look at different thinks to feel that this is something you enjoy cause its something different
you know the band routine routine day and day out?
that kills music
you know the band small parts isolated and destroyed
you can only sometimes stop the time
reduce something what is not good for you or your family
or maximise something what is good for you so everybody feels that you do something right
small parts are the cells
they are isolated like all beeings
but you can save them
not for ever but from time to time
you can not only stop the time
you can do more then that
every beeing has the power that the clock goes slower, then stops, then jumps backwards and slowly move backwards
then you forgot what you did to do that and feel that you are alone and hope that you can do this trick again from time to time ;)

tell some compliments and then hide in a corner and smile
close your eyes and share memories with the music

all tricks have the power to feel ;)

 

RE: Blind testing assumes that one steps in the same river twice, posted on November 8, 2016 at 17:36:07
davidandcandy@gmail.com
Audiophile

Posts: 192
Location: Virginia
Joined: May 12, 2014
 I have no idea what you just said. Not being critical, it just went completely over my head. Or under my feet. Somewhere other than hitting my "oh, yeah, I get it" button.
Everyone thinks I'm strange except my friends deep inside the earth

 

RE: Blind testing assumes that one steps in the same river twice, posted on November 7, 2016 at 19:59:34
hahax@verizon.net
Audiophile

Posts: 4308
Location: New Jersey
Joined: March 22, 2006
 And without blind testing there is a high probability of circular reasoning. And I'm saying this as someone who does do non-blind testing and makes decisions based on non-blind listening.

 

Please elaborate on circular reasoning, posted on November 7, 2016 at 21:17:33
Jay Buridan
Audiophile

Posts: 10275
Location: Michigan
Joined: January 21, 2004
 It's an intriguing comment.

"Horse sense is the thing a horse has which keeps it from betting on people. "
― W.C. Fields

 

I know it's real because I hear it. I know I hear it because it's real, posted on November 19, 2016 at 13:49:05
Analog Scott
Audiophile

Posts: 9933
Joined: January 8, 2002
 In a nut shell.

 

RE: Please elaborate on circular reasoning, posted on November 8, 2016 at 05:21:11
hahax@verizon.net
Audiophile

Posts: 4308
Location: New Jersey
Joined: March 22, 2006
 Google The Zanzibar fallacy. It's an example that comes from one of the early great British audio men, Percy Wilson, who was technical editor of Gramaphone magazine decades ago. He used it because of his fear of using circular reasoning in judging reproduction.

 

I know what circular reasoning is, posted on November 19, 2016 at 19:23:31
Jay Buridan
Audiophile

Posts: 10275
Location: Michigan
Joined: January 21, 2004
 but I don't get how it applies to blind v nonblind listening.

"Horse sense is the thing a horse has which keeps it from betting on people. "
― W.C. Fields

 

Found it, posted on November 19, 2016 at 19:18:28
Jay Buridan
Audiophile

Posts: 10275
Location: Michigan
Joined: January 21, 2004
 Percy and his wife were staying at a downtown hotel. Bud asked me to drive them there, and he made Percy promise to tell me his tale, the Zanzibar Fallacy, which Bud loved.
It goes like this: There was a captain of a freighter who had been sailing the seas for a very long time. He was a fine captain, liked by his crews. He had a tradition. Outside the ship's wheelhouse was a small cannon. Each day at noon the captain shot off the cannon.
One day, the captain got his crew together to announce his retirement. He was going to build a house outside the capital of Zanzibar, and he invited all his crew to visit him after it was finished. One morning a crew member showed up. He and the captain reminisced as the captain showed him around his home. On the second floor was a replica of the wheelhouse with a cannon outside, just like on the ship. It was nearly noon. The captain checked his watch and at noon shot the cannon.
The crewman had to leave, but as he left he asked the captain how he knew it was noon, so the cannon was shot at the correct time. The captain replied that it was simple. In the city was a man who studied time and had a grand collection of clocks. He just made sure his watch was coordinated with those clocks, and he advised the crewman to visit the man in order to see his collection.
The next morning the crewman was in town and visited the man. It was just before noon, and at noon all the alarms of the clock collection went off. The crewman complimented the man on his impressive collection. But he was curious. How did he know the time was correct, that it was really noon when the alarms went off? The man replied that it was easy. Outside the city lived a retired sea captain, and every day at noon he shot off a cannon. The man simply coordinated all his clocks with the captain's cannon shot.

"Horse sense is the thing a horse has which keeps it from betting on people. "
― W.C. Fields

 

Long term listening is the only correct evaluation!, posted on November 7, 2016 at 19:28:25
RealStereo
 How can anyone understand all the complexity of music quality in a short time???

 

nah...., posted on November 19, 2016 at 13:44:19
Analog Scott
Audiophile

Posts: 9933
Joined: January 8, 2002
 The *only* correct evaluation is the one that makes the buyer happy with their purchase. The rest is just an objectivist/subjectivist ego driven circle jerk. The value of audio is in the joy it brings us. Apparently some folks think the point of audio is for them to be right and those who take a different approach to be proven wrong.

I love audio. I hate audiophiles

 

It takes time to KNOW you will be happy! nt, posted on November 21, 2016 at 17:57:04
RealStereo
 nt

 

Not everyone, posted on November 22, 2016 at 02:07:43
Analog Scott
Audiophile

Posts: 9933
Joined: January 8, 2002
 Not every time either. Let's face it audiophiles don't have the best track record of buying and keeping equipment regardless of how they auditioned it. There are no hard fats rules one way or another

 

+1 (nt), posted on November 13, 2016 at 13:04:43
mkuller
Audiophile

Posts: 38130
Location: SF Bay Area
Joined: April 22, 2003
 (nt)

 

Roger that!, posted on November 7, 2016 at 21:21:28
Jay Buridan
Audiophile

Posts: 10275
Location: Michigan
Joined: January 21, 2004
 Magazine reviews are almost useless.. except for the less than useless reviews of Srajan.

"Horse sense is the thing a horse has which keeps it from betting on people. "
― W.C. Fields

 

RE: Blind testing assumes that one steps in the same river twice, posted on November 7, 2016 at 19:24:21
oldmkvi
Audiophile

Posts: 10581
Joined: April 12, 2002
 It's based on Impressions.
Can't get a firm grip on Impressions.

 

Page processed in 0.068 seconds.

[ Top Top of Page ] [ Contact Us ] [ Support/Wish List ] [ Copyright © 2024, Audio Asylum, all rights reserved ]
[ General ] [ Speakers ] [ Tubes ] [ Vinyl ] [ Digital ] [ Hi-Rez ] [ Video Asylum ] [ Cables ] [ Tweaks/DIY ] [ Music ] [ Films ]