Why blind listening tests are crucial

Has been favorite of all deniers, oops ... "objectivists" ... for decades., posted on March 26, 2015 at 10:01:55
Beef Guinness Cutie Pie Audiophile Posts: 136 Joined: December 29, 2011	If someone could explain to them the importance of listening, period, before any "blind" listening - that would be great. That's what is sorely lacking, unfortunately - both the ability (and skills) to listen, and the means (quality audio systems) of listening.

Did you play the video or did blind listening test scare you way like all subjectivists do , posted on March 26, 2015 at 10:14:07
3db Audiophile Posts: 1514 Joined: July 22, 2003	Same question to you.. Were you able to discern a difference between the two?

Problem is, you'd need to keep the blindfold on for one entire month..., posted on March 26, 2015 at 10:33:23
genungo	Long term listening tests reveal subtle sonic nuances that short term listening tests do not reveal. For this reason, review publications like Stereophile are issued but once a month.

RE: Problem is, you'd need to keep the blindfold on for one entire month..., posted on March 26, 2015 at 10:38:15
Jaundiced Ear	"Long term listening tests reveal subtle sonic nuances that short term listening tests do not reveal." What evidence is there of this outside of the claims of the listener? JE

That video, posted on March 26, 2015 at 10:45:25

tomservo
Manufacturer

Posts: 8204
Joined: July 4, 2002

That video is a demonstration that our senses are a combined system that system moment by moment delivers our conscious reality. In that case, that what we see can over ride what we hear, even when we know what is going on.

More directly related to the concept of BT's is the demo below which demonstrates how what you KNOW effects what you hear. Watch along a few min to Poppy Crumm's demo, watch and listen carefully, this one only works once because the second time "you know".

https://www.youtube.com/watch?v=BYTlN6wjcvQ

The issue is, blind testing to be scientifically rigorous is very involved and it must be for life and death things like drug testing where all the people want a good result but only some get the real thing.

The utility in home audio is only when you have two things you want to compare and it's easy / fast to switch between them. To compare two amplifiers say, one would level match and listen to both switching back and forth trying different kinds of music until you had a track or two where the difference was most clear.

Then, in a way you can't see or know the result of, have a friend do the switching and see how much difference there is when you are only using your ears and not what you know about which is which.

Do this at your leisure, with your system, using music of your selection, no pressure, the point is answering the question "can I hear a difference with my ears alone, when I don't know which is which?"
If not, one might ask, how much is this addition worth?

There is a good reason why they don't have a little red light go on or the tester nod to you or have any clues when the tone goes on in a hearing test, it is limited to what your ears tell you.

I am not sure why that concept is so irritating for some.

RE: Problem is, posted on March 26, 2015 at 10:46:34
genungo	What evidence is there that God exists outside of the claims of billions of people and no small number of elves, fairies, and gnomes?

Unfortunately..., posted on March 26, 2015 at 10:47:31

mkuller
Audiophile

Posts: 38130
Location: SF Bay Area
Joined: April 22, 2003

...there isn't much evidence that blind listening reveals subtle audible differences either.

When we talk "evidence" we are usually referring to scientific testing.

First the test must be validated for the use. DBTs were designed in the 1950s in England for testing the efficacy of new drugs.

In order for a double blind test to be valid, all variables except the single one being tested for must be controlled, there must a large group of people in the test and enough trials to reach a statistically significant confidence level in the results.

The way blind tests are used in audio they are more a test of the critical listening skills of the listener than for differences between components.

RE: That video, posted on March 26, 2015 at 11:43:58
3db Audiophile Posts: 1514 Joined: July 22, 2003	I know why its irritating. It calls their bluff and nobody likes being wrong. :)

RE: Problem is, posted on March 26, 2015 at 12:20:10
Jaundiced Ear	In other words, you've got no evidence to support your claim. Are you seriously telling us that you not only hear, but listen to the claims of "no small number of elves, fairies, and gnomes?" JE

RE: Unfortunately..., posted on March 26, 2015 at 12:24:23
Jaundiced Ear	"...there isn't much evidence that blind listening reveals subtle audible differences either." So there is no evidence that long term listening reveals subtle audible differences, and there isn't evidence that blind listening reveals subtle audible differences. Maybe the evidence is telling us that these subtle audible differences don't exist outside the listener's imagination? JE

RE: Why blind listening tests are crucial, posted on March 26, 2015 at 12:57:58
daleda Audiophile Posts: 741 Joined: March 6, 2002	When my wife bends over and moves her thong to the right and then to the left and repeats it always sounds the same - very scientific!

I think many audiophiles close their eyes while listening, posted on March 26, 2015 at 13:07:49
jedrider Audiophile Posts: 15167 Location: No. California Joined: December 26, 2003	I do, although sometimes I can have quite an imagination and feel that I am in the audience without closing my eyes. This still has nothing to do with DBTs. A Bose saleman asked me which system sounded better and I tried to say the more expensive system as I could see the price tags, but my vision isn't what it use to be and the saleman didn't like my answer.

RE: Unfortunately..., posted on March 26, 2015 at 13:12:12
mkuller Audiophile Posts: 38130 Location: SF Bay Area Joined: April 22, 2003	...that's laughable. How do you know the colors you see are what you think? Believe what you want and listen however you like. Just don't tell me me way is wrong unless you can prove it.

Thanks for sharing that!, posted on March 26, 2015 at 13:39:35
Zipcord Manufacturer Posts: 707 Location: No. California Joined: December 19, 2002	Pretty awesome - I sure fell for it

RE: Unfortunately..., posted on March 26, 2015 at 13:40:55

Jaundiced Ear

Requiring me to disprove your claim is fallacious reasoning. Declining to disprove nonsense in no way validates or proves the truth of the nonsense.

JE

http://en.wikipedia.org/wiki/Philosophic_burden_of_proof#Pro

RE: Why blind listening tests are crucial, posted on March 26, 2015 at 14:06:44
kerr Audiophile Posts: 4376 Location: Central Indiana Joined: November 10, 2003	I'll never hear someone repeat "ba" and "fa" the same way again.

RE: Problem is, posted on March 26, 2015 at 14:24:52
genungo	Just sayin'... You can't necessarily prove or disprove a lot of things, but that says little about what might actually be true or untrue. We are all limited in our ability to know.

RE: my favorite Optical equivalent, posted on March 26, 2015 at 14:25:31
BigguyinATL Manufacturer Posts: 3475 Joined: April 10, 2002	A or B which is Lighter.... "The hardest thing of all is to find a black cat in a dark room, especially if there is no cat" - Confucius

Jaundiced reasoning..., posted on March 26, 2015 at 14:38:07
mkuller Audiophile Posts: 38130 Location: SF Bay Area Joined: April 22, 2003	...ok so you can't show I'm wrong. Prove your way (whatever it is) is right.

The Real Problem is..., posted on March 26, 2015 at 14:41:16

BigguyinATL
Manufacturer

Posts: 3475
Joined: April 10, 2002

The result of our Stereo reproduction Process (performing & creating, then recording, producing, publishing, purchasing, installing, playing, amplifying, transducing, and listening) is inherently flawed. Incredible amounts of information are lost (or never existed) in the recording of a performance & Oodles (that's a technical term) of other stuff is added or distorted in time, phase Amplitude and frequency.

Yet, when we sit down, relaxed, with a bit of anticipation, and drop that needle or hit the play button, our minds imagine that were are at an event in real time - with sometimes amazing amounts of reality. It is something you should not take for granted.

"The hardest thing of all is to find a black cat in a dark room, especially if there is no cat" - Confucius

It's white and gold (nt) , posted on March 26, 2015 at 14:51:22
Steve O Audiophile Posts: 12374 Location: SE MI Joined: September 6, 2001

RE: additional comments, posted on March 26, 2015 at 15:03:36

BigguyinATL
Manufacturer

Posts: 3475
Joined: April 10, 2002

I don't mind that many of us hear changes in the playback when the actual signal doesn't change at all. This is because our minds have to have a certain level of confidence in the playback system in order to have the illusion to be believable. The threshold of that confidence varies - and sometimes wains over time. Heyser described that in his "Audio" articles on the "Catastrophe Effect". The multidimensional manifold of our perception and expectations is constantly changing.

I'm an easy sell, most the time my mind eagerly places me in the event as best it can. And even better after a glass of wine. My wife of 32 years, a musician, to her and my frustration many years ago - is the exact opposite. When listening to the finest stereo (and even my modest systems) with even the finest recordings I could find hears the playback and all the performance - yet her mind can't or doesn't place her in the illusion of a performance.

Interesting - if we play back a movie - surround sound and all, Like the McGurk effect, she will enjoy the artificial reality of the movie - even if it is a non-existent colony on Mars!

"The hardest thing of all is to find a black cat in a dark room, especially if there is no cat" - Confucius

RE: Jaundiced reasoning..., posted on March 26, 2015 at 15:51:27

Jaundiced Ear

What are you going on about? My original post in this thread was to ask genungo for evidence supporting his claim that "Long term listening tests reveal subtle sonic nuances that short term listening tests do not reveal." Whether I can not or choose not to disprove his claim has nothing at all to do with whether it is true or not. As the one making the claim, it's incumbent upon genungo to back it up.

Perhaps the attached link will make my position clearer to you.

I really have no idea what you are talking about when you say I can't show you're wrong and that I have to show I'm right or how the position explicated in the link below is somehow "Jaundiced reasoning."

JE

https://yourlogicalfallacyis.com/burden-of-proof

Sorry..., posted on March 26, 2015 at 16:10:53
mkuller Audiophile Posts: 38130 Location: SF Bay Area Joined: April 22, 2003	...I didn't realize I was dealing with a debate champ who has taken no position on the issue here. Obviously you are right.

RE: That video, posted on March 26, 2015 at 16:54:59
ahendler Audiophile Posts: 5151 Location: San Antonio, Texas Joined: January 24, 2003	Quick a/b switching will only tell you if there is a difference in the sound of the two components. It will seldom tell you which sounds better. That takes long term listening with a lot of source material. It usually takes me two weeks to a month before I really know if I like a component. In my experience DBT is only a starting poing and in many cases worthless Alan

Agree, but I would take "long term" even farther, posted on March 26, 2015 at 17:12:37

YRY
Audiophile

Posts: 491
Location: So. California
Joined: April 4, 2002

Reviewers have certain practical limitations with respect to time. With one's own system/recordings; however, we can take our sweet time. Over time (months, or even years), the sonic characteristics of certain recordings (as they are reproduced on one's particular system) become burned into your brain, so that if they are not reproduced as expected (as a result of a system change), it can be readily noticed. Especially in a blind test with 2 unfamiliar components, there can simply be too much new information to process at once in order to make fine judgements. If one can tell a difference in a blind test, then the difference is a large one. IMO, smaller differences can be ferreted out with long term listening.

The discussion seems to be centered a lot on the effect of expectation and non-sonic cues. What if one has no idea of what to expect? I think in that case, any definite differences detected are real. For me, a definite difference is one that is noted on practically any recording.

Of course, I have no data to back up any of this, but I know what I like when I hear it.

But the blind test only works within the blind test..., posted on March 26, 2015 at 17:48:58

RGA
Reviewer

Posts: 15177
Location: Hong Kong
Joined: August 8, 2001

and your video illustrates this to a tee. As he notes - EVEN THOUGH you KNOW that what you are seeing is illusory in that Ba Ba is what is said - when you see the mouth move as Fa you hear Fa.

In a blind test - you can very well fail the DBT statistically but once you are sighted again you go right back to preferring A over B.

Many years back I compared two CD players level matched and blind via headphones through a line level adjustable headphone amplifier. I liked one player over the other sighted - when blind I could not tell them statistically apart to the .05 significance level. So my brain SHOULD tell me that "they're both the same and indistinguishable" but once going back to sighted listening I still wound up preferring the one I preferred before the test. Both models were from Sony and I wasn't in love with either before the test so I didn't have a dog in the hunt .

I think blind tests are hugely informative but unless you intend to listen blind your whole life you will fall prey to this bias - that includes price bias. Some people hear what they want to hear. Bigger price = better sound - but so too are some swayed by "I can't afford it so the cheaper one is better/as good." Ie you may be hearing what you want to be be hearing.

But as the professor points out - no matter how smart you are or how good you are - sound/brain/other senses are a complete package and in normal day to day life you can't get rid of those other senses.

I think of attraction to a woman. There is touch, smell, sight, sound, body language cues etc. Sure you could do a blind mating to some of them but at some point when the lights go on your preference will emerge.

Understanding language does rely , posted on March 26, 2015 at 18:03:09

E-Stat
Audiophile

Posts: 37634
Joined: May 12, 2000
Contributor
Since: April 5, 2002

on lip movements. Years ago, I had a customer who was born in Russia. While I could understand what Angelina said pretty well when I was in her presence, I had the darndest time with her on the phone. I couldn't fill in the missing pieces.

Listening to audio systems exercises different parts of perception. It's not as if we're looking at a symphony and hearing a rock band. I tend to listen to music in the dark or with my eyes closed to focus on the space and the complexity of what I'm hearing. I was trying to imagine watching a speaker driver mimic the human mouth. :)

As for biases, we all have them but they are not as cut and dry as some suggest like the usual "if it costs more, it sounds better". I've been in many a comparison where the value of the components is pretty much the same. My preferences were based upon my perception of how they rendered the music, not the dollar signs since clearly there was no difference. In one particular case, the one I preferred was cosmetically the ugly duckling as compared to the other. Both were unknown brands to me at the time.

The smaller brother to that ugly duckling has been in my main system now for fourteen years. :)

I don't have a problem with blind testing, but do not agree that if you can't hear a difference between two DUT using the instantaneous Cowboy style switching method then no difference can exist. I very much find that long term listening to familiar content is far more valuable.

Similarly, one must ensure that the methodology used is verified (scientific conrols, anyone?) and free from flaws, most usually those based solely upon someone's set of unproven assumptions. I have seen numerous flawed control-free tests where the mechanism used for the testing ends up blending characteristics from each DUT such that there really is no true comparison of A vs B.

RE: Jaundiced reasoning..., posted on March 26, 2015 at 18:03:13

genungo

J.E., I was being a bit facetious in the above post and I admit that it is only my opinion that long term listening sometimes reveals hidden nuances that get passed over in short term listening sessions. I think it's what happens sometimes, for myself and for certain others too. It has not been proven to be true beyond all reasonable doubt, but it has not been unproven either.

I do think it's important that everyone is honest and open about what they *believe to be true*, though. Honesty and forthrightness is the beginning of scientific inquiry.

RE: Jaundiced reasoning..., posted on March 26, 2015 at 19:09:34
Jaundiced Ear	Hey, no worries! I don't take myself very seriously and you shouldn't either. Remember: the point of this hobby is to have fun! JE

McGurk is not relevant outside of its context, posted on March 27, 2015 at 04:40:04
Dave_K	McGurk demonstrates that we have some ability to lip read and we use it as an aid in speech recognition. But anybody who has watched an over-dubbed foreign language film can tell you that we're easily capable of separating the auditory and visual speech cues when they are conflicting. And this really has no relevance outside of speech recognition.

RE: Why blind listening tests are crucial, posted on March 27, 2015 at 05:07:39

3db
Audiophile

Posts: 1514
Joined: July 22, 2003

A few things I noted from reading all the posts.
1) DBT .. never got mentioned by me. I said blind listening test. Many of you caught that, hence "listening to my eyes closed remark". You obviously get it for those who do.
2.)Cause and effect and the ability to follow logic doesn't exist for those of you who can't make the connection of sight bias and how it affects what we hear. This video clearly demonstrates how sight prevented us from hearing the "fa fa fa" when what was really being pronounced is "ba ba ba". Then comes the BS rationalization as to how sight bias doesn't effect your hearing. This thread is indicating that the subjective camp is far more closed minded than the objective camp.

Bingo! Give that man a big cigar. Nt, posted on March 27, 2015 at 06:49:07
geoffkait	Nt

This video clearly demonstrates the effect of sight bias, posted on March 27, 2015 at 08:00:59
3db Audiophile Posts: 1514 Joined: July 22, 2003	RGA, you can rationalize this an infinite number of ways but the bottom line is, you heard fa fa fa when he was saying ba ba ba and only when you removed sight bias by closing your eyes did you here ba ba ba. Its doesn't get any simpler or plainer than that. It is what it is. This same sight bias also influences what we hear in choosing components by sight.

RE: That video, posted on March 27, 2015 at 08:21:26

Dave_K

The McGurk effect and your examples from Poppy Crum are specifically related to speech recognition. I think it is well known that we utilize a wide variety of audible and non-audible cues to understand each other in conversation. We don't just listen to the articulation of words, we also make use of what sentence/context they are used, vocal inflections, tone of voice, accent, lip movements, body language, conversational context, feelings about the person speaking, and so on. Our use of all these cues makes our speech recognition ability a lot more robust than it would be if we only listened to pronunciation.

But I fail to see what relevance this has to blind testing of audio components. In the McGurk video and Poppy's Zeppelin clip, the visual cues are directly correlated with and related to the audio. The brain is "tricked" because it is presented with simultaneous inputs that are correlated but conflicting, and has to choose. There is nothing remotely like that in our music systems, which remain visually static when playing music. And Poppy's "legislatures" clip is about masking, there is no visual component to it.

You and 3db are implying that because we utilize non-auditory cues as part of speech recognition, that we must also use non-auditory cues when comparing audio components. But the kind of visual cues used in your examples are not present in sighted listening comparisons of audio components, so there is no logical basis for using these examples to make your point about blind testing.

RE: That video, posted on March 27, 2015 at 08:54:36

3db
Audiophile

Posts: 1514
Joined: July 22, 2003

"But I fail to see what relevance this has to blind testing of audio components. In the McGurk video and Poppy's Zeppelin clip, the visual cues are directly correlated with and related to the audio. The brain is "tricked" because it is presented with simultaneous inputs that are correlated but conflicting, and has to choose. There is nothing remotely like that in our music systems, which remain visually static when playing music. And Poppy's "legislatures" clip is about masking, there is no visual component to it."

The relevance is that sight changes what you hear whether you staring at lips moving or looking at your components. The fact that one sees a component already sends an expectation in one's mind and has a direct bearing of how we process the sound.

"You and 3db are implying that because we utilize non-auditory cues as part of speech recognition, that we must also use non-auditory cues when comparing audio components. But the kind of visual cues used in your examples are not present in sighted listening comparisons of audio components, so there is no logical basis for using these examples to make your point about blind testing."

The keyword is sighted... and sighted is a visual clue whether it is lips moving or seeing the components being tested. Visual clue isn't relegated only to movement.

RE: Why blind listening tests are crucial, posted on March 27, 2015 at 09:05:44

kerr
Audiophile

Posts: 4376
Location: Central Indiana
Joined: November 10, 2003

>This thread is indicating that the subjective camp is far more closed minded than the objective camp. <

You should check out some of the PropHead threads from a few years back.

The true objective camp doesn't accept blind tests as valid unless they are double blind. They believe everything sounds the same except speakers (and phono cartridges) so they devised a test that would help support their agenda. The true subjective camp thinks everything sounds different. The reason I lean towards the subjective camp is because I've proven to myself that some things do sound different, which renders the true objective camp incorrect. So I don't agree that the objective camp is anything but completely closed-minded. Haven't decided on the subjective camp yet. :)

I agree blind tests are helpful. I've participated in many single blind tests. I don't agree they are crucial but a few of them have indeed failed to support my sighted listening. Typically they support the basic differences I hear but reduce the ease of which I hear them.

Your examples haven't demonstrated that, posted on March 27, 2015 at 09:16:32

Dave_K

Your example demonstrated that when the brain is presented correlated but conflicting auditory and visual information for one syllable of human speech, that in this case, the brain chooses to trust the visual. You haven't demonstrated that the same is true for other syllables and speech patterns, and you haven't demonstrated that our brain's approach to speech recognition has any relevance to sighted audio component comparisons. The McGurk effect is simply not present in those comparisons. There is no way for the brain to be "fooled" by conflicting audio and visual cues when the visual is static and not correlated to the music. Your conclusion is just a non sequitur.

By the way, I do think that our listening impressions are biased in some way by knowing what we're listening to in combination with preconceptions, prejudices, and other baggage we bring into the listening session. It's just that it is has nothing to do with our reliance on a combination of visual & auditory cues in speech recognition. If we're predisposed to like something before hearing it, we're more likely to like it when we do hear it. And vice versa. If you put a sheet in front of the equipment rack when I'm comparing components, but you tell me what's playing, I suspect I would form the same opinions as I would if I could see the equipment.

Dave K. below spent considerable effort, explaining why your points and examples are irrelevant., posted on March 27, 2015 at 09:22:49
Beef Guinness Cutie Pie Audiophile Posts: 136 Joined: December 29, 2011	You may - and most certainly will - ignore the obvious.

RE: Your examples haven't demonstrated that, posted on March 27, 2015 at 10:15:32
3db Audiophile Posts: 1514 Joined: July 22, 2003	Other syllables? Really? How much in denial can someone be in? Honestly man. Get a life. "There is no way for the brain to be "fooled" by conflicting audio and visual cues when the visual is static and not correlated to the music" And you can back this statement up with objective test results, not inconsequential subjective results?

RE: Your examples haven't demonstrated that, posted on March 27, 2015 at 10:51:24
3db Audiophile Posts: 1514 Joined: July 22, 2003	Read http://seanolive.blogspot.ca/2009/04/dishonesty-of-sighted-audio-product.html

Dr Floyd Toole's work has also supported sight bias, posted on March 27, 2015 at 11:23:24

3db
Audiophile

Posts: 1514
Joined: July 22, 2003

http://www.harman.com/EN-US/OurCompany/Innovation/Documents/White%20Papers/AudioScience.pdf

http://www.harman.com/EN-US/OurCompany/Innovation/Documents/

RE: Give that man a big cigar. , posted on March 27, 2015 at 12:59:17
genungo	Why, does he deserve to die?

The logic eluded you, posted on March 27, 2015 at 13:54:45

Dave_K

You were claiming that A implies B without establishing any logical connection between them. It was kind of an "underpants gnomes" argument, which is why I picked on it.

I do not dispute that audio component evaluations are influenced by the knowledge of what component you are listening to and what preconceptions you came in with. But that has NOTHING to do with the McGurk effect or the Poppy Crum stuff that Tom posted.

Cigars, posted on March 28, 2015 at 03:52:12

Dave_K

I have been known to occasionally smoke a big cigar, about once every few years. It's always when a friend or professional acquaintance buys me one. The first half of a cigar is always a nice treat, especially when combined a good Islay malt whiskey. But the second half over stays its welcome, and by the end I'm wondering why I agreed to this and how do I get out. I'd rather have an average cigar where nobody cares if I bin it half-way then a premium cigar where I feel an obligation to finish it.

Who cares? Nobody is saying certain biases don't exist in audio. Just don't blame everything on them. Nt, posted on March 28, 2015 at 06:53:07
geoffkait	Nt

Why the industry can't afford BT or DBT listening tests, posted on March 30, 2015 at 00:19:29

stehno
Manufacturer

Posts: 739
Location: Oregon
Joined: November 8, 2001

1. Reasonbly configured system or even SOTA-level systems generally sound far more alike than they do different.

2. There is no litmus test for participants in such tests and it should be a well-known fact that the majority of enthusiasts and professionals do not have very well-trained ears.

3. Even the industry's best SOTA-level playback systems are pretty dismal when compared to live music or the absolute sound.

This is often times why a reviewer or auditioner is lost without their reference music and why when listening critically, the listener must strain to hear some of the musical characteristics they covet.

Given the above, a blind test of any kind could and probably would prove disastrous at some point because:

1. It could expose an expensive component (with high advertising dollars) for what it isn't. And no mfg'er would like that.

2. It could expose the participant's (often times an "industry expert") hearing for what it isn't.

3. Upon reading the findings of such tests, some consumers could conclude that "high-end" audio is far from being high-end after all and/or the consumer could conclude that the participant's hearing isn't what it ought to be.

It's called risk management and I speculate the stakes are too high for industry professionals to consider such tests as valid.

Anyway, that's my guess.

RE: Why the industry can't afford BT or DBT listening tests, posted on March 30, 2015 at 01:02:56

pictureguy
Audiophile

Posts: 22597
Location: SoCal
Joined: October 19, 2008

Steh,
without regard to the merits of your arguements, ANOTHER reason that DBT is not attempted more often is that it is nearly impossible to actually DO.

It may be possible to arrange such tests for something simple, maybe power cords or interconnects, but as soon as you start chaining stuff together, it gets weird.

I did experiments when working semiconductor processing. One task was to optimize a process. I had maybe 8 or so variables. The number of tests was HUGE so I had to 'make a few assumptions' simply to limit the number of required tests. And this was with output that was ALL MEASURABLE and in no way 'opinion'.

I'd LOVE to figure out how to actually compare stuff for 'best'. Heck, you can't even get 5 'audiophiles' (randomly selected) to agree on break-in or what CABLE sounds best. Good luck with DBT.

One guy who has offered money to anyone who in HIS test can reliably (a statistic) tell the difference between amps has NEVER paid. Few even pass the 'qualifying' test, let alone when the big bucks are on the line.
Too much is never enough

Its clearly evident that , posted on March 30, 2015 at 06:14:39
3db Audiophile Posts: 1514 Joined: July 22, 2003	you can't follow logic and that is evident in your replies. All I see is BS rationalization by the subjective camp stating how there is no logical sequence.

RE: Dave K. below spent considerable effort, explaining why your points and examples are irrelevant., posted on March 30, 2015 at 06:17:51
3db Audiophile Posts: 1514 Joined: July 22, 2003	All I see is irrelevant arguements thrown up in a vain attempt to disprove the logical. Your just another lemming ready to follow the subjective crowd over the cliff.

RE: Who cares? Nobody is saying certain biases don't exist in audio. Just don't blame everything on them. Nt, posted on March 30, 2015 at 06:20:33
3db Audiophile Posts: 1514 Joined: July 22, 2003	Wow. Really?? Your acknowledging that biases exist and yet you can't accept that influences what one hears? Really?????

RE: Why blind listening tests are crucial, posted on March 30, 2015 at 15:10:08
Analog Scott Audiophile Posts: 9933 Joined: January 8, 2002	Kind of late to the discussion. A question, do you do blind listening tests?

RE: Why blind listening tests are crucial, posted on March 30, 2015 at 16:30:42
A.Wayne Audiophile Posts: 2527 Location: Front row center Joined: November 30, 2011	So blind people cant hear or is it the sighted who have an issue .....

How many DBT tests of hearing audio equipment have you done?, posted on March 30, 2015 at 18:23:45

Timbo in Oz
Audiophile

Posts: 23221
Location: Canberra - in the ACT - SE Australia
Joined: January 30, 2002

I want a set of figures for each.

i)The number of trials ie 'n'. ii) the confidence limit you set:- 90/10, 95/5 or 99/1. I'll assume for now that you took 'n' into account when setting this.

And what the values of alpha and beta were for that test when you did the calculations.

What did you do to make sure that nothing else changed apart from the D'sUT; and for each trial - the listener?

Warmest

Tim Bailey

Skeptical Measurer & Audio Scrounger

RE: Why the industry can't afford BT or DBT listening tests, posted on March 30, 2015 at 19:46:32

stehno
Manufacturer

Posts: 739
Location: Oregon
Joined: November 8, 2001

You bring up another very good point, but I hope you didn't think my points were without merit. :)

The problem you at least indirectly mentioned is that many products are not up to speed the moment they are installed.

For example, a friend with well-trained ears was visiting some years ago and after about an hour of listening I installed a new IC that I had ranted and raved to him about and was beta testing for a cable mtg'er. So I installed it and it sounded like crap even to me. My friend asked if this mtg'er knew what they were doing. So I left the system on in repeat mode and we headed out to dinner and 90 minutes later the system was singing a new beautiful song.

There's plenty of other examples far better than this one. But suffice it to say that any audio blind tests where everybody thinks they can just plug-and-play-at-a-product's-optimal is very naive and flat out wrong many time.

Without applying thought or standards, half-assed is all that's given and all that's received. No matter how methodical the participant think they may be.

RE: Why the industry can't afford BT or DBT listening tests, posted on March 30, 2015 at 21:46:14

pictureguy
Audiophile

Posts: 22597
Location: SoCal
Joined: October 19, 2008

Their are 2 (TWO) aspects to not being up to speed the moment of install.

Warmup happens to both SS and Tube gear. Some transitors apparently work best when warmed slightly from ambient. I think it was the Phase Linear 400 of a LONG time ago that had this problem in spades.
Warmup may also effect insulated product, like cables and interconnects. I'm becoming aware of a class of phenom dealing with insulation and 'saturation' of insulation during use.

The other aspect is one of the LISTENER. I've been semi-trashed (most were on the polite side, I'll admit) when I simply asserted that lots of breakin was YOU, not the GEAR. This is over longer time periods and not the short time stuff like cap formation and insulator saturation.

Maybe MORE, but this is what occurs to me, off the top.

To the extent that the above is valid, it's also a reason for DBT to be on the Very Difficult end of the spectrum to get right, IN PRACTICE.

Too much is never enough

RE: Why blind listening tests are worthless, posted on March 30, 2015 at 22:04:19

Jon Risch
Bored Member

Posts: 6659
Joined: April 4, 2000
Contributor
Since: March 1, 1999

Bottom line:

All it takes is a crappy blind test, for it to come up statistically non-significant.

This proves absolutely nothing.

Of course, often times a null result of a blind test is confused as being the same as a negative result, but that is just not true, and not real science.

The second most problematic issue is that it is really hard to design and maintain a truly high resolution blind test that can actually detect subtle sonic differences. It is much harder than most people realize, especially if you have never done it before.

Unless the blind listening test has met a certain metric of sensitivity, that is, it has shown exactly what it CAN detect, then for all we know, it is inherently incapable of detecting the sonic difference in which we are interested in investigating. Unless it has met that baseline metric with statistical proof of it's detection abilities, it is indeed worthless and useless as any sort of detection system or proof of any sort of lack of sonic differences.

These posts below relate to this subject, and go into more detail on what the problems and issues with blind testing are, and why they may not be the be-all, end-all that some folks seem to think they are.

Yes, we can fool ourselves, but it is a fact that this extends to reading more than we should into a blind listening test null result too, that is also fooling ourselves (all in the name of supposed science).

http://www.audioasylum.com/forums/prophead/messages/2190.html
http://www.audioasylum.com/forums/prophead/messages/2579.html
http://www.audioasylum.com/forums/prophead/messages/2580.html
http://www.AudioAsylum.com/forums/prophead/messages/3100.html

Jon Risch

RE: Why the industry can't afford BT or DBT listening tests, posted on March 30, 2015 at 22:33:43
stehno Manufacturer Posts: 739 Location: Oregon Joined: November 8, 2001	Yes, Leo, I'm well aware of your stance here which I always find surprising. Especially with your background. But hey, you're still a swell guy. :)

RE: Why the industry can't afford BT or DBT listening tests, posted on March 31, 2015 at 00:24:21

stehno
Manufacturer

Posts: 739
Location: Oregon
Joined: November 8, 2001

BTW, as mentioned before, I'm well aware how easily some-to-many can be influenced or have their minds play tricks on them as you say.

But you take this fact to such an extreme as though everybody can be easily dup'ed or persuaded.

That's the silly part. That's like saying ALL politicians are corrupt when it takes just one straight shooting politician to negate that entire statement.

You wanna' say some-to-many are potentially dup'ed when they think they hear things, I can live with that.

You wanna' say that every time anybody claims to hear an improvement that it's their mind playing tricks on them, well, my imaginary friend and I think you have very serious issues..

Anyway, good chattin' with you.

RE: Why the industry can't afford BT or DBT listening tests, posted on March 31, 2015 at 00:37:22

pictureguy
Audiophile

Posts: 22597
Location: SoCal
Joined: October 19, 2008

That's a very good question.
IS there anyone with hearing which is beyond even perfect pitch? Will someones memory or a particular 'signiture' be so good as to identify a particular piece of gear MONTHS after last hearing it? Or even longer? Is there anyone out there who can RELIABLY (statistically speaking) identify when SILVER is used in an interconnect? One person I 'spoke' with hates tinned wire in a power cable. And BLACK insulation!
I always think of the movie where a crime is committed and all the witnesses tell different stories.
I think it was a Kurosawa movie?
I know that a movie with Forest Whitaker uses a similar device. Vantage Point is the movie. A crime is committed and you see it from a bunch of different viewpoints and from different observers.
I view stereo the same way. A bunch of persons in a room and all hearing the same thing. Or ARE they?
Experience is subjective.
Too much is never enough

RE: Who cares? Nobody is saying certain biases don't exist in audio. Just don't blame everything on them. Nt, posted on March 31, 2015 at 10:18:24
kerr Audiophile Posts: 4376 Location: Central Indiana Joined: November 10, 2003	>Your acknowledging that biases exist and yet you can't accept that influences what one hears< It (can) influence... it doesn't dictate.

General Asylum

Why blind listening tests are crucial

140.80.199.91