 
|
Audio Asylum Thread Printer Get a view of an entire thread on one page |
For Sale Ads |
|
|
Audio Asylum Thread Printer Get a view of an entire thread on one page |
For Sale Ads |
129.128.119.163
In Reply to: Try to read between the lines, my friend. posted by cheap-Jack on March 16, 2006 at 07:50:19:
I would go about it this way.If I were to conduct this test, I would have a FRIEND label the CD's and change them FOR ME without me seeing which disc was which.
I would then listen to each disc about 3 times to start. So I would write notes for three iterations of three discs or nine trials.
Then I would see if there was ANY similarities between my notes, and then ask my friend to reveal the sequence of which discs were played. This, of course, is if any differences were there to be heard.
If I DID hear any differences at all I would then try an ABX with two discs at a time. I would listen to two discs, one for say 2 minutes and then another for two minutes. Then I would have my friend play one OR the other, and I would try and tell him which one was which. I would try this for at LEAST 10 iterations and probably more like 20. I would want three different ABX tests, so that each disc would be part of at least ONE of the ABX tests. (aka 1-2, 2-3, 1-3)
This test method is one heck of a lot more thorough than just listening to the discs ONCE - not to mention SIGHTED!
The results of the reported method are still totally invalid. Statements about "trusting ears" and "try it yourself" etc etc do not change the fact that the method used is insufficient to prove ANYTHING more that people just don't know how to do listening tests so that any useful data whatsoever is generated.
Sure hearing is subjective - we can't tell what someone else is hearing. But we CAN test to see if the person is hearing the same thing in repeated iterations, and reduce psychological bias and the effect of the power of suggestion. People tend to hear differences when told they are going to hear a difference. The fact that they DO hear a difference in this case should not be automatically accepted and filed under "trust your ears". It should be filed under "test is suspect - did not compensate for human factors".
I'm done with this thread. Feel free to add any closing comments you have - and thank you for a cordial exchange. But I feel I have explained my point as well as I can. Arguing until we both agree is pointless. If that worked, everyone at the Asylum would be in total agreement by now!! lol
Cheers,
Presto
Follow Ups:
Hi.I don't care whatever so-called "more reliable" methodology
of conducting audition tests. After all, our ears get the final say.Mind you, audio perception involves hearing, sight & feeling.
That's why we all enjoy live concerts where we hear, see & involve in emotion of being there in the performance.Missing any of these senses will kind of jeopardize our aural interpetation & render it incomplete.
We enjoy our fine dines as we see, smell & taste the food. Who wants to be blindfolded & nose plugged to eat? Please don't tell me
the so called scientific way of fine food appraisal is to eat with eyes closed & nose plugged.Back to DBTs or whatever "more reliable" aural tests of audio gears,
it is so tedeous & so difficult to carry it out right, resulting null resutls so often. The auditon panel members got to trained how to conduct such audition properly etc etc etc less alone hearing & memory being different from person to person. It is still "more reliable"? I doubt.I would go back to the basic way of sighted hearing which IMO, the nature bestows on us since day one. Anything artificial way depriving our total sense of hearing perception is not good enough & should not used to challenge our hearing.
c-J
...then, I'll be impressed with a measurement of the senses.Meanwhile, with music, I'll stick by my ears.
clark
As you know from reading my earlier posts, we do use human senses to "measure" wine (this is what I do professionally), but we do it in much the way my suggested experiment is structured- blind, with strict protocols.From an instrumentation point of view, we can indeed say a lot of things about a wine from its spectrum and other simple measurements (ebulliometry, FSO2/TSO2, total acidity, residual sugar, tannins...). We can't tell good from great, but we can tell flawed from either, can predict the wine's general sensory impression, and we absolutely can distinguish wine A from similar wine B in double blind human sensory tests, and indeed do so on a routine basis. Good versus great is totally subjective, but ability to reliably tell one wine from another (or the effect of some process on a wine) with only the wine in a glass in front of you is absolutely objective and is the gold standard in wine tasting, as with all other non-voodoo sensory research.
If this guy can do it, great, we have an interesting phenomenon to look at. If he can't, well, there are (surprise, surprise) people who fool themselves and (no surprise, sadly) people who are happy to bullshit others to sell magazines or hardware.
That's more or less true in audio too, wrt. blind testing: You can tell if something's wrong. Nor does any "scientific" examination of wine yet reveal aspects such as its terroir -- I once met a fellow in France who claimed he could tell within a couple hundred meters where most wines come from. Machine, do your thing! NO, NO, NO! That was Grands-Echézeauz, not La Tache. Fool!The use of DBTs in wine is greatly exaggerated. Oh, if there's a competition, yes, of course. But 99.99% of the time wine is tasted fully sighted by the pros, who reach their conclusions dispassionately. As do the coffee guys when they "cup". As do we in audio when we listen.
Worth reiterating: We who practice audio with our ears feel no need to justify ourselves to anyone. There is no point to prove. If proof you need, you are always welcome to seek it -- elsewhere, please. Unless you have funds available, then I for one would sign up as director.
Hope that helps.
clark
> We who practice audio with our ears feel no need to justify ourselves to anyone. There is no point to prove. If proof you need, you are always welcome to seek it <Bingo
c
"But 99.99% of the time wine is tasted fully sighted by the pros, who reach their conclusions dispassionately. "This is absolutely incorrect. I participate in both professional and amateur tastings, at wineries, in analytical settings, during retailer evaluation and selection, as part of quality control, and in educational programs. I do this for a living and on a daily basis. I do this at wineries and institutions around the world. I do this with winemakers, technologists, retailers, critics, and wine educators. After 10 years in this industry, I cannot recall any setting where any of these people would not use blind methods to either evaluate or to confirm impressions.
All Master of Wine and Master Sommelier examinations are done with blind tests. Every last one of them.
The most popular and influential wine critics in the US, The Wine Spectator and The Wine Advocate, do all of their published scoring blind.
I pounded you before about this analogy. It is one that badly weakens your argument. I would think that a bright and articulate guy like you would have found a better one by now.
NB- Sometimes, confusion arises from differing meanings of terms of art. What audiophiles would refer to as "double blind" would just be called "blind" in the wine world. In the wine world, "double blind" has a different meaning, irrelevant to the topic at hand. The separation test I outlined earlier is one we will often use to determine effects of different materials in the wine handling process on the final sensory properties, but it would not be called "double blind" by a wine guy.
So, do you think you could, in the comfort and privacy of your listening room, with the drink or smoke of your choice, your choice of listening material, and all the time you need (months, if you want), be able to separate those two groups of CDs? If so, let's roll.
...during the (delightful) examination of these beverages was the
source masked -- with this exception: Occasionally we would do that
trick just for the hell of it. I was once treated to a group of
wines at Larry Archibald's and he had wrapped them in foil, just to
see how I would do. I didn't mind the test and by the way I did
surprisingly well, pleasing both of us.Also I recall the time, maybe a decade
ago, when I was discovering glasses (meaning, glass shape and
size). Already I knew what the result would be, when I poured the
same wine into two different glasses and handed them to a friend
who is pretty good at this stuff. He didn't see me doing it, so we
had "blind" conditions. Of course he thought they were two
completely different wines.After that the glass factor had to be added to our investigations,
but openly, sighted.I'll stipulate that MW and MS exams are conducted blind. Geez Luiz!
Also I know coffee and tea people who would laugh outright at the
idea that blindness is needed for them to render a fair opinion. A
cup is what it is.And thus audio as well.
clark
SY, do we have to be criticised for everything? I am all for blind tests that WORK! In audio, they don't work very well. This has been my personal experience, along with John Atkinson, and the 'Absolute Sound' reviewers, starting decades ago. Nothing has changed in the testing that would bring out any significant improvement in the test procedure.
John, I think I have an understanding about the sighted/blind/double-blind testing issue. I think it comes down to the psychology of the person. There are people, such as yourself, who have developed a very high level of critical thinking skills and an ability to recognize when their emotional biases are interfering with objective analysis, and to overcome those emotions and remain analytical.So, it is quite likely that some people are able to objectively analyze situations without being double blindfolded. This, I think, is a good thing, because it allows the observer to consider the next step in development and research, and not just play an odds games.
On the other hand, most people don't fall into that category, so viva la double-blind test (for the rest of us).
Tonmeister '74
You say: I am all for blind tests that WORK! In audio, they don't work very well.---
I dispute that. There is damn near a century of work that shows exactly the opposite.
Your evidence, John. Not anecdotes, not hear-say, but evidence. Solid, testable, verifiable, confirmable, falsifiable evidence.
Where?
None of this trying to shift the burden crappola like Mike Kuller, let's see your affirmative evidence for your claim.
d
MKJ's statements appear entirely reasonable to me, regarding wine, beer, or audio.Your claim about "weaselling out" is positively scurrulous, indefensible, and insulting.
You do not appear to be a reasonable person when you engage in such rhetoric.
Are you blind? He weaseled!clark
I am not blind.I have also done wine tasting.
I have also studied how the pros do it when they are serious.
It's blind, Clark, it's blind. No weaselling.
Yes, everyone also drinks wine non-blind. For advertising, you betcha, they want you to react to the label.
Just like audio, if you're SERIOUS, do a blind test, which despite Curl's statement of the other day, works just fine.
If you're wanting to enjoy something, do what thou wilt.
John, c'mon, be fair, I certainly don't criticize you for "everything." In fact, I can't think of many things at all that I've ever criticized you for.I do criticize blatantly false statements like Clark's (I think that you'd admit that I might have slightly more knowledge about what people do in the wine biz than he does) and I criticize people who make outlandish claims, then duck the responsibility of testing them. I find it curious that blind tests purportedly "don't work" in audio, yet they work in every single other field of sensory research...
I take what you say seriously, as you know. I do not take seriously what magazine-peddlers say.
I'm back in town tomorrow, give me a call if you have a chance, I've got an interesting little problem on my hands that you might be able to shed light on.
Larry is a fine guy, but he's not a pro, just an amateur with a good palate. I can't comment on what wine hobbyists do, only tell you the straight dope on what the pros do. That was your earlier false claim.If you don't believe me, ask another inmate, Mark Kelly, who is a first-rate winemaker and a pioneer in micro-oxygenation.
Numerous times I have walked into an office (or room) where dozens of bottles are on the table and "pros" are trying them; none were wrapped. And I have gone on barrel tastings where all participants were fully aware what was going down (as it were).Knowledge does NOT cloud judgement. Especially if you ARE a pro!
And I repeat, professional coffee and tea cuppers also taste unblind.
One reaches a point where one is no longer swayed by labels. Perhaps that's a good working definition of "pro".
clark
Barrel tastings are generally informal and not analytical. They're used to help sell wine. And pros do informal tastings, too, but when things are serious, glasses are unmarked.Label-awareness has little to do with it- I'm an alumnus of the legendary Wine Box, where one quickly learns to not be swayed by labels, but I'm fully aware that as a human being with a human brain (rather than being a gas chromatograph), all sorts of unconscious biases creep in to my analyses. One cannot put aside unconscious biases- they are, after all, unconscious, n'est-ce pas?
There's a pretty interesting bit in the recent bio of Robert Parker ("The Emporer of Wine") on The Callahan Challenge. It's amusing and worth a read. I was there and was, in fact, one of the writer's sources. I didn't do too terribly well, but my cooking/drinking partner scored 27/29 correct and nailed four or five wines dead-on.
Clark, please name some names of people in your area engaged in wine research who are not doing their work using standard blind methods like triangle, forced choice, and the like. As far as I'm aware, Boston is not a center for wine research, but I certainly could be wrong about that.I don't know about you, but if I heard the kind of differences the German guy was describing, I'd like to think that I could nail the test I described earlier.
...and asserted, rather patronizingly I might add, that because Boston so far as you know is "not a center for wine research" I couldn't possibly know what I'm talking about.Within those new confines I'm happy to stipulate that you are correct. That leaves you holding the bag, or swirling the dregs, in all other wine situations, as well as in the tea and coffee cupping areas I have mentioned all along.
And by the way, while Boston is somewhat a center for coffee cupping, and I've been there, for tea it's a leader and I've been there too. No blind tasting!
clark
PS My audio life is too full already to try something that comes from an unknown source. Plus I'm still proselytizing a physical acoustical principle discovered forty-seven years ago, for all the good that does with sloppy listeners. And it isn't even a tweak!
clark
I've seen less sighted than blind tastings myself.Yeah, the club I belonged to was the Bell Labs wine enthusiasts at the time, yes we did have everything from research psychologists to solid-state physicists on board, and yes, the tastings were designed more like professional tastings.
That doesn't mitigate the point that in fact the PUBLICATIONS who do this sort of thing report doing blind tastings.
Yes, everyone does informal tastings, too, but not for serious business. Just like they shouldn't do informal listening (non-blind) for serious audio science.
...and rearrange all the furniture. But before I get home, they put it all back exactly where it was when I left. This is a fact.There is no point to prove. If proof you need, you are always welcome to seek it -- elsewhere, please. Unless you have funds available, then I for one would sign up as director.
> ...and rearrange all the furniture. But before I get home, they put it all back exactly where it was when I left. This is a fact.
There is no point to prove. If proof you need, you are always welcome to seek it -- elsewhere, please. Unless you have funds available, then I for one would sign up as director. <If you want to take on the burden of proof, knock yourself out. If anyone else decides to take that burden on, THEY are welcome to do so. That is, I think, Clark's point. HE doesn't need to prove anything but if you're concerned about it, feel free to seek out the proof you need.
Clark:You see, most "approval and disapproval" I come to express is based on just what you say: by using my ears. And I do think that I can hear AT LEAST as good as the next guy, meaning that my hearing is not impared and I spend quite a bit of time doing analytical listening (as opposed to just sitting down to listen and enjoy).
There can be NO ARGUMENT when someone makes a claim such as "I heard these discs - they are unique and all sound different".
That is fine. The person used his ears, he heard differences, and according to him THERE ARE differences (even if someone else 'does not', 'cannot', or "WILL NOT" hear these differences.
But for that person to go on to explain that the differences are caused by where the discs were stored - c'mon Clark - you must admit that his hearing ability has NOTHING TO DO with making an assertion like that. He simply cannot tell WHY those discs sounded different. There is no validity to the statement. Maybe CPU load was higher while burning one disc compared to the other? Maybe the CD writer was at an increasingly higher temperature for the subsequent burns?. Maybe operating system processes were cycling in and out as they often do (automatically) - did he save a printout of his PC's event viewer? Was the burn speed FORCED to a finite value, or was it automatically set by the software? Was it fixed or did it float? How come nobody else is asking these questions?
***Maybe the discs are different because of a reason completely unaffliliated with where they were stored!***
And maybe the discs AND their pit geometries are bit for bit identical and the differences are not there at all.
There is a REALLY big difference between hearing something and reporting that, and hearing something and automatically being POSITIVE as to what caused it.
I don't doubt that people DO hear many of these differences that they report to hear. (Or at least HONESTLY BELIEVE that they do.) What I DO doubt (and so do many others) is how they came up with the REASON for why things sounded different to them. If the sun was shining that day, they might say "That's IT - it was because it was sunny outside". Basically they can take ANY of 1,000,000 variables and because THEY chose THAT variable it MUST be the variable responsible for the difference? Right? Exactly wrong.
All I am trying to say here is that if we (as a community) don't look for the REAL reasons why things make actual positive sonic differences we're going to end up with a bunch of "placebo effect" products, treatments and methods instead of discovering the REAL cause behind the improved sonics.
And unfortunately, the power of suggestion is powerful enough to make people THINK they knew EXACTLY what causes the difference in the first place.
Don't distrust your ears. Distrust bad science and poor reasoning.
Cheers,
Presto
"All I am trying to say here is that if we (as a community) don't look for the REAL reasons why things make actual positive sonic differences we're going to end up with a bunch of "placebo effect" products, treatments and methods instead of discovering the REAL cause behind the improved sonics."Thanks for re-iterating this oh so important crucial issue.
Music making the painting, recording it the photograph
d
- http://db.audioasylum.com/cgi/m.mpl?forum=general&n=405566&highlight=theaudiohobby+innovation&session= (Open in New Window)
Chris is talking about preference.The claim here is to fact, i.e. "different floors make a difference in how CD's sound".
Offering up preference in reference to fact is nothing but suborned implication that offers up a straw man. I would prefer it if you would be more logical in the future.
...you keep up with your prick-ly ways. Impertinent and aggressive high-handedness is a great spectator sport, I could watch it for hours.The experiment about floors was risibly fraught with ambiguity and error. For instance, how are CDs changed by the shipping method? What if FedEx has plastic floors on their trucks and UPS, wood? Just for instance.
clark
First truth, the high-handed, etc, ways are yours.Second truth, you spout stuff and nonsense about double-blind testing.
Third truth, you routinely say thing about people that are untrue, offensive, and hurtful.
Those are three truths.
The experiment about floors was troublesome for many reasons. Stating that, however, is purely disingenous and evasive on your part, because you didn't say in your dismissal of blind testing that you were dismissing this test, you said, as you are wont to do, that all blind testing of audio is invalid.
And that position, a position that you have often supported, is sheer, absolute quackery.
And in the process becoming a fount, nay a veritable Old Faithful, of UNTRUTH:"You said... that all blind testing of audio is invalid."
And you have me saying that repeatedly.
Where, please, did I make those statements? Remember, it's what I am "wont to do".
If you can't prove your charge, I shall have to lodge a complaint with the Bored that you are full of chutzpah.
clark
Bobby Flay is a joke. He's on that show for marketing reasons, not because of any great cooking skill. I wince every time I see him cook the same damn thing, just with the theme ingredient slotted in. It's a pity that they couldn't retain Puck; he's a brand and a celeb, his restaurants are OK-to-good but no better, but the guy really, really knows how to cook.In any case, this entirely misses the poit. The question here is not about which is "better," it's "is there any difference at all beyond this guy's imagination?" And my question, which is why all of these listening wizzes seem to be afraid to put their claims to the test.
a
PP is not on Iron Chef. I've only eaten his food once, back in the heyday of K-Paul. Terrific stuff.
Pick two floors, ones that the claimant says have the biggest differences. Send him ten coded discs, five from floor A, five from floor B. If he can successfully separate them into two piles, taking as much time as he wants, using whatever method and environment makes him comfortable, he's got something. If he can't, he's just one more human who has managed to fool himself (as we all do from time to time).
nt
This post is made possible by the generous support of people like you and our sponsors:
Top of Page ]
[ Contact Us ]
[ Support/Wish List ]
[ Copyright © 2025, Audio Asylum, all rights reserved ]
