|
Audio Asylum Thread Printer Get a view of an entire thread on one page |
For Sale Ads |
130.164.67.107
In Reply to: RE: That clears up a lot! posted by EBradMeyer on September 22, 2007 at 05:08:50
I've already discussed a lot of this with David Moran on HydrogenAudio (see link).
People who are using my complaint to justify attacking the paper as a whole, who haven't read the paper themselves, definitely don't know how I feel about it. I don't have a problem with the conclusion at all. And for that to be true, ultimately, I don't have a problem with the test setup. I'll probably never be in as good of a position to evaluate the value of high res music as you were with your test setups. It's a result that, even if I were to take your setup with a grain of salt, is still a test well beyond my means both financially and in terms of my listening experience. So I trust it.
Still, I understand that much of the data was omitted either because the editors thought it was impertinent (ie, the music selections), or because it was perceived that no rational debate on the data is possible. The former can't be helped, but the latter seemed like a somewhat cavalier attitude to take, and one that confused me. Even though, as some others have pointed out, it's most likely true.
Like I said to David: "Trust us" is not a very reasonable argument to use in a technical paper. Clearly the editors disagreed, though, and I don't think one must resort to conspiracy theories to explain that. I'm still a little confused as to how much detail is acceptable for peer review, versus how much should ideally exist in a paper for the purposes of test reproduction and establishing trust in the test procedure.
Nevertheless: Thank you very much for doing this test!
- http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=57406&view=findpost&p=517818 (Open in New Window)
Follow Ups:
Initially it was expressed in:
http://www.audioasylum.com/forums/prophead/messages/3/37245.html
See, it's not about "omission of data"!
Then that was expanded upon in:
Howdy
Ironically one of the papers in their references doesn't seem to have these problems, it gives much more detail about equipment used, music selections, details of the test protocol, some demographics of the test subjects, a serious digression into the four outliers who could hear a clear difference, etc.:
D. Blech and M. Yang, “DVD-Audio versus SACD: Perceptual Discrimination of Digital Coding Formats,” presented at the 116th Convention of the Audio Engineering Society, Berlin, Germany, 2004 May 8–11, convention paper 6086
I certainly don't know what the differing constrains are for being published by AES in these two different fora, but after reading the paper which is the subject of this thread reading the cited paper above was a breath of fresh air.
-Ted
One is once again reminded of Todd Krieger's inimitable phrase, "token peers".
Why, I've even gotten E. Brad to treat me high-handedly, in that connection:
"This statement was about whatever should or should not
be written at this point in a particular refereed journal. Unless you're
planning to write for it, which something tells me you aren't, it doesn't
apply to you."
Good to know.
I'll sign off --
clark, AES Life Member
"Sometimes experimenters may make systematic errors during their experiments, unconsciously veer from the scientific method (Pathological science) for various reasons, or, in rare cases, deliberately falsify their results. Consequently, it is a common practice for other scientists to attempt to repeat the experiments in order to duplicate the results, thus further validating the hypothesis." (see link)
It seems clear that documentation in this case was lacking and hence it would not be possible to replicate the experiment based upon the published paper. This is a sad statement for the documentation of an empirical investigation published in a peer reviewed scientific journal.
It would be entirely futile to argue this point as the demands of the scientific method clearly were not met and franlkly it matters little what I, you, or the editors feel about it. The scientific method makes no allowance for lowering standards regardless of whether rational debate may or may not be possible, and it certainly makes no allowance for "Trust us" assertions!
We hope our perceptions ain't slippin
But we swear to God we seen Lou Reed
Cow tippin
I agree. Full disclosure of equipment used and methods is a mandatory and prefunctory part of all scientific papers. It would be like me publishing a paper on detection limits in mass spectrometery for some compound and then not telling them what kind of mass spectrometer I was using...instead simply saying that it was a properly designed and funcitoning unit. I can assure you lots of questions would be forth coming from the reviewers if I had omitted that critical piece of information. Also, omission of sample handling would raise a few eyebrows!
FAQ |
Post a Message! |
Forgot Password? |
|
||||||||||||||
|
This post is made possible by the generous support of people like you and our sponsors: