In Reply to: a tale of bias. The irony of the common held beliefs of many objectivists posted by Analog Scott on July 19, 2010 at 13:30:55:
"What is important is that you select your system in advance"
If you are trying to avoid fooling yourself perhaps this advice has meaning, but if you are trying to do a meta-analysis that will persuade others it is not sufficient to select the system in advance. It is also necessary to provide a convincing argument that your system was actually selected in advance. You can say that was your procedure, but in this day of questionable scientific integrity you will have to convince your readers that was what you actually did.
I don't see how you can provide any evidence that this was your procedure. Indeed, if one is dishonest it would be entirely possible to do a preliminary survey of the literature and come up with a set of criteria that gives the desired results, remember these criteria and then trash all the preliminary work. At this point one would begin the "real" work and just happen to start with "developing" the pre-arranged criteria.
As you can see, I am highly suspect of meta-analysis. They are subject to being gamed and the complex statistics are easily abused. If there is a well defined and reputable collection of studies, then there won't be such a problem. However, I don't believe this is the case in the audio field.
Tony Lauck
"Diversity is the law of nature; no two entities in this universe are uniform." - P.R. Sarkar
This post is made possible by the generous support of people like you and our sponsors:
Follow Ups
- Highly Suspicious of Meta-Analysis - Tony Lauck 09:41:05 07/20/10 (2)
- RE: Highly Suspicious of Meta-Analysis - Analog Scott 10:34:57 07/20/10 (1)
- RE: Highly Suspicious of Meta-Analysis - Tony Lauck 12:47:53 07/20/10 (0)