Article: 1148 of sci.psychology.research Newsgroups: sci.psychology.research From: gsiegle@sunstroke.sdsu.edu (Greg Siegle) Subject: Re: Forum for null results Organization: San Diego State University Date: 16 Feb 1995 02:52:53 GMT Approved: al@debra.dgbt.doc.ca Hi all, Greg Shenaut (in Article: 1128 of sci.psychology.research) has suggested creation of a forum for null-results. I wholeheartedly second the idea. Greg gave three reasons for the forum's creation including: > (1) to get feedback--perhaps > someone may want to build on these attempts, but in a slightly > different direction; (2) to prevent someone else from going over > this same ground; (3) to have something to show for all the work > and expense we went to. I'd add a fourth reason: to obviate the file-drawer problem for meta-analysts. Meta-analysis attempts to combine results found throughout a literature to make robust claims about a phenomenon. A fundamental assumption of meta-analysis is that published studies represent the research done in the field, and thus, a combination of effect sizes culled from published studies represents the state of known research in the field. Unfortunately this assumption is violated if significant numbers of unpublished works exist, and remain unpublished because they provide null results. While there are many dangers in relying on null results as evidence for a null hypothesis, inclusion of null results may provide an important way of increasing the robustness of research syntheses. Greg suggests that the null results forum should not be peer-reviewed. While not having peer review would, indeed, assure that null-results are brought into the public forum, such results might not be citable in research-syntheses. Potentially a peer-reviewed "Journal of Null Results" in which articles were selected for methodological clarity, high enough power to be interesting, etc., but not rejected for having null results might be especially helpful in this respect. I would be interested in seeing both peer-reviewed and non-peer-reviewed forums created. -- Greg Siegle