How to Identify a "Good" Scientific Journal

Everyone says their scientific journal is more reputable than your scientific journal. Who's right?

Filed under General Science

Skeptoid #50
June 15, 2007
Podcast transcript | Listen | Subscribe
Bookmark and Share

Today's episode is about a subject that's as old as debating itself, and it was prompted by the following email:

I would love to look a lot of this up on my own, but am unsure about what sources can be trusted. I know you talked about how scientists are not created equal, but as an average person without the background to fully understand the primary sources or the ability to synthesize a consensus without reading meta-analyses, where can I go for reliable information?

This listener brings up a great point. If you follow any of the conversations on the Skeptalk email discussion list, you've probably heard us banter this back and forth. One guy says "Hey, leprechauns are real, here's an article in a peer reviewed scientific journal that says so," and then someone else replies "No they're not, because there haven't been any such articles in my peer reviewed scientific journals." Almost any debate can degrade into "My peer reviewed scientific journal is better than yours."

Now, a really satisfying answer to this question would be "Here, go to www.legitimate-scientific-journals.com, and you can see at a glance if your source is a credible one." Surely there must be some register like that, right? I will dash your hopes with a simple answer: No. There is no such thing as an authoritative list of reputable scientific journals. There can't be. And the reason is that word "authoritative". Who is qualified to be the authority? No one is. No one must be. The moment that any one group is anointed with the ability to declare a source to be legitimate or not, is the moment that we lose objectivity and impartiality.

It is very important to be aware that there is any number of bodies who do presume to be such an authority. Approach with extreme skepticism! The only reason anyone would compile such a list is to promote an agenda. Someone once commented on one of my episodes, and they tried to shoot me down by pointing out that one of my sources was discredited on a web site called sourcewatch.org. Sourcewatch.org, sounds pretty legitimate, sounds like they do good work, sounds like they're out there looking out for our best interests by rubber stamping some sources and discrediting others. But according to critics, Sourcewatch is a two-man operation that endorses only publications following their own narrow political bias. This is a perfect example of what you should expect from any source that attempts to identify itself as a rubber stamping authority. Be skeptical of any group you find whose purpose is to identify reputable journals.

As long as we're throwing around the word reputable, I might as well give the somewhat disappointing answer to the listener's question, and tell where you can find a reliable journal. Scientific journals achieve their status only through long histories and good reputations. To be broadly accepted within the mainstream scientific community, a journal must have established a long history of responsible reporting, good quality articles detailing well performed research, and exhaustive peer review. Long standing reputation among the scientists who matter the most. If you're not one of those scientists, it can be difficult to know which research is good, which editors and referees are good, and which journals have a long history of publishing them in good standing. For this reputation to have any meaning, it must stand on its own and not be supported by appearing on some simple list. Unfortunately, listener, you just have to know; but I will give you a starting point in a moment.

While it is essential that good journals be peer reviewed, you should be aware that almost every publication hoping for prominence describes itself as peer reviewed. When you hear someone defend their source by stating that it's peer reviewed, be skeptical. By itself, that's meaningless. Think back to our old example of the guy writing a UFO newsletter in his basement who has a couple of his UFOlogist buddies endorse his writing. Suddenly he's "peer reviewed". This is not the type of peer review that carries any meaning within the mainstream scientific community, since the peers have a clear agenda and have not established long histories of scientific acumen by the legitimate community at large. This is an extreme example, but it does accurately portray a lot of what's out there. When you don't know anything else about a journal, the fact that it calls itself peer reviewed cannot, must not, be allowed to carry any weight.

One source that a lot of laypeople are turning to these days is Wikipedia. What about Wikipedia? It's new and it's a very different animal from anything previously available, and is something of a paradigm shift. Wikipedia is not perfect, but it is generally very good. Its principle weakness, so often pointed out by critics, is also its principle strength. Critics of Wikipedia love to charge that any old Joe Blow can go in there and edit any article to say whatever the heck he wants. And this is true, to a point; but they do have multiple layers of redundant checks and balances in place. Every topic has editors, and every edit eventually makes it past several sets of eyeballs. Every article is read by untold zillions of eyeballs, and tempered with suggested edits by many of them. Most of these suggestions are good, and some of them are bad. The volunteer editors include some of the foremost authorities on the subject, and they include crank nitwits, and everyone in between. Wikipedia has tens of thousands of regular editors, over a thousand administrators who enforce the behavior of those editors (eventually weeding out the crank nitwits), and even a judicial committee which resolves any disputes that are not otherwise handled by the process. The underlying software provides a powerful nerve center from which the editors and administrators can track history and changes. This open source process leads to an inevitable effect: Many Wikipedia articles end up being the closest thing to an authoritative consensus that we have on a given subject. Each article continually improves over time until it becomes what Wikipedia describes as the "ideal" article: "balanced, neutral and encyclopedic, containing notable, verifiable knowledge."

When Wikipedia was first conceived, it was a brand new idea that had never been tried on such a scale. No doubt, it had plenty of growing pains. But they've had years to improve the system. They've been dragged through the media more than once over high-profile errors resulting from vandalism, and every day since inception, they've worked to address those loopholes. The process still isn't perfect, but it's one of the most amazing compendiums in human history.

So, I'm going to give our listener a simple answer to his simple question. Start with Wikipedia, or any other encyclopedic resource with a good reputation like Britannica or MSN's Encarta. Nearly always, good articles will include links to additional references (especially in Wikipedia), but these links are of tremendously varying quality. Be careful of their external links, and carefully note why each external reference is being cited. Good articles will often include a section on criticism or skepticism of the subject. Read it.

Note that I'm no doubt going to be criticized for pointing laypeople toward Wikipedia as a starting point for research, mainly due to the usual criticisms of Wikipedia. But, as I said before, Wikipedia's weakness is also its strength, and I do stand by this recommendation, especially for laypeople of a given subject who don't otherwise have the experience to choose a good starting point.

What about identifying which scientific journals are reliable? Since we're not all scientists in the chosen field with the education and experience to know which are the most reputable publications in our field, we need some kind of list. But, as we've discussed, lists are bad things when they come from a source with an agenda. So we turn again to our source with no agenda, Wikipedia. Search Wikipedia for "List of Scientific Journals" and you'll find that they have a page listing a few hundred reputable journals in most scientific fields. Generally, this is an excellent list. The fact that it comes from Wikipedia, and is constantly being revised for objectivity and quality by experts in each field, is its strongest recommendation. Some fields are not listed, and most subsections are partial. You can drill down to find more. But beware: the further you drill down, the broader the quality control becomes, and the more journals of lower repute are included, and more non-scientific fields are listed. If you use this list to gauge the reliability of a source that you see referenced, you are in as good of hands as are available to the inexperienced journeyman; but you must use the list wisely. Stay at the top level, or as close to it as you can. With each click that you drill down, reputation for the listed journals is generally lower.

$2/mo $5/mo $10/mo One time

Again, this recommendation will no doubt be criticized, and the criticism is generally valid. But I maintain that for the average guy off the street, this is the best way to gain a "good enough" grasp of a journal's quality, and to find good research that truly represents the current scientific consensus.

Follow me on Twitter @BrianDunning.

Brian Dunning

© 2007 Skeptoid Media, Inc. Copyright information

References & Further Reading

Davis, Martha. Scientific Papers and Presentations. Burlington: Academic Press, 2005. 35-50.

Garfield, Eugene. "Citation indexes for science. A new dimension in documentation through association of ideas." International Journal of Epidemiology. 19 Sep. 2006, Volume 35, Number 5: 1123-1127.

Library Staff. "How do I evaluate journals?" College of Staten Island Library. College of Staten Island, 1 Jan. 2009. Web. 12 Nov. 2009. <http://www.library.csi.cuny.edu/faqs/169-evaluatingjournals>

Little, John W., Parker, Roy. "How to Read a Scientific Paper." The Department of Chemistry and Biochemistry Department at The University of Arizona. The University of Arizona, 19 Sep. 2009. Web. 12 Nov. 2009. <http://www.biochem.arizona.edu/classes/bioc568/papers.htm>

SourceWatch Team. "SourceWatch About." SourceWatch. SourceWatch.Org, 10 Feb. 2009. Web. 10 Nov. 2009. <http://www.sourcewatch.org/index.php?title=SourceWatch:About>

Wikipedia Team. "List of scientific journals." Wikipedia.org. Wikipedia.org, 7 Nov. 2009. Web. 12 Nov. 2009. <http://en.wikipedia.org/wiki/List_of_scientific_journals>

Reference this article:
Dunning, B. "How to Identify a "Good" Scientific Journal." Skeptoid Podcast. Skeptoid Media, Inc., 15 Jun 2007. Web. 23 Apr 2014. <http://skeptoid.com/episodes/4050>

Discuss!

10 most recent comments | Show all 20 comments

wikipedia is an encyclopedia. It isnt a journal.

Encyclopedias are not the basis for opinion. They are great to get a short term view.

I note errors in wiki quite a lot. It has a lot of view point rather than dispassionate discussion. It likes to present a "balanced" argument. Balanced argument is not real data.

Henk van der Gaast, Sydney
November 28, 2010 7:55pm

I also agree that Wikipedia is a great source of information, and the closest thing we can get to consensus. Read it with skepticism, but it's more right than the average webiste or "peer-reviewed journal" article.

Yes, USA
July 17, 2011 12:26pm

Thank you for giving a thumbs up to Wikipedia Brian.

Just want to mention that Wikipedia even with all its flaws is where most of the world is going for its information. If the site isn't correct, then correct it. My blog Guerrilla Skepticism on Wikipedia is a "call to editing" for skeptics to improve Wiki. http://guerrillaskepticismonwikipedia.blogspot.com/

Susan Gerbic, Salinas, CA
September 19, 2011 2:20pm

Just remember guys..history and belief is consensus..

Science isnt..nor is real philosophy..

Sadly, what you guys think is science and philosophy is managed by the consensus in wiki.

I am sure that veracity, traceability, measurability, proposition and finally theory is missed in 90% of wiki.

Is it a great encyclopedia? YOU BET IT IS!!!

nobody has done any degree using an encyclopedia...ok diviniy degrees and alt mod...but frankly, they dont count and never have been proactive!

Mud, Sin City, Oz
March 14, 2012 12:05am

Wikipedia is simply a good place to start, as Brian says, not the end source of all knowledge. I use it to develop an understanding of the right words to use to correctly research the subject in question.
I would like to point out that wikipedia is as correct as "phone a friend" and "ask the audience" combined into one.

Tyler, Clinton, MA
January 09, 2013 8:05am

I'd like to highlight that for those who have common rense and forget that when reading an article headed "conspiracy" an encyclopedia is dealing with a "conspiracy" posit.

If Brian got a dollar for every time he deals with a subject in a skeptoid and the same is foisetd back at him from...the wiki posts, he would be at least 100 dollars richer.

It all goes to show that some people arent here for the weeks skeptoid podcast.

Mud, Sin City, Oz
January 31, 2013 2:24am

i like alot of the initial points in this article. i also agree with the conclusion that wiki is one of the most accurate sources of information since it is based on global consensus. that seems to me to be a far purer source than special interest 'studies' we read... or any education designed to breed conformity and soften the truths. but you already explained all that. anyway, i agree... it always frustrates me when teachers want us to use 'reputable' sources for our papers.... as more and more hypocrisy and conspiracy unwinds through the internet, the validity of just about everything seems in conjecture. interesting times we live in... perhaps we will see the downfall of democracy as the fallacies and corruption behind capitalism are slowly exposed. there probably wont be anything better, but its getting to a point where we wont survive capitalism anymore. just look at the growing mass scale food intolerance being designed by the monopoly of monsanto. the chairman for the safety of 'gmos' is even an ex monsanto employee. world is ruined. needs to end. needs to get fixed and megacorps need to pay for atrocities against humanity.

syn, ca
March 14, 2013 8:50pm

Its hard not to be flippant and be labelled cynical when the irony of the above sinks in..

"Blame it on the boogie" I have always maintained. OK I was in britches until I was thirty according to some here.

Magnanamous Dinoflagellate, sin city, Oz
May 22, 2013 7:48pm

Don't know how good this is, but I found where Australia ranked 20,000 journals. http://goo.gl/0Xiul

Larry Anderson, Atlanta, GA
June 16, 2013 7:37am

The Royal Society motto should be in here. It roughly translates as "Take nobody's word for it". It is brief and basically all you need to say.
Wikipedia itself has some very balanced findings on one of its pages, which have evidence going both ways. Its unreliability seems to be word-of-mouth, and I think that a large portion of vandalism on Wikipedia is done purely to demonstrate that you can.

Bill, Canberra
November 02, 2013 1:51am

Make a comment about this episode of Skeptoid (please try to keep it brief & to the point). Anyone can post:

Your Name:
City/Location:
Comment:
characters left. Discuss the issues - personal attacks against other commenters, posts containing advertisements or links to commercial services, nonsense, and other useless posts will be deleted.
Answer 4 + 1 =

You can also discuss this episode in the Skeptoid Forum, hosted by the James Randi Educational Foundation, or join the Skeptalk email discussion list.

What's the most important thing about Skeptoid?

Support Skeptoid
 
Skeptoid host, Brian Dunning
Skeptoid is hosted
and produced by
Brian Dunning


Newest
Your Body's Energy Fields
Skeptoid #411, Apr 22 2014
Read | Listen (12:50)
 
The Black Eyed Kids
Skeptoid #410, Apr 15 2014
Read | Listen (11:18)
 
Oil Pulling
Skeptoid #409, Apr 8 2014
Read | Listen (12:24)
 
Skeptoid Media is a 501(c)(3) Public Charity
Apr 4 2014
Listen (1:13)
 
15 Phreaky Phobias
Skeptoid #408, Apr 1 2014
Read | Listen (12:44)
 
Newest
#1 -
Listener Feedback: Alternative Medicine
Read | Listen
#2 -
The JFK Assassination
Read | Listen
#3 -
Asking the Socratic Questions
Read | Listen
#4 -
5 False Arguments for Raw Milk
Read | Listen
#5 -
The Vanishing Village of Angikuni Lake
Read | Listen
#6 -
The Riddle of the L-8 Blimp
Read | Listen
#7 -
The Secrets of MKULTRA
Read | Listen
#8 -
Who Discovered the New World?
Read | Listen

Recent Comments...

[Valid RSS]

  Skeptoid PodcastSkeptoid on Facebook   Skeptoid on Twitter   Brian Dunning on Google+   Skeptoid RSS

Members Portal

 
 


"Ghost Hunting"
inFact with Brian Dunning


Support Skeptoid