How to Tell a Good Website from a Crap Website

When you find a science article on the web, how do you know whether it's reliable or not?

Filed under General Science, Logic & Persuasion

Skeptoid #336
November 13, 2012
Podcast transcript | Listen | Subscribe
Also available in Turkish
Bookmark and Share

It's been said that searching for information on the Internet is like drinking from a firehose. There is a mind-boggling amount of information published that's freely available to anyone and everyone. The Internet grows so quickly that every time you open your web browser, you've got direct access to the largest compilation of information in history, bigger than all the books in all the libraries in all the world; and at current rates, it's growing by 5% every month. Search for information on any given subject, and you're presented with more options than anyone can know what to do with. So when the average person wants to learn some decent information, how can you tell whether the website you've found is giving you good info, or giving you crap? Today we'll find out.

We're going to look at three categories of tools for appraising the validity of the information presented on a website. First, we're going to go through some general rules of thumb, pertaining to the website's style of presentation, that most laypeople should be able to spot. Next, we're going to look at a handful of software tools designed to give you an objective assessment. And finally, we're going to quickly review the "How to Spot Pseudoscience" guide to give you a pretty darned good idea of any given piece of information you're curious about.

Style of Presentation

There actually is a certain amount of "judging a book by its cover" that makes sense, particularly for websites. Websites can be published by anyone, whether they have a large staff of editors and researchers behind them or not. Big slick presentations are found everywhere, from university websites to science journals to mass media consumer portals promoting who knows what. But there are important differences between a science article and a pseudoscience article, even on the slickest website, that you can learn to spot.

Often the most obvious is the list of references at the end of the article. If there isn't one, then you're probably reading a reporter's interpretation of the research, and should try to click through to find the original. If there are no references at all, then it's a big red flag that what you're reading is unlikely to be legitimate science research. If it's not referenced, pass and move on. A lack of references doesn't mean the article is wrong, it just means that there's a better, more original source out there that you haven't found yet.

If there are references, be aware that oftentimes, cranks will list references as well, so there are some things you need to look out for. Be wary of someone who cites himself. Be especially wary of a study or article that's cited, but once you click on it, you find that it actually says something different than what the author described. It's very important to look at what those citations are: Are they articles in legitimate science journals, or are they published in a journal dedicated to the promotion of something?

Many Google results will return not a page on a slick big-budget website, but on an obscure page. For example, university professors will often have a little website on the university's server, describing their research or whatever. Often, those little websites look terrible, because they're not made by a professional web person. A crank who churns out his own website might superficially look really similar. How do you know whether you're looking at an amateurish site made by a crank, or an amateurish site made by a real science expert?

One way is that real science professionals know that there are ways to establish proper credibility, and they generally follow those rules. The citation of sources is important here as well. A proper research scientists knows that he must list sources to be taken seriously. A crank often skips it, or cites himself, or makes vague references to famous names like Einstein (probably the only names he knows).

Grammatical errors are a case of where it's appropriate to judge a book by its cover. Bad spelling and grammar left uncorrected is a sign that you're probably reading the page of a crank, who works in isolation and has nobody double checking his work. A professor's personal website, however, is often checked over and corrected by undergrads or associates. Do be wary of bad grammar.

So we're dancing around the subject a bit of who is the author. First of all, if the author is anonymous, dismiss the article out of hand. If the author is a reporter, which is often the case, then you need to click through to find the lead researcher's name. If he's a legitimate scientist, he'll have plenty of publications out there, and it's easy to look him up by going to Google Scholar and typing in his name. This doesn't prove anything, but having publications in recognized journals gives the author more credibility than someone who doesn't. Be aware that most indexes like Google Scholar also list crap publications, even mass market paperback books that are not vetted in any way, so you do have to be careful about looking exactly at what those publications are.

If the website teases you with a bit of titillating information but then requires a purchase to get the rest of the story, you could be dealing with a crank sales portal, or you could be dealing with a paywall which is still (unfortunately) common for science journals. Universities almost always have accounts that allow them past the paywalls. You should be able to easily tell whether you're looking at a paywall where researcher credentials can be input to download the full article, or whether you're looking at a sales page trying to pitch you on buying the book to learn "the secrets" or whatever it is. A journal paywall is a good indicator that you're probably looking at real science; the sales page is a good indicator that you're probably looking at crap.

A braggadocious domain name like RealScientificInfo.com or LifeRevolution.biz is just like a used car salesman calling himself Honest John. Websites like that are not typical of the way proper science reporting is done. The website should represent an actual, real-world organization, academic institution, or publication, and not be just some random web compilation.

Software Tools

It would be great if there was such a thing as a web browser plugin or something that would simply give you a red X or a green check to tell a layperson whether a website is reliable or crap. But despite a number of efforts to build just such a thing, no great headway has been made.

One good tool is the Quackometer, which uses an automated algorithm to scan a website's pages, looking at its use of language. It comes back with a score telling you how likely it is that the site is misusing scientific sounding language, and is promoting quackery, or whether it generally appears good. Obviously this is an imperfect solution; but when I've used it on sites that I know, I've found that its results are generally correct, with its biggest flaw being that it often gives a little too good of a score to sites that deserve lambasting.

The Web of Trust is a crowdsourced rating system that gives a trustworthiness score for sites. It's a browser plugin that gives you a little icon next to every link on the page, plus a bigger one for the page you're on, that ranges from green to yellow to red based on ratings given by users. In my experience, it's less useful for gauging the reliability of scientific articles on sites, and more useful for metrics like the site's customer service and security; more for detecting spam than bad reporting.

Rbutr is a browser plugin that lets users link articles that rebut whatever's written on the current page. So, if you're reading something that's been rebutted somewhere, rbutr will link you right to it. The downside is that it cuts both ways: it rebuts a bad article with a good, and rebuts that same good article with the bad. According to someone else. There's not really a way for the end user to know which is better, just that they rebut each other.

Somewhat surprisingly, online trustworthiness services, of which TRUSTe is the best known, allow sites to pay for a privacy certification that they can put on their websites. It turns out that sites who pay for these logos are actually more likely to not be very trustworthy; people with less honorable motives are often more highly motivated to convince you that they are honorable. And, in any case, site privacy has nothing to do with the quality of the site's articles. If you see some sort of a logo or certification on a website, it proves nothing whatsoever by itself. By no means should you assume that it makes the information likely to be good.

The best roundup of tools for assessing the validity of online data is Tim Farley's Skeptical Software Tools. You should keep it as a bookmark, and if anything new comes along for helping laypeople evaluate websites, Tim will be among the first to report on it.

How to Spot Pseudoscience

Skeptoid followers may recognize this list from episode 37, way back long ago. This is an abbreviated version that you can apply to the contents of a website. These common red flags don't prove anything, but they're characteristics of pseudoscience. Watch out any time you see these on a website:

Light BulbAncient knowledge, ancient wisdom, statements that ancient people believed or knew about this, or that it's stood the test of time. To test whether an idea's true, we test whether it's true; we don't ask if ancient people believed it.

Light BulbClaims of suppression by authorities, an old dodge to explain away why you've never heard of this before. The biggest red flag of all is that somebody "Doesn't want you to know" this, or "What doctors won't tell you".

Light BulbAnything that sounds too good to be true probably is. Miraculously easy solutions to complicated problems should always set off your skeptical radar.

Light BulbIs the website dedicated to promotion or sales pertaining to a particular product or claim? If so, you're probably reading a sales brochure disguised as a research report.

Light BulbBe especially aware of websites that cite great, famous, well-known names as their inspiration. Albert Einstein, Nikola Tesla, and Stephen Hawking are three of the most abusively co-opted names in history. Real research instead tends to cite current researchers in the field, names that few people have ever heard of. The famous names are mentioned mainly in sales pitches.

Light BulbAlways watch out for the all-natural fallacy, in its many guises. If a website trumps the qualities of being all-natural, organic, green, sustainable, holistic, or any other of the popular marketing buzzwords of the day, it's more likely that you're reading pseudoscience than science.

$2/mo $5/mo $10/mo One time

Light BulbDoes the article fit in with our understanding of the world? Is it claiming a revolutionary development or idea — free energy, super health — things everyone wants but that don't actually exist? Be skeptical.

Light BulbReal research always cites weaknesses and conflicting evidence, which is always present in science. Pseudoscience tends to dismiss all such evidence. If a website claims that scientists or experts all agree on this new discovery, you're probably reading unscientific nonsense.

Light BulbIn general, the word "revolution" is something of an old joke in science fields, along with the phrases "scientists are baffled" and "what they don't want you to know". If the website promises to revolutionize anything, you're almost certainly dealing with a crank who has little connection with genuine science.

Light BulbAnytime someone puts on their web page that they're smart, or that they are a renowned intellectual or thinker, they're not. Click your way elsewhere.

Light BulbFinally, always run screaming from a website by One Guy with All the Answers. The claim to have solved or explained everything with a new, pioneering theory is virtually certain to be crankery.

So there you have it; it's neither perfect nor comprehensive, but it should give most laypeople a fair start on evaluating a website's quality of information. If nothing else, it shows what a difficult task this is, and highlights yet another reason why so many people believe weird things. Bad information is easy to sell, and not always so easy to spot.

Follow me on Twitter @BrianDunning.

Brian Dunning

© 2012 Skeptoid Media, Inc. Copyright information

References & Further Reading

Cohen, L., Jacobson, T. "Evaluating Web Content." University Libraries. University at Albany, 1 Mar. 2009. Web. 7 Nov. 2012. <http://library.albany.edu/usered/eval/evalweb/>

Farley, T. "Skeptical Software Tools." Skeptical Software Tools. Tim Farley, 25 Sep. 2008. Web. 9 Nov. 2012. <http://www.skeptools.com/>

Lewis, A. "Quackometer." Quackometer. Andy Lewis, 11 Apr. 2006. Web. 10 Nov. 2012. <http://www.quackometer.net>

Matwyshyn, A. Harboring Data: Information Security, Law, and the Corporation. Stanford: Stanford University Press, 2009. 222.

Netcraft. "March 2012 Web Server Survey." Netcraft News. Netcraft LTD, 5 Mar. 2012. Web. 9 Nov. 2012. <http://news.netcraft.com/archives/2012/03/05/march-2012-web-server-survey.html>

User Education Services. "Evaluating Web Sites." University Libraries. University of Maryland, 17 Aug. 2012. Web. 10 Nov. 2012. <http://www.lib.umd.edu/ues/guides/evaluating-web>

Reference this article:
Dunning, B. "How to Tell a Good Website from a Crap Website." Skeptoid Podcast. Skeptoid Media, Inc., 13 Nov 2012. Web. 19 Apr 2014. <http://skeptoid.com/episodes/4336>

Discuss!

10 most recent comments | Show all 42 comments

claims of surpression by authorities???????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????

My god! They surpress a lot of good things all the time!

Yes, you can still be critical, but dismiss it because of this????

He realy thinks science is dair, good, honest etc.

When will he be critical about science?

Never is my guess!!!

Science is 'invented' to dumb down the masses when religion didn't work anymore for this purpose.

O yes and all documented!!!

But well, this will be probably
laughed and scoffed at as a knee -jerk reflex without investigating even a little of it,

Btw I laughed my socks off when I read Randi is moderator on the forum!

Hilarious!

Com'on folks, Be real!

Me, Germany
December 14, 2012 12:27pm

So buzzwords like "green" and "sustainable" should set off alarm bells, as should any publication (web or otherwise) that omits or summarily dismisses all contrary evidence?

Too funny! Kind of like the reports issued ad nauseum by the global warming hoaxers and their "intellectual" muscle at the East Anglican Climate research center and the UN Climate Change Panel...two sources that our fearless leader here at Skeptoid has cited numerous times? Get outta here!

Clark, Indiana
December 19, 2012 3:36am

Rotanev Sualocin, I would argue that notion greatly. Any properly scientific study or paper is going to take extensive time, manpower, and usually finances, to complete.

Attaching your name to the paper(s) regarding your research or analysis means that you are putting your full faith and credibility in that what you have is correct. Likewise, in this field, people who publish papers without their names often know what they have won't stand up to scrutiny, and thus hide behind anonymity so they can continue their work.

Science doesn't care much for "controversial topics" or "unconventional view". It only cares that the material published is done logically and concisely.

Ford, Conor, Poway California
January 07, 2013 3:15pm

But science has review and readers...

You can always send off an email stating your claim about the unreliability of a paper to and ed..

Sadly, to date no skeptoider other than myself has said he has asked for retraction of an article.

What does this mean?

I read a lot of Evidence Based articles that beg dismissal.

Why hasnt anyother critic here in skeptoid done the same?

Authority mayhap? If you do not have the authority to make your own representations to a poor article then you cant make claim....Its churlish making the above claims if you dont actively participate.

Even the great Bart Ehrman will tell you the same.

Mud, At virtually missing point, NSW, OZ,
January 10, 2013 2:07am

I'm missing something here.

You claim you're the one to have asked for a retraction, and use that as proof to the fact that you read alot of questionable articles.

One has nothing to do with the other, and the latter is not proven by the former.

I wouldn't be so bold to say that you're the only one. I'm certain there's plenty of private communication regarding retraction of statements.

Ford, Conor, Poway California
January 11, 2013 8:16pm

CRAP WEBSITE:
http://preventdisease.com/news/09/102809_9_arguments_to_win_any_vaccine_debate.shtml

Government Goodies, Secret Government Lab
January 14, 2013 5:31am

As a full-time sceptic I am suspicious of everything - especially my own mind. When I investigate something I first formulate my question. Then I think - what do I want the answer to be? Better be careful then ... you know the story, you can always find something to support a point of view, but it's good to check out the contrary view - maybe I was wrong in the first place!!!

The most reliable tool is "Follow the Pulpit". There is always a pulpit of some sort. Even in a scientific journal academic work may have been funded by a drug company - who don't often publish their failures. And the researcher must always have an eye on future funding.

You can usually spot the ideological pulpit very easily. The money-based pulpits can be much less obvious. If the work is real research, look at the age and status of the main authors. If they are young then career prospects are likely to dominate. If they are old, then they may be struggling to maintain a previously held position. If they are old, and they have changed their view, then it's definitely worth a look. These authors are less likely to be swayed by future careers, but maybe they are just senile?

Difficult, isn't it?

Of course, it doesn't matter anyway, because the people who make the important executive decisions do not do so on the basis of scientific fact, they make the decisions based on the expediencies of their immediate circumstances. That's how the world really works. Why do we bother?

GordonK, NZ
February 17, 2013 11:17am

Most of this article makes sense, but warning against people who cite themselves seems completely wrongheaded. Most scientists work in similar areas over the course of many decades, and many of them are engaged in a continuous program of research that stretches across numerous individual publications. In order to put each part of the work in context, e.g. to make it clear exactly what is novel about this particular paper compared to the author's own previous work, it is absolutely essential to cite the previous work. Indeed, to do otherwise could be considered scientifically fraudulent, because it would leave the impression that this particular paper was novel in every way, when instead it is only novel in some specific aspects, compared to the author's own previous work. Thus having a self-citation is indicative mainly of whether this work is related to previous work from the same author, not whether it is trustworthy. Having ONLY self-citations, of course, would be a huge red flag.

J. Bednar, Edinburgh, Scotland
April 29, 2013 2:20am

@me,Germany... repeated punctuation, misspellings, all caps and self contradiction are certainly red flags. Please re read your comment and see if any of these items jump out.

dod, ny,ny
May 26, 2013 10:26pm

This is a greta article, but I strongly disagree with your statement: "Somewhat surprisingly, online trustworthiness services, of which TRUSTe is the best known, allow sites to pay for a privacy certification that they can put on their websites. It turns out that sites who pay for these logos are actually more likely to not be very trustworthy; people with less honorable motives are often more highly motivated to convince you that they are honorable."

Security certificates should certainly be used on any site that handle financial data such as credit card numbers, and arguably should be used on any site that accepts any personal data.

Further, major Internet companies are pushing us toward universal secure sites; some APIs from Google and other providers require SSL certificates.

So I'm very skeptical of your assertion that sites displaying SSL logos are "more likely to not be very trustworthy."

Bob Nicholson, Sunnyvale, CA
July 31, 2013 7:38am

Make a comment about this episode of Skeptoid (please try to keep it brief & to the point). Anyone can post:

Your Name:
City/Location:
Comment:
characters left. Discuss the issues - personal attacks against other commenters, posts containing advertisements or links to commercial services, nonsense, and other useless posts will be deleted.
Answer 0 + 4 =

You can also discuss this episode in the Skeptoid Forum, hosted by the James Randi Educational Foundation, or join the Skeptalk email discussion list.

What's the most important thing about Skeptoid?

Support Skeptoid
 
Skeptoid host, Brian Dunning
Skeptoid is hosted
and produced by
Brian Dunning


Newest
The Black Eyed Kids
Skeptoid #410, Apr 15 2014
Read | Listen (11:18)
 
Oil Pulling
Skeptoid #409, Apr 8 2014
Read | Listen (12:24)
 
Skeptoid Media is a 501(c)(3) Public Charity
Apr 4 2014
Listen (1:13)
 
15 Phreaky Phobias
Skeptoid #408, Apr 1 2014
Read | Listen (12:44)
 
The Death of Mad King Ludwig
Skeptoid #407, Mar 25 2014
Read | Listen (11:49)
 
Newest
#1 -
Listener Feedback: Alternative Medicine
Read | Listen
#2 -
The JFK Assassination
Read | Listen
#3 -
Asking the Socratic Questions
Read | Listen
#4 -
5 False Arguments for Raw Milk
Read | Listen
#5 -
The Vanishing Village of Angikuni Lake
Read | Listen
#6 -
The Riddle of the L-8 Blimp
Read | Listen
#7 -
The Secrets of MKULTRA
Read | Listen
#8 -
Who Discovered the New World?
Read | Listen

Recent Comments...

[Valid RSS]

  Skeptoid PodcastSkeptoid on Facebook   Skeptoid on Twitter   Brian Dunning on Google+   Skeptoid RSS

Members Portal

 
 


"Are We Alone?"
inFact with Brian Dunning


Support Skeptoid