What does a good Skeptics.SE answer look like? What are the qualities that a perfect Skeptics.SE answer would have?

Ground rules:

  • One quality per answer
  • Provide a brief explanation of why it is important.
  • Use the comments for discussion.
  • If you don't agree, downvote the attribute. If you agree, upvote.

Related Question: What are the attributes of a good question?

share

9 Answers 9

It must actually answer the question.

You might want to talk about a slightly different topic related to the original question - see for example here:

Answers providing tangential/related information but not an actual answer?

Please don't.

Simply put, comments belong in the comment section. If you can't answer the question, but you have something important to add, don't post it as an answer. It'll decrease the visibility of the actual answers. Instead, post it as a comment. If it's long, break it into a few comments.

share
    
How about "It must actually address the question."? Sometimes an answer explains why the question has a false premise or can't be answered. –  Oddthinking Jun 20 '11 at 7:03
1  
@Oddthinking - I agree that some questions are illogical, in which case saying so might be the only answer (though the place to say that might be in a comment and not as an answer). I don't prefer "address" instead of "answer", though. For example, if you give a talk titled "Towards a theory of [whatever]" your audience might guess that you're not going to actually get there. Similarly, to "address" a question is less direct and thorough than to actually answer it or cover it: I'd worry that providing "tangential/related information" might be seen as "addressing" it. –  ChrisW Jun 20 '11 at 12:33
    
Well, that's a given in this context, so it doesn't even have to be said. But: isn't up/downvoting enough for that? Isn't it built into the system already? Does this site really need it in the FAQ? Let's turn this around: do you really expect people to ask "Do I have to actually answer the question?". And *frequently? –  Jürgen A. Erhard Jun 30 '11 at 15:46
    
@jae I expect that sometimes someone will provide an answer that's in some way (e.g. in this way) 'wrong' and that it's then convenient to be able to correct them with a link to a FAQ entry. Here is a whole small discussion on this topic: so it happens or happened enough to be noticeable. It may also be useful to people who read the entire the FAQ in order to know 'everything' (about the rules and expectations). –  ChrisW Jun 30 '11 at 15:52

It must be factual, not based in speculation.

The primary purpose of Skeptics is to verify or falsify claims. If an answer an be summed to "I don't know for sure, but here is what I think the answer is," then it is a bad answer as it unhelpful - it neither verifies nor falsifies anything.

Speculation is not completely bad. For example, an answer that shows, without speculation, that a claim is false, and then provides some speculation as to how the claim might have sprung from a misunderstanding is certainly a good answer.

share

It must be written in an accessible vocabulary.

Most Stack Exchange sites are intended for expert-to-expert exchange. Skeptics is different in that most users will be laymen in the topic discussed, be it biology, physics or psychology. For this reason, an answer should always be written in a plain language void of overly technical terms so that it might be understood my the average user.

share
    
What about references? Is it helpful if the answer is phrased in simple English, with a simple (if possible, yes, I see your "should be written") explanation... if the reference is (necessarily, let's assume) total gibberish to "the average user"? I see a potential conflict/contradiction here. –  Jürgen A. Erhard Jun 30 '11 at 15:42

It must be written in a polite and neutral tone.

We expect users to be respectful of others they disagree with, no matter how frustrated they may be. Rants about young earth creationists, "climate change denialists," etc. will not be tolerated here. If we want to introduce people to the evidence, we have to be nice. Otherwise, we'll drive them away before we even got a chance to educate them.

share

Every answer must have one or more references.

At the present time, we have a strict "no original research" policy. Simply put, hearsay and anecdotes are not enough.

The significant claims in an answer, especially the claim which most directly answers the question, must be backed up by a reference. References should have credibility in the domain (i.e. no encyclopedia, no source which may be biased, etc.) and should preferably be peer-reviewed literature.

share
    
Your answer is contradictory. All ... must ..., but later not all claims have to. Either - or. –  user unknown Jun 19 '11 at 0:24
    
@userunknown: I said "all significant claims." Not all claims are significant. –  Borror0 Jun 19 '11 at 0:52
    
Yes, but you continue with answers on high school level. –  user unknown Jun 19 '11 at 2:29
    
@userunknown: If you see a way to make this clearer, please go ahead and edit. –  Borror0 Jun 19 '11 at 8:08
    
You can write 'Significant claims should have references', or delete the last paragraph. It's your answer, and in it's current form unclear. –  user unknown Jun 19 '11 at 10:50
    
@userunknown: It's also Community Wiki. To be clear, I agree with you. I'm just not sure of how to make my post clearer at the moment. –  Borror0 Jun 19 '11 at 10:53
1  
@user I take it the intent is that a claim based on high-school science knowledge is not 'significant'; though it may be crucial to the answer, it is not 'extraordinary'; nor worthy of scepticism; it's a common-place, ordinary, mundane, not significant: and therefore needn't be referenced. –  ChrisW Jun 20 '11 at 2:31
    
Is every answer expected to include at least one significant claim, though? Would that claim need to be referenced even if it's high-school science? In other words, does every answer need references? –  ChrisW Jun 20 '11 at 12:18
    
@ChrisW: No. When I have had such answers, I have explicitly referenced vague items like "High school chemistry", or similar, to avoid reflex complaints about lack of references. –  Oddthinking Jun 28 '11 at 16:26
    
@Oddthinking: In fact the answer is yes: every answer without exception must include a reference. So I suppose that even answers like this one shouldn't be taken as a guide, or as permission. I'm noticing that whenever I think I can answer a question using high-school science, I might as well just make it into some kind of comment instead of an answer. –  ChrisW Jun 29 '11 at 1:41
    
@odd I agree with @chr here. If an answer's only claims are high-school level, then they should be referenced. However, let's be reasonable. To support, e.g., conservation of energy, I think it is sufficient to link to Wikipedia or any mainstream physics textbook. In other words, the standards for what is authoritative clearly depend on the claim being supported. "Extraordinary claims..." –  Sklivvz Jun 29 '11 at 6:56
    
@Sklivvz, all right then. I will get my old Chemistry texts out ready. @Borror0, worth adding this distinction to the question?: Not just all significant claims, but also, at least, the most significant claims. –  Oddthinking Jun 29 '11 at 7:03
1  
@Oddthinking - I have edited Borror0's answer. –  ChrisW Jun 29 '11 at 13:08

Its references must support the argument, and should be verifiable.

A high-quality reference should have, at least, the following characteristics:

  • Comes from a peer-reviewed source.
  • Is either a primary source, or it, in turn, cites its sources, so the primary sources can be tracked down.
  • Supports the argument being provided in the answer (i.e. not out of context)
  • Draws logical and statistically robust conclusions from any premises or data it offers.

Ideally:

  • The reference is widely available for others to inspect (e.g. not behind a paywall, or out-of-print)
  • The reference is not contradicted by similar articles supporting the opposite claim.

The reference need not be a web-site - it can be printed articles, books, documentaries, etc.

The reference need not come from the list of useful sources, but these are widely-respected sources.

Not all references are going to be able to meet this quality bar, but your arguments will be more convincing, and are more likely to receive votes from fellow skeptics when they do.

[This answer is taken from a meta-question on the topic of good references]

share
    
Perhaps a less subjective and more functional couple of adjectives than 'high-quality' in the title: "acceptable and informative"? "peer reviewed"? "properly constructed"? "suitably referenced"? –  ChrisW Jun 20 '11 at 2:23
    
I'm pondering that one @Chris. I don't want to say "peer reviewed" / "properly constructed"/ "suitably referenced" because they are only part of the picture. "Acceptable" seems to have the same problem as "quality". What is meant by "high quality" is defined lower down in a less subjective and more functional way, so I feel it gets there in the end. So, I think it is fine as it stands, but I am open to an even better suggestion. –  Oddthinking Jun 20 '11 at 4:42
1  
@Oddthinking Well imo you also shouldn't want to say "high-quality" because that's not saying anything (it's uninformative, not prescriptive), and the title is quite the important place to have said something. Try putting the most important thing or two in the title; you can expand on it in the text. Alternatively, "What are the attributes of a good reference?" –  ChrisW Jun 20 '11 at 12:15
    
@ChrisW, I found your argument persuasive. Are you happy with the change? –  Oddthinking Jun 20 '11 at 13:02
    
It should be edited to be shorter: "Its references must support the argument and stand up to scrutiny". And saying "address the question" instead might be more primary than "support the argument". "Stand up to scrutiny" is a difficult phrase (simpler vocabulary might be more 'accessible'), but I can't think of an equally short alternative: perhaps "satisfy sceptical readers who want to learn more"? But anyway, yes: that change was for the better. –  ChrisW Jun 20 '11 at 13:45
    
"Verifiable": that's a good word. "Its references should answer the question and be verifiable." –  ChrisW Jun 20 '11 at 14:07
    
Actually, maybe just "Its references should be peer reviewed and verifiable." I think that 'covering the question', 'supporting the argument', etc., are a different attribute (already written in other answers). –  ChrisW Jun 20 '11 at 14:12
    
You should include from a source who has a certain authority on the matter. –  ChrisW Jun 20 '11 at 14:22
    
@ChrisW. Agreed that "verifiable" is better than the scrutiny line. Updated. Thanks. Supporting the argument is a different attribute that answering the question. (My reference may support my claim of what weapons a ninja might carry, without addressing the question "Who would win between a pirate and a ninja?") I'm not a fan of the "authority" phrase - I couldn't define what it meant without tripping over "Appeal to authority" fallacies. –  Oddthinking Jun 28 '11 at 16:23
    
Can you add 'should' and/or 'must' to each of the two clauses in the title, in order to match all the other answers in this topic, and to match the canonical form of a specification? –  ChrisW Jun 29 '11 at 13:42
    
@ChrisW: Done. Feel free to make your own copy-edits. Be bold! –  Oddthinking Jun 29 '11 at 13:51
1  
I added 'should': you'll complain about or down-vote unverifiable references, but not delete them. –  ChrisW Jun 29 '11 at 14:06

References include the following items:

  • a (permanent) reference (e.g. the book title and author, or the article title and date and journal name and volume, etc.)
  • a web link, where possible
  • a pointer to the page or section
  • a quote of the relevant text, to protect against link-rot and to aid the reader who doesn't want to search further
  • an indication of your opinion on the reliability of the source, and why.

This answer is based on an answer from What constitutes a reference?

share
    
As a special case, Wikipedia links should include a version. –  Oddthinking Jun 18 '11 at 11:58

It should summarize the whole truth.

Some topics have thousands of research papers, published world-wide: for example questions related to cancer; tobacco use; the safety of foodstuffs; etc.

With thousands of papers published, it would be easy (but wrong) to select only a few which show whatever you want, and especially where the conclusion contradicts normal findings from a majority of other studies.

For complex topics, where studies disagree, a good answer should show a review of the literature and not just cite a single study (appropriate references might be 'survey' studies, which summarise the conclusions of other studies).

It's not that the majority opinion is always necessarily correct, but an answer should properly cover the topic (which should include at least an acknowledgement of majority opinions).

share
2  
How about phrasing it in the following manner: "For complex topics, where many studies disagree, a good answer should show should give a review of the literature and not just cite a single study." It's less about the majority, and more about properly covering the topic. –  Borror0 Jun 19 '11 at 8:24

It must explain the methodology of the studies it cites.

Citing studies is not enough. While a study may prove that X has a correlation with Y, the methodology is important - especially when it comes to social sciences where different definition of terms like "happiness" or generosity" would change the answer drastically.

share
1  
Shouldn't cited studies explain their own methodologies? Are you asking that the methodology be transcribed/summarized into the answer? Also, the famous answer about torture doesn't cite any methodologies (instead its references are all "appeals to authority") ... is it a bad answer then? I think not. Therefore there must something wrong with this, and/or this isn't invariably applicable. –  ChrisW Jun 18 '11 at 1:55
    
@ChrisW: The best example I have of the importance of explaining the methodology is Nature's comparison of Wikipedia and Britannica. It puts factual errors and omissions at the same level. Most errors on Wikipedia were factual errors; most errors on Britannica were omissions, things that the experts feel should have been mentioned. Factual errors are obviously graver mistakes, since it is misinformation. If you just cite the conclusion, this information isn't mentioned. –  Borror0 Jun 18 '11 at 2:00
1  
I'm not enamored by this rule. As @ChrisW says, the study itself should explain its methodology. I'll sometimes say in my answer "randomly-controlled" or "survey", for example, to give a very broad indication of how much it should be trusted, but this risks our answers becoming as unreadable to the lay public as abstracts. –  Oddthinking Jun 18 '11 at 11:02
    
@Oddthinking: I don't think so. You don't need to be overly technical - in fact, you shouldn't be - but it's usually relatively easy to sum up what they did. Citing a conclusion without describing the methodology is a major sin, IMO. It allows to misrepresent a study very easily. Even involuntarily. –  Borror0 Jun 19 '11 at 8:21
    
@Oddthinking, @Borror0: Given the down-votes/disagreements which this answer received, I'm going to try editing its title from 'must' to 'may'. –  ChrisW Jun 29 '11 at 13:45

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .