There are a few answers popping up regularly based on «logic».

The problem is obviously not the logic part, it is that the logic depends on uncited facts. Using logic to follow the consequences of cited facts is perfectly acceptable and necessary.

By this I mean answers that:

  • Have little or no sensible references
  • When prodded, the author typically replies with "I can't reference anything because it's just «logic»"

Whereas it is always, always the case that the author is just speculating (and not using «logic» in the context of facts).

In practice these answers are based on speculation, circular logic and no-true-Scotsman fallacies (and not pure «logic»):

Example:
Q: Will you get shocked by lightning in a car?
A: You are isolated, therefore no.

This kind of answer may be OK on other SE sites, but not here. Here's what's wrong with the answer.

  • If you are isolated, then you are safe. This is completely obvious and we have no problem with it.
  • But are you isolated? In theory yes, because of the wheels, but does this effectively work in practice? Has anyone actually tried it and published the results? The answer is not even addressing the question in this respect.

The site must get rid of these answers because they are:

  • low quality and therefore bad for the site in general and bad for the OP
  • mere opinion/speculation and therefore tend to spark useless discussions ("bad subjective")
  • not real answers to the question
  • broken windows, because they are examples of speculative claims being repeated
  • very unlikely to be fixed by anyone because fixing them is equivalent to re-writing new answers (and therefore anyone would probably write a correct separate answer instead)

In many cases it's sufficient to leave a comment, and the author will fix or delete the answers. In some other cases the author refuses to help because they are used to other SE where this style of answers is tolerated.

In most cases, moderators and high rep users are being dragged in pointless discussions in the comments.

What should we do to avoid these problematic bad answers?

Please vote one or more of the answers below (or add your own solution).

** Update:** This has now been folded into the guidelines here. Keep voting if you want to change/confirm this policy.

share
4  
The problem is not the logic part, it is that the logic depends on uncited facts. Using logic to follow the consequences of cited facts is perfectly acceptable, but in nearly all cases a pure logic answer can't work because the facts it needs are not universally accepted. Pure logic could only work for questions where the claim itself is internally contradictory, but we haven't had such a question yet. But the problem is not that those answers use logic, it is that they don't cite the claims the logic is based on. –  Fabian Jul 21 '11 at 8:58
    
Totally agreed, that's why I used quotes. The authors use the word «logic» to justify their answers. –  Sklivvz Jul 21 '11 at 8:59
1  
Maybe post a few example questions, unless that's seen as unkind to the posts that might be negatively highlighted? –  Hendy Jul 21 '11 at 15:18
    
@hen it's probably not a good idea as we shouldn't be discussing the examples, but the policy? –  Sklivvz Jul 21 '11 at 15:39
    
Anyways, this question has got three "logic" answers: skeptics.stackexchange.com/questions/4912/… –  Sklivvz Jul 21 '11 at 16:02
1  
@Slivvz: agreed that the question is the policy... but I just hadn't seen anything like this before and it helps me get what's going on if I can see quotes/specific examples. Thanks for the reference. –  Hendy Jul 21 '11 at 16:48
    
@hen: a fresh one from the main site. skeptics.stackexchange.com/questions/2795/… –  Sklivvz Jul 21 '11 at 17:28
    
@Slivvz: got it. Yes, that's frustrating! –  Hendy Jul 21 '11 at 17:45
    
Related issue on Programmers.SE: Do significant claims require evidence? Hopefully you guys can save yourself at least some of the prolonged discussion we've had over the past few days about it. :) –  user1755 Jul 21 '11 at 20:54
1  
@mar: note that we already settled that here. all answers require references for claims. this question is more about how to handle broken windows. –  Sklivvz Jul 21 '11 at 21:29
    
@Sklivvz It's actually settled across the SE network (your point that it's because people are coming from SE sites that tolerate "logic" or common-knowledge based answers is incorrect); the issue, and why it'll be especially hard to avoid them, is that people, especially those not used to Stack Exchange but even high-rep people who refuse to accept the "back it up" principle, believe there's value in repeating common knowledge. That's the purpose of me linking to the P.SE discussion: to save yourself the trouble of trying to figure out a way to avoid those answers. –  user1755 Jul 21 '11 at 21:35
1  
@mar I disagree on this. The vast majority of answers on SE sites are not backed up by references. E.g. the first question on programmers atm 4 answers, zero references, zero challenges to add references. –  Sklivvz Jul 22 '11 at 10:47
    
See also here etc. etc. –  Sklivvz Jul 22 '11 at 10:49
    
possible duplicate of Must all answers be referenced? –  user unknown Jul 27 '11 at 2:27
    
@user: this is different, it's what to do with unreferenced answers, given that the previous policy is in place. –  Sklivvz Jul 28 '11 at 8:44

4 Answers 4

up vote 4 down vote accepted

Bad answers should be downvoted and commented upon. If the poster refuses to fix, or ignores the comments, a "citation-needed" banner should be added to the answer, to invite others to downvote or edit the question.

share
    
This is the bare minimum that can be acceptable on a skeptics site. –  Sklivvz Jul 23 '11 at 10:19

If possible, an answer should have citation for the facts, it relies on.

But not every answer without citation is a bad answer. Possible reasons:

  • you can't find a study and nobody else does
  • the question handles about a field, where no scientist works on - most esoteric claims talk about such stupid ideas, no scientist gets interested
  • the answer can be given by facts which are common knowledge, and taught at school

If the person, which asks the question, isn't allowed to freely choose a question, but only, if he knows, there is a answer which can use citation, the person doesn't need to ask, but can get the answer directly.

As long as there isn't an answer with citation, we don't know whether there will be an answer tomorrow, next week, next month, at all.

Citing a study can become a fetisch. Answers which don't really fit the question but have citation are upvoted blindly. How valid is the source? We shall not perform original research, but judge publications - by what?

You need common knowledge and logic to decide, whether a citation is appropriate, and whether the research is up to date, and fits to the question. But there is no final answer. Citation isn'T the silver bullet.

If somebody has a sounding explanation, that is more worth than a citation from a paper, which nobody understands. Yes, he might be wrong, so show it, and if you have, use citation, it's fine. But don't make it a dogma.

share
1  
I think this site is not about who is right or about attempting to get to the absolute truth of anything. This site is about trying to get there through a particular method, which involves citations, experiments and facts. If a particular question cannot be address in this way, it's by definition a bad question for this site. –  Sklivvz Jul 27 '11 at 6:13
    
If the person already knows, how a question is addressed, it needn't ask the question. Most people don't ask wrestling questions, where they already know the answer. –  user unknown Jul 28 '11 at 12:18
    
I was thinking the same when the site has started, but I have changed my mind since. Even for a "common school knowledge" it is possible to provide references (to a schoolbook if necessary). Not all references need to be to peer to peer studies. Ordinary claims do not require extraordinary references. See also meta.skeptics.stackexchange.com/questions/80/… –  Suma Aug 2 '11 at 9:25
1  
"Answers which don't really fit the question but have citation are upvoted blindly. How valid is the source? We shall not perform original research, but judge publications - by what?" I absolutely agree here. I think I remember this was already discussed before (with no clear result), but I have failed to find where. –  Suma Aug 2 '11 at 9:33

Your premise that it is always, always the case that the author is just speculating (and not using «logic» in the context of facts) can easily be shown to be false by a single counter-example, due to your ill-advised use of the word always.

That's a simple application of DeMorgan's Theorem, which IS pure logic.

Because of this flawed premise, your whole argument is unsound, if even a single counter-example can be found.

Source: http://en.wikipedia.org/wiki/Soundness


Here is such a counter-example: Security in open source vs. closed source software I'll summarize here:

The original post may be succinctly summarized as:

Our teacher won't allow us to use the open source FreePascal compiler. He argues that Anyone can modify open source software, therefore open source software is insecure and FreePascal is open source, therefore it is insecure. Is this right?

My answer may be succinctly summarized as

The teacher's argument is unsound. He makes a statistical claim about open source software, and then derives a conclusion about a particular piece of software. This is a common fallacy ('Division').

In addition, I linked sources to show that the claim specifically does not apply to FreePascal, but this is not germane to the fact that the teacher's argument is unsound.

share
    
I don't understand: you did use sources for free pascal, so how is that a counterexample for this meta-question? Secondly my complaint in regarding the original question, was due to the fact that the claim is about open source in general, but your reasoning, and your sources, only apply to a particular case. You are overgeneralizing. You have shown that FP is secure, not that F/OSS is secure. You can't use logic to extend a single measurement to a whole class of phenomena. –  Sklivvz Aug 17 '11 at 22:44
    
I don't believe that is a fair summary of the teacher's position. I think it is more "Anyone can modify open source software, therefore open source software cannot be trusted to be secure. FreePascal is open source, therefore it cannot be trusted to be secure." I don't believe he or she is claiming FreePascal has malicious code in it, merely that it cannot be trusted not to have malicious code. (I do think the teacher is being awfully short-sighted.) –  Oddthinking Aug 18 '11 at 9:20
1  
@Sklivvz: You're right that a single example doesn't prove anything about a class of phenomenon. However, a single example is sufficient to disprove a universal claim. My answer provides sufficient sources to disprove the teacher's claim that anyone can insert malicious logic into any open source application. And my logic argument shows that if the teacher is making a weaker claim (about most F/OSS, instead of all) then it wouldn't justify exclusion of FreePascal. –  Ben Voigt Aug 18 '11 at 12:15
    
@Oddthinking: I think I and the teacher agree that a software whose behavior can silently become malicious may be said to be insecure, not just potentially insecure. Given that, do you still disagree with the summary? –  Ben Voigt Aug 18 '11 at 12:16
    
@Sklivvz: If the cited answer isn't applicable to this discussion on meta, why did you leave a link to it, alongside very negative comments threatening to delete my answer, as a comment on exactly that answer? –  Ben Voigt Aug 18 '11 at 12:18
    
Ben, I am happy to define "insecure" that way. In that case, I cannot see the fallacy of division. His or her (false) premise is that all open source software might already contain malicious code and is therefore, by definition, insecure. (That's not a statistical claim; it applies to all members.) FreePascal is a member of the set of open source software. Therefore FreePascal is insecure. –  Oddthinking Aug 18 '11 at 12:43
    
@Oddthinking: My answer covered both variants of the argument. The "all" case is shown to be unsound because I provided sources to show that FreePascal can't be edited by just anyone, disproving the premise. The "majority" case is shown to be unsound by logic, recognizing the fallacy of division. The fallacy of division applies to the subsequent question, and bulk of the answers, which deal with statistics on the F/OSS population as a whole and not with the particular product. –  Ben Voigt Aug 18 '11 at 15:00
    
@Oddthinking: The issue is that Sklivvz has called by answer "bad" because I didn't disprove the statistical premise. But we both agree that the statistical version is unsound (it employs a division fallacy), so I only need evidence to disprove the stronger premise which applies to ALL F/OSS, as you've stated it. And a single counter-example is sufficient to do so, again I need not provide any survey on the general population. –  Ben Voigt Aug 18 '11 at 15:02
    
I'm sorry - you are the only person I can see who is talking about "statistical" claims. Everyone else seems to be sticking "for all" types of claim. Disproving a statistical claim that no-one made isn't advancing your argument. Time to take this to chat. –  Oddthinking Aug 18 '11 at 15:11
    
@Oddthinking: If the downvoters leaving comments like "The question is if an OSS model would produce more secure software than a closed model" aren't talking about a statistical argument (use of the word "more" seems to mean "a greater fraction" here), then I'm not sure what they mean. –  Ben Voigt Aug 18 '11 at 15:13
    
The argument is similar to: "my dad doesn't want to use an italian-plates care because he says it's unsafe. Is it true that italian-plates cars are less safe then german-plates cars (because german-plates cars have regular MOTs)?". The answer cannot be "your dad is wrong because the car he chose has all MOTs done even if italian". The answer should be, "statistically, it has been shown [link] that in practice italian cars are just as safe, etc etc". The question is completely statistical. You cannot argue statistically by examining one data point. –  Sklivvz Aug 19 '11 at 12:54
    
@Oddthinking: Sklivvz's latest comment very clearly states that he IS thinking of a statistical argument. –  Ben Voigt Aug 19 '11 at 12:59

Bad answers should be deleted by moderators (after fair warning).

The answer can still be edited by original author and when fixed, they can flag it to have it reinstated.


Note: while this may seem harsh, I think it's actually the way to go. We are a community of Skeptics, and we should only allow answers that are based on factual evidence. Any other answer is not a skeptical answer. It goes against what we stand for.

share
    
If one of the fair warning answers gets up, we should have another meta question on how long that should be. I am acutely aware that I act faster on answers where the author has seen the complaint and responded but declined to fix it, than answers where the author hasn't responded (and I am forced to assume they haven't been back.) That always struck me as a little unfair. –  Oddthinking Jul 22 '11 at 11:31
    
I agree with this. Having a standard amount of time before it's delete, say 48 hours or something, is both fair and stops bad answers from lingering too long. –  TheEnigmaMachine Jul 25 '11 at 21:28
    
Citation is not proof. You are merely replacing one form of belief with another. Even the special meaning of the word proof comes from Euclid's the Elements, which was entirely based on logic. –  user2767 Jun 20 '12 at 8:41
    
@EdwinBuck Funny that you mention Euclid, since he wasn't really concerned about the real world, but merely an abstraction of it. In the real world, doing an experiment, and peer-review counts as proof. This is the basic postulate of scientific skepticism. Feel free to disagree, however the basic premise of the site is not up for discussion. –  Sklivvz Jun 20 '12 at 8:49
    
@Skilwz Funny how you managed to delete the only non-yours answer to which is the valid interpretation of Sandro Del-Prete's picture. Remember, the appearance of improprietary behaviour is just as damaging as improprietary behaviour. By the way, your post was so much better with a wikipedia link to the author (which had nothing to do with image interpretation), and a second link were we could all buy a print. –  user2767 Jun 20 '12 at 9:00
    
@EdwinBuck deleted answers can be undeleted by the community (high reps) or other mods. All my actions are subject to the review of the community. Please remember to be nice here. –  Sklivvz Jun 20 '12 at 9:21
    
@Skilwz While I am being nice, please take it in the best possible way that your actions, while noble, can carry an air of hypocritisim in the eyes of others. You use the best logical reasoning to denounce logical arguments. The error is never that logic is used, the error is that logic is used without foundation in the concrete world; however, for the post you deleted, it is an argument about interpretation an item which is not possible to found in the concrete world. Demanding citation on an interpretation, or on common knowledge (interpretation is personal) is myopic to the extreme. –  user2767 Jun 20 '12 at 9:33
    
@EdwinBuck while I understand your point, your answer makes unreferenced claims, e.g. "The interpretation of possible meaning behind selecting one pattern over another is entirely sociological." -- on this site, you either back this kind of statements up with research, or you remove them. You have been given almost 10 months to fix your answer. –  Sklivvz Jun 20 '12 at 9:38
    
"Based" on factual evidence is a funny thing. If I start with true facts and then make leaps than require reasoning than I can end up with a false claims. Here's an answer with 12 votes that goes against the current scientific consensus on the question. | There are a lot of answers on the website that cite studies which aren't meta studies. If I just cite individual studies, I have no problem to show that homeopathy works. –  Christian Jun 21 '12 at 11:49
    
@Sklivvz: At the moment there only one non-moderator with high enough reputation to see deleted answers. To make community review of those decisions workable we would need more people with access to moderator tools. Halving the required reputation would help. –  Christian Jun 21 '12 at 11:53
    
@Christian the two questions you posted are different (consumption vs effect of level of glucose). I see no contradiction. Regarding the access to deleted answers, we've tried to argue for that in the past with little success (see meta). –  Sklivvz Jun 21 '12 at 12:03
    
@Christian: also, there are 2 non-mod users capable of seeing them (OddThinking and Oliver_C) and even halving the rep requirement will not allow the next non-mod user to see them (who is MonkeyTuesday at 9k). 2 users are enough to undo the deletion, I think. –  Sklivvz Jun 21 '12 at 12:07
1  
@Sklivvz: Seeing deleted questions comes with the mod tools at 10k reputation. Undeleting comes at 20k. When only 6 people can see the deleted post I don't think it's an accurate description to say that "the community" makes that decision. - The meta discussion around reputation limit was made when the reputation limits were lower because the website was in beta. Restarting the discussion at a time where the current limits where in place was blocked. –  Christian Jun 21 '12 at 12:21
    
As explained here, I tried to flag for undeletion but was refused by the website. –  Cees Timmerman Sep 26 '12 at 15:14
    
That's not what you say there. You can't undelete your answer unless you are the deleter. You can certainly flag your deleted answer. –  Sklivvz Sep 26 '12 at 17:10

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .