-1

Regarding this question, Was Shakespeare Marlowe? What does stylometry say? , I asked it because I noticed from a review of old stylometries (done by Peter Farey) that unlike most conspiracy theories, the idea that Marlowe wrote Shakespeare is strongly supported by the only scientific method to distinguish authors reliably, which is stylometry.

I asked about it here to test this website's scientific honesty--- I know that the stylometries can't distinguish Marlowe from Shakespeare, but I also know that this is politically impossible to say, because the mainstream literary crowd can't accept this.

To my surprise, I got an answer that gave a paper which claimed to distinguish Marlowe from Shakespeare, and I actually believed that I was wrong (for a few hours), that Shakespeare was actually stylometrically different from Marlowe. Then I read the paper.

The paper's conclusions nonewithstanding, the methods and results of the comparison give such stunningly strong evidence that Marlowe wrote Shakespeare that I am now dead certain that they are the same. These guys' program identified all of Marlowe's work by function words and vocabulary as Shakespeare with the exception of Tambourlaine and the Massacre at Paris! Further, the only reason Tambourlaine wasn't identified is that they compared Tambourlaine I and Tambourlaine II.

Their method is made to look less accurate than it is, by two other failures that they include in the analysis. The two other plays that they misidentify as by a different author than the canonical author are both of long-disputed authorship, and I am sure that their program gets it right--- they are not by the author that they are traditionally attributed to. A quick search will reveal that there is no real historical evidence for the attribution of these plays, which were both published anonymously, and their author was guessed by a process of elimination.

So their method doesn't fail on any plays, and it only fails to distinguish Marlowe from Shakespeare, this is true of Mendenhall's letter-distribution stylometry, and, as others have noted, of every stylometry that claims to distinguish Marlowe from Shakespeare. Considering how much time has been spent looking for a definitive stylometric separation between these two authors, the failure is ridiculous.

Looking at the stylometries a year or two ago, I became a Marlovian, meaning I thought it was likely that Marlowe wrote Shakespeare. I didn't care about the historical evidence, or about the conspiracy aspects (although these are interesting, and the ideas are supportable by historical evidence, these sorts of things can't really be supported by any quantifiable scientific evidence). My main focus was on the stylometry.

I use this question as a test of this site--- in those handful of cases where the science and the dogma conflict, can you actually write the conclusion that the science supports, even when this is a fringe opinion in the relevant (nonscience) field? This question is a perfect test case.

What do you do when the science and dogma conflict? If this website was around in 1500, could it support Copernicus? Is it original research to analyze the failure of stylometry to separate Marlowe and Shakespeare?

This is important for the usefulness of this site. I was told I would be banned from this site for rudeness, as is usual for impolitic characters like myself, so I might not be around for much longer. But I hope this site can be useful.

3 Answers 3

8

I asked about it here to test this website's scientific honesty--- I know that the stylometries can't distinguish Marlowe from Shakespeare, but I also know that this is politically impossible to say, because the mainstream literary crowd can't accept this.

So your “knowledge” flies in the face of the scientific opinion of the experts in the field. Which expertise can you demonstrate to offset this?

Now, I’m the first to agree that the particular paper cited has at best a sloppy analysis. And, furthermore, you don’t need to be a recognised authority in a field to offer an informed opinion. On the contrary.

In fact, it was a good thing that you read the paper, and criticised it here (although I’m unsure what format would have been appropriate). And yes, the currently highest voted answer on this question is simply misleading. I’ll go even further: you’ve caused me considerable doubt in this question: without being involved in the debate I’ll grant you that the cited paper probably doesn’t support the hypothesis that Shakespeare and Marlowe were different people.

But it’s disingenuous to pretend that stylometry is so trivial that anybody without an education in the field can overthrow it. Thinking the field is that trivial simply showcases the Dunning–Kruger effect in action. The same sentiment fuels large part of the Intelligent Design movement: people who are convinced that they’ve figured it out without having to read a single biology text book.

If you happen to have expertise in stylometry, by all means, publish a paper. Don’t take your grudge with the rest of the experts out on third parties.

Just to illustrate how this whole line of thinking can go terribly wrong, consider this: your whole analysis of the paper is centered on the flawed statistical analysis. Well, not so much flawed as simply absent.

But your conclusion is as flawed as theirs: you conclude from this lack of evidence that there is no evidence. That is of course wrong. The only thing we know is that their method failed to find any. In fact, their paper even carries this caveat:

Furthermore, it is important to note that our model can only at best imply that two different authors are more similar than any other pairing of authors.

Even if the rest of the paper is flawed, this is correct: their method could at best establish a similarity. And that similarity heavily depends on the features they used for classification. As I’ve said I don’t know the field so I wouldn’t know what biases this could introduce into the analysis.

But precisely because of that ignorance I have to defer judgement to the experts in the field, just as I defer judgement on quantum mechanics to physicist, judgement in bridge stability and nuclear reactor safety to engineers, and health matter to my physician.

You are free to investigate all of these fields (and I do!) but be aware of the shortcomings in your knowledge, rather than jumping to conclusions, like you did:

the methods and results of the comparison give such stunningly strong evidence that Marlowe wrote Shakespeare

Neither the paper, nor you, have quantified this “evidence” in any meaningful way. And while we’re talking statistics: the strong over-representation of Shakespeare in the corpus makes it utterly unsurprising that some works are misclassified as Shakespeare. And it isn’t necessarily surprising either that only one author is strongly affected by this misclassification.

To reiterate: I wouldn’t be terribly surprised if you were right and the current majority of experts is wrong. But your confidence is completely unwarranted by the data and your knowledge or lack thereof.


I use this question as a test of this site--- in those handful of cases where the science and the dogma conflict, can you actually write the conclusion that the science supports, even when this is a fringe opinion in the relevant (nonscience) field? This question is a perfect test case.

No, it was a stupid test case. Nobody involved in the discussion – including you – has shown the necessary expertise to evaluate the issue in an unbiased fashion. We know about this shortcoming. In fact, the whole site’s policies are built around this: we cannot evaluate the scientific merit of theories in most fields. For this reason, original research is strictly banned. Not on a whim, but because it would amount to a carte blanche to sprout unscientific theories.

And yes, this means that in some cases this website will favour established dogma over fresh evidence which we cannot evaluate fairly – but only dogma established by (at least the semblance of) good evidence. And as soon as better evidence emerges, this dogma will be overthrown. Notice that on this particular issue, nobody except you has taken a strong position either way.

If this website was around in 1500, could it support Copernicus?

Yes, definitely. There was never any scientific opposition to Copernicus. And note that your criticism of the methods in the paper itself is entirely welcome here (and yes, despite what I said in a comment on the answer):

Is it original research to analyze the failure of stylometry to separate Marlowe and Shakespeare?

To show errors in the analysis is not original research. To draw any conclusions from this is.

11
  • 2
    Actually there was scientific opposition to Copernicus. It's a mistake to assume that Copernicus 'discovered' heliocentricism one day, and after that only stick-in-the-mud idealogues opposed him. Aug 29, 2012 at 20:56
  • 1
    @DJClayworth Hmm. Maybe. I’d debate that since the usual conception is that “true” scientific pursuit (rather than just blundering about) began no earlier than ~ 400 years ago. But well, that’s not the real point of the post anyway. Aug 29, 2012 at 22:16
  • @Konrad: I made a prediction some months ago (around the election?) that the issue of Original Research will come to a head soon, because I don't think we have a clearly-stated well-reasoned policy against it - just hints here and assumptions there. I find your arguments against it interesting; I'm hoping we can have a proper general discussion (rather than about stylometry and honesty) soon and come to a clear resolution, and I hope you repeat them.
    – Oddthinking Mod
    Aug 30, 2012 at 0:58
  • @KonradRudolph: First, there aren't really "experts" in this field, it's mostly people from other fields. The stylometry never caught on, and I suspect part of the reason is that it fails to distinguish Marlowe from Shakespeare (this is one of the first test-cases for stylometries). If they are the same person, the failure is good. The reason I know this is that I read Marlowe and Shakespeare, and to my eye, they are identical in micro-style, only differening in theme and big-picture issues. I looked at Mendenhall's stylometry, which is trivial to check qualitatively by looking for....
    – Ron Maimon
    Aug 30, 2012 at 1:08
  • 4 letter words, and I checked for myself the anomalous number of 4 letter words in Shakespeare and Marlowe. I looked at every googleable stylometry to see if any distinguish Marlowe from Shakespeare, and the only one I found was "run on lines and feminine endings", which, as Farey points out, distinguishes Early Shakespeare from late Shakespeare just as well, and makes a trend for Shakespeare. Certain function word distributions also work, but these also suffer from drift (unlike Mendenhall, or these guys methods). Clustering tends to separate Shakespeare and Marlowe, but it's qualitative.
    – Ron Maimon
    Aug 30, 2012 at 1:11
  • The basic result of all these cumulative failures of completely different methods that fail nowhere else leads me to complete certainty that these are the same person. The only reason people put caveats on stylometry is because of the baffling (to them) result that it never separates Marlowe and Shakespeare, like it conclusively separates Hamilton from Madison, or Dickens from Thackeray. The reason is, duh, they're the same writer.
    – Ron Maimon
    Aug 30, 2012 at 1:14
  • @KonradRudolph: It isn't "some works" that are classified as Shakespeare, it is the majority of Marlowe's work (and it would be the vast majority if it weren't for the Tambourlaine shenanigans), and the method is essentially dead on (as it should be) in distinguishing the other guys from Shakespeare and from each other.
    – Ron Maimon
    Aug 30, 2012 at 1:17
  • 4
    @Ron, You are essentially ignoring what I and have DJClayworth have said in criticism of the methods. You are simply ignoring potential problems and cherry-picking evidence. That, combined with your lack to understand the importance of quality control via peer review, makes this whole discussion completely one-sided. I will not engage with you in stylometric debate because I recognise that I at least am not qualified. Aug 30, 2012 at 8:28
  • @KonradRudolph: I am not cherry picking anything. It's all the evidence. I am ignoring what you said, because what you said is not important.
    – Ron Maimon
    Aug 30, 2012 at 17:26
  • 2
    @Ron That assertion is simply ridiculous. If you think that what I said is not important aspect then you know much less about the topic than you think. And yes, that is cherry-picking evidence. Aug 30, 2012 at 19:22
  • @KonradRudolph: Suppose a new play is discovered tomorrow, for definiteness sake, pretend it's Richard III. how can one tell if it's by Marlowe or by Shakespeare? You can tell it's not by anyone else easily, but what stylometry should one use? I went through every single paper I could find on the subject over the past few years, and I have not found one quantitative method that is guaranteed to work. This is in contrast to every other pair of authors in human history, even ones emulating each other, like in the Federalist papers. It is not cherry picking, it is every method in every paper.
    – Ron Maimon
    Aug 31, 2012 at 3:01
4

Before I get to the actual scientific issues, allow me to summarize your complaint. You asked a question, already knowing what the answer should be - and when someone cited a valid scientific paper that disagreed with your claims, you attacked that paper and made serious allegations against the authors. And this was all in the name of testing the open-mindedness of this site.

On to the paper.

I have worked in the field of classification and machine learning, and while I think there are some issues in the paper I would completely disagree with your condemnation of it. Examinations of small, fixed-content datasets like this are really difficult to do, and I think the authors approach was a good one under the circumstances. I think their conclusions are on the whole fair.

The section in your answer "Estimated probability of authorship" appears however to be nonsense. You have pulled figures out of thin air ("10% variable") and then combined them in a naive way. You utterly failed to explain the many results which indicate separate authorship. If you had written the same paper, and reached a conclusion that Shakespeare and Marlowe were the same, some guy on an internet forum would be applying the same logic to the results and claiming it even more astronomically probable that you were wrong. Your statement that the method "only fails to distinguish Marlowe from Shakespeare" should be balanced by noting that it completely successfully distinguished Shakespeare from Marlowe. The authors suggestion that the small corpus of Marlowe could account for the misclassification is at least somewhat valid - it's a well-known issue in the field.

Finally, you do not in any way address the 'unsupervised learning' section of the paper, which is probably the strongest evidence.

And really finally, your answer absolutely fails to present any evidence of its own. It's only content is an attack on the result you dislike. To even approach being a 'good' answer you have to present some actual evidence for your position. Since it was you who asked and answered the question I would assume you had evidence of your own.

4
  • Their method didn't successfully distinguish Shakespeare from Marlowe--- it attributed I Henry VI to Marlowe. The biases from the different length make it clear why going in one direction is easier, but if they were different authors, it should never go one way or the other! It is insane to have this kind of stylometry failure, it is unprecedented in all the annals of stylometry. Jefferson Madison and Hamilton were distinguished, in the Federalist papers, and these guys are similar in style.
    – Ron Maimon
    Aug 30, 2012 at 1:02
  • I wasn't 100% sure of the answer until I read the paper, and saw the stylometry failure and the ridiculous political conclusion. By the standards of stylometry, 100,000 words is enormous. Stylometries are definitive on newspaper articles and short essays, for 7 longish plays it is essentially dead certain. I do adress the "unsupervised learning", it's complete nonsense--- there is no quantitative way to give confidence to the clusters, because the cluster distance is not given, nor is the reason for the clustering made apparent, as it is for best-match in function words/vocabulary.
    – Ron Maimon
    Aug 30, 2012 at 1:03
  • Regarding "no evidence of my own", I am using the data in that paper, that is incompetently or fraudulently presented as evidence against Marlovian authorship. The contents of the paper made me 4-sigma sure, before I was only at 95% confidence.
    – Ron Maimon
    Sep 3, 2012 at 3:54
  • I did not pull the 10% out of thin air--- they compared 32 plays by other authors, with absolutely no error on either method! So the failure rate is below detection for 32 plays, 62 trials (two methods)! A 1 in 10 failure rate for both methods would only lead to zero errors in 64 trials 1/e^6.4 or 1 time in 1000. For Shakespeare/Marlow pairs, it is again no errors, and that's another 50 plays. Assuming there is a 10% error is too weak--- you would expect 6 non-Shakespeare/Marlowe misclassifications for 10% error.
    – Ron Maimon
    Sep 3, 2012 at 4:23
2

Separate issue, so separate answer:

I was told I would be banned from this site for rudeness, as is usual for impolitic characters like myself, so I might not be around for much longer.

This is simply false. You were told no such thing. You were warned that your previous behaviour was considered rude by a lot of users, and that such behaviour would not be tolerated. Impoliticity (if that word existed) and impoliteness are not the same. For instance, while the accusation of intellectual dishonesty was appropriate as used by you, the accusation of fraud wasn’t.

As for whether you will be around longer – this is entirely up to you. If you behave according to the rules (or simply common courtesy) nobody will prevent you from staying.

1
  • Impolitic and impolite are synonyms, not technically, they supposedly mean different things, but they don't really. I know how easy it is to get a lynch mob sicced on people, and I have no intention of accomodating or bending to censorious policies. If I think something is fraudulent, I'll say it, but I can live with non-defaming formulations.
    – Ron Maimon
    Aug 30, 2012 at 1:57

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .