History's Replication Crisis

 

Anton Howes recently asked Does History Have a Replication Crisis?  The question is one that Howes has been concerned with for some time, but the immediate impetus for the essay was the publication of Jenny Bulstrode’s Black metallurgists and the making of the industrial revolution published in the journal History & Technology. Bulstrode claims that,

“Between 1783 and 1784, British financier turned ironmaster, Henry Cort, patented a process of rendering scrap metal into valuable bar iron that has been celebrated as one of the most important innovations in the making of the modern world. Here, the concern is the 76 Black metallurgists in Jamaica, who developed the process for which Cort took credit.”

Howes describes the innovation as a process “to more easily convert scrap iron into new bar or wrought iron — a higher-quality iron that had had various impurities beaten out of it with hammers — by bundling the scrap together, heating it, and then passing it through grooved rollers, rather than the more usual flat ones, stretching and smoothing the sides and edges of the heated metal so that the resulting bars became “perfectly welded at the edges and throughout” and “completely welded at the sides, without a crack, into one mass, perfectly sound to the centre”.” Not surprisingly, the discovery that a famous inventor of an important process had in fact stolen his invention from enslaved people spread quickly.

On NPR you can listen to How Henry Cort stole his iron innovation from Black metallurgists in Jamaica

In the Guardian you can read about how Industrial Revolution iron method ‘was taken from Jamaica by Briton’

At The World you can hear how Historian uncovers the Jamaican metal workers behind Industrial Revolution

At New Scientist you can read about how English industrialist stole iron technique from Black metallurgists

     Howes, however, was skeptical of the claim ( The Cort Case) suggesting that the evidence presented in the paper did not warrant the conclusion that the innovation in question had been developed by enslaved workers and then stolen by Cort. Oliver Jelf (The origin of Henry Cort’s iron-rolling process: assessing the evidence) looked at the sources cited by Bulstrode and concluded that there was a more fundamental problem. The sources simply did not say what Bulstrode claimed they did. For instance,

 

Jelf did not simply claim that the sources do not support Bulstrode’s argument, he transcribed and presented the sources in the paper, leading Howes to state that “What I simply cannot fathom, now that I’ve read her sources thanks to Jelf’s transcriptions, is how Bulstrode arrived at her narrative at all (Does History Have a Replication Crisis?).”

Ian Leslie (Stories are bad for your intelligence: How Historians (and Others) Make Themselves Stupid) has theory for how Bulstrode came to the narrative. He traces it to problems with stories and  story telling. Leslie says that,

I doubt that Bulstrode set out to deceive. My guess is that she came across a few suggestive fragments in her reading (the ‘cousin’ of Cort travelling from Jamaica to England) and wanted so badly to make them into a story which fitted her ideologically determined prior - that the British stole ideas from those they enslaved - that she got carried away, fabricating causes and effects where none existed.”

He thinks more of the blame should fall on the peer reviewers. Leslie suggests that,

It’s one thing for a young and passionate academic to make mistakes; it’s quite another for a series of experienced academics to let her make them. The paper had two anonymous peer-reviewers (Bulstrode thanks other historians in an endnote, though they may not have read the paper). Even to an ignorant reader like me, the paper just smells funny - it has the aroma of the fantastical. How on earth did these experts read it without becoming suspicious? Why didn’t they double-check its remarkable claims?

I can’t agree with Leslie’s argument. I don’t think that stories or peer-reviewers are the fundamental problem here. 

We need to tell stories. Often the answer to “Why did this happen?” is a sequence of events, a story about how it came to happen. Nor can the blame for misleadingly citing sources be pushed on to the referees. Although I am an economist, I have probably written more referee reports for books and papers written by historians than economists. I will note it in my report if I think an author incorrectly uses a source that I am familiar with.  But I can’t check every citation. I can’t even check the citations to crucial claims if it requires a trip to the archive. Experts in the field should be familiar with important secondary sources, but you can’t know every primary source. You certainly can’t run off to check on every novel primary sources that someone has discovered. You have to be able to trust the author to honestly report what is in the sources that that they cite.

Leslie’s concern about the siren song of stories makes him overly generous with Bulstrode. A professional historian should not get carried away with enthusiasm to the point that they try to support claims with references to sources that do not actually provide any support for those claims. Actually, amateur historians and undergraduate students shouldn't do that either. Historians must tell stories, but they must tell stories that are constrained by the sources. If you do not want your story telling to be constrained by the historical evidence you should be forthcoming and admit that your genre is historical fiction, not history.

I have frequently said that I think honesty is the most important trait for a historian. In economics and other quantitative social sciences I can say “Send me your data.” Many journals require making the data available. But a historian might cite documents that I would have to travel to multiple cities, states, or even countries to access. To be of any use to me I need to be able to trust that you have honestly represented the sources that you cite. Once you have lost my trust you are worthless to me as a historian. Even if I can point to things that you got right, I can’t be sure about anything that I don’t already know. I can’t learn anything from you.

Anton Howes suggests making history more like quantitative social sciences. Try to make copies of relevant sources available. Now that so many people have digital images of the primary sources they use this is at least imaginable. Still, it is not a panacea, as demonstrated by recent revelations on honesty research (see datacolada.org.) Nevertheless, to the extent that it can be done, it would be great, both for the credibility of current research as well as a resource for future research.

But there should also be repercussions. Sadly, I doubt that there will be. Anton Howes notes other historical myths that seem immune to revision in response to evidence. I and others have written a great deal about one historian who in an influential book did not honestly represent what was in primary or secondary sources, going well beyond honest mistakes driven by youthful enthusiasm. As best I can tell there were absolutely no repercussions for him. Other historians still cite the book and praise the author. 

I hope that I am wrong; I hope that many historians read Howes' Does History Have a Replication Crisis? and take the question seriously.