Thursday, June 4, 2015

On The Run and Social Science Research Methods

There has been a lot more about Alice Goffman’s On the Run the last few days.

Lubet’s response to the response


Although, much of the attention has been focused on the issue of her possible criminal conduct, it is the methodology of her project that really concerns me. I have not yet read the book. I have, however, read her paper in the American Sociological Review that was based upon the same research. She claims to have spent six years studying the residents of a neighborhood in Philadelphia. The name of the neighborhood is a pseudonym as are the names of all the individuals. Consequently, it is not possible to verify any of her claims. It is not even possible to check her account against her own field notes. She claims to have destroyed them. All of this is ostensibly to protect the people who are described in the book.

Her entire methodology is so alien to my view of research in the social sciences I find it hard to comprehend. It is not her immersion in the culture of the people she was studying that concerns me. It seems like a legitimate method of qualitiative research. Whether you use qualitative or quantitative methods should be determined by the question you are trying to answer. What puzzles me is the complete lack of accountability. One of the essential elements of good historical research is to be clear about the relationship between your conclusions and the sources that you use. Anyone should be able to follow your trail of sources to see if it leads to your conclusions. Is anyone going to believe you if you say that you use evidence from a secret notebook at an undisclosed archive? Could you write a history dissertation at Princeton based upon a secret diary that you say you destroyed to protect the author’s privacy?  In economics you are generally expected to be ready to present you data to other researchers or have a very good reason why you cannot. The American Economic Review, for instance, expects authors to make their data available. Reinhart and Rogoff got in trouble a while back for a spreadsheet error, but we should not forget that when a graduate student asked for the data they gave it to him. Goffman’s entire body of research appears to depend on “Just trust me.”

I am not saying that she lied. There are troubling inconsistencies within her accounts and between her accounts and other evidence. And her response to Lubet’s suggestion that she had committed a crime only adds another inconsistency. Her account in the book is completely different than the account in her response. Even if there were not inconsistencies, I would be concerned about a methodology that places so much weight trusting the author. The rewards in the social sciences for coming up with results that are deemed interesting and important are considerable. Goffman got a Ph. D. from Princeton, a best dissertation award, a book contract, a publication in the American Sociological Review, a TED talk, and a job at the University of Wisconsin. The temptations to give people what they want are too great to rely upon a methodology that provides no means for subsequent researchers to evaluate the evidence.


Note about the anonymous critique: Some might wonder why I am willing to link to an anonymous critique when I have such a problem with the anonymity in Goffman’s work. I have seen discussion on the web suggesting that because this critique is anonymous it should be completely disregarded. I don’t know why the author prefers to remain anonymous. As long as their argument is not based upon their authority I do not really care. The Federalist papers were published under a pseudonym. Gosset’s work on the t distribution was published under a pseudonym. I do not regard anonymity itself as a problem. The anonymous author of the critique does not at any point ask me to just trust them. There is nothing in their argument that hinges on their identity rather than the evidence.


No comments: