This is a guest post by Charles Seguin. He is a PhD student in sociology at the University of North Carolina at Chapel Hill.

Sociologists and historians have shown us that national public discourse on lynching underwent a fairly profound transformation during the periods from roughly 1880-1925. My dissertation studies the sources and consequences of this transformation, but in this blog post I’ll just try to sketch some of the contours of this transformation. In my dissertation I use machine learning methods to analyze this discursive transformation, however after reading several hundred lynching articles to train the machine learning algorithms, I think I have a pretty good understanding of key words and phrases that mark the changes in lynching discourse. In this blog post then, I’ll be using basic keyword, bigram (word pair), and trigram searches to illustrate some of the changes in lynching discourse.

To conduct this analysis I wrote a Python script to scrape the archives of three national newspapers that were central to the media representation of lynching at the time: The Atlanta Constitution–the leading national southern paper, The Chicago Tribune–a northern paper with a strong anti-lynching editorial bent, and The New York Times–the national paper of record and staple of scholarly analyses. My scraper went to these archives, searched for several keywords and bigrams by year, and recorded how many articles were written containing these keywords. The figures below, all 3-year moving averages, illustrate changes over time. In most cases the papers were similar, so I simply aggregated them, in the last figure, I break the patterns out by paper.

I first scraped the occurrence of the term “lynched” in all three newspapers. The presence of the term “lynched” generally means that the article referenced some specific lynching event. Other terms such as “lynching” are less useful for this application because they tend also to be in articles that reference lynching as a more general phenomenon. Below is a figure showing how many articles the combined three newspapers ran containing the word “lynched:”

lynched_graph

During the 1880s national newspapers shamelessly legitimated lynchings as a understandable community responses to heinous crimes, what Michael Pfeifer calls a “Rough Justice” frame. Headlines such as “Murderer Lynched” or “Negro Brute Lynched” painted lynching as a natural, even desirable, consequence of black criminality. These articles strove to legitimate the practice of lynching.

Lynched for his crime

To roughly capture the rise and fall of this frame I assembled a dictionary of descriptors that were used at the time to describe lynching victims as criminals, or even sub-human. These terms were: murderer, fiend, brute, slayer, criminal, scoundrel, ruffian, desperado, brutal negro, culprit, firebug, thief, bad negro, ravisher, villain. The figure below shows the total articles that contained any of these descriptors followed by “lynched.” For example, I searched for “brute lynched,” or “thief lynched.” The figure also shows the total articles for the search term “alleged” in articles that also contained the word “lynched.”

criminal_graph

The figure above shows then that the number of articles with criminal descriptors of lynching victims declined precipitously throughout the 1880s and 1890s. It also shows that newspapers began using the word “alleged” to describe the supposed crimes of lynching victims.

Using the “alleged” descriptor was one small step in casting doubt on the guilt of the lynching victim, and also hinted towards the absence of due process. The figure below shows the rise of the articles referencing the terms “lawlessness”, “lawless,” “anarchic,” “anarchy”, and “mob rule.” These terms were found in articles that were critical of the lack of due process in lynchings (even if they were still quite racist in most other respects).

lawlessness_graph

Up until this point, I have been aggregating results from the three newspapers. However, there were important regional differences in lynching discourse. One such difference was the use of alleged sexual crimes to justify lynchings. Lynching apologists, particularly in the southern states, alleged that lynching was necessitated by the threat to white women from black rapists. Here key words were “ravish” or “ravished” and “outrage” or “outraged” which were terms for used euphemistically to refer to rape or sexual assault. While the term “outrage,” could have referred to the lynching itself, in practice it referred to the supposed crime of the lynching victim. The figure below shows how the use of sexual assault rhetoric was more common in the southern paper, The Atlanta Constitution, than in the two northern outlets.

outrage_graph

Lynching discourse in the national media changed in many ways that were pushed by anti-lynching activists, such as Ida B. Wells, W.E.B. Dubois, and the early NAACP. These activists pushed to change the representation of lynching in the national news media, because they saw its representation as being central to the legitimation and continuation of lynching. While the change in lynching discourse over this period fell far short of what these activists hoped for, and fell far short of a full critique of racial violence and domination, the transformation over this period was nevertheless stark and important. Scraping these archives for keywords provides a rough indication of the scope of this transformation.

  • David R

    Interesting. Python seems something of a brute-force technique, for the purpose of this analysis. Did the author search for bigrams such as “brute-force?” Also, I would have liked to see more discussion of potential causes of the shift in the discursive framing of lynching. Overall, I give this a B, B+…

  • Laura Nelson

    Interesting analysis. Did you try normalizing for the number of total articles printed in each year, or producing a graph with the number of total articles by year? When I use similar techniques I wonder if some of the trends reflect the fact that the newspaper is simply printing more or fewer articles year to year.

  • Charles Seguin

    Laura,

    Yes–I did try normalizing, and you’re 100% correct that the total number of article can create spurious trends. It doesn’t do too much in this particular case, but it some other cases the effects are big. For example, if you don’t normalize it looks like African American papers (e.g. The Chicago Defender) pay less attention to lynching than do white papers. When you normalize you can see that the African American papers are writing about lynching at about six times the rate as the white papers!

    John Woolley has a paper that talks about normalizing: “Using Media-Based Data in Studies of Politics,” American Journal of Political Science 44 (January 2000), pp. 156-173.

    David,

    Thanks for the B+! I agree that this sort of keyword searching doesn’t get you very far on its own. However, after having read several hundred of these articles the old-fashioned way, you do get a pretty good sense of what terms mark (not constitute) different types of discourse, and scraping can give you a better sense of the patterns in these terms. In the end of course, these keyword frequencies are no substitute for either a close reading of the texts, or more sophisticated text classification models. Nevertheless, I will say that I learned a great deal from these keyword searches that wasn’t readily apparent to me in my own reading.

    I didn’t end up searching for “brute force” as the bigram didn’t seem important in any of my earlier readings. It is interesting however, that the term “force,” or “forced,” actually marks a fairly specific type of lynching article: those in which mobs forced open jail doors to lynch a victim who was inside.

    As for the causes of this shift, I’m still working on those!

  • Адам Фай

    Do you have any suggestions for resources and tools for self learning?