Is Trump's UN speech an outlier?

Sep 25, 2017 00:00 · 1125 words · 6 minutes read Visualization Scaling Text Analysis #rstats

One of a kind UN speech?

When Donald Trump delivered his now infamous speech to the UN’s General Assembly, with articles highlighting the rather unusual vocabulary used. While the speech might be unusual if contrasted with the rather cautious and timid language of diplomacy I was curious how it stacked up to other US presidents’ speeches in the UN.

In this post I will examine 8 presidential speeches made at the UN’s General Assembly by US presidents. Using the “wordfish” unsupervised scaling algorithm should give us a sense of where this speech is located relative to the others, based on it’s vocabulary. Wordfish will place the documents on a one dimensional scale based on word frequencies in the texts. It was developed by political scientists Slapin and Proksch.1 The original goal was to scale party manifestos on the left-right spectrum, so the algorithm’s output is quite fitting to my question. Moreover, it does not require any reference texts, we just need to specify which two documents are the two ends of the scale.2

Data and analysis

For the data, I selected the first UN General Assembly speech of every US president after their inauguration. Since this is a debut moment for every president it should make the texts more comparable to each other. Also, this means that except Trump and H.W. Bush, we have two speeches from Clinton, W. Bush, and Obama. I decided that the date range of the sample is 1990 - 2017 because this is the start of the final decline (and eventual dissolution) of the USSR so the Cold War rhetoric wouldn’t distort the results. Unfortunately this also limits the sample to 8 speeches made over this time period. However, there are some extraordinary periods in there, like Bush’s 9/11 or Hurricane Katrina speech. I got the speeches from https://2009-2017.state.gov/p/io/potusunga/index.htm and http://www.presidency.ucsb.edu/

I use the implementation that is in the quanteda package. It requires the transformation of text into a document-feature matrix (dfm) for the wordfish to be able to estimate the positions. First, I load all the documents into R and then transform them into a dfm object.

unga <- readtext("*.txt")
unga_dfm <- dfm(corpus(unga))

Now some cleaning is in order, making all the words lowercase and removing stopwords. In other cases removing numbers and special characters are a good idea as well, but these texts contain none of these.

dfm_tolower(unga_dfm)
## Document-feature matrix of: 8 documents, 4,207 features (72.5% sparse).
dfm_remove(unga_dfm, stopwords("english"))
## Document-feature matrix of: 8 documents, 4,073 features (74.1% sparse).

Now comes probably the most subjective part of the whole analysis: choosing the two end of the scale. As a reasonable assumption, I choose W. Bush’s 2001 speech as one end and Obama’s 2009 speech as the other. The justification is that Bush carried his ‘axis of evil’ and ideas into his UN speech and also appealed on existential threats (not unlike Trump): > “Civilization, itself, the civilization we share, is threatened.”

On the other end the Obama speech came 9 month after his election, when he took over the presidency with his election slogans of ‘change’, ‘hope’ that were carried into his UN speech. Finally, this division also represents the political divide of the speakers, with one of them being a Republican and the other a Democrat.

unga_dfm@Dimnames$docs  # locating the positions of each document in the dfm
## [1] "clinton93.txt" "clinton97.txt" "hwbush90.txt"  "obama09.txt"  
## [5] "obama13.txt"   "trump17.txt"   "wbush01.txt"   "wbush05.txt"
wordfish <- textmodel_wordfish(unga_dfm, dir = c(4,7))

The quanteda package can provide a quick visual overview of the results. First, let’s check how different words affect the estimation of the positions. In the below plot, the Estimated beta is the weight of the given word, while the Estimated psi is the word fixed effects. For the interpretation it means that more frequent words should appear in each text because they do not have a political meaning (such as prepositions). So while a common word might have a high fixed effect, it has a weight of zero because of it’s lack of political meaning.

textplot_scale1d(wordfish, margin = "features",
                 highlighted = c("terror","sovereignity", "islam", "war", "nuclear", "iran"),
                 highlighted_color = "orangered2")

Somewhat surprisingly, words like ‘war’, ‘terror’ or ‘islam’ are assigned a weight close to 0, with only ‘iran’ having a sizable weight and effect of the highlighted words. Finally, let’s plot the positions of the speeches on the political scale, with bootstrapped confidence intervals.

textplot_scale1d(wordfish, margin = "documents")

Well. Um. So this is not exactly what I was expecting. The result that I was interested in is that the Trump ‘17 speech is at the same point of the scale (the ’Estimated theta’) that the W. Bush ‘01 speech. This is somewhat unsurprising but it does show that Trump’s speech was not off the charts of US Presidential addresses to the UN General Assembly. Quite the contrary, it is very similar to another Republican speech made by an ’establishment’ President. We should keep in mind, that this scale measures the foreign policy positions of these speeches, as foreign policy is at the core of the UN speeches.

Given this, what I was not expecting is the two Clinton speech being on the opposite end from Obama’s 2013 appearance. I am no foreign policy expert, so I cannot really comment on whether this is a result of CLinton’s genuinely different policy stance or just some other dimensions in the texts messing up the estimation (see footnote 2).3

While the results are not that conclusive, it clearly shows that Trump’s speech is not an ideological outlier compared to his predecessors. The script and the text files are uploaded on github, so if anyone is interested in playing with it or expanding on this idea is more than welcome.

Some caveats of this short exercise

The wordfish algorithm is used for unsupervised ideological scaling and takes the ‘bag of words’ assumption: that is, the order and the context of the words do not matter within the document. While this might be an unreasonable assumption (and surely, languages do not work without order) it allows for a simplified model. Similarly, validating the above results are somewhat problematic, as we would need human coding to match against the algorithm’s results. Unfortunately, I don’t have time, or disposable funds for this (this is why its a half-baked idea blog post and not an article).


  1. Slapin, J. B., & Proksch, S. O. (2008). A scaling model for estimating time‐series party positions from texts. American Journal of Political Science, 52(3), 705-722.

  2. Of course it is possible that the noise in the text is not noise at the end but other dimensions (as elaborated by Will Lowe here)

  3. I tried setting different endpoints for the algorithm, but the results are the same regarding the Bush-Trump overlap and the Clinton puzzle.