Danny Ebanks, "Topic Models Estimators Are Statistically Inconsistent and Substantively Misleading: Evidence and Solutions”
We demonstrate that the most commonly used topic model estimators are statistically inconsistent and yield systematically misleading substantive conclusions when used for measurement. Specifically, (1) Latent Dirichlet Allocation models using variational inference diverges from the truth in the large-sample limit and is statistically inconsistent, and (2) this inconsistency is degenerate, which we define as converging to erroneous estimates as if they were meaningful. As a consequence, studies that rely on such models produce misleading descriptions of fundamental quantities of interest in American politics, such as legislative polarization and party-leader alignment. We then provide a consistent, spectral alternative that scales to large corpora and is easily applied to a wide variety of social science applications using text. We furnish new evidence from 14 million Congressional floor speeches, which reveals that polarization and party-leader alignment actually diverge substantially over the last 120 years. These findings challenge conventional theories of legislative power and underscore that modern statistical architectures fundamentally shape our substantive understanding of American politics.
The American Politics Research Workshop (Gov 3004) meets all academic year, Tuesdays, 12:00–2:00 PM, in CGIS K354. This workshop presents an opportunity for graduate students and Harvard faculty to present and receive feedback on their current research. The workshop highlights key theoretical and empirical findings from Harvard affiliates on topics related to American politics.
All interested Harvard affiliates are invited to attend.
LOCATION:CGIS Knafel room K354, or virtual via Zoom STATUS:CONFIRMED DTSTART:20250401T160000Z DTEND:20250401T170000Z END:VEVENT END:VCALENDAR