Elff, Martin, Jan Paul Heisig, Merlin Schaeffer, and Susumu Shikano. 2021. “Multilevel Analysis with Few Clusters: Improving Likelihood-based Methods to Provide Unbiased Estimates and Accurate Inference”. British Journal of Political Science 51(1): 412-426.

Abstract:

Quantitative comparative social scientists have long worried about the performance of multilevel models when the number of upper-level units is small. Adding to these concerns, an influential Monte Carlo study by Stegmüller (2013) suggests that standard maximum likelihood methods yield biased point estimates and severely anti-conservative inference with few upper-level units. In this paper, we seek to rectify this negative assessment. First, we show that maximum likelihood estimators of coefficients are unbiased in linear multilevel models. The apparent bias in coefficient estimates found by Stegmueller can be attributed to Monte Carlo Error and a flaw in the design of his simulation study. Second, we show how inferential problems can be overcome by using restricted maximum likelihood estimators for variance parameters and a t-distribution with appropriate degrees of freedom for statistical inference. Thus, accurate multilevel analysis is possible within the framework that most practitioners are familiar with, even if there are only a few upper-level units.

/article/multilevel-improving/multilevel-ci-coverage.png

Figure 1: Performance of Likelihood-based Confidence Intervals of Upper-Level Covariate Effect in Multilevel Linear and Probit Models

View article (local copy)
View online appendix (local copy)
Supplementary material (local copies)
Open Access at Cambridge University Press