Limited overlap between the covariate distributions of groups with different treatment assignments does not only make estimates of average treatment effects rather imprecise, but can also lead to substantially distorted confidence intervals. This paper argues that this is because the coverage error of traditional confidence intervals is driven by the number of observations in the areas of limited overlap. Some of these “local sample sizes” can be very small in applications, up to the point that distributional approximations derived from classical asymptotic theory become unreliable. Building on this observation, this paper constructs confidence intervals based on classical approaches to small sample inference. The approach is easy to implement, and has superior theoretical and practical properties relative to standard methods in empirically relevant settings.