Properties of instrumental variable estimators are sensitive to the choice of valid instruments, even in large cross‐section applications. In this paper we address this problem by deriving simple mean‐square error criteria that can be minimized to choose the instrument set. We develop these criteria for two‐stage least squares (2SLS), limited information maximum likelihood (LIML), and a bias adjusted version of 2SLS (B2SLS). We give a theoretical derivation of the mean‐square error and show optimality. In Monte Carlo experiments we find that the instrument choice generally yields an improvement in performance. Also, in the Angrist and Krueger (1991) returns to education application, when the instrument set is chosen in the way we consider, it turns out that both 2SLS and LIML give similar (large) returns to education.