General characterizations of valid confidence sets and tests in problems which involve locally almost unidentified (LAU) parameters are provided and applied to several econometric models. Two types of inference problems are studied: (i) inference about parameters which are not identifiable on certain subsets of the parameter space, and (ii) inference about parameter transformations with discontinuities. When a LAU parameter or parametric function has an unbounded range, it is shown under general regularity conditions that any valid confidence set with level $1 - \alpha$ for this parameter must be unbounded with probability close to $1 - \alpha$ in the neighborhood of nonidentification subsets and will have a nonzero probability of being unbounded under any distribution compatible with the model: no valid confidence set which is almost surely bounded does exist. These properties hold even if "identifying restrictions" are imposed. Similar results also obtain for parameters with bounded ranges. Consequently, a confidence set which does not satisfy this characterization has zero coverage probability (level). This will be the case in particular for Wald-type confidence intervals based on asymptotic standard errors. Furthermore, Wald-type statistics for testing given values of a LAU parameter cannot be pivotal functions (i.e., they have distributions which depend on unknown nuisance parameters) and even cannot be usefully bounded over the space of the nuisance parameters. These results are applied to several econometric problems: inference in simultaneous equations (instrumental variables (IV) regressions), linear regressions with autoregressive errors, inference about long-run multipliers and cointegrating vectors. For example, it is shown that standard "asymptotically justified" confidence intervals based on IV estimators (such as two-stage least squares) and the associated "standard errors" have zero coverage probability, and the corresponding $t$ statistics have distributions which cannot be bounded by any finite set of distribution functions, a result of interest for interpreting IV regressions with "weak instruments." Furthermore, expansion methods (e.g., Edgeworth expansions) and bootstrap techniques cannot solve these difficulties. Finally, in a number of cases where Wald-type methods are fundamentally flawed (e.g., IV regressions with poor instruments), it is observed that likelihood-based methods (e.g., likelihood-ratio tests and confidence sets) combined with projection techniques can easily yield valid tests and confidence sets.