In most statistical estimation problems, the distribution of errors is unknown, and the traditional assumption of normality is used for convenience. We investigate here the fragility of the inferences based on normality by hypothesizing a neighborhood of distributions around the normal distribution, and by identifying the set of alternative maximum likelihood estimates corresponding to the set of error distributions.