We establish conditions which (in various settings) guarantee the existence of equilibria described by ergodic Markov processes with a Borel state space $S$. Let $\mathscr{P}(S)$ denote the probability measures on $S$, and let $s \mapsto G(s) \subset \mathscr{P}(S)$ be a (possibly empty-valued) correspondence with closed graph characterizing intertemporal consistency, as prescribed by some particular model. A nonempty measurable set $J \subset S$ is self-justified if $G(s) \cap \mathscr{P}(J)$ is not empty for all $s \in J$. A time-homogeneous Markov equilibrium (THME) for $G$ is a self-justified set $J$ and a measurable selection $\Pi: J \rightarrow \mathscr{P}(J)$ from the restriction of $G$ to $J$. The paper gives sufficient conditions for existence of compact self-justified sets, and applies the theorem: If $G$ is convex-valued and has a compact self-justified set, then $G$ has an THME with an ergodic measure. The applications are (i) stochastic overlapping generations equilibria, (ii) an extension of the Lucas (1978) asset market equilibrium model to the case of heterogeneous agents, and (iii) equilibria for discounted stochastic games with uncountable state spaces.