This paper shows that there is more scope for a borrower to engage in a sustainable infinite debt rollover (a “Ponzi scheme”) when interest/growth rates are stochastic. In this context, I prove that the relevant “r vs. g” comparison uses the yield r_long to an infinite‐maturity zero‐coupon bond. I show that r_long is lower than the risk‐neutral expectation of the short‐term yield when it is variable, and that r_long is close to the minimal realization of the short‐term yield when it is highly persistent. The paper applies these results to illustrative heterogeneous agent dynamic stochastic general equilibrium models to obtain similarly weakened sufficient conditions for the existence of public debt bubbles.