This paper concerns the relationship between uniform boundedness and convergence of various generalized inverses. Using the stable perturbation for generalized inverse and the gap between closed linear subspaces, we prove the equivalence of the uniform boundedness and convergence for generalized inverses. Based on this, we consider the cases for the Moore-Penrose inverses and group inverses. Some new and concise expressions and convergence theorems are provided. The obtained results extend and improve known ones in operator theory and matrix theory.