In this paper we investigate perturbations by holomorphically and polynomially Riesz operators concerning the sets of upper and lower semi-Fredholm operators, the sets of left and right Fredholm operators, as well as the sets of upper and lower semi-Browder operators and the sets of left and right Browder operators. We consider in particular perturbations of shifts by polynomially Riesz operators.