A graph $\Gamma$ is called an $n$-Cayley graph over a group $G$ if its automorphism contains a semi-regular subgroup isomorphic to $G$ with $n$ orbits. Every $n$-Cayley graph over a group $G$ is completely determined by $n^2$ suitable subsets of $G$. If each of these subsets is a union of conjugacy classes of $G$, then it is called a quasi-abelian $n$-Cayley graph over $G$. In this paper, we determine the characteristic polynomial of quasi-abelian $n$-Cayley graphs. Then we exactly determine the eigenvalues and the number of closed walks of quasi-abelian semi-Cayley graphs. Furthermore, we construct some integral graphs.