In this paper, a specially defined automorphism group $\Gamma(G)$ of a connected countable simple infinite graph is considered. As the main result, we prove that $\Gamma(G)$ contains at most one non-trivial element. All infinite graphs with a non-trivial automorphism group are completely described. Finally, for graphs with odd, or with a small even number (2 or 4) of non-zero eignevalues, the corresponding automorphism groups are characterized.