We prove that if $G$ is a finite non-abelain $p$-group of order $p^n$ ($p$ a prime number), $\leq 6$, then the order of $G$ devides the order of the group of automorphisms of $G$.