For a finite group $G$, let $\operatorname{cd}(G)$ be the set of irreducible complex character degrees of $G$ forgetting multiplicities and $X_1(G)$ be the set of all irreducible complex character degrees of $G$ counting multiplicities. Suppose that $p$ is a prime number. We prove that if $G$ is a finite group such that $|G|=|\operatorname{PGL}(2,p)|$, $p\in\operatorname{cd}(G)$ and $\max(\operatorname{cd}(G))=p+1$, then $G\cong\operatorname{PGL}(2,p),~SL(2,p)$ or $\operatorname{PSL}(2,p)\times A$, where $A$ is a cyclic group of order $(2,p-1)$. Also, we show that if $G$ is a finite group with $X_1(G)=X_1(\operatorname{PGL}(2,p^n))$, then $G\cong\operatorname{PGL}(2,p^n)$. In particular, this implies that $\operatorname{PGL}(2,p^n)$ is uniquely determined by the structure of its complex group algebra.