In this paper we consider famous Euclidean theorem which is mentioned in the title. Also, we consider numerous proofs of this theorem and its modifications. These modifications were given by famous mathematicians like Kummer, Stieltjes, Fermat, Sylvester, Legendre, Dirichlet, Bertrand and Euler. Proofs are various and they use number theory, analysis, algebra, combinatorics and topology. Indeed, this paper is a complete overview of all relevant fact connected with Euclidean theorem that the set of all prime numbers is infinite and his proof of this theorem.