At present, the traditional machine learning methods and convolutional neural network (CNN) methods are mostly used in image recognition. The feature extraction process in traditional machine learning for image recognition is mostly executed by manual, and its generalization ability is not strong enough. The earliest convolutional neural network also has many defects, such as high hardware requirements, large training sample size, long training time, slow convergence speed and low accuracy. To solve the above problems, this paper proposes a novel deep LeNet-5 convolutional neural network model for image recognition. On the basis of Lenet-5 model with the guaranteed recognition rate, the network structure is simplified and the training speed is improved. Meanwhile, we modify the Logarithmic Rectified Linear Unit (L ReLU) of the activation function. Finally, the experiments are carried out on the MINIST character library to verify the improved network structure. The recognition ability of the network structure in different parameters is analyzed compared with the state-of-the-art recognition algorithms. In terms of the recognition rate, the proposed method has exceeded 98%. The results show that the accuracy of the proposed structure is significantly higher than that of the other recognition algorithms, which provides a new reference for the current image recognition.