As an important part of the evaluation reform, online evaluation system can effectively improve the efficiency of evaluation work, which has been paid attention by teaching institutions. The online evaluation system needs to support the safe and stable transmission of information between the client and the server, and socket protocol establishes the connection through the listening port, which can easily carry out the message transmission and process control. Because it can well meet the construction requirements of online evaluation system, it is applied in our study. The building of online evaluation system based on socket protocol includes the function design of students and teachers, data flow design, evaluation difficulty grading design and system implementation. The system uses Java language and MVC mode for development, which has good scalability and platform-independence. It realizes the paperless examination process and greatly reduces the workload of teachers. The contribution of this paper is mainly reflected in two aspects. One is to explore the construction of an online evaluation system based on the socket protocol, and it provide an Asynchronous IO technical solution for the network communication between the student and the server, which provides a reference for the development of similar systems. The second is to give the realization method of the difficulty classification of the evaluation, and classify the difficulty of the test questions, which lays the foundation for carrying out personalized testing and evaluation.