In this paper, we develop a fast compact difference scheme for the fourth-order multi-term fractional sub-diffusion equation with Neumann boundary conditions. Combining L1 formula on graded meshes and the efficient sum-of-exponentials approximation to the kernels, the proposed scheme recovers the losing temporal convergence accuracy and spares the computational costs. Meanwhile, difficulty caused by the Neumann boundary conditions and fourth-order derivative is also carefully handled. The unique solvability, unconditional stability and convergence of the proposed scheme are analyzed by the energy method. At last, the theoretical results are verified by numerical experiments.