Atkinson proved in 1941 that $\int^\infty_0 e^{-t/T} |\zeta(1/2+it)|^4dt = TQ_4(\log T)+O(T^c)$ with $c = 8/9+\varepsilon$, where $Q_4(y)$ is a suitable polynomial in $y$ of degree four. We improve Atkinson's result by showing that $c=1/2$ is possible, and we provide explicit expressions for all the coefficients of $Q_4(y)$ and the closely related polynomial $P_4(y)$.