In this paper we study the convergence of finite difference schemes to weak solutions of the third boundary value problem for Poisson's equation on the unit square. Using the theory of interpolation of function spaces, we obtain error estimates in a discrete $W^1_2$ Sobolev norm consistent, or ``almost'' consistent, with the smoothness of the data.