The role of thermodynamics in continuum mechanics and the derivation of proper constitutive relations is a topic discussed in Rational Mechanics. The classical literature did not use the accumulated knowledge of thermostatics and was very critical of the heuristic methods of irreversible thermodynamics. In this paper, a small strain gradient elasticity theory is constructed with memory effects and dissipation. The method is nonequilibrium thermodynamics with internal variables; therefore, the constitutive relations are compatible with thermodynamics by construction. The thermostatic Gibbs relation is introduced for elastic bodies with a single tensorial internal variable. The thermodynamic potentials are first-order weakly nonlocal, and the entropy production is calculated. The constitutive functions and the evolution equation of the internal variable are then constructed. The second law analysis has shown a contribution of gradient terms to the stress, also without dissipation.