In this paper, novel second order and fourth order diffusion models are proposed for image denoising. Both models are based on the gradient vector convolution (GVC) model. The second model is coined by incorporating the GVC model into the anisotropic diffusion model and the fourth order one is by introducing the GVC to the You-Kaveh fourth order model. Since the GVC model can be implemented in real time using the FFT and possesses high robustness to noise, both proposed models have many advantages over traditional ones, such as low computational cost, high numerical stability and remarkable denoising effect. Moreover, the proposed fourth order model is an anisotropic filter, so it can obviously improve the ability of edge and texture preserving except for further improvement of denoising. Some experiments are presented to demonstrate the effectiveness of the proposed models.