Connections and covariant derivatives are usually taught as a basic concept of differential geometry, or more precisely, of differential calculus on smooth manifolds. In this article we show that the need for covariant derivatives may arise, or at lest be motivated, even in a linear situation. We show how a generalization of the notion of a derivative of a function to a derivative of a map between affine spaces naturally leads to the notion of a connection. Covariant derivative is defined in the framework of vector bundles and connections in a way which preserves standard properties of derivatives. A special attention is paid on the role played by zero-sets of a first derivative in several contexts.