We study the seismic response of two adjacent structures connected with a dry friction damper. Each of them consists of a viscoelastic rod and a rigid block, which can slide without friction along the moving base. A simplified earthquake model is used for modeling the horizontal ground motion. Energy dissipation is taken by the presence of the friction damper, which is modeled by the set-valued Coulomb friction law. Deformation of viscoelastic rods during the relative motion of the blocks represents another way of energy dissipation. The constitutive equation of a viscoelastic body is described by a fractional Zener model, which includes fractional derivatives of stress and strain. The problem merges fractional derivatives as non-local operators and theory of set-valued functions as the non-smooth ones. Dynamical behaviour of the problem is governed by a pair of coupled multi-valued differential equations. The posed Cauchy problem is solved by use of the Grünwald--Letnikov numerical scheme. The behaviour of the system is analyzed for different values of system parameters.