The harmonic index $H(G)$ of a graph $G$ is defined as the sum of the weights $\frac2{du+dv}$ of all edges $uv$ of $G$, where du denotes the degree of a vertex $u$ in $G$. In this paper, we determine (i) the trees of order $n$ and $m$ pendant vertices with the second smallest harmonic index, (ii) the trees of order $n$ and diameter $r$ with the smallest and the second smallest harmonic indices, and (iii) the trees of order $n$ with the second, the third and the fourth smallest harmonic index, respectively.