The goal of this paper is to study conformal Yamabe soliton and $*$-Yamabe soliton, whose potential vector field is torse forming. Here, we have characterized conformal Yamabe soliton admitting potential vector field as torse forming with respect to Riemannian connection, semi-symmetric metric connection and projective semi-symmetric connection on Riemannian manifold. We have also shown the nature of $*$-Yamabe soliton with torse forming vector field on Riemannian manifold admitting Riemannian connection. Lastly we have developed an example to corroborate some theorems regarding Riemannian connection on Riemannian manifold.