Incorporating the interfacial damage and thermal residual stresses, an elastoplastic damage formulation is proposed to predict the overall transverse mechanical behavior of continuous elliptical-fiber reinforced ductile matrix composites within the framework of micromechanics and homogenization. Based on the concept of equivalent inclusion and taking the progressive interfacial debonding angle into consideration, partially debonded fibers are replaced by equivalent orthotropic, perfectly bonded fibers. Three interfacial damage modes are considered. The Weibull's probabilistic function is adopted to describe the varying probability of progressive partial fiber debonding. The effective elastic moduli of four-phase composites, composed of a ductile matrix and randomly located yet unidirectionally aligned fibers are derived by a micromechanical formulation. Thermal residual stresses are taken into account through the concept of thermal eigenstrain to investigate the effects of the manufacturing process-induced residual stresses. Employing the micromechanical approximation, the overall stress-strain responses and the effective yield function are formulated with the thermal eigenstrain. When comparing with the available experimental data, significant effects of thermal residual stresses are discussed. Moreover, the effects of the interfacial strengths and the cross-sectional shapes of fibers on the mechanical behaviors of composites are systematically investigated.