Attacks based on the application layer of the cloud storage system have been dramatically increasing nowadays. However, the present detection studies of attacks are mainly focused on the network and transmission layer instead of the application layer. In this paper, we proposed an anomaly attack detection method based on the hidden semi-Markov model (HsMM) to secure the cloud storage system from the application-layer-based attacks. In this proposed method, observation serials are constituted by the time intervals between the I/O requests made by normal users and their characterization using the hidden semi-Markov model based on each protocol for application layer. By applying this technique in the cloud storage system, it is able to effectively detect and correct their abnormal behaviors. In addition, to ensure the QoS(Quality of Service), a Priority Queuing and flow controlling module is proposed in this paper, which can allocate more I/O bandwidths and resources to normal users. Besides, the experimental results have shown that the proposed method can describe such normal I/O behaviors of users based on each protocol for the application layer in the cloud storage system with 99.2% higher detection ratio and 0.7% lower false positive ratio when detecting abnormal behaviors of users, and it can ensure the QoS for normal uses.