This paper proposes an accelerometer-based gesture recognition algorithm. As a pre-process procedure, raw data output by accelerometer should be quantized, and then use discrete Hidden Markov Model to train and recognize them. Based upon this recognition algorithm, we treat gesture as a method of human-computer interaction and use it in 3D interaction subsystem in VR system named VDOM by following steps: establish Gesture-Semantic Map, train standard gestures, finally do recognition. Experimental results show that the system can recognize input gestures quickly with a reliable recognition rate. The users are able to perform most of the typical interaction tasks in virtual environment by this accelerometer-based device.