MEMS Accelerometer Based Hand Gesture Recognition

MEMS Accelerometer Based Hand Gesture Recognition MEMS ACCELEROMETER BASED HAND GESTURE RECOGNITION Meenaakumari.M1, M.Muthulakshmi2 1Dept.of ECE, Sri Lakshmi Aammal Engineering College, Chennai, 2Asst.Prof, Dept.of ECE, Sri Lakshmi Aammal Engineering College, Chennai, Abstract This paper presents an MEMS accelerometer mostly based on gesture recognition algorithm and its applications. The hardware module consists of a triaxial mems accelerometer, microcontroller, and zigbee wireless transmission module for sensing and collecting accelerations of handwriting and hand gesture trajectories. Users will use this hardware module to write down digits, alphabets in digital kind by making four hand gestures. The accelerations of hand motions measured by the accelerometer are transmitted wirelessly to a personal computer for trajectory recognition. The trajectory algorithm composed of information assortment collection, signal preprocessing for reconstructing the trajectories to attenuate the cumulative errors caused by drift of sensors. So, by changing the position of MEMS (micro electro mechanical systems) we can able to show the alphabetical characters and numerical within the PC. Keywords MEMS accelerometer, gesture, handwritten recognition, trajectory algorithm. INTRODUCTION NOW A DAYS, the expansion of human machine interaction technologies in electronic circuits has been greatly reduced the dimension and weight of consumer electronics products such as smart phones and handheld computers, and therefore will increases our day to day convenience. Recently, an attractive alternative, a conveyable embedded device with inertial sensors, has been projected to sense the activities of human and to capture their motion trajectory information from accelerations for handwriting and recognizing gestures. The foremost necessary advantage of inertial sensors for general motion sensing is that they can be operated without any external reference and limitation in operating conditions. However, motion trajectory recognition is comparatively tough for different users since they have different speeds and styles to generate various motion trajectories. Thus, several researchers have tried to avoid the problem domain for increasing the  accuracy of handwriting recognition systems. During this work a miniature MEMS accelerometer based recognition systems which acknowledge four hand gestures in 3-D is constructed by using this four gestures, numerical and alphabets will be recognized in the digital format. MEMS are termed as micro electro mechanical system where mechanical parts like cantilevers or membranes have been manufactured at microelectronics circuits. It uses the technology known as micro-fabrication technology. It has holes, cavity, channels, cantilevers, membranes and additionally imitates mechanical parts. The emphasis on MEMS is based on silicon. The explanation that prompt that prompt the utilization of MEMS technology are for example miniaturization of existing devices, development of new devices based on principal that do not work at large scale and to interact with micro world. Miniaturization reduces cost by decreasing material consumption. It also increases applicability by reducing mass and size allowing placing the, MEMS in places where a traditional system. Instead of having a series of external components connected by wire or soldered to printed circuit board the MEMS on silicon can be integrated directly with the electronics. These are called smart integrated ME MS already include data acquisition, filtering, data storage, communication interfacing and networking. MEMS technology not only makes the things smaller but often makes them better. A typical example is brought by the accelerometer development. An accelerometer is a device that measures the physical acceleration. The physical parameters are temperature, pressure, force, light etc. it measures the weight per unit mass. By contrast, accelerometers in free fall or at rest in outer space will measure zero. Another term for the type of acceleration that accelerometers can measure is g-force. It works on the principle of displacement of a small proof mass etched into the silicon surface of the integrated circuit and suspended by small beams. RELATED WORK There are mainly two existing types of gesture recognition methods, i.e., vision-based and accelerometer and/or gyroscope based. Due to some limitations like ambient optical noise, slower dynamic response, and relatively large data collections/processing of vision-based method [1], our recognition system is implemented based on an inertial measurement unit based on MEMS acceleration sensors. If gyroscopes are used for inertial measurement [2] it causes heavy computational burden, thus our system is based on MEMS accelerometers only and gyroscopes are not implemented. Many researchers have focused on developing effective algorithms for error compensation of inertial sensors to improve the recognition accuracy. For few examples, Yang et al. [3] proposed a pen-type input device to track trajectories in 3-D space by using accelerometers and gyroscopes. An efficient acceleration error compensation algorithm based on zero velocity compensation was developed to decrease the acceleration err ors for acquiring accurate reconstructed trajectory. An extended Kalman filter with magnetometers (micro inertial measurement unit (ÃŽÂ ¼IMU) with magnetometers), proposed by Luo et al. [10], was employed to compensate the orientation of the proposed digital writing instrument. If the orientation of the instrument was estimated precisely, the motion trajectories of the instrument were reconstructed accurately. Dong et al. [4] proposed an optical tracking calibration method based on optical tracking system (OTS) to calibrate 3-D accelerations, angular velocities, and space attitude of handwriting motions. The OTS was developed for the following two goals: 1) to obtain accelerations of the proposed ubiquitous digital writing instrument (UDWI) by calibrating 2-D trajectories and 2) to obtain the accurate attitude angles by using the multiple camera calibration. However, in order to recognize or reconstruct motion trajectories accurately, the aforementioned approaches introduce other sensors such as gyroscopes or magnetometers to obtain precise orientation. This increases additional cost for motion trajectory recognition systems as well as computational burden of their algorithms. In this paper, a portable device has been developed with a trajectory recognition algorithm. The portable device consists of a triaxial accelerometer, a microprocessor, and an zigbee wireless transmission module. The acceleration signals measured from the triaxial accelerometer are transmitted to a computer via the zigbee wireless module. Users can utilize this portal device to write digits and make hand gestures at normal speed. The measured acceleration signals of these motions can be recognized by the trajectory recognition algorithm. The recognition procedure is composed of acceleration acquisition, signal preprocessing, feature generation, feature selection, and feature extraction. The acceleration signals of hand motions are measured by the portable device. The signal preprocessing procedure consists of calibration, a moving average filter, a high-pass filter, and normalization. First, the accelerations are calibrated to remove drift errors and offsets from the raw signals. The se two filters are applied to remove high frequency noise and gravitational acceleration from the raw data, respectively. The features of the preprocessed acceleration signals of each axis include mean, correlation among axes, interquartile range (IQR), mean absolute deviation (MAD), root mean square (rms), VAR, standard deviation (STD), and energy. Before classifying the hand motion trajectories, we perform the procedures of feature selection and extraction methods. In general, feature selection aims at selecting a subset of size m from an original set of d features (d > m). Therefore, the criterion of kernel-based class separability (KBCS) with best individual N (BIN) is to select significant features from the original features (i.e., to pick up some important features from d) and that of linear discriminate analysis (LDA) is to reduce the dimension of the feature space with a better recognition performance (i.e., to reduce the size of m). The objective of the feature selection an d featureextraction methods is not only to eradicate the burden of computational load but also to increase the accuracy of classification. The reduced features are used as the inputs of classifiers. The contributions of this paper include the following: 1) the development of a portable device with a trajectory recognition algorithm, i.e., with the hardware module , can give desired commands by hand motions to control electronics devices anywhere without space limitations, and 2) an effective trajectory recognition algorithm, i.e., the proposed algorithm can efficiently select significant features from the time and frequency domains of acceleration signals and project the feature space into a smaller feature dimension for motion recognition with high recognition accuracy. III.HARDWARE DESIGN OF  PORTABLE DEVICE The portable device consists of a triaxial accelerometer (MMA2240), a microcontroller (C8051F206 with a 12-b A/D converter), and a wireless transceiver (nRF2401, Nordic). The triaxial accelerometer measures the acceleration signals generated by a users hand motions. The microcontroller collects the analog acceleration signals and converts the signals to digital ones via the A/D converter. The wireless transceiver transmits the acceleration signals wirelessly to a personal computer (PC).The MMA2240 is a low-cost capacitive micro machined accelerometer with a temperature compensation function and a g-select function for a full-scale selection of +_}2 g to +_}6 gand is able to measure accelerations over the bandwidth of 0.5 kHz for all axes. The accelerometers sensitivity is set from à ¢Ã‹â€ Ã¢â‚¬â„¢2 g to +2 g. The C8051F206 integrates a high-performance 12-b A/D converter and an optimized signal cycle 25-MHz 8-b microcontroller unit (MCU) (8051 instruction set compatible) on a signal chip. The output signals of the accelerometer are sampled at 100 Hz by the 12-b A/D converter. Then, all the data sensed by the accelerometer are transmitted wirelessly to a PC by an zigbee transceiver at 2.4-GHz transmission band with 1-Mb/s transmission rate. The overall power consumption of the digital pen circuit is 30 mA at 3.7 V. The block diagram of the portable device is shown in Fig. 1. MEMS PIC ACCELEROMTER MICROCON ZIGBEE TX TROLLER PC RS 232 ZIGBEE RX Fig.1. Block diagram of the portable device. IV. TRAJECTORY RECOGNITION ALGORITHM The proposed trajectory recognition algorithm consisting of acceleration acquisition, signal preprocessing, feature generation, feature selection, and feature extraction. In this paper, the motions for recognition include Arabic numerals alphabets. The acceleration signals of the hand motions are measured by a triaxial accelerometer and then preprocessed by filtering and normalization. Consequently, the features are extracted from the preprocessed data to represent the characteristics of different motion signals, and the feature selection process based on KBCS picks p features out of the original extracted features. To reduce the computational load and increase the recognition accuracy of the classifier, LDA is utilized to decrease the dimension of the selected features. The reduced feature vectors are then fed into a PNN classifier to recognize the motion to which the feature vector it belongs. A. Signal Preprocessing The microcontroller collects the acceleration signals of hand motions which are generated by the accelerometer. Due to slight tremble movement of hand certain amount of noise is generated. The signal preprocessing consists of calibration, a moving average filter, a high-pass filter, and normalization. First, the accelerations are calibrated to remove drift errors and offsets from the raw signals. The second step of the signal preprocessing is to use a moving average filter to reduce the high-frequency noise of the calibrated accelerations, and the filter is expressed as where x[t] is the input signal, y[t] is the output signal, and N is the number of points in the average filter. In this paper, we set N = 8. The decision of using an eight-point moving average filter is based on our empirical tests. Then, a high-pass filter is used to remove the gravitational acceleration from the filtered acceleration to obtain accelerations caused by hand movement. In general, the size of samples of each movement between fast and slow writers is different. Therefore, after filtering the data, we first segment each movement signal properly to extract the exact motion interval. Then, we normalize each segmented motion interval into equal sizes via interpolation. B. Feature Generation The characteristics of different hand movement signals can be obtained by extracting features from the preprocessed x-, Fig 2 Block diagram of the trajectory recognition algorithm. 5) Correlation among axes: The correlation among axes is computed as the ratio of the covariance to the product of the STD for each pair of axes. For example, the correlation (corrxy) between two variables x on x-axis and y on y-axis is defined as where E represents the expected value, à Ã†â€™x and à Ã†â€™x are STDs, and mx and my are the expected values of x and y, respectively. 6)MAD 7)rms Y-, and z-axis signals, and we extract eight features where xi is the acceleration instance and m is from the triaxial acceleration signals, including mean, the mean value of xi in (6) to (7). STD, VAR, IQR [6], correlation between axes [7], MAD, rms, and energy [8] . They are explicated as follows. 8) Energy: Energy is calculated as the sum of 1) Mean: The mean value of the acceleration the magnitudes of squared discrete fast signals of each hand motion is the dc Fourier transform (FFT) components of the component of the signal signal in a window. The equation is defined as where W is the length of each hand motion. 2) STD: STD is the square root of VAR where Fi is the ith FFT component of the window and |Fi| is the magnitude of Fi. C. Feature Selection Feature selection comprises a selection criterion. The KBCS can be computed as follows: Let (x, y) (Rd ÃÆ'- 3) VAR Y) represents a sample, where Rd denotes a d dimensional feature space, Y symbolizes the set of class labels, and the size of Y is the number of class c. This method projects the samples onto a kernel space, where xi is the acceleration instance and m is and m i is defined as the mean vector for the I th class in the kernel space, ni denotes the number of the mean value of xi in (3) and (4). samples in the ith class, m denotes the mean vector 4) IQR: When different classes have similar for all classes in the kernel space, S B denotes the between-class scatter matrix in the kernel space, and mean values, the interquartile range S/ Wdenotes the within-class scatter matrix in the represents the dispersion of the data and kernel space. Let (à £Ã†â€™Ã‚ ») be a possible nonlinear eliminates the influence of outliers in the mapping from the feature space Rd to a kernel space data. ÃŽÂ º and tr(A) represents the trace of a square matrixA. 1889 www.ijarcet.org ISSN: 2278 1323 International Journal of Advanced Research in Computer Engineering Technology (IJARCET) Volume 2, No 5, May 2013 The following two equations are used in the class separability measure: 1 2 3 4 The class separability in the kernel space can be measured as To maintain the numerical stability in the maximization of J à ¢Ã‹â€ Ã¢â‚¬ ¦ , the denominator tr(Sà ¢Ã‹â€ Ã¢â‚¬ ¦ W ) has to be prevented from approaching zero. IV. EXPERIMENTAL RESULTS In this section, the effectiveness of trajectory recognition algorithm is validated. A.Handwritten Digit Recognition The acceleration signals after the signal preprocessing procedure of the proposed trajectory recognition algorithm for the digit 0. The calibrated acceleration signals acquired from the accelerometer module are shown. With the preprocessed accelerations, alphabets and numerical features are generated by the feature generation procedure. Subsequently, the KBCS is adopted to choose characteristic features from the generated features. We choose digits 0 and 6 to illustrate the effectiveness of the KBCS, since their accelerations and handwritten trajectories are pretty similar and difficult to classify. The IQR features of these two digits are closely overlapped. Thus, the features are not effective for 1 2 3 4 5 6 7 8 9 10 Fig. 4. Trajectories of four hand gestures. corrxy, meanz, energyx, energyy, and energyz selected by the KBCS. Finally, the dimension of the selected features was further reduced by the LDA not only to ease the burden of computational load but also to increase the accuracy of classification. Fig. 5.a Trajectories of alphabets Fig. 5.b. Trajectories of alphabets. Fig. 6. IQR features of (red star) digit 0 and (blue diamond) digit 6. Fig. 3. Generation of numerical 1890 www.ijarcet.org ISSN: 2278 1323 International Journal of Advanced Research in Computer Engineering Technology (IJARCET) Volume 2, No 5, May 2013 Fig. 6.a. Mean feature of (red star) digit 0 and digit (blue diamond) 6. Therefore, the total testing samples were 100 (10 ÃÆ'- 10 ÃÆ'- 1) for the testing procedure, and the total training samples were 900 (10 ÃÆ'- 10 ÃÆ'- 9) for the raining procedure. Because there are ten digits needed to be classified, the maximum of the dimension of the feature extraction by the LDA was nine. To see the performance variation caused by feature dimensions, we varied the dimensions of the LDA from one to nine. In Fig. 10, the best average recognition rate of Fig. 7. Average recognition rates versus the feature dimensions of the PNN classifier by using the LDA. Fig. 8. Average recognition rates versus the feature dimensions of the PNN classifier by using the KBCS. V. CONCLUSION The development of a portable device, is used to generate desired commands by hand motions to control electronic devices without space limitations. The time and frequency domains of acceleration signals of motion recognition, which has high recognition accuracy. The acceleration made by the hand gesture is measured by accelerometer are wirelessly transmitted to computer. In the experiments, we used 2-D handwriting digits, alphabets by using four hand gestures to validate the effectiveness of the proposed device and algorithm. The overall handwritten digit recognition rate was 98%, and the gesture recognition rate was also 98.75%. This result encourages us to further investigate the possibility of using our digital pen as an effective tool for HCI applications. In this project, an additional button can be used to allow users to indicate the starting point and ending point of motion. That is, the limitation of the proposed trajectory recognition algorithm is that it can only recognize a letter or a number finished with a single stroke. VI. FUTURE ENHANCEMENT The algorithms can be developed for letters or words with multistrokes which involve more challenging problems. REFERENCES S. Zhou, Q. Shan, F. Fei, W. J. Li, C. P. Kwong, and C. K. Wu et al.,Gesture recognition for interactive controllers using MEMS motion sensors, in Proc. IEEE Int. Conf. Nano/Micro Engineered and MolecularSystems, Jan. 2009,pp. 935-940. S. Zhang, C. Yuan, and V. Zhang, Handwritten character recognition using orientation quantization based on 3-D accelerometer, presented at the 5th Annu. Int. Conf. Ubiquitous Systems, Jul. 25th, 2008. J. Yang, W. Chang, W. C. Bang, E. S. Choi, K. H.Kang, S. J. Cho, and D. Y. Kim, Analysis and compensation of errors in the input device based on inertial sensors, in Proc. IEEE Int. L. Wang, Feature selection with kernel class separability, IEEE Trans.Pattern Anal. Mach. Intell., vol. 30, no. 9, pp. 1534-1546, Sep. 2008. Z. Dong, U. C. Wejinya, and W. J. Li, An optical-tracking calibration method for MEMS-based digital writing instrument, IEEE Sens. J.,vol. 10, no. 10, pp. 1543-1551, Oct. 2010. J. S.Wang, Y. L. Hsu, and J. N. Liu, An inertial-measurement-unit-based pen with a trajectory reconstruction algorithm and its applications, IEEE Trans. Ind. Electron., vol. 57, no. 10, pp. 3508-3521, Oct. 2010. S. Zhou, Z. Dong, W. J. Li, and C. P. Kwong, Hand-written character recognition using   MEMSmotionsensingtechnology,in Proc.IEEE/ASME Int. Conf. Advanced Intelligent Mechatronics, 2008, pp.1418-1423. J. K. Oh, S. J. Cho, and W. C. Bang et al., Inertial sensor based recognition of 3-D character gestures with an ensemble of classifiers, presented at the 9th Int. Workshop on Frontiers in Handwriting Recognition, 2004. Conf. Inf.Technol.-Coding and Computing,2004, pp. 790-796. Y. Luo, C. C. Tsang, G. Zhang, Z. Dong, G. Shi, Y. Kwok, W. J. Li, P. H. W. Leong, and M. Y. Wong, An attitude compensation technique for a MEMS motion sensor based digital writing instrument, in Proc.IEEE Int. Conf. Nano/Micro Eng. Mol. Syst., 2006, pp. 909-914. Z. Dong, G. Zhang, Y. Luo, C. C. Tsang, G. Shi, Y. Kwok, W. J. Li,P. H. W. Leong, and M. Y. Wong, A calibration method for MEMS inertial sensors based on optical tracking, in Proc. IEEE Int. Conf.Nano/Micro Eng. Mol. Syst., 2007, pp.542-547. S. J. Preece, J. Y. Goulermas, L. P. J. Kenney, and D. Howard, A comparison of feature extraction methods for the classification of dynamic activities from accelerometer data, IEEE Trans. Biomed. Eng., vol. 56,no. 3, pp.871-879, Mar. 2009. L. Bao and S. S. Intille, Activity recognition from user-annotated acceleration data, Pervasive, Lecture Notes in Computer Science,no. 3001,pp. 1-17, 2004. Y. P. Chen, J. Y. Yang, S. N. Liou, G. Y. Lee, and J. S. Wang, Online classifier construction algorithm for human activity detection using a triaxial accelerometer, Appl. Math. Comput., pp. 849-860, Nov. 2008. L. Wang, Feature selection with kernel class separability, IEEE Trans.Pattern Anal. Mach. Intell., vol. 30, no. 9, pp. 1534-1546, Sep. 2008. Z. Dong, U. C. Wejinya, and W. J. Li, An optical-tracking calibration method for MEMS-based digital writing instrument, IEEE Sens. ,vol. 10, no. 10, pp. 1543-1551, Oct. 2010. J. S.Wang, Y. L. Hsu, and J. N. Liu, An inertial-measurement-unit-based pen with a trajectory reconstruction algorithm and its applications, IEEE Trans. Ind. Electron., vol. 57, no. 10, pp. 3508-3521, Oct. 2010. S. Zhou, Z. Dong, W. J. Li, and C. P. Kwong, Hand-written character recognition using MEMS motion sensing technology, in Proc.IEEE/ASME Int. Conf. Advanced Intelligent Mechatronics, 2008, pp.1418-1423. J. K. Oh, S. J. Cho, and W. C. Bang et al., Inertial sensor based recognition of 3-D character gestures with an ensemble of classifiers, presented at the 9th Int. Workshop on Frontiers in Handwriting Recognition, 2004. A. H. F. Lam, W. J. Li, Y. Liu, and N. Xi, MIDS: Micro input devices system using MEMS sensors, presented at the IEEE/RSJ Int. Conf.Intelligent Robots an