Sensor fusion for motion detection and estimation

This project implemented machine learning strategies to determine a user's ambulation state through a locomotion mode classifier that feeds a continuous parameter estimator for level ground, ramp and stair modes. The system was applied over a very wide range of environmental parameters (6 ramp inclines, 4 stair heights and 28 walking speeds) to increase robustness and better represent the continuous nature of real-world environments.

Context

With an aging population and the prevalence of disabilities resulting from spinal cord injury, stroke or disease, proper rehabilitation therapy is a very important process for regaining mobility and independence and for reducing the risk of further health complications.

As technology becomes more sophisticated and portable, applying wearable sensors to rehabilitation has become a prevalent research field. These sensors have been used to provide additional information in clinical studies, as well as to improve the control of rehabilitation devices such as prostheses, orthoses and exoskeletons. The machine learning strategy presented in this study focuses on the latter application, although it could be adapted for monitoring patient activity.

Powered rehabilitation devices are designed to replicate or augment biological torques at the ankle, knee and hip joints using advanced sensing and control modes. Currently, many research devices operate with variable power and damping parameters to assist walking in various community ambulation modes – level ground, ramps and stairs. These control parameters are tuned manually for each ambulation mode based on patient comfort and performance feedback, a time-consuming process carried out during initial testing, usually under a single set of environmental parameters – walking speed, ramp incline or stair height.

Sensors

Various sensors, such as inertial measurement units (IMU), goniometers (GON) and electromyography (EMG) systems, can be used to collect data on body dynamics.

An inertial measurement unit (IMU) is a device that measures and reports an object's velocity, orientation and gravitational forces. IMUs typically combine accelerometers, gyroscopes and sometimes magnetometers to provide a complete representation of motion in three dimensions. A goniometer is a device that measures the angle of a joint, such as the elbow, knee or ankle. It can be manual, using a graduated scale and a movable arm, or electronic, providing more precise and easily recorded measurements. Electromyography (EMG) is a technique that measures the electrical activity produced by muscles when they contract. It uses electrodes placed on the skin (surface EMG) or inserted into the muscle (intramuscular EMG) to record these electrical signals.

More recent studies show promise in using EMG to predict human intent, online mode classification algorithms, and model-based torque control that leverages kinematic and force data, although these systems mostly assume a static environment. Because gait kinematics vary with each parameter of the locomotion environment, using static control parameters results in a device that cannot adapt to changing environments. This adaptability is also important for monitoring rehabilitation, since walking speed is a common outcome measure, and stair and ramp exercises improve muscle strength, balance and walking speed.

Machine learning

For adaptation to happen within a device's control architecture without user intervention, both an accurate estimate of the relevant environmental parameter and a classification of the user's ambulation mode are required. Locomotion mode classifiers have been widely analyzed in the field of rehabilitation robotics, the most common models being linear discriminant analysis (LDA) and support vector machines (SVM). Other models such as kNN, k-means and Gaussian mixture models have also been studied, but without significant improvements over LDA. A common technique to improve classification accuracy beyond the base model is to add temporal history information. This is typically implemented as a majority-vote filter, although studies have shown that a dynamic Bayesian network (DBN) – a type of hidden Markov model (HMM) that uses a probabilistic approach to incorporate information from past and current decisions based on the model's confidence in each decision – performs better than an LDA with majority voting.

Although locomotion type classification has been analyzed extensively, research on continuous parameter estimation using wearable sensors is comparatively scarce. The most common estimation methods are integration of IMU data, kinematic models and machine learning approaches. Although integration techniques are easily applied to speed and slope estimation, accurate height estimation is more error-prone because of the double integration required. Of the three estimation methods, kinematic-model-based methods are the most accurate for speed estimation; however, building kinematic models requires detailed analysis of each ambulation mode. ML methods for walking parameter estimation have recently received attention because the input-output nature of wearable sensors and walking parameters can be framed as a regression problem. This relationship lends itself well to a generalized approach for estimation across ambulation modes and easily expands to incorporate information from many sensors.

The process of building this system is detailed through the feature selection, parameter sweep, model selection and model optimization steps. We also aim to give designers of new and existing systems insight into which sensors matter most for each model, so that the effort of incorporating wearable sensors is used more effectively. Previous studies have shown that EMG information helps classification, since these signals relate to user intent. In contrast, we hypothesize that parameter estimation can be performed largely through mechanical sensors, since these sensors reflect the geometry and kinematics of each mode. The main contributions of this work include: the development and rigorous optimization of an ML approach for locomotion classification and environmental parameter regression; and a detailed analysis of the best sensor types and locations for four ML problems: ambulation mode classification, walking speed estimation, ramp incline estimation and stair height estimation.

Methods

Data from the wearable sensors are fed into a feature extractor, where different patterns are extracted depending on the sensor type and the user's gait phase. The features are first used in the intent classifier to determine the user's intent, and then the locomotion parameter to be estimated. The estimated values are then filtered by a Kalman filter. The final outputs of both the user intent classifier and the Kalman filter are fed back into their respective blocks as prior data for the next step.

Results

This study illustrates the construction of a combined locomotion mode classifier and locomotion parameter estimator to provide accurate information about a user's current ambulation state. Through the feature selection, parameter sweep, model selection and model optimization steps, the proposed system achieved error rates similar to or lower than the current state of the art for all four problems: mode classification, walking speed estimation, ground slope estimation and stair height estimation. This is despite evaluating the proposed system over a much wider range of environmental parameters than previous studies, which makes it more robust across the full domain of possible locomotion. The feature dataset and Matlab code are provided to facilitate reproducibility and future work. The sensor importance analysis provides valuable insight for designers, as there were significant differences in the effect of each sensor type across these four problems.

For mode classification, the proposed system averaged over 98% accuracy (0.70% steady-state error, 3.55% transition error). An important difference between the proposed system and prior literature is the presence of multiple environmental parameter conditions for each mode. Leveraging the robust dataset, this classifier was able to identify a wide range of inclines and heights as the correct mode. For parameter estimation, the root mean square error (RMSE) of the estimate was computed during steady-state locomotion.

The sensor importance analysis showed that mechanical sensors (IMU, GON) were generally more important than EMG sensors. This was especially true for regression, since no EMG features were selected for estimating any parameter. For classification, although the effect of removing EMG data was less pronounced than removing either type of mechanical sensor, removing EMG still caused a significant increase in error.

This study also showed that for parameter estimation, selecting the right sensor type is crucial. Goniometers were best for estimating ramp incline and stair height, while IMUs should be used for speed estimation.

The estimation improvements from Kalman filtering the neural network output also strongly suggest filtering regression outputs. In addition, future research could apply the proposed parameter estimation method to more environmental variables, such as user heading or clinical measures like step length, which could enable new device adaptation and better rehabilitation.