MARS: mmWave-based Assistive Rehabilitation System for Smart Healthcare
This article appears as part of the ESWEEK-TECS special issue and will be presented in the International Conference on Hardware/Software Codesign and System Synthesis (CODES+ISSS), 2021. The proper publication information is in the reference section.
The figure below shows ten different movements we evaluate in our dataset.
They are:
1) Left upper limb extension
2) Right upper limb extension
3) Both upper limb extension
4) Left front lunge
5) Right front lunge
6) Squad
7) Left side lunge
8) Right side lunge
9) Left limb extension
10) Right limb extension
We also give an example demo of real-time joint angle estimation for left front lunge movement from mmWave point cloud:
Dataset
The folder structure is described as below.
${ROOT}
|-- synced_data
| |-- wooutlier
| | |-- subject1
| | | |-- timesplit
| | |-- subject2
| | | |-- timesplit
| | |-- subject3
| | | |-- timesplit
| | |-- subject4
| | | |-- timesplit
| |-- woutlier
| | |-- subject1
| | | |-- timesplit
| | |-- subject2
| | | |-- timesplit
| | |-- subject3
| | | |-- timesplit
| | |-- subject4
| | | |-- timesplit
|-- feature
|-- model
| |-- Accuracy
synced_data folder contains all data with outlier/without outlier. Under the subject folder, there are synced kinect_data.mat and radar_data.mat if the readers want to play with individual movements. Under timesplit folder, there are train, validate, the test data and labels for each user. Note that labels here have all 25 joints from Kinect. In the paper, we only use 19 of them. Please refer to the paper for details of the 19 joints.
feature folder contains train, validate, the test feature and labels for all users. The features are generated from the synced data.
Dimension of the feature is (frames, 8, 8, 5). The final 5 means x, y, z-axis coordinates, Doppler velocity, and intensity.
Dimension of the label is (frames, 57). 57 means 19 coordinates in x, y, and z-axis. The order of the joints is shown in the paper.
model folder contains the pretrained model and and recorded accuracy.
Dependencies
- Keras 2.3.0
- Python 3.7
- Tensorflow 2.2.0
Run the code
The code contains load data, compile model, training, and testing. Readers can also choose to load the pretrained model and just do the testing.
python MARS_model.py
License
A MIT license is used for this repository.
Reference
@article{10.1145/3477003,
author = {An, Sizhe and Ogras, Umit Y.},
title = {MARS: MmWave-Based Assistive Rehabilitation System for Smart Healthcare},
year = {2021},
issue_date = {October 2021},
volume = {20},
number = {5s},
issn = {1539-9087},
url = {https://doi.org/10.1145/3477003},
doi = {10.1145/3477003},
journal = {ACM Trans. Embed. Comput. Syst.},
month = sep,
articleno = {72},
numpages = {22},
keywords = {millimeter wave, Human pose estimation, point cloud, smart healthcare}
}