Methodology & Results

Method

mmWave sensor triggers 150 frames over 10 seconds and captures data
Process mmWave data and perform 2-D Fast Fourier Transform (FFT)
Camera takes a picture in sync with mmWave sensor
Images are further processed using open-source project OpenPose [1] to be used as labels
Classification model is a teacher-student network similar to [2] that is composed of a Convolutional Neural Network (see Figure 2), using built-in Python packages
Model is further tested using dynamic data
Provides a human pose estimated figure performing activity and classification of activity

Results

We trained our classification model for 150 epochs with an Adam optimizer and a total of 1050 data samples. Our current model can classify amongst three different activities: stretching, raising dumbbells, and sitting down. The experiments for these activities have 450, 300, and 300 samples respectively.