SensAI

SensAI is a yoga classification app. We collaborated with a start-up, YogiFi, which manufactures smart yoga mats, to improve its yoga pose classification abilities. Using the heatmap data collected from the smart mat, in addition to landmark keypoints extracted from a yoga pose, the app is able to classify the yoga poses accurately.

Motivation

To encourage fitness enthusiasts to utilize a device equipped with sensors, such as a smartphone with camera or a pressure sensitive mat, that can collect valuable data from the user during training sessions and then use this data to analyze posture, and provide corrective feedback.  

Yoga is about 5000 years old and is one of the ancient and proven forms of exercise for gaining fitness. It is the most used complementary health approach in the US. The yoga industry is projected to reach $215 billion by 2025. In spite of this, there is a lack of on-demand access to quality feedback while performing yoga.

YogiFi is a start-up that manufactures smart yoga mats and uses the pressure data collected from its smart mat to analyze posture, and provide corrective feedback to the practitioner. A limitation with the data from the smart mat is that there are many yoga poses that have the same heatmap signature and hence, being able to distinctly classify the yoga poses becomes a challenge. SensAI addresses this classification problem by evaluating additional deep learning models that can classify the poses distinctly using just the smart data. In addition, SensAI aims to improve the classification accuracy using landmark keypoints extracted from the yoga pose along with the heat map data.

Data Source

The primary data source was the priorietary heatmap data that was shared by YogiFi. Additionally, we use landmark keypoints extracted from Yoga-82 dataset using MoveNet to train a multi-modal neural network model.

Modeling

We use denoising techniques to remove noise from the heatmap data. We then extract the contours of the heatmap using OpenCV library which allows the heatmap signature to be location invariant . We train an EfficientNet-B0 convolutional neural network using the transformed heatmap data to classify the yoga poses. We also train multi-modal models using the heatmap data along with the landmark keypoints extracted from Yoga-82 using MoveNet. We use early fusion and late fusion techniques to train the multi-modal models.

Acknowledgements

We thank our instructors Fred Nugen and Ramesh Sarukkai for their feedback and guidance. We are grateful to Muralidhar Somisetty and Sankar Dasiga from YogiFi for providing us with its proprietary training data and a smart yoga mat to test.