As a self-proclaimed training nerd, the temporary closure of gyms in Ireland during the pandemic gave me an opportunity to expand my knowledge of weightlifting/powerlifting technique. I ordered the renowned strength training book 'Starting Strength' by Mark 'Rip' Rippetoe. Rip placed an emphasis on the importance of checking the path of the barbell in your lifts. This can highlight possible inefficiencies, as moving the bar away from your centre of gravity introduces extra forces for you to work against. While learning about this area of technique analysis, one quote stuck with me:

"There are several advanced movement-analysis instruments that record and interpret bar path information"

Knowing the advancements made in mobile technology since the release of this book, I went on to the android Play Store and downloaded a number of apps which attempted to provide bar-path tracking functionality. I was disappointed to find that they had an average rating of around 2 stars and didn't seem to make use of the power of machine learning. I figured I could make something better and decided to make my own bar-path tracking android app. I knew that if I could train an object detection model to find the position of weight-plates in a photo, the centre of these plates would be the position of the bar.

Bumps in the Road

This project was doable but difficult. I will include a number of problems I faced and their relative solutions.

Finding the Right Data

Initially I was using frames from videos of my friends and myself to train my object detection model. I soon realised that the small number of videos wouldn't provide a wide enough variety of images. After scouring Instagram and YouTube to no avail, I finally found the holy grail of lifting videos - Reddit. There are a number of subreddits where people help eachother with their lifting form. I downloaded 200 videos and extracted a frame from each with the help of some Python scripts. I could not fully automate the process though, because I needed to ensure that the frames collected had an adequate side-view of the lift.

Money

As a college student, spending all of my money on expensive GPUs or costly cloud services would leave me with no funds left for mint-chocolate protein. With regards to training my model, I was able to use the free resources of Google Colab. There were some limitations, but scarcity often leads to creativity. As for inference on users' videos, a faster and more accurate approach would have been to use REST/gRPC with an online server. This is still very expensive for machine learning models so I decided to perform all inference on-device.

Accuracy of the Model

Having decided to perform inference on-device, the most practical solution was to do transfer learning using TensorFlow Lite with an SSD Mobilenet checkpoint. These small models sacrifice accuracy for speed and didn't perform to the standard I had hoped for. My solution was to prompt the user to select the area in which the plates are moving. Inference is then performed on this restricted area, blocking out any other noise in the rest of the frame. This approach vastly improved results and made the tool more robust.

Video Processing on Android

Phones are generally still quite limited in their speeds for video processing, which is necessary for performing inference on each frame of a video. I used the FFmpeg library for trimming, decoding and encoding the users' videos. Initially this was very slow but I managed to halve the processing time by choosing to reduce the video resolution of the output.

BarPath

Time to show you the finished product. BarPath was launched on the Play Store on June 12th 2020. It is a completely free app and I won't be spamming you with Candy Crush ads either. My only satisfaction will come from people getting use and enjoyment out of the app. My metric of project success will be user ratings.

Here are some demos of the app:

And here are the results from some videos of people who are way better lifters than I am:

Well, what do you think? Do you like it? Do you loath it entirely? Please let me know! It is difficult to get feedback during the pandemic so all comments and messages will be appreciated. Furthermore, if you have any ideas you would like to discuss, I would be very interested to hear them.

Link to the app: play.google.com/store/apps/details?id=com.d..