Introduction: Real-Time Push-Up & Squat Recognition With Form and Deepness Scoring

By
About: Student at Howest

This project is a complete AI-powered fitness tracker built from scratch. It recognizes exercises (push-ups and squats) in real-time using a webcam, calculates repetition count, evaluates form and depth using pose estimation, and displays feedback on a small LCD screen.

The system is optimized for low latency and minimal hardware overhead, running real-time inference and visual feedback with only a webcam, Raspberry Pi, and a display module.

Supplies

Supplies

Hardware

  1. Raspberry Pi 5 with active cooling (fan or heatsink)
  2. Logitech C270 USB webcam
  3. LCD1602 I2C display module
  4. Jumper wires
  5. 5V/3A USB-C power supply for Raspberry Pi
  6. MicroSD card (16GB or more)

Software and Tools

  1. Python 3 (on Pi and laptop)
  2. OpenCV for image processing
  3. MediaPipe or MoveNet for pose estimation
  4. YOLOv8 (Ultralytics) for exercise classification
  5. Roboflow for dataset annotation
  6. Jupyter Notebook for training
  7. Git + VS Code or any preferred IDE

Step 1: Define the Project Goal and Requirements

The goal is to create a real-time exercise recognition system that:

  1. Detects squats and push-ups
  2. Counts repetitions
  3. Evaluates form and depth
  4. Displays feedback on an LCD1602

Tools Required: Raspberry Pi 5 (139.90€), webcam (Logitech C270 (24.99€)), LCD1602 I2C(8.95€), Python, YOLOv8, pose model (MediaPipe, MoveNet, BlazePose...).

Step 2: Collect the Dataset

  1. Gathering Data:
  2. Collected a variety of push-up and squat images from public datasets on Kaggle and Roboflow.
  3. Additionally recorded some custom footage using a webcam to increase variation and improve generalization.
  4. Labeling:
  5. Labeled the data using Roboflow (if needed).
  6. Defined two classes: pushup and squat.
  7. Exported labeled data in YOLO format, ensuring consistent folder structure and naming (train/val/test with /pushup and /squat in all of them).


Step 3: Train YOLOv11 Classifier

  1. Environment Setup:
  2. Used a Jupyter Notebook for step-by-step monitoring.
  3. Installed dependencies and used CUDA for faster training
  4. Model Training:
model = YOLO('yolov11n-cls.pt') # lightweight for faster inference
model.train(data=data_path, epochs=40, imgsz=640)
  1. Export and Test:
  2. Saved the best model as exercise_classifier.pt
  3. Verified performance on test images and confusion matrix.


Step 4: Write Core System Components


  1. Server (Laptop):
  2. Handles classification and pose analysis.
  3. Receives compressed frames from client.
  4. Runs inference using YOLO model and pose model.
  5. Sends back exercise name, rep count, form score, and depth analysis.
  6. Client (Raspberry Pi):
  7. Captures frames using the webcam.
  8. Sends them to the server.
  9. Receives feedback and displays it on the LCD.
  10. Camera Handler Module:
  11. Designed to handle video capture using OpenCV.
  12. Includes error logging, resolution control, and compression.
  13. LCD Display Logic:
  14. Created a dedicated class for LCD control.
  15. Displayed current rep count, detected exercise, and form score in real time.


Step 5: Implement the Analysis Engine

  1. YOLO Classification:
  2. Used the YOLO model to detect whether the person is doing a push-up or a squat in each frame.
  3. Smoothed results using frame intervals to improve stability.
  4. Pose Estimation with MediaPipe (or MoveNet):
  5. Extracted keypoints such as hips, knees, shoulders.
  6. Wrote mathematical functions to calculate:
  7. Joint angles (e.g., knee bend)
  8. Vertical displacement to determine depth
  9. Smooth rep detection logic using Savitzky-Golay filter based on angle thresholds and movement patterns
  10. Repetition Counter:
  11. Implemented state machine logic to count reps only when full motion is completed.
  12. Applied filters to avoid miscounts from jitter or false poses.
  13. Output Integration:
  14. Combined results (exercise type + form score + depth) and sent them back to the Raspberry Pi client.
  15. Displayed feedback on the LCD in real time

Step 6: Final Testing and Debugging

  1. Ensured network communication over TCP
  2. Tested webcam capture speed and server processing latency
  3. Verified LCD updates with correct rep count and exercise form feedback
  4. Adjusted thresholds for squats and push-ups based on live testing