AVA Action Recognition (Undergraduate Thesis)

Machine Learning, 2025 to 2026. Role: Model architecture, training and ablations (team of three).

Code on GitHub

A lightweight model for AVA-style action recognition: for every person in a short video clip it predicts which of the 80 AVA actions they are doing. It was the team's undergraduate thesis at AIUB, built to stay small enough for edge devices.

Highlights

• X3D-XS video backbone with RoIAlign over 8 frames per person
• YOLOv8s person boxes cached once, then linked into tracks by IoU matching
• Five variants trained 20 epochs each on the same split: 9.61% to 11.80% validation mAP
• Box-geometry gate alone added about 1.5 points over the keyframe baseline
• About 3.15 GFLOPs and over 90 FPS on one consumer GPU, as reported in the thesis
• Team of three; thesis graded A+

Main challenge

Video action models are usually heavy, and a single keyframe misses motion. The model had to use several frames per person and know where each person is, while staying cheap to run. Only part of AVA 2.1 could be downloaded, so every comparison had to be made on that subset.

How it was solved

People are detected once with YOLOv8s, cached, and linked into 8-frame tracks with IoU matching. A frozen X3D-XS backbone gives video features; RoIAlign pools each person's features frame by frame and a learned scorer weights the frames. A small network turns the box geometry into a gate on the pooled features. Each component was added in its own run and measured against the previous one.

Outcomes

• Validation mAP rose from 9.61% (keyframe only) to 11.80% (final variant)
• The geometry gate alone added about 1.5 points over the keyframe baseline
• Thesis graded A+

Built with

• Python
• PyTorch
• Computer Vision
• Action Recognition
• X3D
• YOLOv8
• RoIAlign
• TensorBoard