We present the BARD dataset (Basketball Action Recognition Dataset). It is designed to advance video action recognition in basketball through high-quality annotations and enriched contextual data. BARD improves upon existing datasets by including player jersey numbers, team colors and a novel output format supporting multi-label classification. To ensure annotation quality, we conducted a human validation study on a subsample of the annotations, with expert reviewers assessing the labeling quality and reporting the evaluation results, thereby providing human validated independent benchmarks. Moreover, in addition to standard caption-based action recognition metrics, we introduce Basketball Caption Evaluation Framework (BaCEF), a new application-oriented evaluation framework. Finally, to demonstrate the quality and challenging nature of the dataset, as well as the utility of our evaluation framework and its potential applications, we evaluate both proprietary models (e.g., Gemini 2.5 Pro) and open-source models (Qwen2.5-VL-7B-Instruct, Qwen2.5-VL-3B-Instruct), including BQwen2.5-VL-3B, a BARD fine-tuned variant of Qwen2.5-VL-3B-Instruct, across our defined benchmarks.
| Property | Value | Description |
|---|---|---|
| Season | 2024–2025 | Most updated season |
| Teams | 30 | Selected NBA teams |
| Games | 60 | Total number of games sampled |
| Initial clips | 24,692 | Raw video segments collected |
| Final clips | 14,676 | After filtering and consolidation |
| Resolution | 720p | High-definition video |
| Labels | Structured JSON | Multi-label format |
| Action recognition | Coarse and Event-level | Play-by-play annotation |
| New fields | Player numbers, team colors | Anonymous identification metadata |
Clip:
Green Tip Layup Shot (21 PTS)
[
{ "player": "00", "action": "2PT Shot", "result": false, "assisted": false, "other_player": null, "color": "blue" },
{ "player": "23", "action": "Rebound", "result": null, "assisted": null, "other_player": null, "color": "blue" },
{ "player": "23", "action": "2PT Shot", "result": false, "assisted": false, "other_player": null, "color": "blue" },
{ "player": "23", "action": "Rebound", "result": null, "assisted": null, "other_player": null, "color": "blue" },
{ "player": "23", "action": "2PT Shot", "result": true, "assisted": false, "other_player": null, "color": "blue" }
]
We have open-sourced the code for downloading the data and reproducing the results presented in our paper.
If you use BARD in your work, please cite the associated paper.
