Research Article
SafePLUG: empowering multimodal LLMs with pixel-level insight and temporal grounding for traffic accident understanding
Chain | Vol.3, Issue 1, 2026 |
pp. 53-72
Multimodal Large Language Models (MLLMs) have achieved remarkable progress across
a range of vision-language tasks and demonstrate strong potential for traffic accident
understanding. However, existing MLLMs in this domain primarily focus on coarse-grained
image-level or video-level comprehension and often struggle to handle fine-grained
visual details or localized scene components, limiting their applicability in complex
accident scenarios. To address these limitations, we propose SafePLUG, a novel framework
that empowers MLLMs with both pixel-level understanding and temporal grounding for
comprehensive traffic accident analysis. SafePLUG supports both arbitrary-shaped visual
prompts for region-aware question answering and pixel-level segmentation based on
language instructions, while also enabling the recognition of temporally anchored
events in traffic accident scenarios. To advance the development of MLLMs for traffic
accident understanding, we curate a new dataset, SafePLUG-Bench, which contains diverse
multimodal question–answer pairs with detailed pixel-level annotations and temporal
event boundaries across a wide range of accident scenarios. Experimental results show
that SafePLUG achieves strong performance on multiple tasks, including region-based
question answering, pixel-level segmentation, temporal event localization, and accident
event understanding. These capabilities lay a foundation for a fine-grained understanding
of complex traffic scenes, with the potential to improve driving safety and enhance
situational awareness in smart transportation systems.
DOI: 10.23919/CHAIN.2026.000005 Cited: 5 Download: 2
