Multi-Stage Verification-Centric Framework for Mitigating Hallucination in Multi-Modal RAG

Abstract

This paper presents the technical solution developed by team CRUISE for the KDD Cup 2025 Meta Comprehensive RAG Benchmark for Multi-modal, Multi-turn (CRAG-MM) challenge. The challenge aims to address a critical limitation of modern Vision Language Models (VLMs): their propensity to hallucinate, especially when faced with egocentric imagery, long-tail entities, and complex, multi-hop questions. This issue is particularly problematic in real-world applications where users pose fact-seeking queries that demand high factual accuracy across diverse modalities. To tackle this, we propose a robust, multi-stage framework that prioritizes factual accuracy and truthfulness over completeness. Our solution integrates a lightweight query router for efficiency, a query-aware retrieval and summarization pipeline, a dual-pathways generation and a post-hoc verification. This conservative strategy is designed to minimize hallucinations, which incur a severe penalty in the competition’s scoring metric. Our approach achieved 3rd place in Task 1, demonstrating the effectiveness of prioritizing answer reliability in complex multi-modal RAG systems. Our implementation is available at https://github.com/Breezelled/KDD-Cup-2025-Meta-CRAG-MM.

BibTeX Citation

@inproceedings{chen2025multistage,
    title={Multi-Stage Verification-Centric Framework for Mitigating Hallucination in Multi-Modal {RAG}},
    author={Baiyu Chen and Wilson Wongso and Xiaoqian Hu and Yue Tan and Flora D. Salim},
    booktitle={2025 KDD Cup Workshop for Multimodal Retrieval Augmented Generation},
    year={2025},
    url={https://openreview.net/forum?id=NTsCbT2jAb}
}

Recommended citation: Chen, B., Wongso, W., Hu, X., Tan, Y., & Salim, F. (2025). Multi-Stage Verification-Centric Framework for Mitigating Hallucination in Multi-Modal RAG. In Proceedings of 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD ’25).
Download Paper