Education, Science, Technology, Innovation and Life
Open Access
Sign In
  1. Home
  2. Journals
  3. Journal of Electronics and Information Science
  4. Vol 11, Issue 1, 2026

DIVE: A Training-free Hallucination Mitigation Mechanism for Complex Scenes

Download as PDF

DOI: 10.23977/jeis.2026.110108 | Downloads: 0 | Views: 26

Author(s)

Shuguo Jiang 1

Affiliation(s)

1 School of Cyberspace Security, Hangzhou Dianzi University, Hangzhou, China

Corresponding Author

Shuguo Jiang

ABSTRACT

When facing real-world scenes that are densely populated with objects or contain complex occlusions, Vision-Language Models are often constrained by the language prior in autoregressive decoding, producing severe hallucination phenomena. To address this pain point that limits the reliable deployment of large models, this paper proposes Dual-branch Inference for Visual-prior Elimination, a training- free hallucination mitigation mechanism for complex scenes. By constructing a dual-branch inference structure at the inference stage and introducing a dynamic visual-confidence penalty, this mechanism effectively quantifies and suppresses the overconfidence in the content generation process, forcing the model's output to be deeply aligned with the underlying visual features. Results on the object hallucination evaluation benchmark POPE show that, without consuming computing power for model fine-tuning, the proposed method reduces the model's hallucination rate when objects are dense or complex occlusions exist, and brings a slight improvement in the question-answering accuracy of the model on the MSCOCO and VG datasets. 

KEYWORDS

Language prior; Hallucination mitigation; Feature alignment

CITE THIS PAPER

Shuguo Jiang. DIVE: A Training-free Hallucination Mitigation Mechanism for Complex Scenes. Journal of Electronics and Information Science (2026). Vol. 11, No. 1, 59-66. DOI: http://dx.doi.org/10.23977/10.23977/jeis.2026.110108.

REFERENCES

[1] Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., Krueger, G. and Sutskever I. (2021) Learning transferable visual models from natural language supervision. In 2021 International conference on machine learning, PMLR, pp. 8748–8763.
[2] Jia, C., Yang, Y.F., Xia, Y., Chen, Y.T., Parekh, Z., Pham, H., Le, Q., Sung, Y.H., Li, Z. and Duerig, T. (2021) Scaling up visual and vision-language representation learning with noisy text super-vision. In 2021 International conference on machine learning, PMLR, pp. 4904–4916.
[3] Naveed, H., Ullah Khan, A., Qiu, S., Saqib, M., Anwar, S., Usman, M., Akhtar, N., Barnes, N. and Mian, A. (2025) A comprehensive overview of large language models. ACM Trans. Intell. Syst. Technol., 16(5), 1–72.
[4] Zhang, J., Huang, J., Jin, S. and Lu, S. (2024) Vision-language models for vision tasks: A survey. IEEE transactions on pattern analysis and machine intelligence, 46(8), 5625–5644.
[5] Li, J., Li, D., Savarese, S. and Hoi, S. (2023) Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. In 2023 International conference on machine learning, PMLR, 19730–19742.
[6] Liu, H., Li, C., Wu, Q. and Lee, Y.J. (2023) Visual instruction tuning. Advances in neural information processing systems, 36, 34892–34916.
[7] Bai, J.Z., Bai, S., Yang, S.S., Wang, S.J., Tan, S.N., Wang, P., Lin, J.Y., Zhou, C. and Zhou, J.R. (2023) Qwen-vl: A versatile vision-language model for understanding, localization, text reading, and beyond. arXiv:2308.12966 [cs.CV]. 
[8] Huang,, L., Yu, W.J., Ma, W.T., Zhong, W.H., Feng, Z.Y., Wang, H.T. Chen, Q.L., Peng, W.H., Feng, X.C., Qin, B. and Liu, T. (2025) A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions. ACM Transactions on Information Systems, 43(2), 1–55.
[9] Gunjal, A., Yin, J. and Bas, E. (2024) Detecting and preventing hallucinations in large vision language models. In Proceedings of the Thirty-Eighth AAAI Conference on Artificial Intelligence, 38, 18135– 18143.
[10] Liu, F., Lin, K., Li, L., Wang, J., Yacoob, Y. and Wang, L. (2024) Mitigating hallucination in large multimodal models via robust instruction tuning. In International Conference on Learning Representations 2024, 57689–57733.
[11] Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C.L., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., Schulman, J., Hilton, J., Kelton, F., Miller, L., Simens, M., Askell, A., Welinder, P., Christiano, P., Leike, J. and Lowe R. (2022) Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35, 27730–27744.
[12] Rafailov, R., Sharma, A., Mitchell, E., Manning, C.D., Ermon, S. and Finn, C. (2023) Direct preference optimization: Your language model is secretly a reward model. Adv. Neural Inf. Process. Syst., 36, 53728–53741.
[13] Yin, S., Fu, C., Zhao, S., Xu, T., Wang, H., Sui, D., Shen, Y., Li, K., Sun, X. & Chen, E. (2024) Woodpecker: Hallucination correction for multimodal large language models. Science China Information Sciences, 67(12), 220105.
[14] Lee. N., Ping, W., Xu, P., Patwary, M., Fung, P.N., ShoeybI, M. and Catanzaro, B. (2022) Factuality enhanced language models for open-ended text generation. Adv. Neural Inf. Process. Syst., 35, 34586–34599.
[15] Li, K., Patel, O., Viégas, F., Pfister, H. and Wattenberg, M. (2023) Inference-time intervention: Eliciting truthful answers from a language model. Adv. Neural Inf. Process. Syst., 36, 41451–41530.
[16] Chuang, Y.S., Xie, Y., Luo, H., Kim, Y., Glass, J.R. and He, P. (2024) Dola: Decoding by contrasting layers improves factuality in large language models. In International Conference on Learning Representations 2024, 54158–54183.
[17] Leng, S.C., Zhang, H., Chen, G.Z., Li, X., Lu, S.J., Miao, C.Y. and Bing, L.D. (2024) Mitigating object hallucinations in large vision-language models through visual contrastive decoding. In Proceedings of the 2024 IEEE/CVF CVPR, 13872–13882.
[18] Shi, W., Han, X., Lewis, M., Tsvetkov, Y., Zettlemoyer, L. and Yih, W.T. (2024) Trusting your evidence: Hallucinate less with context-aware decoding. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2 (Short Papers), 783–791.
[19] Huang, Q., Dong, X.Y, Zhang, P., Wang, B., He, C.H., Wang, J.Q., Lin, D.H., Zhang, W.M. and Yu N.H. (2024) Opera: Alleviating hallucination in multi-modal large language models via overtrust penalty and retrospection-allocation. In Proceedings of the 2024 IEEE/CVF - CVPR, 13418–13427.
[20] Dhuliawala, S., Komeili, M., Xu, J., Raileanu, R., Li, X., Celikyilmaz, A. and Weston J. (2024) Chain-of-verification reduces hallucination in large language models. In Findings of the association for computational linguistics: ACL 2024, 3563–3578.
[21] Li, Y., Du, Y., Zhou, K., Wang, J., Zhao, X. and Wen, J.R. (2023) Evaluating object hallucination in large vision-language models. In Proceedings of the 2023 conference on empirical methods in natural language processing, 292–305.
[22] Zheng, L.M., Chiang, W.L., Sheng, Y., Zhuang, S.Y., Wu, Z.H., Zhuang, Y.H., Lin, Z., Li, Z.H., Li, D.C., Xing, E.P., Zhang, H., Gonzalez, J.E., Stoica, I. (2023) Judging llm-as-a-judge with mt-bench and chatbot arena. Advances in neural information processing systems, 36, 46 595–46 623.

Subscription
E-Mail Alert
Most popular papers
Call for Special Issue
Publication Ethics & OA Statement
Plagiarism Policy
Contact Us
Downloads: 17143
Visits: 767275

Sponsors, Associates, and Links

All published work is licensed under a Creative Commons Attribution 4.0 International License.

Copyright © 2016 - 2031 Clausius Scientific Press Inc. All Rights Reserved.