이채린, 최대선, 나현식,

Vol. 35, No. 3, pp. 585-599, 6월. 2025
10.13089/JKIISC.2025.35.3.585, Full Text:
Keywords: Backdoor attack, Multi-Modal Model, VQA, Foundation Model Security, Patch-based Attack
Abstract

This study analyzes the security vulnerabilities of multimodal AI models and empirically evaluates the effectiveness of unimodal backdoor attacks on the BLIP VQA model. Unlike previous work, we demonstrate that backdoor attacks manipulating only the image modality can still succeed. We also show that backdoors can persist even when the model is fine-tuned on an unseen dataset (VQAv2). By varying the poisoning rate from 0.1% to 10%, we find that a high Backdoor Success Rate (BSR) can be achieved with minimal poisoned data, while maintaining high Clean Accuracy (CA). Furthermore, our analysis on the impact of answer types reveals that the Binary type is the most vulnerable, achieving over 99% BSR across all poisoning rates. In contrast, the Other type maintains a lower BSR (85.3%–96.2%) at rates below 1%, and only reaches near 99% BSR at rates above 3%, indicating greater resistance. These findings highlight the backdoor vulnerabilities of foundation models in the AI supply chain and underscore the need for further research into detection and defense methods for multimodal AI systems.

Statistics
Cite this article