Mike Luu

Vol. 2,  No. 2, pp. 0-0, Oct.  2025
10.23246/AAIRJ.2025.02.02.02

Alignment LLM Tool Uses

PDF

Large Language Models (LLMs) equipped with tool-calling capabilities present new security vulnerabilities when performing seemingly benign language processing tasks. This paper demonstrates how translation, summarization, and explanation requests containing malicious foreign-language instructions can trigger un-safe tool activations in LLMs. Using a benchmark dataset of 150 unsafe instructions across Korean, Japanese, and Vietnamese, we evalu- ate tool activation rates across different task types. Our findings reveal that translation tasks achieve the highest vulnerability with 86.67% tool activation rate, followed by summarization (86.00%) and explanation (69.33%). Surprisingly, prompt engineering safeguards ("make sure to only translate the content") prove highly effective for explanation tasks, reducing activation rates from 69.33% to 0.00%, while remaining less effective for translation and summarization tasks. This research highlights security considerations for deploying tool-enabled LLMs in multilingual environments and demonstrates varying effectiveness of mitigation strategies across different task types. The benchmark dataset is available at Hugging Face Datasets and our experimental code is available at GitHub.