Adversarial Misuse of Generative AI: Google's Gemini Analysis

Image: Dall-e 3

Google's Threat Intelligence Group (GTIG) analyzed how advanced persistent threat (APT) and coordinated information operations (IO) actors attempted to misuse its Gemini AI assistant.

The analysis revealed that while threat actors used Gemini for tasks like research, code troubleshooting, and content creation, they did not develop novel attack capabilities.

Iranian APT actors were the most frequent users, employing Gemini across various attack lifecycle phases, including reconnaissance, vulnerability research, and payload development.

IO actors primarily used Gemini for research, content generation (including persona development), translation, and expanding their reach; Iranian actors were again the most prevalent.

Attempts to exploit Gemini through prompt injection attacks or jailbreaks were largely unsuccessful, with Gemini's safety measures preventing the generation of malicious content.

Gemini's assistance with common tasks accelerated threat actors' operations, but did not fundamentally alter their capabilities; it served as a productivity tool rather than a game-changer.

GTIG's findings highlight the need for continued vigilance and collaboration across the industry to mitigate the potential for AI misuse in cybersecurity, while acknowledging the evolving AI landscape and its potential impact on future threats.

https://cloud.google.com/blog/topics/threat-intelligence/adversarial-misuse-generative-ai