AI Threat Awareness: Defending Against a New Class of Attack