GPT-Red is an automated red-teaming model that OpenAI trains to find prompt injection weaknesses. It works the way a human red-teamer does. It sends a prompt, watches how a GPT model responds, and iterates toward a goal such as a successful data exfiltration. Training runs on self-play reinforcement learning, with GPT-Red and a set of defender models learning at the same time across many scenarios. The attacker earns reward for eliciting a valid failure. The … More →
The post GPT-Red beat human red teamers on a prompt injection test appeared first on Help Net Security.