RedHit: Adaptive Red-Teaming of Large Language Models via Search, Reasoning, and Preference Optimization

Existing LLM red-teaming methods often depend on hard-coded strategies that quickly become obsolete against novel attack patterns, requiring constant manual updates, while current automated approaches typically lack effective reasoning capabilities, leading to lower attack success rates and longer training times. RedHit is a multi-round, automated, and adaptive red-teaming framework that integrates Monte Carlo Tree Search, chain-of-thought reasoning, and Direct Preference Optimization to enhance the adversarial capabilities of an adversarial LLM. RedHit formulates prompt injection as a tree-search problem, incrementally constructing a Prompt Search Tree in which each node stores an adversarial prompt, its response, and a reward, guided by an evaluator module that dynamically scores model responses using multi-detector feedback. This yields fine-grained reward signals that let the adversarial model iteratively improve using preference data from previous attack rounds, demonstrating robust and adaptive adversarial behavior across multiple attack rounds.