Forest Global Training Partners

Cyber Risks · Cisco Talos

Choose your fighter: Balancing competing requirements to select models for your AI SOC

Summary: Cisco Talos tested 66 model and reasoning combinations from Anthropic and OpenAI on a tool-assisted log-review task, using a synthetic 80,054-record corpus generated with EvidenceForge and four independently prompted reviewer personas across five rounds per condition. The top condition, GPT-5.6 Sol Ultra, scored a median 96.25 but took 33.72 minutes and cost $55.48 per panel, while GPT-5.6 Luna Low scored 58.25 at 3.24 minutes and $0.39.

What's compelling: Talos found that raising reasoning effort raised cost but not score - GPT-5.6 Luna's scores fell at every higher level, and Claude Opus 4.8 lost 9.5 points going from high to xhigh - so the model you assume is strongest may deliver worse analysis at greater expense. Format violations and safety refusals also blocked usable answers outright: Claude Sonnet 4.6 produced no complete panels at max, and Anthropic's Fable was excluded after safeguards blocked 21 of 31 attempts even under relaxed CVP guardrails.

Read the original at Cisco Talos ↗

Topics
Detection & Response
Related
YARA-X 1.23.0 Release, (Sun, Nov 15th) Cyber Risks · Nov 15, 2026