Post by Simbian

Claude Sonnet 5 just shipped. On Simbian's Cyber Defense Benchmark (CDB), it's the lowest-scoring Anthropic model we've tested.   CDB is a threat-hunting eval, not a trivia leaderboard: agents query 100K+ real attack-log events per environment across 105 MITRE-mapped procedures, hunting for what actually happened.   Five Anthropic models have run it. Opus 4.6: 44.5% coverage. Sonnet 4.6: 36.9%. Opus 4.8: 32.7%. Opus 4.7: 30.9%. Sonnet 5: 19.9% — just over half of Sonnet 4.6's score (CDB, arXiv 2604.19533, as of 2026-06).   It's not a cost trade either. Sonnet 5 runs 14% cheaper than Sonnet 4.6 per investigation, but delivers 46% less coverage — worse value, not better. Its score, cost, and wall-clock time all land within 2-5% of Gemini 3.5 Flash. #ThreatHunting #CyberSecurity