Post by Mercor

Earlier today, we published OpenAI GPT-5.6 Sol's APEX results. Now let's dive into what makes Sol distinctive: cost efficiency, how it achieves superior token economy, and what are the limits. Among frontier models, Sol (xHigh) delivers 49.8 Pass@1 points per million tokens on APEX-Agents. The next best is GPT-5.5 at 33.1. Fable 5 sits at 31.6, Grok 4.5 at 31.5, and Opus 4.8 at 30.2. That's roughly 1.5x the field. Sol reaches near-frontier accuracy on 757k tokens per trajectory while everyone else spends 0.9M to 1.4M. The mechanism is per-step leanness, not fewer steps. Sol averages about 28k tokens per tool call. Opus 4.8 averages 63k. Sol actually takes more steps than most of the frontier, but each one is less than half the weight. It works in many light moves rather than a few heavy ones. That is a different agent style, not just a smaller bill. So where does the efficiency run out? Quant modeling. On APEX-Agents, Sol's Pass@1 is 37.7%, compared with Fable 5 leading at 43.3%. The gap is not spread evenly across domains. Sol matches Opus 4.8 and GPT-5.5 in corporate law (60.2%) and management consulting (59.5%), but gives up 6 to 8 points to the frontier in investment banking (42.3%). Its headroom is concentrated in one place: heavy quantitative modeling work. There is another cost to running leaner steps: consistency. Given 4 attempts, Sol solved 48.5% of tasks (Pass@4). But it solved only 28.1% on all four attempts (Pass^4). That is a 20.4 point capability-to-reliability gap, wider than GPT-5.5's 15.0 points (45.8 vs 30.8). Sol can do more than its Pass@1 suggests. It just does not do it the same way twice. It is inexpensive and capable, but more variable across multiple runs. Finally, we wanted to see whether more reasoning spend would close any of these gaps, so we reran Sol on APEX-SWE at Max reasoning. Pass@1 rose from 39.7% to 41.2%. Integration held flat at 47.3%. Observability rose from 32.0% to 35.0%. These are real but slight gains, well inside the error bars. On APEX-Agents, Max effort actually scored lower than xHigh. Across both benchmarks the pattern holds: extra reasoning spend does not necessarily convert proportionally into higher scores.