Post by Turing

Rubrics-Graded Reasoning is here. Today, Turing is releasing the Advanced PhD Reasoning Rubrics Data Pack, a new rubric-based reasoning dataset built for frontier AI teams developing and evaluating advanced reasoning models. The dataset spans Computer Science, Data Science, and Chemistry and includes 1,106 expert-authored PhD-level tasks paired with weighted atomic rubrics and golden answers. 𝐖𝐡𝐲 𝐝𝐨𝐞𝐬 𝐭𝐡𝐢𝐬 𝐦𝐚𝐭𝐭𝐞𝐫? Most benchmarks measure whether a model reached the correct answer. They tell you what happened, but not how the model reasoned. This dataset goes deeper by evaluating: • Intermediate reasoning steps • Derivations and calculations • Scientific mechanisms and structures • Code and data workflows • Methodological decisions • Edge-case handling • Structured outputs and multi-step pipelines Each task is designed to transform expert evaluation into machine-verifiable training signals, making it particularly valuable for: -Reinforcement Learning (RL) -Reward Modeling -Post-Training -Process-Level Evaluation -Benchmarking & Regression Testing -Reasoning Failure Analysis -Scientific & Engineering QA To ensure the dataset remains challenging for frontier systems, it was calibrated across 16 evaluation rounds, with pass rates ranging from 0% to 50% on state-of-the-art models. This release also showcases the quality bar behind the reasoning datasets Turing delivers to frontier-model partners. If you're working on model reasoning, evaluation, or RL infrastructure, we'd love to hear what you think. Explore the dataset: On Hugging Face:  https://lnkd.in/e-HW5SrV Learn more: https://lnkd.in/eC2px-sS