Post by Yuzhe Yang

📈 Can LLMs really reason over health time series? Introducing 𝗛𝗘𝗔𝗥𝗧𝗦 ❤️ — the first 𝘭𝘪𝘷𝘪𝘯𝘨 benchmark for health time-series reasoning. Most current evaluations of health time series are still narrow in scope. With 𝗛𝗘𝗔𝗥𝗧𝗦, we move beyond that and study how modern LLMs handle real physiological data at scale. We built a large-scale benchmark with  • 🧪 𝟮𝟬𝗞+ test samples  • 🧩 𝟭𝟭𝟬 tasks  • 🏥 𝟭𝟮 health domains (metabolism, motion, cardiac, sleep, audio, ...)  • 📡 𝟮𝟬 signal modalities (ECG, PPG, EEG, IMU, EMG, CGM, ...) 📊 It enables to date the broadest coverage of  • sequence lengths (up to 1M+ steps),  • sampling frequencies (up to 48kHz),  • time spans (from seconds to years). 🚀 Rather than focusing on a narrow slice of prediction, 𝗛𝗘𝗔𝗥𝗧𝗦 covers four levels of reasoning in one unified benchmark: 🧠 𝘗𝘦𝘳𝘤𝘦𝘱𝘵𝘪𝘰𝘯 🔍 𝘐𝘯𝘧𝘦𝘳𝘦𝘯𝘤𝘦 ✍️ 𝘎𝘦𝘯𝘦𝘳𝘢𝘵𝘪𝘰𝘯 ⚙️ 𝘋𝘦𝘥𝘶𝘤𝘵𝘪𝘰𝘯 Across 14 state-of-the-art LLMs 🤖, we find that strong general capability does not yet translate into strong health time-series reasoning. Many models still struggle with long-range temporal structure, high-frequency signals, and tasks that require more than simple pattern matching or heuristic shortcuts. 𝗛𝗘𝗔𝗥𝗧𝗦 is designed as a living and evolving community benchmark. We hope it will continue to grow with community inputs on new datasets / tasks / models, and help push toward AI that can better understand and reason over health time series in the real world! 👇 📄 Paper: https://lnkd.in/gf5-UBeA 🌐 Website: https://lnkd.in/gNEnwjXB 🕵️ Code: https://lnkd.in/gzcfmYCZ 🤗 Dataset: https://lnkd.in/g7Ea6zvj 🏆 Leaderboard: https://lnkd.in/gc5y_8EX Great work led by my students Sirui Li, Shuhan Xiao, Mihir Joshi and collaborators Ahmed Abdelhadi Metwally, Daniel McDuff, and Wei Wang! We are also grateful for generous compute support from Google, OpenAI, Anthropic, and xAI. UCLA UCLA Computer Science Computational Medicine Department UCLA Henry Samueli School of Engineering and Applied Science #AI #HealthAI #LLM #TimeSeries #MultimodalAI #FoundationModels