Paper Accepted at ICML: There Are Futures That Benchmark-Driven AI Cannot See

The position paper "There Are Futures That Benchmark-Driven AI Cannot See," co-authored by Sobhan Lotfi, Ava Iranmanesh, Lachin Naghashyar, Ali Shirali, Fateme Nateghi Haredasht, Sanmi Koyejo, Philip Torr, Yong Suk Lee, Fazl Barez, Joel Lehman, Peter Norvig, and Arvind Narayanan, has been accepted at the International Conference on Machine Learning (ICML) 2026.

The paper argues that AI's benchmark-centered selection environment, while successful at bypassing complex debates about the nature of intelligence, taxes exaptation — the process by which traits evolved for one function become decisive for another. When one selection rule dominates, ideas that do not fit it have nowhere to persist. The paper proposes mechanisms to restore exaptive capacity without abandoning benchmarking: plural evaluation regimes, protected venues for non-comparable work, long-horizon funding, and training norms that encourage researchers to question selection rules, not only optimize within them.