Research Meetup: Benchmarking Coding Agents (NYC)
- WHO IT IS FOR: AI researchers and practitioners specializing in coding agents and software engineering automation.
- WHAT IT IS ABOUT: An exploration of benchmarking frameworks for coding agents featuring insights from industry and academic leaders.
- FORMAT: An in-person gathering featuring a series of lightning talks and a professional networking mixer.
Join us at Scale AI in New York City for the third installment of the Scale Labs Research Meetup Series. This exclusive gathering brings together the brightest minds working at the frontiers of coding agents to discuss the critical challenge of benchmarking and evaluation in the field.
The evening will feature a series of high-impact lightning talks from experts at Meta, Columbia University, and Scale AI, covering specialized frameworks such as ProgramBench, BranchBench, Terminal Bench, and SWE Atlas. It is a unique opportunity to dive deep into the technical nuances of how we measure agentic performance in software engineering.
Beyond the presentations, attendees can enjoy food, drinks, and official swag while connecting with fellow researchers. With no rigid agenda, the focus remains entirely on the cutting-edge work and the community driving the evolution of AI coding tools. Please note that this is a strictly in-person event with limited capacity.