Director of Engineering, Simulation Evaluation

We are seeking an experienced Director, Simulation Evaluation to lead our efforts in rigorously assessing the safety, compliance, and service quality of the Waymo Driver. This critical role will be instrumental in scaling our autonomous vehicle technology by developing and overseeing the advanced quantitative methodologies and infrastructure required to evaluate performance in new cities, novel situations, and address issues identified in the field.

You will lead a team of highly skilled quantitative evaluation engineers and data scientists focused on developing innovative workflows and tools that accelerate the evaluation of the Waymo Driver. Your leadership will be pivotal in ensuring the robustness and scalability of our evaluation processes as we expand our operations.

• Define and Drive Strategy: Develop and execute the strategic roadmap for quantitative evaluation, aligning it with Waymo's business objectives and product development lifecycle. Drive the technical strategy for how we measure the Waymo driver's behavior in all situations, from interactions with pedestrians, to navigating construction zones.
• Lead and Mentor a High-Performing Team: Recruit, build, and lead a team of quantitative evaluation specialists, fostering a culture of technical excellence, innovation, and continuous improvement. This includes guiding individual contributors who are focused on:




• Creating metrics and evaluations for the areas that matter most to Waymo.
• Developing ML-based tooling for data mining, identification, sampling, and optimization of evaluation workflows using simulations of logged and synthetically generated driving scenarios.
• Improving product requirements for "eval of eval" tools and measurements to understand and control the quality of Waymo's driving evaluation tools.
• Translating product requirements into engineering and data science tasks, and implementing these tasks.
• Critically examining and improving the data and statistical methods used to evaluate our tools.
• Extending and improving underlying infrastructure to meet product needs and ensure consistency across the stack.
• Working with partners to integrate evaluation tools into workflows and processes.




• Champion Quantitative Rigor: Oversee the design, development, and implementation of sophisticated quantitative methods and statistical models for evaluating the Waymo Driver's performance across diverse operational domains.
• Innovate Evaluation Workflows: Drive the creation of scalable and efficient workflows for evaluating safety, compliance, and service quality, crucial for expanding into new operational design domains (ODDs) and addressing emergent challenges.
• Collaborate Cross-Functionally: Partner closely with onboard engineering, system engineering, and commercial teams, product management, operations, and legal teams to integrate evaluation insights into product development cycles and inform strategic decision-making.
• Ensure Data Integrity and Infrastructure: Guide the development and maintenance of robust data processing pipelines and underlying infrastructure to support large-scale quantitative evaluation, ensuring data quality and consistency.
• Communicate Insights: Clearly communicate complex quantitative findings and recommendations to diverse stakeholders, including senior leadership, to drive product improvements and business outcomes.

• 10+ years of industry experience in quantitative analysis, data science, or a related field, with a significant portion in a leadership or management role.
• 5+ years of experience managing and growing high-performing quantitative or data science teams, including managing managers, within a complex engineering environment.
• Masters in Machine Learning, Statistics, Mathematics, Physics, Robotics, a related quantitative field, or equivalent practical experience.
• Deep expertise in statistical modeling, experimental design, and quantitative methods, with a proven track record of applying these to real-world problems delivered into production.
• Proficiency in programming languages such as C++ and/or Python, with experience building scalable data and evaluation tools.
• Proven ability to translate complex technical requirements into actionable engineering and data science tasks.
• Experience developing evaluation tools, metrics, statistical algorithms, and related data infrastructure is highly preferred.
• Experience iterating on product vision with Product Managers and leadership.
• Excellent communication, interpersonal, and leadership skills, with the ability to influence and inspire at all levels.

• PhD in one of the quantitative fields listed above.
• Strong understanding of machine learning principles and their application in data mining, sampling, and optimization.
• Experience with evaluation of autonomous vehicle systems.
• Deep expertise in architecting solutions that efficiently utilize massive-scale infrastructure to achieve high-performance and scalability.

#LI-Hybrid