The wide scale adoption of Artificial Intelligence (AI) will require that AI engineers and developers can provide assurances to the user base that an algorithm will perform as intended and without failure. High levels of assurance stem from all the planned, systematic activities applied at all stages of the AI engineering lifecycle with the intent of ensuring that any intelligent system is producing outcomes that are valid, verified, data-driven, trustworthy, and explainable to a layman, ethical in the context of its deployment, unbiased in its learning, and fair to its users. In this talk I will discuss how existing test and evaluation (T&E) processes needed to be updated for systems-enabled by AI. I will show how T&E is critical for the assurance of systems enabled by AI. A point of emphasis is that I focus on systems conducting missions that leverage AI. The implication is that algorithm performance should be characterized relative to the deployed systems, and that assurance should reflect the deployed environment and operating envelope.
About the Johns Hopkins Institute for Assured Autonomy: Led by APL and the Whiting School of Engineering, the IAA is becoming a nationally recognized center of excellence in autonomous systems, showcasing the robust portfolio of research and work from two premier divisions of JHU and creating strategic external partnerships. The IAA seeks to ensure the safe, secure, and reliable integration of autonomous systems and artificial intelligence (AI) in society. As autonomous systems proliferate, both physically and virtually, the institute seeks to ensure the systems will be trusted and safe in their operations, will withstand corruption by adversaries, and will integrate seamlessly into ecosystems and communities. In this burgeoning field, JHU strives to advance a clear vision for an autonomous future.
Speaker Biography
Dr. Laura Freeman is a Research Associate Professor of Statistics and the Director of the Intelligent Systems Lab at the Virginia Tech Hume Center. Her research leverages experimental methods in research that brings together cyber-physical systems, data science, artificial intelligence (AI), and machine learning to address critical challenges in national security. She is a hub faculty member in the Commonwealth Cyber Initiative and leads research in AI Assurance. She develops new methods for test and evaluation focusing on emerging system technology. She is also the Assistant Dean for Research, in that capacity she works to shape research directions and collaborations in across the College of Science in the National Capital Region. Previously, Dr. Freeman was the Assistant Director of the Operational Evaluation Division at the Institute for Defense Analyses. In that position, she established and developed an interdisciplinary analytical team of statisticians, psychologists, and engineers to advance scientific approaches to DoD test and evaluation. During 2018, Dr. Freeman served as that acting Senior Technical Advisor for Director Operational Test and Evaluation (DOT&E). As the Senior Technical Advisor, Dr. Freeman provided leadership, advice, and counsel to all personnel on technical aspects of testing military systems. She reviewed test strategies, plans, and reports from all systems on DOT&E oversight. Dr. Freeman has a B.S. in Aerospace Engineering, a M.S. in Statistics and a Ph.D. in Statistics, all from Virginia Tech. Her Ph.D. research was on design and analysis of experiments for reliability data.