Testing and repairing machine learning systems in adversarial environment

Yinzhi Cao, Lehigh University
Host: Abhishek Jain

Machine learning (ML) systems are increasingly deployed in safety- and security-critical domains such as self-driving cars and malware detection, where the system correctness for corner case inputs are crucial. Existing testing of ML system correctness depends heavily on manually labeled data and therefore often fails to expose erroneous behaviors for rare inputs.

In this talk, I will present the first framework to test and repair ML systems, especially in an adversarial environment. In the first part, I will introduce DeepXplore, a whitebox testing framework of real-world deep learning (DL) systems. Our evaluation shows that DeepXplore can successfully find thousands of erroneous corner case behaviors, e.g., self-driving cars crashing into guard rails and malware masquerading as benign software. In the second part, I will introduce machine unlearning, a general, efficient approach to repair an ML system exhibiting erroneous behaviors. Our evaluation, on four diverse learning systems and real-world workloads, shows that machine unlearning is general, effective, fast, and easy to use.

Speaker Biography

Yinzhi Cao is an assistant professor at Lehigh University. He earned his Ph.D. in Computer Science at Northwestern University and worked at Columbia University as a postdoc. Before that, he obtained his B.E. degree in Electronics Engineering at Tsinghua University in China. His research mainly focuses on the security and privacy of the Web, smartphones, and machine learning. He has published many papers at various security and system conferences, such as IEEE S&P (Oakland), NDSS, CCS, and SOSP. His JShield system has been adopted by Huawei, the world’s largest telecommunication company. His past work was widely featured by over 30 media outlets, such as NSF Science Now (Episode 38), CCTV News, IEEE Spectrum, Yahoo! News and ScienceDaily. He received two best paper awards at SOSP’17 and IEEE CNS’15 respectively. He is one of the recipients of 2017 Amazon Research Awards (ARA).