Undergraduate Research – 2024

Faculty Research Advisor: Benjamin Van Durme

Abstract: Legal professionals need to write analyses that rely on citations to relevant precedents (i.e., previous case decisions). Intelligent systems assisting legal professionals in writing such documents provide great benefits, but are challenging to design. Such systems need to help locate, summarize, and reason over salient precedents in order to be useful. To enable systems for such tasks, we work with legal professionals to transform a large, open-source legal corpus into a dataset supporting two important backbone tasks: information retrieval and retrieval-augmented generation. This dataset, which we term CLERC (Case Law Evaluation Retrieval Corpus), is constructed for training and evaluating models on their ability to (1) find corresponding citations for a given piece of legal analysis and to (2) compile the text of these citations (as well as previous context) into a cogent analysis that supports a reasoning goal. We benchmark state-of-the-art models on CLERC, showing that current approaches still struggle: GPT-4o generates analyses with the highest ROUGE F-scores but hallucinates the most, while zero-shot IR models only achieve 48.3% recall@1000.

Full paper »

About Abe: Abe Hou is an undergraduate senior, triple majoring in computer science, sociology, and pure math. He is interested the bidirectional interactions between AI and society, specifically on 1) building natural language processing (NLP) and AI applications for public policies and law and 2) AI safety and governance. He has published four papers at top computer science conferences and founded the Technology and Policy Society at Johns Hopkins, a student group sponsored by the Johns Hopkins Data Science and AI Institute. Abe has also served as the director of programming at the Johns Hopkins Undergraduate Law Review for two years. He is currently working on NLP for law and generative agent-based modeling for public policy. In his free time, Abe loves poetry and soccer and hosts a book club at the Center for Language and Speech Processing.

Faculty Research Advisor: Scott Smith

Abstract: Our work develops novel minimal-state operational semantics for higher-order functional languages that use only the call stack and a program point or lexical level as complete state information—there is no environment, substitution, or continuation. We carry out a proof that this form of operational semantics is equivalent to standard presentations. This maximal compression of the program state opens the door to potential new applications; we define a program analysis as a direct finitization of these operational semantics. The program analysis that naturally emerges has a number of novel and interesting properties compared to standard program analyses for higher-order programs; for example, it can infer recurrences and does not need value widening. We formally define the analysis and implement it in OCaml. Evaluating our program analysis against related work, it analyzes most benchmarks orders of magnitude faster and more accurately, and thanks to our analysis being purely rule-based, it makes for much more straightforward implementation and mechanization.

Full paper »

About Robert: Robert Zhang recently graduated from John Hopkins with a combined BS/MSE degree in computer science. They are interested in many topics spanning programming languages and formal methods—particularly program analysis verification, and synthesis and their applications to real-world software. Since 2022, Robert has been collaborating with Scott Smith on developing novel operational semantics and an associated program analysis technique, which led to a publication at Object-Oriented Programming, Systems, Languages, and Applications—a top conference in programming languages. For this work, Robert was selectively chosen as the sole recipient of the 2023 Masson Fellowship based on research merit. Aside from research, Robert served as a course assistant for six consecutive semesters over the past three years, during which they helped with Principles of Programming Languages, Functional Programming in Software Engineering, and Full-Stack JavaScript. They served on the executive board of the ACM JHU Chapter, in which they helped organize and participated in student-oriented events bridging academia and industry. They are also a member of Upsilon Pi Epsilon, the nation’s first computer science honor society. This fall, Robert will join the University of Texas at Austin to pursue a PhD in computer science, specializing in programming languages and formal methods.