This work broadens the space of rich yet practical models for structured prediction. We introduce a general framework for modeling with four ingredients: (1) latent variables, (2) structural constraints, (3) learned (neural) feature representations of the inputs, and (4) training that takes the approximations made during inference into account. We build up to this framework through an empirical study of three NLP tasks: semantic role labeling, relation extraction, and dependency parsing – obtaining state-of-the-art results on the former two. We apply the resulting graphical models with structured and neural factors, and approximation-aware learning to jointly model part-of-speech tags, a syntactic dependency parse, and semantic roles in a low-resource setting where the syntax is unobserved. We also present an alternative view of these models as neural networks with a topology inspired by inference on graphical models that encode our intuitions about the data.
Speaker Biography
Matt Gormley is a final year PhD candidate in Computer Science at Johns Hopkins University, co-advised by Mark Dredze and Jason Eisner. His current research focuses on joint modeling of multiple linguistic strata in learning settings where supervised resources are scarce. He has authored papers in a variety of areas including global optimization, joint inference and learning, topic modeling, and neural networks. He holds a Bachelor’s degree in Computer Science from Carnegie Mellon University (CMU). He will return to CMU this spring to join the faculty of the Machine Learning department.