Contact

Education

Professional Experience

Publications

1. Sheng Zhang, Xutai Ma, Kevin Duh, and Benjamin Van Durme. 2019. Broad-Coverage Semantic Parsing as Transduction. In Empirical Methods in Natural Language Processing (EMNLP). [bibtex]

   @inproceedings{zhang-emnlp-19,
     title = {Broad-Coverage Semantic Parsing as Transduction},
     author = {Zhang, Sheng and Ma, Xutai and Duh, Kevin and {Van Durme}, Benjamin},
     year = {2019},
     booktitle = {Empirical Methods in Natural Language Processing (EMNLP)}
   }
   

2. Elias Stengel-Eskin, Tzu-ray Su, Matt Post, and Benjamin Van Durme. 2019. A Discriminative Neural Model for Cross-Lingual Word Alignment. In Empirical Methods in Natural Language Processing (EMNLP). [bibtex]

   @inproceedings{stengel-eskin-emnlp-19,
     title = {A Discriminative Neural Model for Cross-Lingual Word Alignment},
     author = {Stengel-Eskin, Elias and Su, Tzu-ray and Post, Matt and {Van Durme}, Benjamin},
     year = {2019},
     booktitle = {Empirical Methods in Natural Language Processing (EMNLP)}
   }
   

3. Venkata Govindarajan, Benjamin Van Durme, and Aaron Steven White. 2019. Decomposing Generalization: Models of Generic, Habitual, and Episodic Statements. Transactions of the Association for Computational Linguistics, 7:501–517. [pdf] [bibtex] [abstract]

   @article{govindarajan-tacl-19,
     author = {Govindarajan, Venkata and {Van Durme}, Benjamin and White, Aaron Steven},
     title = {Decomposing Generalization: Models of Generic, Habitual, and Episodic                     Statements},
     journal = {Transactions of the Association for Computational Linguistics},
     volume = {7},
     pages = {501-517},
     year = {2019},
     url = {https://doi.org/10.1162/tacl_a_00285},
     eprint = {https://doi.org/10.1162/tacl_a_00285}
   }
   

    We present a novel semantic framework for modeling linguistic expressions of generalization— generic, habitual, and episodic statements—as combinations of simple, real-valued referential properties of predicates and their arguments. We use this framework to construct a dataset covering the entirety of the Universal Dependencies English Web Treebank. We use this dataset to probe the efficacy of type-level and token-level information—including hand-engineered features and static (GloVe) and contextual (ELMo) word embeddings—for predicting expressions of generalization. 

4. J. Edward Hu, Abhinav Singh, Nils Holzenberger, Matt Post, and Benjamin Van Durme. 2019. Large-scale, Diverse, Paraphrastic Bitexts via Sampling and Clustering. In The SIGNLL Conference on Computational Natural Language Learning (CoNLL). [bibtex]

   @inproceedings{hu-conll-2019,
     title = {Large-scale, Diverse, Paraphrastic Bitexts via Sampling and Clustering},
     author = {Hu, J. Edward and Singh, Abhinav and Holzenberger, Nils and Post, Matt and {Van Durme}, Benjamin},
     booktitle = {The SIGNLL Conference on Computational Natural Language Learning (CoNLL)},
     year = {2019}
   }
   

5. Siddharth Vashishtha, Benjamin Van Durme, and Aaron Steven White. 2019. Fine-Grained Temporal Relation Extraction. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL). [bibtex]

   @inproceedings{Vashishtha-acl-19,
     title = {Fine-Grained Temporal Relation Extraction},
     author = {Vashishtha, Siddharth and {Van Durme}, Benjamin and White, Aaron Steven},
     booktitle = {Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL)},
     year = {2019}
   }
   

6. Alex Wang, Jan Hula, Patrick Xia, Raghavendra Pappagari, R. Thomas McCoy, Roma Patel, Najoung Kim, Ian Tenney, Yinghui Huang, Katherin Yu, Shuning Jin, Berlin Chen, Benjamin Van Durme, Edouard Grave, Ellie Pavlick, and Samuel R. Bowman. 2019. Can You Tell Me How to Get Past Sesame Street? Sentence-Level Pretraining Beyond Language Modeling. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL). [bibtex]

   @inproceedings{wang-acl-19,
     booktitle = {Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL)},
     title = {Can You Tell Me How to Get Past Sesame Street? Sentence-Level Pretraining Beyond Language Modeling},
     author = {Wang, Alex and Hula, Jan and Xia, Patrick and Pappagari, Raghavendra and McCoy, R. Thomas and Patel, Roma and Kim, Najoung and Tenney, Ian and Huang, Yinghui and Yu, Katherin and Jin, Shuning and Chen, Berlin and {Van Durme}, Benjamin and Grave, Edouard and Pavlick, Ellie and Bowman, Samuel R.},
     year = {2019}
   }
   

7. Yonatan Belinkov, Adam Poliak, Stuart Shieber, Benjamin Van Durme, and Alexander Rush. 2019. Don’t Take the Premise for Granted: Mitigating Artifacts in Natural Language Inference. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL). [bibtex]

   @inproceedings{belinkov-acl-19,
     title = {Don't Take the Premise for Granted: Mitigating Artifacts in Natural Language Inference},
     author = {Belinkov, Yonatan and Poliak, Adam and Shieber, Stuart and {Van Durme}, Benjamin and Rush, Alexander},
     booktitle = {Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL)},
     year = {2019}
   }
   

8. Sheng Zhang, Xutai Ma, Kevin Duh, and Benjamin Van Durme. 2019. AMR Parsing as Sequence-to-Graph Transduction. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL). [bibtex]

   @inproceedings{zhang-acl-19,
     title = {AMR Parsing as Sequence-to-Graph Transduction},
     author = {Zhang, Sheng and Ma, Xutai and Duh, Kevin and {Van Durme}, Benjamin},
     booktitle = {Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL)},
     year = {2019}
   }
   

9. Zhongyang Li, Tongfei Chen, and Benjamin Van Durme. 2019. Learning to Rank for Plausible Plausibility. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL). [bibtex]

   @inproceedings{li-acl-19,
     title = {Learning to Rank for Plausible Plausibility},
     author = {Li, Zhongyang and Chen, Tongfei and {Van Durme}, Benjamin},
     booktitle = {Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL)},
     year = {2019}
   }
   

10. Najoung Kim, Roma Patel, Adam Poliak, Alex Wang, Patrick Xia, R. Thomas McCoy, Ian Tenney, Alexis Ross, Tal Linzen, Benjamin Van Durme, Samuel R. Bowman, and Ellie Pavlick. 2019. Probing What Different NLP Tasks Teach Machines about Function Word Comprehension. In Proceedings of the Eighth Joint Conference on Lexical and Computational Semantics (*SEM). [bibtex] [abstract]

    @inproceedings{kimStarSem19,
      title = {Probing What Different NLP Tasks Teach Machines about Function Word Comprehension},
      booktitle = {Proceedings of the Eighth Joint Conference on Lexical and Computational Semantics (*{SEM})},
      author = {Kim, Najoung and Patel, Roma and Poliak, Adam and Wang, Alex and Xia, Patrick and McCoy, R. Thomas and Tenney, Ian and Ross, Alexis and Linzen, Tal and {Van Durme}, Benjamin and Bowman, Samuel R. and Pavlick, Ellie},
      pdf = {https://www.aclweb.org/anthology/S19-1026},
      year = {2019},
      numpages = {15}
    }
    

    We introduce a set of nine challenge tasks that test for the understanding of function words. These tasks are created by structurally mutating sentences from existing datasets to target the comprehension of specific types of function words (e.g., prepositions, wh-words). Using these probing tasks, we explore the effects of various pretraining objectives for sentence encoders (e.g., language modeling, CCG supertagging and natural language inference (NLI)) on the learned representations. Our results show that pretraining on CCG—our most syntactic objective—performs the best on average across our probing tasks, suggesting that syntactic knowledge helps function word comprehension. Language modeling also shows strong performance, supporting its widespread use for pretraining state-of-the-art NLP models. Overall, no pretraining objective dominates across the board, and our function word probing tasks highlight several intuitive differences between pretraining objectives, e.g., that NLI helps the comprehension of negation.

11. Yonatan Belinkov, Adam Poliak, Stuart Shieber, Benjamin Van Durme, and Alexander Rush. 2019. On Adversarial Removal of Hypothesis-only Bias in Natural Language Inference. In Proceedings of the Eighth Joint Conference on Lexical and Computational Semantics (*SEM). [pdf] [bibtex] [abstract]

    @inproceedings{belinkov-etal-2019-adversarial,
      title = {On Adversarial Removal of Hypothesis-only Bias in Natural Language Inference},
      author = {Belinkov, Yonatan and Poliak, Adam and Shieber, Stuart and {Van Durme}, Benjamin and Rush, Alexander},
      booktitle = {Proceedings of the Eighth Joint Conference on Lexical and Computational Semantics (*{SEM})},
      year = {2019},
      url = {https://www.aclweb.org/anthology/S19-1028}
    }
    

    Popular Natural Language Inference (NLI) datasets have been shown to be tainted by hypothesis-only biases. Adversarial learning may help models ignore sensitive biases and spurious correlations in data. We evaluate whether adversarial learning can be used in NLI to encourage models to learn representations free of hypothesis-only biases. Our analyses indicate that the representations learned via adversarial learning may be less biased, with only small drops in NLI accuracy.

12. J. Edward Hu, Huda Khayrallah, Ryan Culkin, Patrick Xia, Tongfei Chen, Matt Post, and Benjamin Van Durme. 2019. Improved Lexically Constrained Decoding for Translation and Monolingual Rewriting. In Proceedings of the Annual Meeting of the North American Association of Computational Linguistics (NAACL). [bibtex] [abstract]

    @inproceedings{improved-lexically-constrained-decoding-for-translation-and-monolingual-rewriting,
      author = {Hu, J. Edward and Khayrallah, Huda and Culkin, Ryan and Xia, Patrick and Chen, Tongfei and Post, Matt and {Van Durme}, Benjamin},
      title = {Improved Lexically Constrained Decoding for Translation and Monolingual Rewriting},
      pdf = {https://www.aclweb.org/anthology/N19-1090},
      booktitle = {Proceedings of the Annual Meeting of the North American Association of Computational Linguistics (NAACL)},
      year = {2019},
      numpages = {12}
    }
    

    Lexically-constrained sequence decoding allows for explicit positive or negative phrase-based constraints to be placed on target output strings in generation tasks such as machine translation or monolingual text rewriting. We describe vectorized dynamic beam allocation, which extends work in lexically-constrained decoding to work with batching, leading to a five-fold improvement in throughput when working with positive constraints. Faster decoding enables faster exploration of constraint strategies: we illustrate this via data augmentation experiments with a monolingual rewriter applied to the tasks of natural language inference, question answering and machine translation, showing improvements in all three.

13. Ian Tenney, Patrick Xia, Berlin Chen, Alex Wang, Adam Poliak, R Thomas McCoy, Najoung Kim, Benjamin Van Durme, Sam Bowman, Dipanjan Das, and Ellie Pavlick. 2019. What do you learn from context? Probing for sentence structure in contextualized word representations. In International Conference on Learning Representations. [pdf] [bibtex] [abstract]

    @inproceedings{tenney2018iclr,
      title = {What do you learn from context? Probing for sentence structure in contextualized word representations},
      author = {Tenney, Ian and Xia, Patrick and Chen, Berlin and Wang, Alex and Poliak, Adam and McCoy, R Thomas and Kim, Najoung and {Van Durme}, Benjamin and Bowman, Sam and Das, Dipanjan and Pavlick, Ellie},
      booktitle = {International Conference on Learning Representations},
      year = {2019},
      url = {https://openreview.net/forum?id=SJzSgnRcKX},
      numpages = {14}
    }
    

    Contextualized representation models such as CoVe (McCann et al., 2017) and ELMo (Peters et al., 2018a) have recently achieved state-of-the-art results on a diverse array of downstream NLP tasks. Building on recent token-level probing work, we introduce a novel edge probing task design and construct a broad suite of sub-sentence tasks derived from the traditional structured NLP pipeline. We probe word-level contextual representations from three recent models and investigate how they encode sentence structure across a range of syntactic, semantic, local, and long-range phenomena. We find that ELMo encodes linguistic structure at the word level better than other comparable models, and that existing models trained on language modeling and translation produce strong representations for syntactic phenomena, but only offer small improvements on semantic tasks over a non-contextual baseline.

14. Yonatan Belinkov, Adam Poliak, Stuart M. Shieber, Benjamin Van Durme, and Alexander Rush. 2019. On Adversarial Removal of Hypothesis-only Bias in Natural Language Inference. In Joint Conference on Lexical and Computational Semantics (StarSem). [pdf] [bibtex] [abstract]

    @inproceedings{on-adv-removal-hypothesis-only-bias-in-natural-language-inference,
      title = {On Adversarial Removal of Hypothesis-only Bias in Natural Language Inference},
      author = {Belinkov, Yonatan and Poliak, Adam and Shieber, {Stuart M.} and {Van Durme}, Benjamin and Rush, Alexander},
      year = {2019},
      booktitle = {Joint Conference on Lexical and Computational Semantics (StarSem)},
      numpages = {7},
      url = {https://www.cs.jhu.edu/~apoliak1/papers/ON-ADVERSARIAL-REMOVAL-HYPOTHESIS-ONLY-BIAS-NLI.pdf}
    }
    

    Popular Natural Language Inference (NLI) datasets have been shown to be tainted by hypothesis-only biases. Adversarial learning may help models ignore sensitive biases and spurious correlations in data. We evaluate whether adversarial learning can be used in NLI to encourage models to learn representations free of hypothesis-only biases. Our analyses indicate that the representations learned via adversarial learning may be less biased, with only small drops in NLI accuracy.

15. Najoung Kim, Kyle Rawlins, Benjamin Van Durme, and Paul Smolensky. 2019. Predicting Argumenthood of English Preposition Phrases. In AAAI. [pdf] [bibtex] [abstract]

    @inproceedings{predicting-argumenthood-of-english-preposition-phrases,
      author = {Kim, Najoung and Rawlins, Kyle and {Van Durme}, Benjamin and Smolensky, Paul},
      title = {{Predicting Argumenthood of English Preposition Phrases}},
      year = {2019},
      url = {http://arxiv.org/abs/1809.07889},
      booktitle = {AAAI},
      numpages = {9}
    }
    

    Distinguishing between arguments and adjuncts of a verb is a longstanding, nontrivial problem. In natural language processing, argumenthood information is important in tasks such as semantic role labeling (SRL) and prepositional phrase (PP) attachment disambiguation. In theoretical linguistics, many diagnostic tests for argumenthood exist but they often yield conflicting and potentially gradient results. This is especially the case for syntactically oblique items such as PPs. We propose two PP argumenthood prediction tasks branching from these two motivations: (1) binary argument-adjunct classification of PPs in VerbNet, and (2) gradient argumenthood prediction using human judgments as gold standard, and report results from prediction models that use pretrained word embeddings and other linguistically informed features. Our best results on each task are (1) acc. = 0.955, F1 = 0.954 (ELMo+BiLSTM) and (2) Pearson’s r = 0.624 (word2vec+MLP). Furthermore, we demonstrate the utility of argumenthood prediction in improving sentence representations via performance gains on SRL when a sentence encoder is pretrained with our tasks.

16. J. Edward Hu, Rachel Rudinger, Matt Post, and Benjamin Van Durme. 2019. ParaBank: Monolingual Bitext Generation and Sentential Paraphrasing via Lexically-constrained Neural Machine Translation. In Proceedings of AAAI. [code] [data] [bibtex] [abstract]

    @inproceedings{parabank-monolingual-bitext-generation-and-sentential-paraphrasing-via-lexically-constrained-neural-machine-translation,
      author = {Hu, J. Edward and Rudinger, Rachel and Post, Matt and {Van Durme}, Benjamin},
      year = {2019},
      code = {http://decomp.io/projects/parabank},
      data = {http://decomp.io/projects/parabank},
      pdf = {https://arxiv.org/pdf/1901.03644.pdf},
      booktitle = {Proceedings of AAAI},
      title = {Para{B}ank: Monolingual Bitext Generation and Sentential Paraphrasing via Lexically-constrained Neural Machine Translation},
      numpages = {8}
    }
    

    We present PARABANK, a large-scale English paraphrase dataset that surpasses prior work in both quantity and quality. Following the approach of PARANMT (Wieting and Gimpel, 2018), we train a Czech-English neural machine translation (NMT) system to generate novel paraphrases of English reference sentences. By adding lexical constraints to the NMT decoding procedure, however, we are able to produce multiple high-quality sentential paraphrases per source sentence, yielding an English paraphrase resource with more than 4 billion generated tokens and exhibiting greater lexical diversity. Using human judgments, we also demonstrate that PARABANK’s paraphrases improve over PARANMT on both semantic similarity and fluency. Finally, we use PARABANK to train a monolingual NMT model with the same support for lexically-constrained decoding for sentence rewriting tasks.

17. Michelle Yuan, Benjamin Van Durme, and Jordan Boyd-Graber. 2018. Multilingual Anchoring: Interactive Topic Modeling and Alignment Across Languages. In Neural Information Processing Systems. [pdf] [bibtex] [abstract]

    @inproceedings{multilingual-anchoring-interactive-topic-modeling-and-alignment-across-languages,
      author = {Yuan, Michelle and {Van Durme}, Benjamin and Boyd-Graber, Jordan},
      booktitle = {Neural Information Processing Systems},
      location = {Montreal, Quebec},
      year = {2018},
      title = {Multilingual Anchoring: Interactive Topic Modeling and Alignment  Across Languages},
      url = {https://papers.nips.cc/paper/8083-multilingual-anchoring-interactive-topic-modeling-and-alignment-across-languages.pdf},
      numpages = {11}
    }
    

    Multilingual topic models can reveal patterns in cross-lingual document collections. However, existing models lack speed and interactivity, which prevents adoption in everyday corpora exploration or quick moving situations (e.g., natural disasters, political instability). First, we propose a multilingual anchoring algorithm that builds an anchor-based topic model for documents in different languages. Then, we incorporate interactivity to develop MTAnchor (Multilingual Topic Anchors), a system that allows users to refine the topic model. We test our algorithms on labeled English, Chinese, and Sinhalese documents. Within minutes, our methods can produce interpretable topics that are useful for specific classification tasks.

18. Sheng Zhang, Xutai Ma, Rachel Rudinger, Kevin Duh, and Benjamin Van Durme. 2018. Cross-lingual Decompositional Semantic Parsing. In Empirical Methods in Natural Language Processing (EMNLP). [pdf] [bibtex] [abstract]

    @inproceedings{cross-lingual-decompositional-semantic-parsing,
      title = {{Cross-lingual Decompositional Semantic Parsing}},
      author = {Zhang, Sheng and Ma, Xutai and Rudinger, Rachel and Duh, Kevin and {Van Durme}, Benjamin},
      booktitle = {Empirical Methods in Natural Language Processing (EMNLP)},
      year = {2018},
      url = {http://aclweb.org/anthology/D18-1194},
      numpages = {12}
    }
    

    We introduce the task of cross-lingual decompositional semantic parsing: mapping content provided in a source language into a decompositional semantic analysis based on a target language. We present: (1) a form of decompositional semantic analysis designed to allow systems to target varying levels of structural complexity (shallow to deep analysis), (2) an evaluation metric to measure the similarity between system output and reference semantic analysis, (3) an end-to-end model with a novel annotating mechanism that supports intra-sentential coreference, and (4) an evaluation dataset on which our model outperforms strong baselines by at least 1.75 F1 score.

19. Rachel Rudinger, Adam Teichert, Ryan Culkin, Sheng Zhang, and Benjamin Van Durme. 2018. Neural Davidsonian Semantic Proto-role Labeling. In Empirical Methods in Natural Language Processing (EMNLP). [pdf] [bibtex] [abstract]

    @inproceedings{neural-davidsonian-semantic-proto-role-labeling,
      title = {{Neural Davidsonian Semantic Proto-role Labeling}},
      author = {Rudinger, Rachel and Teichert, Adam and Culkin, Ryan and Zhang, Sheng and {Van Durme}, Benjamin},
      booktitle = {Empirical Methods in Natural Language Processing (EMNLP)},
      year = {2018},
      url = {http://aclweb.org/anthology/D18-1114},
      numpages = {12}
    }
    

    We present a model for semantic proto-role labeling (SPRL) using an adapted bidirectional LSTM encoding strategy that we call Neural-Davidsonian: predicate-argument structure is represented as pairs of hidden states corresponding to predicate and argument head tokens of the input sequence. We demonstrate: (1) state-of-the-art results in SPRL, and (2) that our network naturally shares parameters between attributes, allowing for learning new attribute types with limited added supervision.

20. Adam Poliak, Aparajita Haldar, Rachel Rudinger, J. Edward Hu, Ellie Pavlick, Aaron Steven White, and Benjamin Van Durme. 2018. Collecting Diverse Natural Language Inference Problems for Sentence Representation Evaluation. In Empirical Methods in Natural Language Processing (EMNLP). [pdf] [bibtex] [abstract]

    @inproceedings{collecting-diverse-natural-language-inference-problems-for-sentence-representation-evaluation,
      title = {{Collecting Diverse Natural Language Inference Problems for Sentence Representation Evaluation}},
      author = {Poliak, Adam and Haldar, Aparajita and Rudinger, Rachel and Hu, J. Edward and Pavlick, Ellie and White, Aaron Steven and {Van Durme}, Benjamin},
      booktitle = {Empirical Methods in Natural Language Processing (EMNLP)},
      year = {2018},
      url = {http://aclweb.org/anthology/D18-1007},
      numpages = {15}
    }
    

    We present a large scale collection of diverse natural language inference (NLI) datasets that help provide insight into how well a sentence representation captures distinct types of reasoning. The collection results from recasting 13 existing datasets from 7 semantic phenomena into a common NLI structure, resulting in over half a million labeled context-hypothesis pairs in total. We refer to our collection as the DNC: Diverse Natural Language Inference Collection. The DNC is available online at http://www.decomp.net, and will grow over time as additional resources are recast and added from novel sources.

21. Aaron Steven White, Rachel Rudinger, Kyle Rawlins, and Benjamin Van Durme. 2018. Lexicosyntactic inference in neural models. In Empirical Methods in Natural Language Processing (EMNLP). [pdf] [bibtex] [abstract]

    @inproceedings{lexicosyntactic-inference-in-neural-models,
      title = {{Lexicosyntactic inference in neural models}},
      author = {White, Aaron Steven and Rudinger, Rachel and Rawlins, Kyle and {Van Durme}, Benjamin},
      booktitle = {Empirical Methods in Natural Language Processing (EMNLP)},
      year = {2018},
      url = {http://aclweb.org/anthology/D18-1501},
      numpages = {8}
    }
    

    We investigate neural models’ ability to capture lexicosyntactic inferences: inferences triggered by the interaction of lexical and syn- tactic information. We take the task of event factuality prediction as a case study and build a factuality judgment dataset for all English clause-embedding verbs in various syntactic contexts. We use this dataset, which we make publicly available, to probe the behavior of current state-of-the-art neural systems, showing that these systems make certain systematic errors that are clearly visible through the lens of factuality prediction.

22. Keisuke Sakaguchi and Benjamin Van Durme. 2018. Efficient Online Scalar Annotation with Bounded Support. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL). [pdf] [code] [bibtex] [abstract]

    @inproceedings{efficient-online-scalar-annotation-with-bounded-support,
      title = {{Efficient Online Scalar Annotation with Bounded Support}},
      author = {Sakaguchi, Keisuke and {Van Durme}, Benjamin},
      booktitle = {Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL)},
      year = {2018},
      numpages = {11},
      code = {https://github.com/decomp-sem/EASL},
      url = {http://www.aclweb.org/anthology/P18-1020}
    }
    

    We describe a novel method for efficiently eliciting scalar annotations for dataset construction and system quality estimation by human judgments. We contrast direct assessment (annotators assign scores to items directly), online pairwise ranking aggregation (scores derive from annotator comparison of items), and a hybrid approach (EASL: Efficient Annotation of Scalar Labels) proposed here. Our proposal leads to increased correlation with ground truth, at far greater annotator efficiency, suggesting this strategy as an improved mechanism for dataset creation and manual system evaluation.

23. Travis Wolfe, Annabelle Carrell, Mark Dredze, and Benjamin Van Durme. 2018. Summarizing Entities using Distantly Supervised Information Extractors. In Proceedings of the SIGIR Workshop on Knowledge Graphs and Semantics for Text Retrieval, Analysis, and Understanding (KG4IR). [pdf] [bibtex] [abstract]

    @inproceedings{summarizing-entities-using-distantly-supervised-information-extractors,
      title = {{Summarizing Entities using Distantly Supervised Information Extractors}},
      author = {Wolfe, Travis and Carrell, Annabelle and Dredze, Mark and {Van Durme}, Benjamin},
      year = {2018},
      url = {http://ceur-ws.org/Vol-2127/paper3-kg4ir.pdf},
      booktitle = {Proceedings of the SIGIR Workshop on Knowledge Graphs and Semantics for Text Retrieval, Analysis, and Understanding (KG4IR)},
      numpages = {8}
    }
    

    We describe entity summarization, the task of producing informative text summaries for an entity described across multiple documents in a collection. Existing (multi-)document summarization techniques are applied as baselines to this task, which we generalize to allow for joint information extraction and summarization. Through user evaluations across a variety of approaches, we discover what is most preferred in an entity summary. In particular we find that distantly supervised information extractors lead to significant improvements over lexical approaches, demonstrating the utility of extraction technologies for a task other than knowledge base population.

24. Pushpendre Rastogi, Adam Poliak, Vince Lyzinski, and Benjamin Van Durme. 2018. Neural Variational Entity Set Expansion for Automatically Populated Knowledge Graphs. In Information Retrieval Journal. October. [pdf] [bibtex] [abstract]

    @inproceedings{neural-variational-entity-set-expansion-for-automatically-populated-knowledge-graphs,
      title = {{Neural Variational Entity Set Expansion for Automatically Populated Knowledge Graphs}},
      author = {Rastogi, Pushpendre and Poliak, Adam and Lyzinski, Vince and {Van Durme}, Benjamin},
      year = {2018},
      journal = {Information Retrieval Journal},
      month = oct,
      day = {25},
      issn = {1573-7659},
      doi = {10.1007/s10791-018-9342-1},
      url = {https://rdcu.be/98BY},
      numpages = {24}
    }
    

    We propose Neural variational set expansion to extract actionable information from a noisy knowledge graph (KG) and propose a general approach for increasing the interpretability of recommendation systems. We demonstrate the usefulness of applying a variational autoencoder to the Entity set expansion task based on a realistic automatically generated KG.

25. Rashmi Sankepally, Tongfei Chen, Benjamin Van Durme, and Douglas W. Oard. 2018. A Test Collection for Coreferent Mention Retrieval. In Proceedings of SIGIR. [pdf] [data] [bibtex] [abstract]

    @inproceedings{a-test-collection-for-coreference-mention-retrieval,
      title = {{A Test Collection for Coreferent Mention Retrieval}},
      author = {Sankepally, Rashmi and Chen, Tongfei and {Van Durme}, Benjamin and Oard, Douglas W.},
      year = {2018},
      booktitle = {Proceedings of SIGIR},
      url = {https://dl.acm.org/citation.cfm?id=3210139},
      data = {https://github.com/rashmisankepally/CoreferentMentionRetrieval},
      numpages = {4}
    }
    

    This paper introduces the coreferent mention retrieval task, in which the goal is to retrieve sentences that mention a specific entity based on a query by example in which one sentence mentioning that entity is provided. The development of a coreferent mention retrieval test collection is then described. Results are presented for five coreferent mention retrieval systems, both to illustrate the use of the collection and to specify the results that were pooled on which human coreference judgments were performed. The new test collection is built from content that is available from the Linguistic Data Consortium; the partitioning and human annotations used to create the test collection atop that content are being made freely available.

26. Adam Poliak, Jason Naradowsky, Aparajita Haldar, Rachel Rudinger, and Benjamin Van Durme. 2018. Hypothesis Only Baselines in Natural Language Inference. In Joint Conference on Lexical and Computational Semantics (StarSem). [pdf] [code] [bibtex] [abstract]

    @inproceedings{hypothesis-only-baselines-in-natural-language-inference,
      title = {{Hypothesis Only Baselines in Natural Language Inference}},
      author = {Poliak, Adam and Naradowsky, Jason and Haldar, Aparajita and Rudinger, Rachel and {Van Durme}, Benjamin},
      year = {2018},
      url = {http://aclweb.org/anthology/S18-2023},
      code = {https://github.com/azpoliak/hypothesis-only-NLI},
      booktitle = {Joint Conference on Lexical and Computational Semantics (StarSem)},
      numpages = {12}
    }
    

    We propose a hypothesis only baseline for diagnosing Natural Language Inference (NLI). Especially when an NLI dataset assumes inference is occurring based purely on the relationship between a context and a hypothesis, it follows that assessing entailment relations while ignoring the provided context is a degenerate solution. Yet, through experiments on ten distinct NLI datasets, we find that this approach, which we refer to as a hypothesis-only model, is able to significantly outperform a majority-class baseline across a number of NLI datasets. Our analysis suggests that statistical irregularities may allow a model to perform NLI in some datasets beyond what should be achievable without access to the context.

27. Hongyuan Mei, Sheng Zhang, Kevin Duh, and Benjamin Van Durme. 2018. Halo: Learning Semantics-Aware Representations for Cross-Lingual Information Extraction. In Joint Conference on Lexical and Computational Semantics (StarSem). [pdf] [bibtex] [abstract]

    @inproceedings{halo-learning-semantics-aware-representations-for-cross-lingual-information-extraction,
      title = {{Halo: Learning Semantics-Aware Representations for Cross-Lingual Information Extraction}},
      author = {Mei, Hongyuan and Zhang, Sheng and Duh, Kevin and {Van Durme}, Benjamin},
      year = {2018},
      numpages = {6},
      booktitle = {Joint Conference on Lexical and Computational Semantics (StarSem)},
      url = {http://aclweb.org/anthology/S18-2017}
    }
    

    Cross-lingual information extraction (CLIE) is an important and challenging task, especially in low resource scenarios. To tackle this challenge, we propose a training method, called Halo, which enforces the local region of each hidden state of a neural model to only generate target tokens with the same semantic structure tag. This simple but powerful technique enables a neural model to learn semantics-aware representations that are robust to noise, without introducing any extra parameter, thus yielding better generalization in both high and low resource settings.

28. Sheng Zhang, Kevin Duh, and Benjamin Van Durme. 2018. Fine-grained Entity Typing through Increased Discourse Context and Adaptive Classification Thresholds. In Joint Conference on Lexical and Computational Semantics (StarSem). [pdf] [bibtex] [abstract]

    @inproceedings{fine-grained-entity-typying-through-increased-discourse-context-and-adaptive-classification-thresholds,
      title = {{Fine-grained Entity Typing through Increased Discourse Context and Adaptive Classification Thresholds}},
      author = {Zhang, Sheng and Duh, Kevin and {Van Durme}, Benjamin},
      year = {2018},
      numpages = {7},
      booktitle = {Joint Conference on Lexical and Computational Semantics (StarSem)},
      url = {http://www.aclweb.org/anthology/S18-2022}
    }
    

    Fine-grained entity typing is the task of assigning fine-grained semantic types to entity mentions. We propose a neural architecture which learns a distributional semantic representation that leverages a greater amount of semantic context – both document and sentence level information – than prior work. We find that additional context improves performance, with further improvements gained by utilizing adaptive classification thresholds. Experiments show that our approach without reliance on hand-crafted features achieves the state-of-the-art results on three benchmark datasets.

29. Adam Poliak, Yonatan Belinkov, James Glass, and Benjamin Van Durme. 2018. Evaluating Fine-grained Semantic Phenomena in Neural Machine Translation Encoders Using Entailment. In Proceedings of the Annual Meeting of the North American Association of Computational Linguistics (NAACL). [pdf] [bibtex] [abstract]

    @inproceedings{evaluating-fine-grained-semantic-phenomena-in-neural-machine-translation-encoders-using-entailment,
      author = {Poliak, Adam and Belinkov, Yonatan and Glass, James and {Van Durme}, Benjamin},
      title = {Evaluating Fine-grained Semantic Phenomena in Neural Machine Translation Encoders Using Entailment},
      year = {2018},
      numpages = {7},
      url = {http://aclweb.org/anthology/N18-2082},
      booktitle = {Proceedings of the Annual Meeting of the North American Association of Computational Linguistics (NAACL)}
    }
    

    We propose a process for investigating the extent to which sentence representations arising from neural machine translation (NMT) systems encode distinct semantic phenomena. We use these representations as features to train a natural language inference (NLI) classifier based on datasets recast from existing semantic annotations. In applying this process to a representative NMT system, we find its encoder appears most suited to supporting inferences at the syntax-semantics interface, as compared to anaphora resolution requiring world-knowledge. We conclude with a discussion on the merits and potential deficiencies of the existing process, and how it may be improved and extended as a broader framework for evaluating semantic coverage.

30. Rachel Rudinger, Jason Naradowsky, Brian Leonard, and Benjamin Van Durme. 2018. Gender Bias in Coreference Resolution. In Proceedings of the Annual Meeting of the North American Association of Computational Linguistics (NAACL). [pdf] [data] [bibtex] [abstract]

    @inproceedings{gender-bias-in-coreference-resolution,
      author = {Rudinger, Rachel and Naradowsky, Jason and Leonard, Brian and {Van Durme}, Benjamin},
      title = {Gender Bias in Coreference Resolution},
      booktitle = {Proceedings of the Annual Meeting of the North American Association of Computational Linguistics (NAACL)},
      year = {2018},
      numpages = {7},
      data = {https://github.com/rudinger/winogender-schemas},
      url = {http://www.aclweb.org/anthology/N18-2002}
    }
    

    We present an empirical study of gender bias in coreference resolution systems. We first introduce a novel, Winograd schema-style set of minimal pair sentences that differ only by pronoun gender. With this evaluation set, we confirm systematic gender bias in three publicly-available coreference resolution systems, and correlate this bias with real-world and textual gender statistics.

31. Rachel Rudinger, Aaron Steven White, and Benjamin Van Durme. 2018. Neural Models of Factuality. In Proceedings of the Annual Meeting of the North American Association of Computational Linguistics (NAACL). [pdf] [data] [bibtex] [abstract]

    @inproceedings{neural-models-of-factuality,
      author = {Rudinger, Rachel and White, Aaron Steven and {Van Durme}, Benjamin},
      title = {Neural Models of Factuality},
      booktitle = {Proceedings of the Annual Meeting of the North American Association of Computational Linguistics (NAACL)},
      year = {2018},
      numpages = {13},
      url = {http://aclweb.org/anthology/N18-1067},
      data = {http://decomp.net}
    }
    

    We present two neural models for event factuality prediction, which yield significant performance gains over previous models on three event factuality datasets: FactBank, UW, and MEANTIME. We also present a substantial expansion of the Decomp dataset to cover the entirety of the English Universal Dependencies treebank, yielding the largest event factuality dataset to date. We report model results on this extended Decomp dataset as well.

32. Sheng Zhang, Rachel Rudinger, Kevin Duh, and Benjamin Van Durme. 2017. Ordinal Common-sense Inference. Transactions of the Association for Computational Linguistics, 5:379–395. [pdf] [bibtex] [abstract]

    @article{ordinal-common-sense-inference,
      author = {Zhang, Sheng and Rudinger, Rachel and Duh, Kevin and {Van Durme}, Benjamin},
      title = {Ordinal Common-sense Inference},
      journal = {Transactions of the Association for Computational Linguistics},
      volume = {5},
      year = {2017},
      numpages = {16},
      keywords = {inference,select},
      issn = {2307-387X},
      url = {https://transacl.org/ojs/index.php/tacl/article/view/1082},
      pages = {379--395}
    }
    

    Humans have the capacity to draw common-sense inferences from natural language: various things that are likely but not certain to hold based on established discourse, and are rarely stated explicitly. We propose an evaluation of automated common-sense inference based on an extension of recognizing textual entailment: predicting ordinal human responses on the subjective likelihood of an inference holding in a given context. We describe a framework for extracting common-sense knowledge from corpora, which is then used to construct a dataset for this ordinal entailment task. We train a neural sequence-to-sequence model on this dataset, which we use to score and generate possible inferences. Further, we annotate subsets of previously established datasets via our ordinal annotation protocol in order to then analyze the distinctions between these and what we have constructed.

33. Keisuke Sakaguchi, Matt Post, and Benjamin Van Durme. 2017. Grammatical Error Correction with Neural Reinforcement Learning. In Proceedings of the 8th International Conference on Natural Language Processing (IJCNLP). [pdf] [bibtex] [abstract]

    @inproceedings{grammatical-error-correction-with-neural-reinforcement-learning,
      title = {{Grammatical Error Correction with Neural Reinforcement Learning}},
      author = {Sakaguchi, Keisuke and Post, Matt and {Van Durme}, Benjamin},
      booktitle = {Proceedings of the 8th International Conference on Natural Language Processing (IJCNLP)},
      year = {2017},
      keywords = {extraction,select},
      numpages = {7},
      url = {http://www.aclweb.org/anthology/I17-2062}
    }
    

    We propose a neural encoder-decoder model with reinforcement learning (NRL) for grammatical error correction (GEC). Unlike conventional maximum likelihood estimation (MLE), the model directly optimizes towards an objective that considers a sentence-level, task-specific evaluation metric, avoiding the exposure bias issue in MLE. We demonstrate that NRL outperforms MLE both in human and automated evaluation metrics, achieving the state-of-the-art on a fluency-oriented GEC corpus.

34. Sheng Zhang, Kevin Duh, and Benjamin Van Durme. 2017. Selective Decoding for Cross-Lingual Open Information Extraction. In Proceedings of the 8th International Conference on Natural Language Processing (IJCNLP). [pdf] [bibtex] [abstract]

    @inproceedings{selective-decoding-for-cross-lingual-open-information-extraction,
      title = {{Selective Decoding for Cross-Lingual Open Information Extraction}},
      author = {Zhang, Sheng and Duh, Kevin and {Van Durme}, Benjamin},
      booktitle = {Proceedings of the 8th International Conference on Natural Language Processing (IJCNLP)},
      year = {2017},
      numpages = {11},
      keywords = {extraction,select},
      url = {http://www.aclweb.org/anthology/I17-1084}
    }
    

    Cross-lingual open information extraction is the task of distilling facts from the source language into representations in the target language. We propose a novel encoder-decoder model for this problem. It employs a novel selective decoding mechanism, which explicitly models the sequence labeling process as well as the sequence generation process on the decoder side. Compared to a standard encoder-decoder model, selective decoding significantly increases the performance on a Chinese-English cross-lingual open IE dataset by 3.87-4.49 BLEU and 1.91-5.92 F1. We also extend our approach to low-resource scenarios, and gain promising improvement.

35. Aaron Steven White, Pushpendre Rastogi, Kevin Duh, and Benjamin Van Durme. 2017. Inference is Everything: Recasting Semantic Resources into a Unified Evaluation Framework. In Proceedings of the 8th International Conference on Natural Language Processing (IJCNLP). [pdf] [bibtex] [abstract]

    @inproceedings{inference-is-everything-recasting-semantic-resources-into-a-unified-evaluation-framework,
      title = {{Inference is Everything: Recasting Semantic Resources into a Unified Evaluation Framework}},
      author = {White, Aaron Steven and Rastogi, Pushpendre and Duh, Kevin and {Van Durme}, Benjamin},
      booktitle = {Proceedings of the 8th International Conference on Natural Language Processing (IJCNLP)},
      year = {2017},
      numpages = {10},
      keywords = {inference,select},
      url = {http://www.aclweb.org/anthology/I17-1100}
    }
    

    We propose to unify a variety of existing semantic classification tasks, such as semantic role labeling, anaphora resolution, and paraphrase detection, under the heading of Recognizing Textual Entailment (RTE). We present a general strategy to automatically generate one or more sentential hypotheses based on an input sentence and pre-existing manual semantic annotations. The resulting suite of datasets enables us to probe a statistical RTE model?s performance on different aspects of semantics. We demonstrate the value of this approach by investigating the behavior of a popular neural network RTE model.

36. Rachel Rudinger, Kevin Duh, and Benjamin Van Durme. 2017. Skip-Prop: Representing Sentences with One Vector Per Proposition. In Proceedings of the 12th International Conference on Computational Semantics (IWCS). [pdf] [bibtex] [abstract]

    @inproceedings{skip-prop-representing-sentences-with-one-vector-per-proposition,
      title = {{Skip-Prop: Representing Sentences with One Vector Per Proposition}},
      author = {Rudinger, Rachel and Duh, Kevin and {Van Durme}, Benjamin},
      booktitle = {Proceedings of the 12th International Conference on Computational Semantics (IWCS)},
      year = {2017},
      keywords = {semantics,representation,select},
      numpages = {7},
      url = {http://www.aclweb.org/anthology/W17-6936}
    }
    

    We introduce the notion of a multi-vector sentence representation based on a one vector per proposition philosophy, which we term skip-prop vectors. By representing each predicate-argument structure in a complex sentence as an individual vector, skip-prop is (1) a response to empirical evidence that single-vector sentence representations degrade with sentence length, and (2) a representation that maintains a semantically useful level of granularity. We demonstrate the feasibility of training skip-prop vectors, introducing a method adapted from skip-thought vectors, and compare skip-prop with one vector per sentence and one vector per token approaches.

37. Benjamin Van Durme, Tom Lippincott, Kevin Duh, Deana Burchfield, Adam Poliak, Cash Costello, Tim Finin, Scott Miller, James Mayfield, Philipp Koehn, Craig Harman, Dawn Lawrie, Chandler May, Max Thomas, Julianne Chaloux, Annabelle Carrell, Tongfei Chen, Alex Comerford, Mark Dredze, et al. 2017. CADET: Computer Assisted Discovery Extraction and Translation. In Proceedings of the 8th International Conference on Natural Language Processing (IJCNLP): System Demonstrations. [pdf] [bibtex] [abstract]

    @inproceedings{cadet-computer-assisted-discovery-extraction-and-translation,
      title = {{CADET: Computer Assisted Discovery Extraction and Translation}},
      author = {{Van Durme}, Benjamin and Lippincott, Tom and Duh, Kevin and Burchfield, Deana and Poliak, Adam and Costello, Cash and Finin, Tim and Miller, Scott and Mayfield, James and Koehn, Philipp and Harman, Craig and Lawrie, Dawn and May, Chandler and Thomas, Max and Chaloux, Julianne and Carrell, Annabelle and Chen, Tongfei and Comerford, Alex and Dredze, Mark and Glass, Benjamin and Hao, Shudong and Martin, Patrick and Sankepally, Rashmi and Rastogi, Pushpendre and Wolfe, Travis and Tran, Ying-Ying and Zhang, Ted},
      booktitle = {Proceedings of the 8th International Conference on Natural Language Processing (IJCNLP): System Demonstrations},
      year = {2017},
      numpages = {4},
      url = {http://www.aclweb.org/anthology/I17-3002},
      keywords = {extraction,interactive,systems,select}
    }
    

    Computer Assisted Discovery Extraction and Translation (CADET) is a workbench for helping knowledge workers find, label, and translate documents of interest. It combines a multitude of analytics together with a flexible environment for customizing the workflow for different users. This open-source framework allows for easy development of new research prototypes using a micro-service architecture based atop Docker and Apache Thrift.

38. Keisuke Sakaguchi, Kevin Duh, Matt Post, and Benjamin Van Durme. 2017. Robsut Wrod Reocginiton via semi-Character Recurrent Neural Network. In AAAI Conference on Artificial Intelligence (AAAI). [pdf] [bibtex] [abstract]

    @inproceedings{robsut-wrod-reocgnition-via-semi-character-recurrent-neural-network,
      title = {{Robsut Wrod Reocginiton via semi-Character Recurrent Neural Network}},
      author = {Sakaguchi, Keisuke and Duh, Kevin and Post, Matt and {Van Durme}, Benjamin},
      booktitle = {AAAI Conference on Artificial Intelligence (AAAI)},
      year = {2017},
      numpages = {7},
      url = {https://engineering.jhu.edu/hltcoe/wp-content/uploads/sites/92/2017/01/2016_Robust_paper.pdf}
    }
    

    The Cmabrigde Uinervtisy (Cambridge University) effect from the psycholinguistics literature has demonstrated a robust word processing mechanism in humans, where jumbled words (e.g. Cmabrigde / Cambridge) are recognized with little cost. Inspired by the findings from the Cmabrigde Uinervtisy effect, we propose a word recognition model based on a semi-character level recursive neural network (scRNN). In our experiments, we demonstrate that scRNN has significantly more robust performance in word spelling correction (i.e. word recognition) compared to existing spelling checkers. Furthermore, we demonstrate that the model is cognitively plausible by replicating a psycholinguistics experiment about human reading difficulty using our model.

39. Chandler May, Kevin Duh, Benjamin Van Durme, and Ashwin Lall. 2017. Streaming Word Embeddings with the Space-Saving Algorithm. Preprint, arXiv:1704.07463. [pdf] [bibtex] [abstract]

    @unpublished{streaming-word-embeddings-with-the-space-saving-algorithm,
      author = {May, Chandler and Duh, Kevin and {Van Durme}, Benjamin and Lall, Ashwin},
      title = {{Streaming Word Embeddings with the Space-Saving Algorithm}},
      journal = {CoRR},
      volume = {abs/1704.07463},
      note = {Preprint, arXiv:1704.07463},
      numpages = {16},
      year = {2017},
      url = {http://arxiv.org/abs/1704.07463},
      archiveprefix = {arXiv},
      eprint = {1704.07463},
      timestamp = {Wed, 07 Jun 2017 14:43:09 +0200},
      biburl = {http://dblp.org/rec/bib/journals/corr/MayDDL17},
      bibsource = {dblp computer science bibliography, http://dblp.org},
      keywords = {streaming,representation,select}
    }
    

    We develop a streaming (one-pass, bounded-memory) word embedding algorithm based on the canonical skip-gram with negative sampling algorithm implemented in word2vec. We compare our streaming algorithm to word2vec empirically by measuring the cosine similarity between word pairs under each algorithm and by applying each algorithm in the downstream task of hashtag prediction on a two-month interval of the Twitter sample stream. We then discuss the results of these experiments, concluding they provide partial validation of our approach as a streaming replacement for word2vec. Finally, we discuss potential failure modes and suggest directions for future work.

40. Pushpendre Rastogi, Adam Poliak, and Benjamin Van Durme. 2017. Training Relation Embeddings under Logical Constraints. In E. Meij L. Dietz C. Xiong, editor, Proceedings of the First Workshop on Knowledge Graphs and Semantics for Text Retrieval and Analysis (KG4IR). [pdf] [bibtex] [abstract]

    @inproceedings{training-relation-embeddings-under-logical-constraints,
      title = {{Training Relation Embeddings under Logical Constraints}},
      author = {Rastogi, Pushpendre and Poliak, Adam and {Van Durme}, Benjamin},
      booktitle = {Proceedings of the First Workshop on Knowledge Graphs and Semantics for Text Retrieval and Analysis (KG4IR)},
      editor = {L. Dietz, C. Xiong, E. Meij},
      year = {2017},
      numpages = {7},
      keywords = {semantics,inference},
      url = {https://pdfs.semanticscholar.org/2341/78756b8bf3b2694671583084b22c76c47560.pdf}
    }
    

    We present ways of incorporating logical rules into the construction of embedding based Knowledge Base Completion (KBC) systems. Enforcing ?logical consistency? in the predictions of a KBC system guarantees that the predictions comply with logical rules such as symmetry, implication and generalized transitivity. Our method encodes logical rules about entities and relations as convex constraints in the embedding space to enforce the condition that the score of a logically entailed fact must never be less than the minimum score of an antecedent fact. Such constraints provide a weak guarantee that the predictions made by our KBC model will match the output of a logical knowledge base for many types of logical inferences. We validate our method via experiments on a knowledge graph derived from WordNet.

41. Travis Wolfe, Mark Dredze, and Benjamin Van Durme. 2017. Feature Generation for Robust Semantic Role Labeling. arXiv:1702.07046. [pdf] [bibtex] [abstract]

    @unpublished{feature-generation-for-robust-semantic-role-labeling,
      title = {{Feature Generation for Robust Semantic Role Labeling}},
      author = {Wolfe, Travis and Dredze, Mark and {Van Durme}, Benjamin},
      url = {https://arxiv.org/pdf/1702.07046.pdf},
      year = {2017},
      numpages = {10},
      note = {arXiv:1702.07046}
    }
    

    Hand-engineered feature sets are a well understood method for creating robust NLP models, but they require a lot of expertise and effort to create. In this work we describe how to automatically generate rich feature sets from simple units called featlets, requiring less engineering. Using information gain to guide the generation process, we train models which rival the state of the art on two standard Semantic Role Labeling datasets with almost no task or linguistic insight.

42. Adam Teichert, Adam Poliak, Benjamin Van Durme, and Matthew Gormley. 2017. Semantic Proto-Role Labeling. In AAAI Conference on Artificial Intelligence (AAAI). [pdf] [bibtex] [abstract]

    @inproceedings{semantic-proto-role-labeling,
      title = {{Semantic Proto-Role Labeling}},
      author = {Teichert, Adam and Poliak, Adam and {Van Durme}, Benjamin and Gormley, Matthew},
      booktitle = {AAAI Conference on Artificial Intelligence (AAAI)},
      year = {2017},
      numpages = {7},
      url = {https://aaai.org/ocs/index.php/AAAI/AAAI17/paper/view/14997/14053}
    }
    

    We present the first large-scale, corpus based verification of Dowty?s seminal theory of proto-roles. Our results demonstrate both the need for and the feasibility of a property-based annotation scheme of semantic relationships, as opposed to the currently dominant notion of categorical roles.

43. Pushpendre Rastogi and Benjamin Van Durme. 2017. Predicting Asymmetric Transitive Relations in Knowledge Bases. In E. Meij L. Dietz C. Xiong, editor, Proceedings of the First Workshop on Knowledge Graphs and Semantics for Text Retrieval and Analysis (KG4IR). [pdf] [bibtex] [abstract]

    @inproceedings{predicting-asymmetric-transitive-relations-in-knowledge-bases,
      title = {{Predicting Asymmetric Transitive Relations in Knowledge Bases}},
      author = {Rastogi, Pushpendre and {Van Durme}, Benjamin},
      booktitle = {Proceedings of the First Workshop on Knowledge Graphs and Semantics for Text Retrieval and Analysis (KG4IR)},
      editor = {L. Dietz, C. Xiong, E. Meij},
      year = {2017},
      numpages = {6},
      keywords = {semantics,inference},
      url = {https://pdfs.semanticscholar.org/27a5/a2da3e41b694870c91c36c430b031d6070ba.pdf}
    }
    

    Knowledge Base Completion (KBC), or link prediction, is the task of inferring missing edges in an existing knowledge graph. Although a number of methods have been evaluated empirically on select datasets for KBC, much less attention has been paid to understanding the relationship between the logical properties encoded by a given KB and the KBC method being evaluated. In this paper we study the effect of the logical properties of a relation on the performance of a KBC method, and we present a theorem and empirical results that can guide researchers in choosing the KBC algorithm for a KB.

44. Sheng Zhang, Rachel Rudinger, and Benjamin Van Durme. 2017. An Evaluation of PredPatt and Open IE via Stage 1 Semantic Role Labeling. In Proceedings of the 12th International Conference on Computational Semantics (IWCS). [pdf] [bibtex] [abstract]

    @inproceedings{an-evaluation-of-predpatt-and-open-ie-via-stage-1-semantic-role-labeling,
      author = {Zhang, Sheng and Rudinger, Rachel and {Van Durme}, Benjamin},
      title = {{An Evaluation of PredPatt and Open IE via Stage 1 Semantic Role Labeling}},
      booktitle = {Proceedings of the 12th International Conference on Computational Semantics (IWCS)},
      year = {2017},
      numpages = {7},
      keywords = {semantics,extraction},
      url = {http://www.aclweb.org/anthology/W17-6944}
    }
    

    PredPatt is a pattern-based framework for predicate-argument extraction. While it works across languages and provides a well-formed syntax-semantics interface for NLP tasks, a large-scale and reproducible evaluation has been lacking, which prevents comparisons between PredPatt and other related systems, and inhibits the updates of the patterns in PredPatt. In this work, we improve and evaluate PredPatt by introducing a large set of high-quality annotations converted from PropBank, which can also be used as a benchmark for other predicate-argument extraction systems. We compare PredPatt with other prominent systems and shows that PredPatt achieves the best precision and recall.

45. Francis Ferraro, Adam Poliak, Ryan Cotterell, and Benjamin Van Durme. 2017. Frame-Based Continuous Lexical Semantics through Exponential Family Tensor Factorization and Semantic Proto-Roles. In Proceedings of the 6th Joint Conference on Lexical and Computational Semantics (*SEM 2017). [pdf] [bibtex] [abstract]

    @inproceedings{frame-based-continuous-lexical-semantics-through-exponential-family-tensor-factorization-and-semantic-proto-roles,
      author = {Ferraro, Francis and Poliak, Adam and Cotterell, Ryan and {Van Durme}, Benjamin},
      title = {{Frame-Based Continuous Lexical Semantics through Exponential Family Tensor Factorization and Semantic Proto-Roles}},
      booktitle = {Proceedings of the 6th Joint Conference on Lexical and Computational Semantics (*SEM 2017)},
      year = {2017},
      numpages = {6},
      keywords = {semantics},
      url = {https://aclweb.org/anthology/S/S17/S17-1011.pdf}
    }
    

    We study how different frame annotations complement one another when learning continuous lexical semantics. We learn the representations from a tensorized skip-gram model that consistently encodes syntactic-semantic content better, with multiple 10% gains over baselines.

46. Keisuke Sakaguchi, Matt Post, and Benjamin Van Durme. 2017. Error-repair Dependency Parsing for Ungrammatical Texts. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL). [pdf] [bibtex] [abstract]

    @inproceedings{error-repair-dependency-parsing-for-ungrammatical-texts,
      title = {{Error-repair Dependency Parsing for Ungrammatical Texts}},
      author = {Sakaguchi, Keisuke and Post, Matt and {Van Durme}, Benjamin},
      booktitle = {Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL)},
      year = {2017},
      numpages = {7},
      keywords = {extraction},
      url = {http://aclweb.org/anthology/P17-2030}
    }
    

    We propose a new dependency parsing scheme which jointly parses a sentence and repairs grammatical errors by extending the non-directional transition-based formalism of Goldberg and Elhadad (2010) with three additional actions: SUBSTITUTE, DELETE, INSERT. Because these actions may cause an infinite loop in derivation, we also introduce simple constraints that ensure the parser termination. We evaluate our model with respect to dependency accuracy and grammaticality improvements for ungrammatical sentences, demonstrating the robustness and applicability of our scheme.

47. Nicholas Andrews, Benjamin Van Durme, Mark Dredze, and Jason Eisner. 2017. Bayesian Modeling of Lexical Resources for Low-Resource Settings. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL). [pdf] [bibtex] [abstract]

    @inproceedings{bayesian-modeling-of-lexical-resources-for-low-resource-settings,
      title = {{Bayesian Modeling of Lexical Resources for Low-Resource Settings}},
      author = {Andrews, Nicholas and {Van Durme}, Benjamin and Dredze, Mark and Eisner, Jason},
      booktitle = {Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL)},
      year = {2017},
      numpages = {11},
      keywords = {extraction},
      url = {http://www.aclweb.org/anthology/P17-1095}
    }
    

    Lexical resources such as dictionaries and gazetteers are often used as auxiliary data for tasks such as part-of-speech induction and named-entity recognition. However, discriminative training with lexical features requires annotated data to reliably estimate the lexical feature weights and may result in overfitting the lexical features at the expense of features which generalize better. In this paper, we investigate a more robust approach: we stipulate that the lexicon is the result of an assumed generative process. Practically, this means that we may treat the lexical resources as observations under the proposed generative model. The lexical resources provide training data for the generative model without requiring separate data to estimate lexical feature weights. We evaluate the proposed approach in two settings: part-of-speech induction and low-resource named-entity recognition.

48. Travis Wolfe, Mark Dredze, and Benjamin Van Durme. 2017. Pocket Knowledge Base Population. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL). [pdf] [bibtex] [abstract]

    @inproceedings{pocket-knowledge-base-population,
      title = {{Pocket Knowledge Base Population}},
      author = {Wolfe, Travis and Dredze, Mark and {Van Durme}, Benjamin},
      booktitle = {Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL)},
      year = {2017},
      numpages = {7},
      url = {http://www.aclweb.org/anthology/P17-2048}
    }
    

    Existing Knowledge Base Population methods extract relations from a closed relational schema with limited coverage, leading to sparse KBs. We propose Pocket Knowledge Base Population (PKBP), the task of dynamically constructing a KB of entities related to a query and finding the best characterization of relationships between entities. We describe novel Open Information Extraction methods which leverage the PKB to find informative trigger words. We evaluate using existing KBP shared-task data as well as new annotations collected for this work. Our methods produce high quality KBs from just text with many more entities and relationships than existing KBP systems.

49. Tongfei Chen and Benjamin Van Durme. 2017. Discriminative Information Retrieval for Question Answering Sentence Selection. In The 15th Conference of the European Chapter of the Association for Computational Linguistics (EACL). [pdf] [bibtex] [abstract]

    @inproceedings{discriminative-information-retrieval-for-question-answering-sentence-selection,
      title = {{Discriminative Information Retrieval for Question Answering Sentence Selection}},
      author = {Chen, Tongfei and {Van Durme}, Benjamin},
      booktitle = {The 15th Conference of the European Chapter of the Association for Computational Linguistics (EACL)},
      year = {2017},
      numpages = {7},
      keywords = {retrieval,extraction,qa},
      url = {http://www.aclweb.org/anthology/E17-2114}
    }
    

    We propose a framework for discriminative IR atop linguistic features, trained to improve the recall of answer candidate passage retrieval, the initial step in text-based question answering. We formalize this as an instance of linear feature-based IR, demonstrating a 34% - 43% improvement in recall for candidate triage for QA.

50. Ryan Cotterell, Adam Poliak, Benjamin Van Durme, and Jason Eisner. 2017. Explaining and Generalizing Skip-Gram through Exponential Family Principal Component Analysis. In The 15th Conference of the European Chapter of the Association for Computational Linguistics (EACL). [pdf] [bibtex] [abstract]

    @inproceedings{explaining-and-generalizing-skip-gram-through-exponential-family-principal-component-analysis,
      title = {{Explaining and Generalizing Skip-Gram through Exponential Family Principal Component Analysis}},
      author = {Cotterell, Ryan and Poliak, Adam and {Van Durme}, Benjamin and Eisner, Jason},
      booktitle = {The 15th Conference of the European Chapter of the Association for Computational Linguistics (EACL)},
      year = {2017},
      numpages = {7},
      keywords = {semantics},
      url = {http://www.aclweb.org/anthology/E17-2028}
    }
    

    The popular skip-gram model induces word embeddings by exploiting the signal from word-context coocurrence. We offer a new interpretation of skip-gram based on exponential family PCA?a form of matrix factorization. This makes it clear that we can extend the skip-gram method to tensor factorization, in order to train embeddings through richer higher-order coocurrences, e.g., triples that include positional information (to incorporate syntax) or morphological information (to share parameters across related words). We experiment on 40 languages and show that our model improves upon skip-gram.

51. Sheng Zhang, Kevin Duh, and Benjamin Van Durme. 2017. MT/IE: Cross-lingual Open Information Extraction with Neural Sequence-to-Sequence Models. In The 15th Conference of the European Chapter of the Association for Computational Linguistics (EACL). [pdf] [bibtex] [abstract]

    @inproceedings{mt-ie-cross-lingual-open-information-extraction-with-neural-sequence-to-sequence-models,
      author = {Zhang, Sheng and Duh, Kevin and {Van Durme}, Benjamin},
      title = {{MT/IE: Cross-lingual Open Information Extraction with Neural Sequence-to-Sequence Models}},
      booktitle = {The 15th Conference of the European Chapter of the Association for Computational Linguistics (EACL)},
      year = {2017},
      numpages = {7},
      keywords = {extraction},
      url = {http://www.aclweb.org/anthology/E17-2011}
    }
    

    Cross-lingual information extraction is the task of distilling facts from foreign language (e.g. Chinese text) into representations in another language that is preferred by the user (e.g. English tuples). Conventional pipeline solutions decompose the task as machine translation followed by information extraction (or vice versa). We propose a joint solution with a neural sequence model, and show that it outperforms the pipeline in a cross-lingual open information extraction setting by 1-4 BLEU and 0.5-0.8 F1.

52. Adam Poliak, Pushpendre Rastogi, and Benjamin Van Durme. 2017. Efficient, Compositional, Order-sensitive n-gram Embeddings. In The 15th Conference of the European Chapter of the Association for Computational Linguistics (EACL). [pdf] [bibtex] [abstract]

    @inproceedings{efficient-compositional-order-sensitive-n-gram-embeddings,
      author = {Poliak, Adam and Rastogi, Pushpendre and {Van Durme}, Benjamin},
      title = {{Efficient, Compositional, Order-sensitive n-gram Embeddings}},
      booktitle = {The 15th Conference of the European Chapter of the Association for Computational Linguistics (EACL)},
      year = {2017},
      numpages = {6},
      keywords = {semantics},
      url = {http://www.aclweb.org/anthology/E17-2081}
    }
    

    We propose ECO: a new way to generate embeddings for phrases that is Efficient, Compositional, and Order-sensitive. Our method creates decompositional embeddings for words offline and combines them to create new embeddings for phrases in real time. Unlike other approaches, ECO can create embeddings for phrases not seen during training. We evaluate ECO on supervised and unsupervised tasks and demonstrate that creating phrase embeddings that are sensitive to word order can help downstream tasks.

53. Aaron Steven White, Kyle Rawlins, and Benjamin Van Durme. 2017. The Semantic Proto-Role Linking Model. In The 15th Conference of the European Chapter of the Association for Computational Linguistics (EACL). [pdf] [bibtex] [abstract]

    @inproceedings{the-semantic-proto-role-linking-model,
      author = {White, Aaron Steven and Rawlins, Kyle and {Van Durme}, Benjamin},
      title = {{The Semantic Proto-Role Linking Model}},
      booktitle = {The 15th Conference of the European Chapter of the Association for Computational Linguistics (EACL)},
      year = {2017},
      numpages = {7},
      keywords = {semantics,decomp},
      url = {http://www.aclweb.org/anthology/E17-2015}
    }
    

    We propose the semantic proto-role linking model, which jointly induces both predicate-specific semantic roles and predicate-general semantic proto-roles based on semantic proto-role property likelihood judgments. We use this model to empirically evaluate Dowty?s thematic proto-role linking theory.

54. Rachel Rudinger, Chandler May, and Benjamin Van Durme. 2017. Social Bias in Elicited Natural Language Inferences. In The 15th Conference of the European Chapter of the Association for Computational Linguistics (EACL): Workshop on Ethics in NLP. [pdf] [bibtex] [abstract]

    @inproceedings{social-bias-in-elicited-natural-language-inferences,
      title = {{Social Bias in Elicited Natural Language Inferences}},
      author = {Rudinger, Rachel and May, Chandler and {Van Durme}, Benjamin},
      booktitle = {The 15th Conference of the European Chapter of the Association for Computational Linguistics (EACL): Workshop on Ethics in NLP},
      year = {2017},
      numpages = {7},
      url = {http://www.aclweb.org/anthology/W17-1609}
    }
    

    We analyze the Stanford Natural Language Inference (SNLI) corpus in an investigation of bias and stereotyping in NLP data. The human-elicitation protocol employed in the construction of the SNLI makes it prone to amplifying bias and stereotypical associations, which we demonstrate statistically (using pointwise mutual information) and with qualitative examples.

55. Mohamed Al-Badrashiny, Jason Bolton, Arun Tejavsi Chaganty, Kevin Clark, Craig Harman, Lifu Huang, Matthew Lamm, Jinhao Lei, Di Lu, Xiaoman Pan, Ashwin Paranjape, Ellie Pavlick, Haoruo Peng, Peng Qi, Pushpendre Rastogi, Abigail See, Kai Sun, Max Thomas, Chen-Tse Tsai, et al. 2017. TinkerBell: Cross-lingual Cold-Start Knowledge Base Construction. In Text Analysis Conference (TAC). [pdf] [bibtex] [abstract]

    @inproceedings{tinkerbell-cross-lingual-cold-start-knowledge-base-construction,
      author = {Al-Badrashiny, Mohamed and Bolton, Jason and Chaganty, Arun Tejavsi and Clark, Kevin and Harman, Craig and Huang, Lifu and Lamm, Matthew and Lei, Jinhao and Lu, Di and Pan, Xiaoman and Paranjape, Ashwin and Pavlick, Ellie and Peng, Haoruo and Qi, Peng and Rastogi, Pushpendre and See, Abigail and Sun, Kai and Thomas, Max and Tsai, Chen-Tse and Wu, Hao and Zhang, Boliang and Callison-Burch, Chris and Cardie, Claire and Ji, Heng and Manning, Christopher and Muresan, Smaranda and Rambow, Owen C. and Roth, Dan and Sammons, Mark and {Van Durme}, Benjamin},
      title = {{TinkerBell: Cross-lingual Cold-Start Knowledge Base Construction}},
      year = {2017},
      numpages = {12},
      url = {http://nlp.cs.rpi.edu/paper/kbp2017tinkerbellsystem.pdf},
      booktitle = {Text Analysis Conference (TAC)}
    }
    

    In this paper we present TinkerBell, a state-of-the-art end-to-end cold-start knowledge base construction system that extracts entity, relation, event and sentiment knowledge from three languages (English, Chinese and Spanish).

56. Travis Wolfe, Mark Dredze, and Benjamin Van Durme. 2016. A Study of Imitation Learning Methods for Semantic Role Labeling. In Empirical Methods in Natural Language Processing (EMNLP), Workshop on Structured Prediction for NLP. [pdf] [bibtex] [abstract]

    @inproceedings{a-study-of-imitation-learning-methods-for-semantic-role-labeling,
      title = {{A Study of Imitation Learning Methods for Semantic Role Labeling}},
      author = {Wolfe, Travis and Dredze, Mark and {Van Durme}, Benjamin},
      url = {http://www.aclweb.org/anthology/W/W16/W16-5905.pdf},
      booktitle = {Empirical Methods in Natural Language Processing (EMNLP), Workshop on Structured Prediction for NLP},
      year = {2016}
    }
    

    Global features have proven effective in a wide range of structured prediction problems but come with high inference costs. Imitation learning is a common method for train- ing models when exact inference isn’t feasible. We study imitation learning for Semantic Role Labeling (SRL) and analyze the ef- fectiveness of the Violation Fixing Perceptron (VFP) (Huang et al., 2012) and Locally Optimal Learning to Search (LOLS) (Chang et al., 2015) frameworks with respect to SRL global features. We describe problems in applying each framework to SRL and evaluate the effectiveness of some solutions. We also show that action ordering, including easy first inference, has a large impact on the quality of greedy global models.

57. Chandler May, Ryan Cotterell, and Benjamin Van Durme. 2016. Analysis of Morphology in Topic Modeling. Preprint, arXiv:1608.03995. [pdf] [bibtex] [abstract]

    @unpublished{analysis-of-morphology-in-topic-modeling,
      author = {{May}, Chandler and {Cotterell}, Ryan and {Van Durme}, Benjamin},
      title = {Analysis of Morphology in Topic Modeling},
      journal = {ArXiv e-prints},
      archiveprefix = {arXiv},
      eprint = {1608.03995},
      primaryclass = {cs.CL},
      keywords = {extraction},
      year = {2016},
      adsurl = {http://adsabs.harvard.edu/abs/2016arXiv160803995M},
      adsnote = {Provided by the SAO/NASA Astrophysics Data System},
      note = {Preprint, arXiv:1608.03995},
      url = {https://arxiv.org/pdf/1608.03995.pdf}
    }
    

    Topic models make strong assumptions about their data. In particular, different words are implicitly assumed to have different meanings: topic models are often used as human-interpretable dimensionality reductions and a proliferation of words with identical meanings would undermine the utility of the top-m word list representation of a topic. Though a number of authors have added preprocessing steps such as lemmatization to better accommodate these assumptions, the effects of such data massaging have not been publicly studied. We make first steps toward elucidating the role of morphology in topic modeling by testing the effect of lemmatization on the interpretability of a latent Dirichlet allocation (LDA) model. Using a word intrusion evaluation, we quantitatively demonstrate that lemmatization provides a significant benefit to the interpretability of a model learned on Wikipedia articles in a morphologically rich language.

58. Aaron Steven White, Drew Reisinger, Keisuke Sakaguchi, Tim Vieira, Sheng Zhang, Rachel Rudinger, Kyle Rawlins, and Benjamin Van Durme. 2016. Universal Decompositional Semantics on Universal Dependencies. In Empirical Methods in Natural Language Processing (EMNLP). [pdf] [bibtex] [abstract]

    @inproceedings{universal-decompositional-semantics-on-universal-dependencies,
      title = {{Universal Decompositional Semantics on Universal Dependencies}},
      author = {White, Aaron Steven and Reisinger, Drew and Sakaguchi, Keisuke and Vieira, Tim and Zhang, Sheng and Rudinger, Rachel and Rawlins, Kyle and {Van Durme}, Benjamin},
      booktitle = {Empirical Methods in Natural Language Processing (EMNLP)},
      year = {2016},
      numpages = {11},
      url = {https://www.aclweb.org/anthology/D16-1177}
    }
    

    We present a framework for augmenting data sets from the Universal Dependencies project with Universal Decompositional Semantics. Where the Universal Dependencies project aims to provide a syntactic annotation standard that can be used consistently across many languages as well as a collection of corpora that use that standard, our extension has similar aims for semantic annotation. We describe results from annotating the English Universal Dependencies treebank, dealing with word senses, semantic roles, and event properties.

59. Svitlana Volkova, Ilia Chetviorkin, Dustin Arendt, and Benjamin Van Durme. 2016. Contrasting Public Opinion Dynamics and Emotional Response during Crisis. In International Conference on Social Informatics (SocInfo16). [pdf] [bibtex] [abstract]

    @inproceedings{contrasting-public-opinion-dynamics-and-emotional-response-during-crisis,
      title = {{Contrasting Public Opinion Dynamics and Emotional Response during Crisis}},
      author = {Volkova, Svitlana and Chetviorkin, Ilia and Arendt, Dustin and {Van Durme}, Benjamin},
      booktitle = {International Conference on Social Informatics (SocInfo16)},
      year = {2016},
      numpages = {17},
      url = {http://www.cs.jhu.edu/~svitlana/papers/VCAV_SocInfo2016.pdf}
    }
    

    We propose an approach for contrasting spatiotemporal dynamics of public opinions expressed toward targeted entities, also known as stance detection task, in Russia and Ukraine during crisis. Our analysis relies on a novel corpus constructed from posts on the VKontakte social network, centered on local public opinion of the ongoing RussianUkrainian crisis, along with newly annotated resources for predicting expressions of fine-grained emotions including joy, sadness, disgust, anger, surprise and fear. Akin to prior work on sentiment analysis we align traditional public opinion polls with aggregated automatic predictions of sentiments for contrastive geo-locations. We report interesting observations on emotional response and stance variations across geo-locations. Some of our findings contradict stereotypical misconceptions imposed by media, for example, we found posts from Ukraine that do not support Euromaidan but support Putin, and posts from Russia that are against Putin but in favor USA. Furthermore, we are the first to demonstrate contrastive stance variations over time across geo-locations using storyline visualization1 technique.

60. Tom Lippincott and Benjamin Van Durme. 2016. Fluency Detection on Communication Networks. In Empirical Methods in Natural Language Processing (EMNLP). [pdf] [bibtex] [abstract]

    @inproceedings{fluency-detection-on-communication-networks,
      title = {{Fluency Detection on Communication Networks}},
      author = {Lippincott, Tom and {Van Durme}, Benjamin},
      booktitle = {Empirical Methods in Natural Language Processing (EMNLP)},
      year = {2016},
      numpages = {5},
      url = {https://www.aclweb.org/anthology/D16-1107}
    }
    

    When considering a social media corpus, we often have access to structural information about how messages are flowing between people or organizations. This information is particularly useful when the linguistic evidence is sparse, incomplete, or of dubious quality. In this paper we construct a simple model to leverage the structure of Twitter data to help determine the set of languages each user is fluent in. Our results demonstrate that imposing several intuitive constraints leads to improvements in performance and stability. We release the first annotated data set for exploring this task, and discuss how our approach may be extended to other applications.

61. Francis Ferraro and Benjamin Van Durme. 2016. A Unified Bayesian Model of Scripts, Frames and Language. In AAAI Conference on Artificial Intelligence (AAAI). [pdf] [bibtex] [abstract]

    @inproceedings{a-unified-bayesian-model-of-scripts-frames-and-language,
      author = {Ferraro, Francis and {Van Durme}, Benjamin},
      title = {{A Unified Bayesian Model of Scripts, Frames and Language}},
      booktitle = {AAAI Conference on Artificial Intelligence (AAAI)},
      year = {2016},
      numpages = {7},
      url = {http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.721.3566&rep=rep1&type=pdf}
    }
    

    We present the first probabilistic model to capture all levels of the Minsky Frame structure, with the goal of corpus-based induction of scenario definitions. Our model unifies prior efforts in discourse-level modeling with that of Fillmore?s related notion of frame, as captured in sentence-level, FrameNet semantic parses; as part of this, we resurrect the coupling among Minsky?s frames, Schank?s scripts and Fillmore?s frames, as originally laid out by those authors. Empirically, our approach yields improved scenario representations, reflected quantitatively in lower surprisal and more coherent latent scenarios.

62. Rebecca Mason, Benjamin Gaska, Benjamin Van Durme, Pallavi Choudhury, Ted Hart, Bill Dolan, Kristina Toutanova, and Margaret Mitchell. 2016. Microsummarization of Online Reviews: An Experimental Study. In AAAI Conference on Artificial Intelligence (AAAI). [pdf] [bibtex] [abstract]

    @inproceedings{microsummarization-of-online-reviews-an-experimental-study,
      author = {Mason, Rebecca and Gaska, Benjamin and {Van Durme}, Benjamin and Choudhury, Pallavi and Hart, Ted and Dolan, Bill and Toutanova, Kristina and Mitchell, Margaret},
      title = {{Microsummarization of Online Reviews: An Experimental Study}},
      booktitle = {AAAI Conference on Artificial Intelligence (AAAI)},
      year = {2016},
      numpages = {7},
      url = {https://pdfs.semanticscholar.org/f902/646b34f3ad6981ee868a5e3a1ec8eaee33a4.pdf}
    }
    

    Mobile and location-based social media applications provide platforms for users to share brief opinions about products, venues, and services. These quickly typed opinions, or microreviews, are a valuable source of current sentiment on a wide variety of subjects. However, there is currently little research on how to mine this information to present it back to users in easily consumable way. In this paper, we introduce the task of microsummarization, which combines sentiment analysis, summarization, and entity recognition in order to surface key content to users. We explore unsupervised and supervised methods for this task, and find we can reliably extract relevant entities and the sentiment targeted towards them using crowdsourced labels as supervision. In an end-to-end evaluation, we find our best-performing system is vastly preferred by judges over a traditional extractive summarization approach. This work motivates an entirely new approach to summarization, incorporating both sentiment analysis and item extraction for modernized, at-a-glance presentation of public opinion.

63. Svitlana Volkova, Yoram Bachrach, and Benjamin Van Durme. 2016. Mining User Interests to Predict Perceived Psycho-Demographic Traits on Twitter. In IEEE Big Data Service. [pdf] [bibtex] [abstract]

    @inproceedings{mining-user-interests-to-predict-perceived-psycho-demographic-traits-on-twitter,
      author = {Volkova, Svitlana and Bachrach, Yoram and {Van Durme}, Benjamin},
      title = {{Mining User Interests to Predict Perceived Psycho-Demographic Traits on Twitter}},
      booktitle = {IEEE Big Data Service},
      year = {2016},
      numpages = {8},
      url = {https://www.microsoft.com/en-us/research/wp-content/uploads/2016/07/VBV_IEEEBigData16.pdf}
    }
    

    We analyze the relation between user interests and their perceived psycho-demographic attributes using Twitter data, training models for predicting various personal traits of users. In contrast to existing work, which bases predictions on the textual tweets produced by users, we leverage the fact that users are embedded in the Twitter social network. We examine the accounts that our users follow, and use them to determine the high-level interests of these users, then use these areas of interest as features for predicting perceived personal traits. We cover target attributes such as gender, age, educational background, political stand and personality. We evaluate our technique on a dataset of over 4,000 Twitter user profiles. We use crowdsourcing to annotate these user profiles with perceptions regarding their personal traits, and correlate these with user interests, as captured by the accounts they follow and their classification in the Twitter ?Who To Follow? hierarchy. We compare the accuracy of our personal trait prediction methods with the state-of-the-art approaches that solely rely on user tweets, and discuss the correlations between perceived user demographics and interests.

64. Drew Reisinger, Rachel Rudinger, Francis Ferraro, Craig Harman, Kyle Rawlins, and Benjamin Van Durme. 2015. Semantic Proto-roles. Transactions of the Association of Computational Linguistics. [pdf] [bibtex] [abstract]

    @article{semantic-proto-roles,
      author = {Reisinger, Drew and Rudinger, Rachel and Ferraro, Francis and Harman, Craig and Rawlins, Kyle and {Van Durme}, Benjamin},
      title = {{Semantic Proto-roles}},
      journal = {Transactions of the Association of Computational Linguistics},
      year = {2015},
      numpages = {14},
      url = {http://www.aclweb.org/anthology/Q15-1034}
    }
    

    We present the first large-scale, corpus based verification of Dowty’s seminal theory of proto-roles. Our results demonstrate both the need for and the feasibility of a property-based annotation scheme of semantic relationships, as opposed to the currently dominant notion of categorical roles.

65. Rachel Rudinger, Pushpendre Rastogi, Francis Ferraro, and Benjamin Van Durme. 2015. Script Induction as Language Modeling. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP). [pdf] [bibtex] [abstract]

    @inproceedings{script-induction-as-language-modeling,
      author = {Rudinger, Rachel and Rastogi, Pushpendre and Ferraro, Francis and {Van Durme}, Benjamin},
      title = {{Script Induction as Language Modeling}},
      booktitle = {Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP)},
      year = {2015},
      numpages = {6},
      url = {http://www.aclweb.org/anthology/D15-1195}
    }
    

    The narrative cloze is an evaluation metric commonly used for work on automatic script induction. While prior work in this area has focused on count-based methods from distributional semantics, such as pointwise mutual information, we argue that the narrative cloze can be productively reframed as a language modeling task. By training a discriminative language model for this task, we attain improvements of up to 27 percent over prior methods on standard narrative cloze metrics.

66. Chandler May, Francis Ferraro, Alan McCree, Jonathan Wintrode, Daniel Garcia-Romero, and Benjamin Van Durme. 2015. Topic Identification and Discovery on Text and Speech. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP). [pdf] [bibtex] [abstract]

    @inproceedings{topic-identification-and-discovery-on-text-and-speech,
      author = {May, Chandler and Ferraro, Francis and McCree, Alan and Wintrode, Jonathan and Garcia-Romero, Daniel and {Van Durme}, Benjamin},
      title = {{Topic Identification and Discovery on Text and Speech}},
      booktitle = {Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP)},
      year = {2015},
      numpages = {11},
      url = {http://www.aclweb.org/anthology/D15-1285}
    }
    

    We compare the multinomial i-vector framework from the speech community with LDA, SAGE, and LSA as feature learners for topic ID on multinomial speech and text data. We also compare the learned representations in their ability to discover topics, quantified by distributional similarity to gold-standard topics and by human interpretability. We find that topic ID and topic discovery are competing objectives. We argue that LSA and i-vectors should be more widely considered by the text processing community as pre-processing steps for downstream tasks, and also speculate about speech processing tasks that could benefit from more interpretable representations like SAGE.

67. Ellie Pavlick, Johan Bos, Malvina Nissim, Charley Beller, Benjamin Van Durme, and Chris Callison-Burch. 2015. Adding Semantics to Data-Driven Paraphrasing. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL). [pdf] [bibtex] [abstract]

    @inproceedings{adding-semantics-to-data-driven-paraphrasing,
      author = {Pavlick, Ellie and Bos, Johan and Nissim, Malvina and Beller, Charley and {Van Durme}, Benjamin and Callison-Burch, Chris},
      title = {{Adding Semantics to Data-Driven Paraphrasing}},
      booktitle = {Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL)},
      year = {2015},
      numpages = {10},
      url = {http://www.aclweb.org/anthology/P15-1146}
    }
    

    We add an interpretable semantics to the paraphrase database (PPDB). To date, the relationship between phrase pairs in the database has been weakly defined as approximately equivalent. We show that these pairs represent a variety of relations, including directed entailment (little girl/girl) and exclusion (nobody/someone). We automatically assign semantic entailment relations to entries in PPDB using features derived from past work on discovering inference rules from text and semantic taxonomy induction. We demonstrate that our model assigns these relations with high accuracy. In a downstream RTE task, our labels rival relations from WordNet and improve the coverage of a proof-based RTE system by 17%.

68. Ellie Pavlick, Travis Wolfe, Pushpendre Rastogi, Chris Callison-Burch, Mark Dredze, and Benjamin Van Durme. 2015. FrameNet+: Fast Paraphrastic Tripling of FrameNet. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL). [pdf] [bibtex] [abstract]

    @inproceedings{framenet-fast-paraphrastic-tripling-of-framenet,
      author = {Pavlick, Ellie and Wolfe, Travis and Rastogi, Pushpendre and Callison-Burch, Chris and Dredze, Mark and {Van Durme}, Benjamin},
      title = {{FrameNet+: Fast Paraphrastic Tripling of FrameNet}},
      booktitle = {Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL)},
      year = {2015},
      numpages = {5},
      url = {http://www.aclweb.org/anthology/P15-2067}
    }
    

    We increase the lexical coverage of FrameNet through automatic paraphrasing. We use crowdsourcing to manually filter out bad paraphrases in order to ensure a high-precision resource. Our expanded FrameNet contains an additional 22K lexical units, a 3-fold increase over the current FrameNet, and achieves 40% better coverage when evaluated in a practical setting on New York Times data.

69. Ellie Pavlick, Juri Ganitkevitch, Tsz Ping Chan, Xuchen Yao, Benjamin Van Durme, and Chris Callison-Burch. 2015. Domain-Specific Paraphrase Extraction. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL). [pdf] [bibtex] [abstract]

    @inproceedings{domain-specific-paraphrase-extraction,
      author = {Pavlick, Ellie and Ganitkevitch, Juri and Chan, {Tsz Ping} and Yao, Xuchen and {Van Durme}, Benjamin and Callison-Burch, Chris},
      title = {{Domain-Specific Paraphrase Extraction}},
      booktitle = {Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL)},
      year = {2015},
      numpages = {5},
      url = {http://www.aclweb.org/anthology/P15-2010}
    }
    

    The validity of applying paraphrase rules depends on the domain of the text that they are being applied to. We develop a novel method for extracting domain-specific paraphrases. We adapt the bilingual pivoting paraphrase method to bias the training data to be more like our target domain of biology. Our best model results in higher precision while retaining complete recall, giving a 10% relative improvement in AUC.

70. Ellie Pavlick, Pushpendre Rastogi, Juri Ganitkevitch, Benjamin Van Durme, and Chris Callison-Burch. 2015. PPDB 2.0: Better paraphrase ranking, fine-grained entailment relations, word embeddings, and style classification. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL). [pdf] [bibtex] [abstract]

    @inproceedings{ppdb-2.0-better-paraphrase-ranking-fine-grained-entailment-relations-word-embeddings-and-style-classification,
      author = {Pavlick, Ellie and Rastogi, Pushpendre and Ganitkevitch, Juri and {Van Durme}, Benjamin and Callison-Burch, Chris},
      title = {{PPDB 2.0: Better paraphrase ranking, fine-grained entailment relations, word embeddings, and style classification}},
      booktitle = {Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL)},
      year = {2015},
      numpages = {5},
      url = {http://www.aclweb.org/anthology/P15-2070}
    }
    

    We present a new release of the Paraphrase Database. PPDB 2.0 includes a discriminatively re-ranked set of paraphrases that achieve a higher correlation with human judgments than PPDB 1.0?s heuristic rankings. Each paraphrase pair in the database now also includes fine-grained entailment relations, word embedding similarities, and style annotations.

71. Keith Levin, Aren Jansen, and Benjamin Van Durme. 2015. Segmental Acoustic Indexing for Zero Resource Keyword Search. In IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). [pdf] [bibtex] [abstract]

    @inproceedings{segmental-acoustic-indexing-for-zero-resource-keyword-search,
      author = {Levin, Keith and Jansen, Aren and {Van Durme}, Benjamin},
      title = {{Segmental Acoustic Indexing for Zero Resource Keyword Search}},
      booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)},
      year = {2015},
      numpages = {5},
      url = {http://www.clsp.jhu.edu/~ajansen/papers/ICASSP2015a.pdf}
    }
    

    The task of zero resource query-by-example keyword search has received much attention in recent years as the speech technology needs of the developing world grow. These systems traditionally rely upon dynamic time warping (DTW) based retrieval algorithms with runtimes that are linear in the size of the search collection. As a result, their scalability substantially lags that of their supervised counterparts, which take advantage of efficient word-based indices. In this paper, we present a novel audio indexing approach called Segmental Randomized Acoustic Indexing and Logarithmic-time Search (S-RAILS). S-RAILS generalizes the original frame-based RAILS methodology to word-scale segments by exploiting a recently proposed acoustic segment embedding technique. By indexing word-scale segments directly, we avoid higher cost frame-based processing of RAILS while taking advantage of the improved lexical discrimination of the embeddings. Using the same conversational telephone speech benchmark, we demonstrate major improvements in both speed and accuracy over the original RAILS system.

72. Pushpendre Rastogi, Benjamin Van Durme, and Raman Arora. 2015. Multiview LSA: Representation Learning via Generalized CCA. In Proceedings of the Annual Meeting of the North American Association of Computational Linguistics (NAACL). [pdf] [bibtex] [abstract]

    @inproceedings{multiview-lsa-representation-learning-via-generalized-cca,
      author = {Rastogi, Pushpendre and {Van Durme}, Benjamin and Arora, Raman},
      title = {{Multiview LSA: Representation Learning via Generalized CCA}},
      booktitle = {Proceedings of the Annual Meeting of the North American Association of Computational Linguistics (NAACL)},
      year = {2015},
      numpages = {11},
      url = {http://www.aclweb.org/anthology/N15-1058}
    }
    

    Multiview LSA (MVLSA) is a generalization of Latent Semantic Analysis (LSA) that supports the fusion of arbitrary views of data and relies on Generalized Canonical Correlation Analysis (GCCA). We present an algorithm for fast approximate computation of GCCA, which when coupled with methods for handling missing values, is general enough to approximate some recent algorithms for inducing vector representations of words. Experiments across a comprehensive collection of test-sets show our approach to be competitive with the state of the art.

73. Travis Wolfe, Mark Dredze, and Benjamin Van Durme. 2015. Predicate Argument Alignment using a Global Coherence Model. In Proceedings of the Annual Meeting of the North American Association of Computational Linguistics (NAACL). [pdf] [bibtex] [abstract]

    @inproceedings{predicate-argument-alignment-using-a-global-coherence-model,
      author = {Wolfe, Travis and Dredze, Mark and {Van Durme}, Benjamin},
      title = {{Predicate Argument Alignment using a Global Coherence Model}},
      booktitle = {Proceedings of the Annual Meeting of the North American Association of Computational Linguistics (NAACL)},
      year = {2015},
      numpages = {11},
      url = {http://www.aclweb.org/anthology/N15-1002}
    }
    

    We present a joint model for predicate argument alignment. We leverage multiple sources of semantic information, including temporal ordering constraints between events. These are combined in a max-margin framework to find a globally consistent view of entities and events across multiple documents, which leads to improvements over a very strong local baseline.

74. Rachel Rudinger, Vera Demberg, Ashutosh Modi, Benjamin Van Durme, and Manfred Pinkal. 2015. Learning to predict script events from domain-specific text. In Joint Conference on Lexical and Computational Semantics (StarSem). [pdf] [bibtex] [abstract]

    @inproceedings{learning-to-predict-script-events-from-domain-specific-text,
      author = {Rudinger, Rachel and Demberg, Vera and Modi, Ashutosh and {Van Durme}, Benjamin and Pinkal, Manfred},
      title = {{Learning to predict script events from domain-specific text}},
      booktitle = {Joint Conference on Lexical and Computational Semantics (StarSem)},
      year = {2015},
      numpages = {6},
      url = {http://www.aclweb.org/anthology/S15-1024}
    }
    

    The automatic induction of scripts (Schank and Abelson, 1977) has been the focus of many recent works. In this paper, we employ a variety of these methods to learn Schank and Abelson?s canonical restaurant script, using a novel dataset of restaurant narratives we have compiled from a website called ?Dinners from Hell.? Our models learn narrative chains, script-like structures that we evaluate with the ?narrative cloze? task (Chambers and Jurafsky, 2008).

75. Travis Wolfe, Mark Dredze, James Mayfield, Paul McNamee, Craig Harman, Tim Finin, and Benjamin Van Durme. 2015. Interactive Knowledge Base Population. Preprint, arXiv:1506.00301. [pdf] [bibtex] [abstract]

    @unpublished{interactive-knowledge-base-population,
      author = {Wolfe, Travis and Dredze, Mark and Mayfield, James and McNamee, Paul and Harman, Craig and Finin, Tim and {Van Durme}, Benjamin},
      title = {Interactive Knowledge Base Population},
      note = {Preprint, arXiv:1506.00301},
      year = {2015},
      url = {https://arxiv.org/abs/1506.00301}
    }
    

    Most work on building knowledge bases has focused on collecting entities and facts from as large a collection of documents as possible. We argue for and describe a new paradigm where the focus is on a high-recall extraction over a small collection of documents under the supervision of a human expert, that we call Interactive Knowledge Base Population (IKBP).

76. Nanyun Peng, Francis Ferraro, Mo Yu, Nicholas Andrews, Jay DeYoung, Max Thomas, Matthew R. Gormley, Travis Wolfe, Craig Harman, Benjamin Van Durme, and Mark Dredze. 2015. A Concrete Chinese NLP Pipeline. In Proceedings of the Annual Meeting of the North American Association of Computational Linguistics (NAACL), Demonstration Track. [pdf] [bibtex] [abstract]

    @inproceedings{a-concrete-chinese-nlp-pipeline,
      title = {{A Concrete Chinese NLP Pipeline}},
      author = {Peng, Nanyun and Ferraro, Francis and Yu, Mo and Andrews, Nicholas and DeYoung, Jay and Thomas, Max and Gormley, Matthew R. and Wolfe, Travis and Harman, Craig and {Van Durme}, Benjamin and Dredze, Mark},
      booktitle = {Proceedings of the Annual Meeting of the North American Association of Computational Linguistics (NAACL), Demonstration Track},
      year = {2015},
      numpages = {5},
      url = {http://www.aclweb.org/anthology/N15-3018}
    }
    

    Natural language processing research increasingly relies on the output of a variety of syntactic and semantic analytics. Yet integrating output from multiple analytics into a single framework can be time consuming and slow research progress. We present a CONCRETE Chinese NLP Pipeline: an NLP stack built using a series of open source systems integrated based on the CONCRETE data schema. Our pipeline includes data ingest, word segmentation, part of speech tagging, parsing, named entity recognition, relation extraction and cross document coreference resolution. Additionally, we integrate a tool for visualizing these annotations as well as allowing for the manual annotation of new data. We release our pipeline to the research community to facilitate work on Chinese language tasks that require rich linguistic annotations.

77. Svitlana Volkova and Benjamin Van Durme. 2015. Online Bayesian Models for Personal Analytics in Social Media. In AAAI Conference on Artificial Intelligence (AAAI). [pdf] [bibtex] [abstract]

    @inproceedings{online-bayesian-models-for-personal-analytics-in-social-media,
      author = {Volkova, Svitlana and {Van Durme}, Benjamin},
      title = {{Online Bayesian Models for Personal Analytics in Social Media}},
      booktitle = {AAAI Conference on Artificial Intelligence (AAAI)},
      year = {2015},
      numpages = {7},
      url = {https://pdfs.semanticscholar.org/f08b/86e0ae9903011df031babb42c12c2328df6a.pdf}
    }
    

    Latent author attribute prediction in social media provides a novel set of conditions for the construction of supervised classification models. With individual authors as training and test instances, their associated content (?features?) are made available incrementally over time, as they converse over discussion forums. We propose various approaches to handling this dynamic data, from traditional batch training and testing, to incremental bootstrapping, and then active learning via crowdsourcing. Our underlying model relies on an intuitive application of Bayes rule, which should be easy to adopt by the community, thus allowing for a general shift towards online modeling for social media.

78. Adrian Benton, Jay Deyoung, Adam Teichert, Mark Dredze, Benjamin Van Durme, Stephen Mayhew, and Max Thomas. 2014. Faster (and Better) Entity Linking with Cascades. In NIPS Workshop on Automated Knowledge Base Construction (AKBC). [pdf] [bibtex] [abstract]

    @inproceedings{faster-and-better-entity-linking-with-cascades,
      author = {Benton, Adrian and Deyoung, Jay and Teichert, Adam and Dredze, Mark and {Van Durme}, Benjamin and Mayhew, Stephen and Thomas, Max},
      title = {{Faster (and Better) Entity Linking with Cascades}},
      booktitle = {NIPS Workshop on Automated Knowledge Base Construction (AKBC)},
      year = {2014},
      numpages = {6},
      url = {https://www.cs.jhu.edu/~mdredze/publications/2014_nips_slinky_cascades.pdf}
    }
    

    Entity linking requires ranking thousands of candidates for each query, a time consuming process and a challenge for large scale linking. Many systems rely on prediction cascades to efficiently rank candidates. However, the design of these cascades often requires manual decisions about pruning and feature use, limiting the effectiveness of cascades. We present Slinky, a modular, flexible, fast and accurate entity linker based on prediction cascades. We adapt the web-ranking prediction cascade learning algorithm, Cronus, in order to learn cascades that are both accurate and fast. We show that by balancing between accurate and fast linking, this algorithm can produce Slinky configurations that are significantly faster and more accurate than a baseline configuration and an alternate cascade learning method with a fixed introduction of features.

79. Xuchen Yao and Benjamin Van Durme. 2014. Information Extraction over Structured Data: Question Answering with Freebase. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL). [pdf] [bibtex] [abstract]

    @inproceedings{information-extraction-over-structured-data-question-answering-with-freebase,
      author = {Yao, Xuchen and {Van Durme}, Benjamin},
      title = {{Information Extraction over Structured Data: Question Answering with Freebase}},
      booktitle = {Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL)},
      year = {2014},
      numpages = {11},
      url = {http://www.aclweb.org/anthology/P14-1090}
    }
    

    Answering natural language questions using the Freebase knowledge base has recently been explored as a platform for advancing the state of the art in open domain semantic parsing. Those efforts map questions to sophisticated meaning representations that are then attempted to be matched against viable answer candidates in the knowledge base. Here we show that relatively modest information extraction techniques, when paired with a web-scale corpus, can outperform these sophisticated approaches by roughly 34% relative gain.

80. Matthew R. Gormley, Margaret Mitchell, Benjamin Van Durme, and Mark Dredze. 2014. Low-Resource Semantic Role Labeling. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL). [pdf] [bibtex] [abstract]

    @inproceedings{low-resource-semantic-role-labeling,
      author = {Gormley, Matthew R. and Mitchell, Margaret and {Van Durme}, Benjamin and Dredze, Mark},
      title = {{Low-Resource Semantic Role Labeling}},
      booktitle = {Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL)},
      year = {2014},
      numpages = {11},
      url = {https://www.aclweb.org/anthology/P/P14/P14-1111.pdf}
    }
    

    We explore the extent to which high-resource manual annotations such as treebanks are necessary for the task of semantic role labeling (SRL). We examine how performance changes without syntactic supervision, comparing both joint and pipelined methods to induce latent syntax. This work highlights a new application of unsupervised grammar induction and demonstrates several approaches to SRL in the absence of supervised syntax. Our best models obtain competitive results in the high-resource setting and state-of-the-art results in the low resource setting, reaching 72.48% F1 averaged across languages. We release our code for this work along with a larger toolkit for specifying arbitrary graphical structure.

81. Svitlana Volkova, Glen Coppersmith, and Benjamin Van Durme. 2014. Inferring User Political Preferences from Streaming Communications. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL). [pdf] [bibtex] [abstract]

    @inproceedings{inferring-user-political-preferences-from-streaming-communications,
      author = {Volkova, Svitlana and Coppersmith, Glen and {Van Durme}, Benjamin},
      title = {{Inferring User Political Preferences from Streaming Communications}},
      booktitle = {Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL)},
      year = {2014},
      numpages = {11},
      url = {http://www.aclweb.org/anthology/P14-1018}
    }
    

    Existing models for social media personal analytics assume access to thousands of messages per user, even though most users author content only sporadically over time. Given this sparsity, we: (i) leverage content from the local neighborhood of a user; (ii) evaluate batch models as a function of size and the amount of messages in various types of neighborhoods; and (iii) estimate the amount of time and tweets required for a dynamic model to predict user preferences. We show that even when limited or no self-authored data is available, language from friend, retweet and user mention communications provide sufficient evidence for prediction. When updating models over time based on Twitter, we find that political preference can be often be predicted using roughly 100 tweets, depending on the context of user selection, where this could mean hours, or weeks, based on the author?s tweeting frequency.

82. Chandler May, Alex Clemmer, and Benjamin Van Durme. 2014. Particle Filter Rejuvenation and Latent Dirichlet Allocation. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL). [pdf] [bibtex] [abstract]

    @inproceedings{particle-filter-rejuvenation-and-latent-dirichlet-allocation,
      author = {May, Chandler and Clemmer, Alex and {Van Durme}, Benjamin},
      title = {{Particle Filter Rejuvenation and Latent Dirichlet Allocation}},
      booktitle = {Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL)},
      year = {2014},
      numpages = {6},
      url = {http://www.aclweb.org/anthology/P14-2073}
    }
    

    Previous research has established several methods of online learning for latent Dirichlet allocation (LDA). However, streaming learning for LDA? allowing only one pass over the data and constant storage complexity?is not as well explored. We use reservoir sampling to reduce the storage complexity of a previously-studied online algorithm, namely the particle filter, to constant. We then show that a simpler particle filter implementation performs just as well, and that the quality of the initialization dominates other factors of performance.

83. Miles Osborne, Ashwin Lall, and Benjamin Van Durme. 2014. Exponential Reservoir Sampling for Streaming Language Models. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL). [pdf] [bibtex] [abstract]

    @inproceedings{exponential-reservoir-sampling-for-streaming-language-models,
      author = {Osborne, Miles and Lall, Ashwin and {Van Durme}, Benjamin},
      title = {{Exponential Reservoir Sampling for Streaming Language Models}},
      booktitle = {Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL)},
      year = {2014},
      numpages = {6},
      url = {http://www.aclweb.org/anthology/P14-2112}
    }
    

    We show how rapidly changing textual streams such as Twitter can be modelled in fixed space. Our approach is based upon a randomised algorithm called Exponential Reservoir Sampling, unexplored by this community until now. Using language models over Twitter and Newswire as a testbed, our experimental results based on perplexity support the intuition that recently observed data generally outweighs that seen in the past, but that at times, the past can have valuable signals enabling better modelling of the present.

84. Alex Fine, Austin Frank, T. Florian Jaeger, and Benjamin Van Durme. 2014. Biases in Predicting the Human Language Model. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL). [pdf] [bibtex] [abstract]

    @inproceedings{biases-in-predicting-the-human-language-model,
      author = {Fine, Alex and Frank, Austin and Jaeger, T. Florian and {Van Durme}, Benjamin},
      title = {{Biases in Predicting the Human Language Model}},
      booktitle = {Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL)},
      year = {2014},
      numpages = {6},
      url = {http://www.aclweb.org/anthology/P14-2002}
    }
    

    We consider the prediction of three human behavioral measures ? lexical decision, word naming, and picture naming ? through the lens of domain bias in language modeling. Contrasting the predictive ability of statistics derived from 6 different corpora, we find intuitive results showing that, e.g., a British corpus over-predicts the speed with which an American will react to the words ward and duke, and that the Google n-grams over-predicts familiarity with technology terms. This study aims to provoke increased consideration of the human language model by NLP practitioners: biases are not limited to differences between corpora (i.e. ?train? vs. ?test?); they can exist as well between corpora and the intended user of the resultant technology.

85. Charley Beller, Rebecca Knowles, Craig Harman, Shane Bergsma, Margaret Mitchell, and Benjamin Van Durme. 2014. I’m a Belieber: Social Roles via Self-identification and Conceptual Attributes. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL). [pdf] [bibtex] [abstract]

    @inproceedings{im-a-belieber-social-roles-via-self-identification-and-conceptual-attributes,
      author = {Beller, Charley and Knowles, Rebecca and Harman, Craig and Bergsma, Shane and Mitchell, Margaret and {Van Durme}, Benjamin},
      title = {{I'm a Belieber: Social Roles via Self-identification and Conceptual Attributes}},
      booktitle = {Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL)},
      year = {2014},
      numpages = {6},
      url = {http://www.aclweb.org/anthology/P14-2030}
    }
    

    Motivated by work predicting coarse-grained author categories in social media, such as gender or political preference, we explore whether Twitter contains information to support the prediction of fine-grained categories, or social roles. We find that the simple self-identification pattern ?I am a ? supports significantly richer classification than previously explored, successfully retrieving a variety of fine-grained roles. For a given role (e.g., writer), we can further identify characteristic attributes using a simple possessive construction (e.g., writer?s ). Tweets that incorporate the attribute terms in first person possessives (my ) are confirmed to be an indicator that the author holds the associated social role.

86. Xuchen Yao, Jonathan Berant, and Benjamin Van Durme. 2014. Freebase QA: Information Extraction or Semantic Parsing? In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL) Workshop: Semantic Parsing. [pdf] [bibtex] [abstract]

    @inproceedings{freebase-qa-information-extraction-or-semantic-parsing,
      author = {Yao, Xuchen and Berant, Jonathan and {Van Durme}, Benjamin},
      title = {{Freebase QA: Information Extraction or Semantic Parsing?}},
      booktitle = {Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL) Workshop: Semantic Parsing},
      year = {2014},
      numpages = {5},
      url = {http://aclweb.org/anthology/W14-2416}
    }
    

    We contrast two seemingly distinct approaches to the task of question answering (QA) using Freebase: one based on information extraction techniques, the other on semantic parsing. Results over the same test-set were collected from two state-of-the-art, open-source systems, then analyzed in consultation with those systems? creators. We conclude that the differences between these technologies, both in task performance, and in how they get there, is not significant. This suggests that the semantic parsing community should target answering more compositional open-domain questions that are beyond the reach of more direct information extraction methods.

87. Keisuke Sakaguchi, Matt Post, and Benjamin Van Durme. 2014. Efficient Elicitation of Annotations for Human Evaluation of Machine Translation. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL) Workshop: WMT. [pdf] [bibtex] [abstract]

    @inproceedings{efficient-elicitation-of-annotations-for-human-evaluation-of-machine-translation,
      author = {Sakaguchi, Keisuke and Post, Matt and {Van Durme}, Benjamin},
      title = {{Efficient Elicitation of Annotations for Human Evaluation of Machine Translation}},
      booktitle = {Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL) Workshop: WMT},
      year = {2014},
      numpages = {11},
      url = {http://aclweb.org/anthology/W14-3301}
    }
    

    A main output of the annual Workshop on Statistical Machine Translation (WMT) is a ranking of the systems that participated in its shared translation tasks, produced by aggregating pairwise sentence-level comparisons collected from human judges. Over the past few years, there have been a number of tweaks to the aggregation formula in attempts to address issues arising from the inherent ambiguity and subjectivity of the task, as well as weaknesses in the proposed models and the manner of model selection. We continue this line of work by adapting the TrueSkillTM algorithm ? an online approach for modeling the relative skills of players in ongoing competitions, such as Microsoft?s Xbox Live ? to the human evaluation of machine translation output. Our experimental results show that TrueSkill outperforms other recently proposed models on accuracy, and also can significantly reduce the number of pairwise annotations that need to be collected by sampling non-uniformly from the space of system competitions.

88. Jacqueline Aguilar, Charley Beller, Paul McNamee, Benjamin Van Durme, Stephanie Strassel, Zhiyi Song, and Joe Ellis. 2014. A Comparison of the Events and Relations Across ACE, ERE, TAC-KBP, and FrameNet Annotation Standards. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL) Workshop: EVENTS. [pdf] [bibtex] [abstract]

    @inproceedings{a-comparison-of-the-events-and-relations-across-ace-ere-tac-kbp-and-framenet-annotation-standards,
      author = {Aguilar, Jacqueline and Beller, Charley and McNamee, Paul and {Van Durme}, Benjamin and Strassel, Stephanie and Song, Zhiyi and Ellis, Joe},
      title = {{A Comparison of the Events and Relations Across ACE, ERE, TAC-KBP, and FrameNet Annotation Standards}},
      booktitle = {Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL) Workshop: EVENTS},
      year = {2014},
      numpages = {9},
      url = {http://www.aclweb.org/anthology/W14-2907}
    }
    

    The resurgence of effort within computational semantics has led to increased interest in various types of relation extraction and semantic parsing. While various manually annotated resources exist for enabling this work, these materials have been developed with different standards and goals in mind. In an effort to develop better general understanding across these resources, we provide a summary overview of the standards underlying ACE, ERE, TAC-KBP Slot-filling, and FrameNet.

89. Rachel Rudinger and Benjamin Van Durme. 2014. Is the Stanford Dependency Representation Semantic? In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL) Workshop: EVENTS. [pdf] [bibtex] [abstract]

    @inproceedings{is-the-stanford-dependency-representation-semantic?,
      author = {Rudinger, Rachel and {Van Durme}, Benjamin},
      title = {{Is the Stanford Dependency Representation Semantic?}},
      booktitle = {Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL) Workshop: EVENTS},
      year = {2014},
      numpages = {5},
      url = {http://www.aclweb.org/anthology/W14-2908}
    }
    

    The Stanford Dependencies are a deep syntactic representation that are widely used for semantic tasks, like Recognizing Textual Entailment. But do they capture all of the semantic information a meaning representation ought to convey? This paper explores this question by investigating the feasibility of mapping Stanford dependency parses to Hobbsian Logical Form, a practical, event-theoretic semantic representation, using only a set of deterministic rules. Although we find that such a mapping is possible in a large number of cases, we also find cases for which such a mapping seems to require information beyond what the Stanford Dependencies encode. These cases shed light on the kinds of semantic information that are and are not present in the Stanford Dependencies.

90. Pushpendre Rastogi and Benjamin Van Durme. 2014. Augmenting FrameNet Via PPDB. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL) Workshop: EVENTS. [pdf] [bibtex] [abstract]

    @inproceedings{augmenting-framenet-via-ppdb,
      author = {Rastogi, Pushpendre and {Van Durme}, Benjamin},
      title = {{Augmenting FrameNet Via PPDB}},
      booktitle = {Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL) Workshop: EVENTS},
      year = {2014},
      numpages = {5},
      url = {http://www.aclweb.org/anthology/W14-2901}
    }
    

    FrameNet is a lexico-semantic dataset that embodies the theory of frame semantics. Like other semantic databases, FrameNet is incomplete. We augment it via the paraphrase database, PPDB, and gain a threefold increase in coverage at 65% precision.

91. Francis Ferraro, Max Thomas, Matthew R. Gormley, Travis Wolfe, Craig Harman, and Benjamin Van Durme. 2014. Concretely Annotated Corpora. In NIPS Workshop on Automated Knowledge Base Construction (AKBC). [pdf] [bibtex]

    @inproceedings{concretely-annotated-corpora,
      author = {Ferraro, Francis and Thomas, Max and Gormley, Matthew R. and Wolfe, Travis and Harman, Craig and {Van Durme}, Benjamin},
      title = {{Concretely Annotated Corpora}},
      booktitle = {NIPS Workshop on Automated Knowledge Base Construction (AKBC)},
      year = {2014},
      numpages = {7},
      url = {http://www.cs.cmu.edu/~mgormley/papers/ferraro+al.nipsw.2014.pdf},
      _abstract = {missing}
    }
    

92. Jennifer Drexler, Pushpendre Rastogi, Jacqueline Aguilar, Benjamin Van Durme, and Matt Post. 2014. A Wikipedia-based Corpus for Contextualized Machine Translation. In The International Conference on Language Resources and Evaluation (LREC). [pdf] [bibtex] [abstract]

    @inproceedings{a-wikipedia-based-corpus-for-contextualized-machine-translation,
      author = {Drexler, Jennifer and Rastogi, Pushpendre and Aguilar, Jacqueline and {Van Durme}, Benjamin and Post, Matt},
      title = {{A Wikipedia-based Corpus for Contextualized Machine Translation}},
      booktitle = {The International Conference on Language Resources and Evaluation (LREC)},
      year = {2014},
      numpages = {4},
      url = {http://www.lrec-conf.org/proceedings/lrec2014/pdf/1217_Paper.pdf}
    }
    

    We describe a corpus for and experiments in target-contextualized machine translation (MT), in which we incorporate language models from target-language documents that are comparable in nature to the source documents. This corpus comprises (i) a set of curated English Wikipedia articles describing news events along with (ii) their comparable Spanish counterparts, (iii) a number of the Spanish source articles cited within them, and (iv) English reference translations of all the Spanish data. In experiments, we evaluate the effect on translation quality when including language models built over these English documents and interpolated with other, separately-derived, more general language model sources. We find that even under this simplistic baseline approach, we achieve significant improvements as measured by BLEU score.

93. Charley Beller, Craig Harman, and Benjamin Van Durme. 2014. Predicting Fine-grained Social Roles with Selectional Preferences. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL) Workshop: LACSS. [pdf] [bibtex] [abstract]

    @inproceedings{predicting-fine-grained-social-roles-with-selectional-preferences,
      author = {Beller, Charley and Harman, Craig and {Van Durme}, Benjamin},
      title = {{Predicting Fine-grained Social Roles with Selectional Preferences}},
      booktitle = {Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL) Workshop: LACSS},
      year = {2014},
      numpages = {6},
      url = {http://www.aclweb.org/anthology/W14-2515}
    }
    

    Selectional preferences, the tendencies of predicates to select for certain semantic classes of arguments, have been successfully applied to a number of tasks in computational linguistics including word sense disambiguation, semantic role labeling, relation extraction, and textual inference. Here we leverage the information encoded in selectional preferences to the task of predicting fine-grained categories of authors on the social media platform Twitter. First person uses of verbs that select for a given social role as subject (e.g. I teach ... for teacher) are used to quickly build up binary classifiers for that role.

94. Xuchen Yao, Benjamin Van Durme, Chris Callison-Burch, and Peter Clark. 2013. Semi-Markov Phrase-based Monolingual Alignment. In Empirical Methods in Natural Language Processing (EMNLP). [pdf] [bibtex] [abstract]

    @inproceedings{semi-markov-phrase-based-monolingual-alignment,
      author = {Yao, Xuchen and {Van Durme}, Benjamin and Callison-Burch, Chris and Clark, Peter},
      title = {{Semi-Markov Phrase-based Monolingual Alignment}},
      booktitle = {Empirical Methods in Natural Language Processing (EMNLP)},
      year = {2013},
      numpages = {11},
      url = {http://www.aclweb.org/anthology/D13-1056}
    }
    

    We introduce a novel discriminative model for phrase-based monolingual alignment using a semi-Markov CRF. Our model achieves state-of-the-art alignment accuracy on two phrase-based alignment datasets (RTE and paraphrase), while doing significantly better than other strong baselines in both non-identical alignment and phrase-only alignment. Additional experiments highlight the potential benefit of our alignment model to RTE, paraphrase identification and question answering, where even a naive application of our model?s alignment score approaches the state of the art.

95. Margaret Mitchell, Jacqui Aguilar, Theresa Wilson, and Benjamin Van Durme. 2013. Open Domain Targeted Sentiment. In Empirical Methods in Natural Language Processing (EMNLP). [pdf] [bibtex] [abstract]

    @inproceedings{open-domain-targeted-sentiment,
      author = {Mitchell, Margaret and Aguilar, Jacqui and Wilson, Theresa and {Van Durme}, Benjamin},
      title = {{Open Domain Targeted Sentiment}},
      booktitle = {Empirical Methods in Natural Language Processing (EMNLP)},
      year = {2013},
      numpages = {12},
      url = {http://www.aclweb.org/anthology/D13-1171}
    }
    

    We propose a novel approach to sentiment analysis for a low resource setting. The intuition behind this work is that sentiment expressed towards an entity, targeted sentiment, may be viewed as a span of sentiment expressed across the entity. This representation allows us to model sentiment detection as a sequence tagging problem, jointly discovering people and organizations along with whether there is sentiment directed towards them. We compare performance in both Spanish and English on microblog data, using only a sentiment lexicon as an external resource. By leveraging linguistically-informed features within conditional random fields (CRFs) trained to minimize empirical risk, our best models in Spanish significantly outperform a strong baseline, and reach around 90% accuracy on the combined task of named entity recognition and sentiment prediction. Our models in English, trained on a much smaller dataset, are not yet statistically significant against their baselines.

96. Juri Ganitkevitch, Benjamin Van Durme, and Chris Callison-Burch. 2013. PPDB: The Paraphrase Database. In Proceedings of the Annual Meeting of the North American Association of Computational Linguistics (NAACL). [pdf] [bibtex] [abstract]

    @inproceedings{ppdb-the-paraphrase-database,
      author = {Ganitkevitch, Juri and {Van Durme}, Benjamin and Callison-Burch, Chris},
      title = {{PPDB: The Paraphrase Database}},
      booktitle = {Proceedings of the Annual Meeting of the North American Association of Computational Linguistics (NAACL)},
      year = {2013},
      numpages = {7},
      url = {http://www.aclweb.org/anthology/N13-1092}
    }
    

    We present the 1.0 release of our paraphrase database, PPDB. Its English portion, PPDB:Eng, contains over 220 million paraphrase pairs, consisting of 73 million phrasal and 8 million lexical paraphrases, as well as 140 million paraphrase patterns, which capture many meaning-preserving syntactic transformations. The paraphrases are extracted from bilingual parallel corpora totaling over 100 million sentence pairs and over 2 billion English words. We also release PPDB:Spa, a collection of 196 million Spanish paraphrases. Each paraphrase pair in PPDB contains a set of associated scores, including paraphrase probabilities derived from the bitext data and a variety of monolingual distributional similarity scores computed from the Google n-grams and the Annotated Gigaword corpus. Our release includes pruning tools that allow users to determine their own precision/recall tradeoff.

97. Jonathan Gordon and Benjamin Van Durme. 2013. Reporting Bias and Knowledge Extraction. In Automated Knowledge Base Construction (AKBC) 2013: The 3rd Workshop on Knowledge Extraction, at CIKM. [pdf] [bibtex] [abstract]

    @inproceedings{reporting-bias-and-knowledge-extraction,
      author = {Gordon, Jonathan and {Van Durme}, Benjamin},
      title = {{Reporting Bias and Knowledge Extraction}},
      booktitle = {Automated Knowledge Base Construction (AKBC) 2013: The 3rd Workshop on Knowledge Extraction, at CIKM},
      year = {2013},
      numpages = {5},
      url = {https://openreview.net/pdf?id=AzxEzvpdE3Wcy}
    }
    

    Much work in knowledge extraction from text tacitly assumes that the frequency with which people write about actions, outcomes, or properties is a reflection of real-world frequencies or the degree to which a property is characteristic of a class of individuals. In this paper, we question this idea, examining the phenomenon of reporting bias and the challenge it poses for knowledge extraction. We conclude with discussion of approaches to learning commonsense knowledge from text in spite of this distortion.

98. Shane Bergsma and Benjamin Van Durme. 2013. Using Conceptual Class Attributes to Characterize Social Media Users. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL). [pdf] [bibtex] [abstract]

    @inproceedings{using-conceptual-class-attributes-to-characterize-social-media-users,
      author = {Bergsma, Shane and {Van Durme}, Benjamin},
      title = {{Using Conceptual Class Attributes to Characterize Social Media Users}},
      booktitle = {Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL)},
      year = {2013},
      numpages = {11},
      url = {http://www.aclweb.org/anthology/P13-1070}
    }
    

    We describe a novel approach for automatically predicting the hidden demographic properties of social media users. Building on prior work in common-sense knowledge acquisition from third-person text, we first learn the distinguishing attributes of certain classes of people. For example, we learn that people in the Female class tend to have maiden names and engagement rings. We then show that this knowledge can be used in the analysis of first-person communication; knowledge of distinguishing attributes allows us to both classify users and to bootstrap new training examples. Our novel approach enables substantial improvements on the widely-studied task of user gender prediction, obtaining a 20% relative error reduction over the current state-of-the-art.

99. Xuchen Yao, Benjamin Van Durme, Chris Callison-Burch, and Peter Clark. 2013. A Lightweight and High Performance Monolingual Word Aligner. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL). [pdf] [bibtex] [abstract]

    @inproceedings{a-lightweight-and-high-performance-monolingual-word-aligner,
      author = {Yao, Xuchen and {Van Durme}, Benjamin and Callison-Burch, Chris and Clark, Peter},
      title = {{A Lightweight and High Performance Monolingual Word Aligner}},
      booktitle = {Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL)},
      year = {2013},
      numpages = {6},
      url = {http://www.aclweb.org/anthology/P13-2123}
    }
    

    Fast alignment is essential for many natural language tasks. But in the setting of monolingual alignment, previous work has not been able to align more than one sentence pair per second. We describe a discriminatively trained monolingual word aligner that uses a Conditional Random Field to globally decode the best alignment with features drawn from source and target sentences. Using just part-of-speech tags and WordNet as external resources, our aligner gives state-of-the-art result, while being an order-of-magnitude faster than the previous best performing system.

100. Xuchen Yao, Benjamin Van Durme, and Peter Clark. 2013. Automatic Coupling of Answer Extraction and Information Retrieval. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL). [pdf] [bibtex] [abstract]

     @inproceedings{automatic-coupling-of-answer-extraction-and-information-retrieval,
       author = {Yao, Xuchen and {Van Durme}, Benjamin and Clark, Peter},
       title = {{Automatic Coupling of Answer Extraction and Information Retrieval}},
       booktitle = {Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL)},
       year = {2013},
       numpages = {7},
       url = {http://www.aclweb.org/anthology/P13-2029}
     }
     

     Information Retrieval (IR) and Answer Extraction are often designed as isolated or loosely connected components in Question Answering (QA), with repeated over-engineering on IR, and not necessarily performance gain for QA. We propose to tightly integrate them by coupling automatically learned features for answer extraction to a shallow-structured IR model. Our method is very quick to implement, and significantly improves IR for QA (measured in Mean Average Precision and Mean Reciprocal Rank) by 10%-20% against an uncoupled retrieval baseline in both document and passage retrieval, which further leads to a downstream 20% improvement in QA F1.

101. Travis Wolfe, Benjamin Van Durme, Mark Dredze, Nicholas Andrews, Charley Beller, Chris Callison-Burch, Jay DeYoung, Justin Snyder, Jonathan Weese, Tan Xu, and Xuchen Yao. 2013. PARMA: A Predicate Argument Aligner. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL). [pdf] [bibtex] [abstract]

     @inproceedings{parma-a-predicate-argument-aligner,
       author = {Wolfe, Travis and {Van Durme}, Benjamin and Dredze, Mark and Andrews, Nicholas and Beller, Charley and Callison-Burch, Chris and DeYoung, Jay and Snyder, Justin and Weese, Jonathan and Xu, Tan and Yao, Xuchen},
       title = {{PARMA: A Predicate Argument Aligner}},
       booktitle = {Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL)},
       year = {2013},
       numpages = {6},
       url = {http://www.aclweb.org/anthology/P13-2012}
     }
     

     We introduce PARMA, a system for cross-document, semantic predicate and argument alignment. Our system combines a number of linguistic resources familiar to researchers in areas such as recognizing textual entailment and question answering, integrating them into a simple discriminative model. PARMA achieves state of the art results on an existing and a new dataset. We suggest that previous efforts have focussed on data that is biased and too easy, and we provide a more difficult dataset based on translation data with a low baseline which we beat by 17% F1.

102. David Etter, Francis Ferraro, Ryan Cotterell, Olivia Buzek, and Benjamin Van Durme. 2013. Nerit: Named Entity Recognition for Informal Text. Technical Report 11, Human Language Technology Center of Excellence, Johns Hopkins University, Baltimore, Maryland. [pdf] [bibtex] [abstract]

     @techreport{nerit-named-entity-recognition-for-informal-text,
       author = {Etter, David and Ferraro, Francis and Cotterell, Ryan and Buzek, Olivia and {Van Durme}, Benjamin},
       title = {{Nerit: Named Entity Recognition for Informal Text}},
       number = {11},
       institution = {Human Language Technology Center of Excellence, Johns Hopkins University},
       address = {Baltimore, Maryland},
       year = {2013},
       numpages = {8},
       url = {http://www.cs.jhu.edu/~ferraro/papers/etter-nerit-2013.pdf}
     }
     

     We describe a multilingual named entity recognition system using language independent feature templates, designed for processing short, informal media arising from Twitter and other microblogging services. We crowdsource the annotation of tens of thousands of English and Spanish tweets and present classification results on this resource.

103. Francis Ferraro, Benjamin Van Durme, and Yanif Ahmad. 2013. Evaluating Progress in Probabilistic Programming through Topic Models. In NIPS Workshop on Topic Models: Computation, Application, and Evaluation. [pdf] [bibtex] [abstract]

     @inproceedings{evaluating-progress-in-probabilistic-programming-through-topic-models,
       author = {Ferraro, Francis and {Van Durme}, Benjamin and Ahmad, Yanif},
       title = {{Evaluating Progress in Probabilistic Programming through Topic Models}},
       booktitle = {NIPS Workshop on Topic Models: Computation, Application, and Evaluation},
       year = {2013},
       numpages = {5},
       url = {https://pdfs.semanticscholar.org/5847/79cfaf2e32b462d604ecd2bba01bd4e7a149.pdfhttps://pdfs.semanticscholar.org/5847/79cfaf2e32b462d604ecd2bba01bd4e7a149.pdf}
     }
     

     Topic models have proven versatile over the past decade, particularly as partial embeddings within more intricate models. These models present challenges that are analytic, computational and engineering in nature. Advances in probabilistic programming have the potential to circumvent a number of these issues, but researchers need a way to coarsely evaluate these frameworks. We identify three axes of a successful framework and argue that computational efficiency of straight LDA provides one such lens. We provide and release a modular open-source testbed to systematically capture one aspect of current probabilistic programming and discuss initial results on both heavily-controlled and ?real? data.

104. Shane Bergsma, Mark Dredze, Benjamin Van Durme, Theresa Wilson, and David Yarowsky. 2013. Broadly Improving User Classification via Communication-Based Name and Location Clustering on Twitter. In Proceedings of the Annual Meeting of the North American Association of Computational Linguistics (NAACL). [pdf] [bibtex] [abstract]

     @inproceedings{broadly-improving-user-classification-via-communication-based-name-and-location-clustering-on-twitter,
       author = {Bergsma, Shane and Dredze, Mark and {Van Durme}, Benjamin and Wilson, Theresa and Yarowsky, David},
       title = {{Broadly Improving User Classification via Communication-Based Name and Location Clustering on Twitter}},
       booktitle = {Proceedings of the Annual Meeting of the North American Association of Computational Linguistics (NAACL)},
       year = {2013},
       numpages = {10},
       url = {http://www.aclweb.org/anthology/N13-1121}
     }
     

     Hidden properties of social media users, such as their ethnicity, gender, and location, are often reflected in their observed attributes, such as their first and last names. Furthermore, users who communicate with each other often have similar hidden properties. We propose an algorithm that exploits these insights to cluster the observed attributes of hundreds of millions of Twitter users. Attributes such as user names are grouped together if users with those names communicate with other similar users. We separately cluster millions of unique first names, last names, and user-provided locations. The efficacy of these clusters is then evaluated on a diverse set of classification tasks that predict hidden users properties such as ethnicity, geographic location, gender, language, and race, using only profile names and locations when appropriate. Our readily-replicable approach and publicly-released clusters are shown to be remarkably effective and versatile, substantially outperforming state-of-the-art approaches and human accuracy on each of the tasks studied.

105. Xuchen Yao, Benjamin Van Durme, Peter Clark, and Chris Callison-Burch. 2013. Answer Extraction as Sequence Tagging with Tree Edit Distance. In Proceedings of the Annual Meeting of the North American Association of Computational Linguistics (NAACL). [pdf] [bibtex] [abstract]

     @inproceedings{answer-extraction-as-sequence-tagging-with-tree-edit-distance,
       author = {Yao, Xuchen and {Van Durme}, Benjamin and Clark, Peter and Callison-Burch, Chris},
       title = {{Answer Extraction as Sequence Tagging with Tree Edit Distance}},
       booktitle = {Proceedings of the Annual Meeting of the North American Association of Computational Linguistics (NAACL)},
       year = {2013},
       numpages = {10},
       url = {http://www.aclweb.org/anthology/N13-1106}
     }
     

     Our goal is to extract answers from pre-retrieved sentences for Question Answering (QA). We construct a linear-chain Conditional Random Field based on pairs of questions and their possible answer sentences, learning the association between questions and answer types. This casts answer extraction as an answer sequence tagging problem for the first time, where knowledge of shared structure between question and source sentence is incorporated through features based on Tree Edit Distance (TED). Our model is free of manually created question and answer templates, fast to run (processing 200 QA pairs per second excluding parsing time), and yields an F1 of 63.3% on a new public dataset based on prior TREC QA evaluations. The developed system is open-source, and includes an implementation of the TED model that is state of the art in the task of ranking QA pairs.

106. Courtney Napoles, Matthew R. Gormley, and Benjamin Van Durme. 2012. Annotated Gigaword. In NAACL Workshop: AKBC-WEKEX. [pdf] [bibtex] [abstract]

     @inproceedings{annotated-gigaword,
       author = {Napoles, Courtney and Gormley, Matthew R. and {Van Durme}, Benjamin},
       title = {{Annotated Gigaword}},
       booktitle = {NAACL Workshop: AKBC-WEKEX},
       year = {2012},
       numpages = {6},
       url = {http://www.aclweb.org/anthology/W12-3018}
     }
     

     We have created layers of annotation on the English Gigaword v.5 corpus to render it useful as a standardized corpus for knowledge extraction and distributional semantics. Most existing large-scale work is based on inconsistent corpora which often have needed to be re-annotated by research teams independently, each time introducing biases that manifest as results that are only comparable at a high level. We provide to the community a public reference set based on current state-of-the-art syntactic analysis and coreference resolution, along with an interface for programmatic access. Our goal is to enable broader involvement in large-scale knowledge-acquisition efforts by researchers that otherwise may not have had the ability to produce such a resource on their own.

107. Benjamin Van Durme. 2012. Streaming Analysis of Discourse Participants. In Empirical Methods in Natural Language Processing (EMNLP). [pdf] [bibtex] [abstract]

     @inproceedings{streaming-analysis-of-discourse-participants,
       author = {{Van Durme}, Benjamin},
       title = {{Streaming Analysis of Discourse Participants}},
       booktitle = {Empirical Methods in Natural Language Processing (EMNLP)},
       year = {2012},
       numpages = {11},
       url = {http://www.aclweb.org/anthology/D12-1005}
     }
     

     Inferring attributes of discourse participants has been treated as a batch-processing task: data such as all tweets from a given author are gathered in bulk, processed, analyzed for a particular feature, then reported as a result of academic interest. Given the sources and scale of material used in these efforts, along with potential use cases of such analytic tools, discourse analysis should be reconsidered as a streaming challenge. We show that under certain common formulations, the batch-processing analytic framework can be decomposed into a sequential series of updates, using as an example the task of gender classification. Once in a streaming framework, and motivated by large data sets generated by social media services, we present novel results in approximate counting, showing its applicability to space efficient streaming classification.

108. Aren Jansen and Benjamin Van Durme. 2012. Indexing Raw Acoustic Features for Scalable Zero Resource Search. In Annual Conference of the International Speech Communication Association (InterSpeech). [pdf] [bibtex] [abstract]

     @inproceedings{indexing-raw-acoustic-features-for-scalable-zero-resource-search,
       author = {Jansen, Aren and {Van Durme}, Benjamin},
       title = {{Indexing Raw Acoustic Features for Scalable Zero Resource Search}},
       booktitle = {Annual Conference of the International Speech Communication Association (InterSpeech)},
       year = {2012},
       numpages = {4},
       url = {https://pdfs.semanticscholar.org/9f14/8ed7440c974ec8e54da1546f4986f3e4114a.pdf}
     }
     

     We present a new speech indexing and search scheme called Randomized Acoustic Indexing and Logarithmic-time Search (RAILS) that enables scalable query-by-example spoken term detection in the zero resource regime. RAILS is derived from our recent investigation into the application of randomized hashing and approximate nearest neighbor search algorithms to raw acoustic features. Our approach permits an approximate search through hundreds of hours of speech audio in a matter of seconds, and may be applied to any language without the need of a training corpus, acoustic model, or pronunciation lexicon. The fidelity of the approximation is controlled through a small number of easily interpretable parameters that allow a trade-off between search accuracy and speed.

109. Aren Jansen, Benjamin Van Durme, and Pascal Clark. 2012. The JHU-HLTCOE Spoken Web Search System for MediaEval 2012. In MediaEval Workshop. [pdf] [bibtex] [abstract]

     @inproceedings{the-jhu-hltcoe-spoken-web-search-system-for-mediaeval-2012,
       author = {Jansen, Aren and {Van Durme}, Benjamin and Clark, Pascal},
       title = {{The JHU-HLTCOE Spoken Web Search System for MediaEval 2012}},
       booktitle = {MediaEval Workshop},
       year = {2012},
       numpages = {2},
       url = {http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.307.8503&rep=rep1&type=pdf}
     }
     

     We present an overview for a truly zero resource query-by-example search system designed for the 2012 MediaEval Spoken Web Search task. Our system is based on the recently proposed randomized acoustic indexing and logarithmic-time search (RAILS) framework. The input is merely the raw acoustic observations for the query and search collection, requiring no trained models whatsoever, not even unsupervised ones. Even still the system is capable of search speeds of at least a thousand times faster than real time, and capable of producing competent zero resource performance.

110. Juri Ganitkevitch, Benjamin Van Durme, and Chris Callison-Burch. 2012. Monolingual Distributional Similarity for Text-to-Text Generation. In Joint Conference on Lexical and Computational Semantics (STARSEM). [pdf] [bibtex] [abstract]

     @inproceedings{monolingual-distributional-similarity-for-text-to-text-generation,
       author = {Ganitkevitch, Juri and {Van Durme}, Benjamin and Callison-Burch, Chris},
       title = {{Monolingual Distributional Similarity for Text-to-Text Generation}},
       booktitle = {Joint Conference on Lexical and Computational Semantics (STARSEM)},
       year = {2012},
       numpages = {9},
       url = {http://www.aclweb.org/anthology/S12-1034}
     }
     

     Previous work on paraphrase extraction and application has relied on either parallel datasets, or on distributional similarity metrics over large text corpora. Our approach combines these two orthogonal sources of information and directly integrates them into our paraphrasing system?s log-linear model. We compare different distributional similarity feature-sets and show significant improvements in grammaticality and meaning retention on the example text-to-text generation task of sentence compression, achieving state-of-the-art quality.

111. Matthew R. Gormley, Mark Dredze, Benjamin Van Durme, and Jason Eisner. 2012. Shared Components Topic Models. In Proceedings of the Annual Meeting of the North American Association of Computational Linguistics (NAACL). [pdf] [bibtex] [abstract]

     @inproceedings{shared-components-topic-models,
       author = {Gormley, Matthew R. and Dredze, Mark and {Van Durme}, Benjamin and Eisner, Jason},
       title = {{Shared Components Topic Models}},
       booktitle = {Proceedings of the Annual Meeting of the North American Association of Computational Linguistics (NAACL)},
       year = {2012},
       numpages = {10},
       url = {http://www.aclweb.org/anthology/N12-1096}
     }
     

     With a few exceptions, extensions to latent Dirichlet allocation (LDA) have focused on the distribution over topics for each document. Much less attention has been given to the underlying structure of the topics themselves. As a result, most topic models generate topics independently from a single underlying distribution and require millions of parameters, in the form of multinomial distributions over the vocabulary. In this paper, we introduce the Shared Components Topic Model (SCTM), in which each topic is a normalized product of a smaller number of underlying component distributions. Our model learns these component distributions and the structure of how to combine subsets of them into topics. The SCTM can represent topics in a much more compact representation than LDA and achieves better perplexity with fewer parameters.

112. Francis Ferraro, Benjamin Van Durme, and Matt Post. 2012. Toward Tree Substitution Grammars with Latent Annotations. In Proceedings of the Annual Meeting of the North American Association of Computational Linguistics (NAACL), Workshop: Inducing Linguistic Structure. [pdf] [bibtex] [abstract]

     @inproceedings{toward-tree-substitution-grammars-with-latent-annotations,
       author = {Ferraro, Francis and {Van Durme}, Benjamin and Post, Matt},
       title = {{Toward Tree Substitution Grammars with Latent Annotations}},
       booktitle = {Proceedings of the Annual Meeting of the North American Association of Computational Linguistics (NAACL), Workshop: Inducing Linguistic Structure},
       year = {2012},
       numpages = {8},
       url = {http://www.aclweb.org/anthology/W12-1904}
     }
     

     We provide a model that extends the split-merge framework of Petrov et al. (2006) to jointly learn latent annotations and Tree Substitution Grammars (TSGs). We then conduct a variety of experiments with this model, first inducing grammars on a portion of the Penn Treebank and the Korean Treebank 2.0, and next experimenting with grammar refinement from a single nonterminal and from the Universal Part of Speech tagset. We present qualitative analysis showing promising signs across all experiments that our combined approach successfully provides for greater flexibility in grammar induction within the structured guidance provided by the treebank, leveraging the complementary natures of these two approaches.

113. Xuchen Yao, Benjamin Van Durme, and Chris Callison-Burch. 2012. Expectations of Word Sense in Parallel Corpora. In Proceedings of the Annual Meeting of the North American Association of Computational Linguistics (NAACL). [pdf] [bibtex] [abstract]

     @inproceedings{expectations-of-word-sense-in-parallel-corpora,
       author = {Yao, Xuchen and {Van Durme}, Benjamin and Callison-Burch, Chris},
       title = {{Expectations of Word Sense in Parallel Corpora}},
       booktitle = {Proceedings of the Annual Meeting of the North American Association of Computational Linguistics (NAACL)},
       year = {2012},
       numpages = {5},
       url = {http://www.aclweb.org/anthology/N12-1078}
     }
     

     Given a parallel corpus, if two distinct words in language A, a1 and a2, are aligned to the same word b1 in language B, then this might signal that b1 is polysemous, or it might signal a1 and a2 are synonyms. Both assumptions with successful work have been put forward in the literature. We investigate these assumptions, along with other questions of word sense, by looking at sampled parallel sentences containing tokens of the same type in English, asking how often they mean the same thing when they are: 1. aligned to the same foreign type; and 2. aligned to different foreign types. Results for French-English and Chinese-English parallel corpora show similar behavior: Synonymy is only very weakly the more prevalent scenario, where both cases regularly occur.

114. Brian Kjersten and Benjamin Van Durme. 2012. Space Efficiencies in Discourse Modeling via Conditional Random Sampling. In Proceedings of the Annual Meeting of the North American Association of Computational Linguistics (NAACL). [pdf] [bibtex] [abstract]

     @inproceedings{space-efficiencies-in-discourse-modeling-via-conditional-random-sampling,
       author = {Kjersten, Brian and {Van Durme}, Benjamin},
       title = {{Space Efficiencies in Discourse Modeling via Conditional Random Sampling}},
       booktitle = {Proceedings of the Annual Meeting of the North American Association of Computational Linguistics (NAACL)},
       year = {2012},
       numpages = {5},
       url = {http://www.aclweb.org/anthology/N12-1056}
     }
     

     Recent exploratory efforts in discourse-level language modeling have relied heavily on calculating Pointwise Mutual Information (PMI), which involves significant computation when done over large collections. Prior work has required aggressive pruning or independence assumptions to compute scores on large collections. We show the method of Conditional Random Sampling, thus far an underutilized technique, to be a space-efficient means of representing the sufficient statistics in discourse that underly recent PMI-based work. This is demonstrated in the context of inducing Shankian script-like structures over news articles.

115. Frank Ferraro, Matthew Post, and Benjamin Van Durme. 2012. Judging Grammaticality with Count-Induced Tree Substitution Grammars. In Proceedings of the Annual Meeting of the North American Association of Computational Linguistics (NAACL) Workshop: BEA. [pdf] [bibtex] [abstract]

     @inproceedings{judging-grammaticality-with-count-induced-tree-substitution-grammars,
       author = {Ferraro, Frank and Post, Matthew and {Van Durme}, Benjamin},
       title = {{Judging Grammaticality with Count-Induced Tree Substitution Grammars}},
       booktitle = {Proceedings of the Annual Meeting of the North American Association of Computational Linguistics (NAACL) Workshop: BEA},
       year = {2012},
       numpages = {6},
       url = {http://www.aclweb.org/anthology/W12-2013}
     }
     

     Prior work has shown the utility of syntactic tree fragments as features in judging the grammaticality of text. To date such fragments have been extracted from derivations of Bayesian-induced Tree Substitution Grammars (TSGs). Evaluating on discriminative coarse and fine grammaticality classification tasks, we show that a simple, deterministic, count-based approach to fragment identification performs on par with the more complicated grammars of Post (2011). This represents a significant reduction in complexity for those interested in the use of such fragments in the development of systems for the educational domain.

116. Benjamin Van Durme. 2012. Jerboa: A Toolkit for Randomized and Streaming Algorithms. Technical Report 7, Human Language Technology Center of Excellence, Johns Hopkins University, Baltimore, Maryland. [pdf] [bibtex] [abstract]

     @techreport{jerboa-a-toolkit-for-randomized-and-streaming-algorithms,
       author = {{Van Durme}, Benjamin},
       title = {{Jerboa: A Toolkit for Randomized and Streaming Algorithms}},
       number = {7},
       institution = {Human Language Technology Center of Excellence, Johns Hopkins University},
       address = {Baltimore, Maryland},
       year = {2012},
       numpages = {7},
       url = {http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.435.1859&rep=rep1&type=pdf}
     }
     

     Recent studies have shown the applicability of streaming and randomized algorithms in a variety of large-scale language mining tasks. However, lack of many publicly available implementations of these methods has limited the use of these techniques beyond initial proof-of-concept, despite the growing interest in large-scale data. Jerboa is a Java-based toolkit aimed at providing reference implementations of a number of methods seen in recent literature, meant at enabling a greater adoption of these techniques for tasks in human language technology.

117. Vinodkumar Prabhakaran, Michael Bloodgood, Mona Diab, Bonnie Dorr, Lori Levin, Christine D. Piatko, Owen Rambow, and Benjamin Van Durme. 2012. Statistical Modality Tagging from Rule-based Annotations and Crowdsourcing. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL), Workshop: ExProM. [pdf] [bibtex] [abstract]

     @inproceedings{statistical-modality-tagging-from-rule-based-annotations-and-crowdsourcing,
       author = {Prabhakaran, Vinodkumar and Bloodgood, Michael and Diab, Mona and Dorr, Bonnie and Levin, Lori and Piatko, Christine D. and Rambow, Owen and {Van Durme}, Benjamin},
       title = {{Statistical Modality Tagging from Rule-based Annotations and Crowdsourcing}},
       booktitle = {Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL), Workshop: ExProM},
       year = {2012},
       numpages = {8},
       url = {http://www.aclweb.org/anthology/W12-3807}
     }
     

     We explore training an automatic modality tagger. Modality is the attitude that a speaker might have toward an event or state. One of the main hurdles for training a linguistic tagger is gathering training data. This is particularly problematic for training a tagger for modality because modality triggers are sparse for the overwhelming majority of sentences. We investigate an approach to automatically training a modality tagger where we first gathered sentences based on a high-recall simple rule-based modality tagger and then provided these sentences to Mechanical Turk annotators for further annotation. We used the resulting set of training data to train a precise modality tagger using a multi-class SVM that delivers good performance.

118. Matthew R. Gormley, Mark Dredze, Benjamin Van Durme, and Jason Eisner. 2011. Shared Components Topic Models with Application to Selectional Preference. In NIPS Workshop: Learning Semantics. [pdf] [bibtex]

     @inproceedings{shared-components-topic-models-with-application-to-selectional-preference,
       author = {Gormley, Matthew R. and Dredze, Mark and {Van Durme}, Benjamin and Eisner, Jason},
       title = {{Shared Components Topic Models with Application to Selectional Preference}},
       booktitle = {NIPS Workshop: Learning Semantics},
       year = {2011},
       numpages = {3},
       url = {https://www.cs.jhu.edu/~jason/papers/gormley+al.nipsw11.pdf},
       _abstract = {missing}
     }
     

119. Aren Jansen and Benjamin Van Durme. 2011. Efficient Spoken Term Discovery using Randomized Algorithms. In Automatic Speech Recognition and Understanding Workshop (ASRU). [pdf] [bibtex] [abstract]

     @inproceedings{efficient-spoken-term-discovery-using-randomized-algorithms,
       author = {Jansen, Aren and {Van Durme}, Benjamin},
       title = {{Efficient Spoken Term Discovery using Randomized Algorithms}},
       booktitle = {Automatic Speech Recognition and Understanding Workshop (ASRU)},
       year = {2011},
       numpages = {6},
       url = {http://www.clsp.jhu.edu/~ajansen/papers/ASRU2011a.pdf}
     }
     

     Spoken term discovery is the task of automatically identifying words and phrases in speech data by searching for long repeated acoustic patterns. Initial solutions relied on exhaustive dynamic time warping-based searches across the entire similarity matrix, a method whose scalability is ultimately limited by the O(n^2 ) nature of the search space. Recent strategies have attempted to improve search efficiency by using either unsupervised or mismatched-language acoustic models to reduce the complexity of the feature representation. Taking a completely different approach, this paper investigates the use of randomized algorithms that operate directly on the raw acoustic features to produce sparse approximate similarity matrices in O(n) space and O(n log n) time. We demonstrate these techniques facilitate spoken term discovery performance capable of outperforming a model-based strategy in the zero resource setting.

120. Juri Ganitkevitch, Chris Callison-Burch, Courtney Napoles, and Benjamin Van Durme. 2011. Learning Sentential Paraphrases from Bilingual Parallel Corpora for Text-to-Text Generation. In Empirical Methods in Natural Language Processing (EMNLP). [pdf] [bibtex] [abstract]

     @inproceedings{learning-sentential-paraphrases-from-bilingual-parallel-corpora-for-text-to-text-generation,
       author = {Ganitkevitch, Juri and Callison-Burch, Chris and Napoles, Courtney and {Van Durme}, Benjamin},
       title = {{Learning Sentential Paraphrases from Bilingual Parallel Corpora for Text-to-Text Generation}},
       booktitle = {Empirical Methods in Natural Language Processing (EMNLP)},
       year = {2011},
       numpages = {12},
       url = {http://www.aclweb.org/anthology/D11-1108}
     }
     

     Previous work has shown that high quality phrasal paraphrases can be extracted from bilingual parallel corpora. However, it is not clear whether bitexts are an appropriate resource for extracting more sophisticated sentential paraphrases, which are more obviously learnable from monolingual parallel corpora. We extend bilingual paraphrase extraction to syntactic paraphrases and demonstrate its ability to learn a variety of general paraphrastic transformations, including passivization, dative shift, and topicalization. We discuss how our model can be adapted to many text generation tasks by augmenting its feature set, development data, and parameter estimation routine. We illustrate this adaptation by using our paraphrase model for the task of sentence compression and achieve results competitive with state-of-the-art compression systems.

121. Tsz Ping Chan, Chris Callison-Burch, and Benjamin Van Durme. 2011. Reranking Bilingually Extracted Paraphrases Using Monolingual Distributional Similarity. In Empirical Methods in Natural Language Processing (EMNLP) Workshop: GEMS. [pdf] [bibtex] [abstract]

     @inproceedings{reranking-bilingually-extracted-paraphrases-using-monolingual-distributional-similarity,
       author = {Chan, Tsz Ping and Callison-Burch, Chris and {Van Durme}, Benjamin},
       title = {{Reranking Bilingually Extracted Paraphrases Using Monolingual Distributional Similarity}},
       booktitle = {Empirical Methods in Natural Language Processing (EMNLP) Workshop: GEMS},
       year = {2011},
       numpages = {9},
       url = {http://www.aclweb.org/anthology/W11-2504}
     }
     

     This paper improves an existing bilingual paraphrase extraction technique using monolingual distributional similarity to rerank candidate paraphrases. Raw monolingual data provides a complementary and orthogonal source of information that lessens the commonly observed errors in bilingual pivot-based methods. Our experiments reveal that monolingual scoring of bilingually extracted paraphrases has a significantly stronger correlation with human judgment for grammaticality than the probabilities assigned by the bilingual pivoting method does. The results also show that monolingual distribution similarity can serve as a threshold for high precision paraphrase selection.

122. Xuchen Yao and Benjamin Van Durme. 2011. Nonparametric Bayesian Word Sense Induction. In Proceedings of the Annual Meeting of the Association of Computational Linguistics (ACL), Workshop: Textgraphs. [pdf] [bibtex] [abstract]

     @inproceedings{nonparametric-bayesian-word-sense-induction,
       author = {Yao, Xuchen and {Van Durme}, Benjamin},
       title = {{Nonparametric Bayesian Word Sense Induction}},
       booktitle = {Proceedings of the Annual Meeting of the Association of Computational Linguistics (ACL), Workshop: Textgraphs},
       year = {2011},
       numpages = {5},
       url = {http://www.aclweb.org/anthology/W11-1102}
     }
     

     We propose the use of a nonparametric Bayesian model, the Hierarchical Dirichlet Process (HDP), for the task of Word Sense Induction. Results are shown through comparison against Latent Dirichlet Allocation (LDA), a parametric Bayesian model employed by Brody and Lapata (2009) for this task. We find that the two models achieve similar levels of induction quality, while the HDP confers the advantage of automatically inducing a variable number of senses per word, as compared to manually fixing the number of senses a priori, as in LDA. This flexibility allows for the model to adapt to terms with greater or lesser polysemy, when evidenced by corpus distributional statistics. When trained on out-of-domain data, experimental results confirm the model?s ability to make use of a restricted set of topically coherent induced senses, when then applied in a restricted domain.

123. Courtney Napoles, Chris Callison-Burch, Juri Ganitkevitch, and Benjamin Van Durme. 2011. Paraphrastic Sentence Compression with a Character-based Metric: Tightening without Deletion. In Proceedings of the Annual Meeting of the Association of Computational Linguistics (ACL), Workshop on Monolingual Text-To-Text Generation. [pdf] [bibtex] [abstract]

     @inproceedings{paraphrastic-sentence-compression-with-a-character-based-metric-tightening-without-deletion,
       author = {Napoles, Courtney and Callison-Burch, Chris and Ganitkevitch, Juri and {Van Durme}, Benjamin},
       title = {{Paraphrastic Sentence Compression with a Character-based Metric: Tightening without Deletion}},
       booktitle = {Proceedings of the Annual Meeting of the Association of Computational Linguistics (ACL), Workshop on Monolingual Text-To-Text Generation},
       year = {2011},
       numpages = {7},
       url = {https://aclanthology.info/pdf/W/W11/W11-1610.pdf}
     }
     

     We present a substitution-only approach to sentence compression which ?tightens? a sentence by reducing its character length. Replacing phrases with shorter paraphrases yields paraphrastic compressions as short as 60% of the original length. In support of this task, we introduce a novel technique for re-ranking paraphrases extracted from bilingual corpora. At high compression rates1 paraphrastic compressions outperform a state-of-the-art deletion model in an oracle experiment. For further compression, deleting from oracle paraphrastic compressions preserves more meaning than deletion alone. In either setting, paraphrastic compression shows promise for surpassing deletion-only methods.

124. Courtney Napoles, Chris Callison-Burch, and Benjamin Van Durme. 2011. Evaluating sentence compression: Pitfalls and suggested remedies. In Proceedings of the Annual Meeting of the Association of Computational Linguistics (ACL), Workshop on Monolingual Text-To-Text Generation. [pdf] [bibtex] [abstract]

     @inproceedings{evaluating-sentence-compression-pitfalls-and-suggested-remedies,
       author = {Napoles, Courtney and Callison-Burch, Chris and {Van Durme}, Benjamin},
       title = {{Evaluating sentence compression: Pitfalls and suggested remedies}},
       booktitle = {Proceedings of the Annual Meeting of the Association of Computational Linguistics (ACL), Workshop on Monolingual Text-To-Text Generation},
       year = {2011},
       numpages = {5},
       url = {http://www.aclweb.org/anthology/W11-1611}
     }
     

     This work surveys existing evaluation methodologies for the task of sentence compression, identifies their shortcomings, and proposes alternatives. In particular, we examine the problems of evaluating paraphrastic compression and comparing the output of different models. We demonstrate that compression rate is a strong predictor of compression quality and that perceived improvement over other models is often a side effect of producing longer output.

125. Byung Gyu Ahn, Benjamin Van Durme, and Chris Callison-Burch. 2011. WikiTopics: What is popular on Wikipedia and why. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL), Workshop on Summarization. [pdf] [bibtex] [abstract]

     @inproceedings{wikitopics-what-is-popular-on-wikipedia-and-why,
       author = {Ahn, Byung Gyu and {Van Durme}, Benjamin and Callison-Burch, Chris},
       title = {{WikiTopics: What is popular on Wikipedia and why}},
       booktitle = {Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL), Workshop on Summarization},
       year = {2011},
       numpages = {5},
       url = {http://www.aclweb.org/anthology/W11-0505}
     }
     

     We establish a novel task in the spirit of news summarization and topic detection and tracking (TDT): daily determination of the topics newly popular with Wikipedia readers. Central to this effort is a new public dataset consisting of the hourly page view statistics of all Wikipedia articles over the last three years. We give baseline results for the tasks of: discovering individual pages of interest, clustering these pages into coherent topics, and extracting the most relevant summarizing sentence for the reader. When compared to human judgements, our system shows the viability of this task, and opens the door to a range of exciting future work.

126. Benjamin Van Durme and Ashwin Lall. 2011. Efficient Online Locality Sensitive Hashing via Reservoir Counting. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL). [pdf] [bibtex] [abstract]

     @inproceedings{efficient-online-locality-sensitive-hashing-via-reservoir-counting,
       author = {{Van Durme}, Benjamin and Lall, Ashwin},
       title = {{Efficient Online Locality Sensitive Hashing via Reservoir Counting}},
       booktitle = {Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL)},
       year = {2011},
       numpages = {5},
       url = {http://www.aclweb.org/anthology/P11-2004}
     }
     

     We describe a novel mechanism called Reservoir Counting for application in online Locality Sensitive Hashing. This technique allows for significant savings in the streaming setting, allowing for maintaining a larger number of signatures, or an increased level of approximation accuracy at a similar memory footprint.

127. Shane Bergsma and Benjamin Van Durme. 2011. Learning Bilingual Lexicons using the Visual Similarity of Labeled Web Images. In International Joint Conference on Artificial Intelligence (IJCAI-11). [pdf] [bibtex] [abstract]

     @inproceedings{learning-bilingual-lexicons-using-the-visual-similarity-of-labeled-web-images,
       author = {Bergsma, Shane and {Van Durme}, Benjamin},
       title = {{Learning Bilingual Lexicons using the Visual Similarity of Labeled Web Images}},
       booktitle = {International Joint Conference on Artificial Intelligence (IJCAI-11)},
       year = {2011},
       numpages = {6},
       url = {https://www.aaai.org/ocs/index.php/IJCAI/IJCAI11/paper/viewFile/3239/3743}
     }
     

     Speakers of many different languages use the Internet. A common activity among these users is uploading images and associating these images with words (in their own language) as captions, filenames, or surrounding text. We use these explicit, monolingual, image-to-word connections to successfully learn implicit, bilingual, word-to-word translations. Bilingual pairs of words are proposed as translations if their corresponding images have similar visual features. We generate bilingual lexicons in 15 language pairs, focusing on words that have been automatically identified as physical objects. The use of visual similarity substantially improves performance over standard approaches based on string similarity: for generated lexicons with 1000 translations, including visual information leads to an absolute improvement in accuracy of 8-12% over string edit distance alone.

128. Lenhart K. Schubert, Benjamin Van Durme, and Marzieh Bazrafshan. 2010. Entailment Inference in a Natural Logic-like General Reasoner. In AAAI Fall Symposium on Commonsense Knowledge (CSK10). [pdf] [bibtex] [abstract]

     @inproceedings{entailment-inference-in-a-natural-logic-like-general-reasoner,
       author = {Schubert, Lenhart K. and {Van Durme}, Benjamin and Bazrafshan, Marzieh},
       title = {{Entailment Inference in a Natural Logic-like General Reasoner}},
       booktitle = {AAAI Fall Symposium on Commonsense Knowledge (CSK10)},
       year = {2010},
       numpages = {6},
       url = {https://www.aaai.org/ocs/index.php/FSS/FSS10/paper/viewFile/2306/2597}
     }
     

     Recent work on entailment suggests that natural logics are well-suited to determining whether one sentence lexically entails another. We show how the EPILOG reasoning engine, designed for a natural language-like meaning representation (Episodic Logic, or EL), can be used to emulate natural logic inferences, while also enabling more general inferences such as ones from multiple premises, or ones based on world knowledge. Thus, to exploit the capabilities of EPILOG, we are working to populate its knowledge base with the kinds of lexical knowledge on which natural logics rely.

129. Benjamin Van Durme and Ashwin Lall. 2010. Online Generation of Locality Sensitive Hash Signatures. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL). [pdf] [bibtex] [abstract]

     @inproceedings{online-generation-of-locality-sensitive-hash-signatures,
       author = {{Van Durme}, Benjamin and Lall, Ashwin},
       title = {{Online Generation of Locality Sensitive Hash Signatures}},
       booktitle = {Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL)},
       year = {2010},
       numpages = {5},
       url = {http://www.aclweb.org/anthology/P10-2043}
     }
     

     Motivated by the recent interest in streaming algorithms for processing large text collections, we revisit the work of Ravichandran et al. (2005) on using the Locality Sensitive Hash (LSH) method of Charikar (2002) to enable fast, approximate comparisons of vector cosine similarity. For the common case of feature updates being additive over a data stream, we show that LSH signatures can be maintained online, without additional approximation error, and with lower memory requirements than when using the standard offline technique.

130. Benjamin Van Durme. 2010. Extracting Implicit Knowledge from Text. PhD thesis, University of Rochester, Rochester, NY 14627. [pdf] [bibtex]

     @phdthesis{extracting-implicit-knowledge-from-text,
       author = {{Van Durme}, Benjamin},
       title = {{Extracting Implicit Knowledge from Text}},
       school = {University of Rochester},
       address = {Rochester, NY 14627},
       year = {2010},
       numpages = {155},
       url = {http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.332.3964&rep=rep1&type=pdf},
       asbtract = {The everyday intelligence of both humans and machines relies on a large store of background, or common-sense, knowledge. That such a knowledge base is not yet available to machines helps partially explain the community?s inability to provide society with the sort of synthetic intelligence described by futurists such as Turing, or Asimov. In response, there have emerged a variety of methods for automated Knowledge Acquisition (KA) that are now being actively explored. Here I consider the extraction of knowledge that is conveyed implicitly, both within everyday texts and queries posed to internet search engines. Through recognizing certain forms of existential predicative patterns, and abstracting from these to more strongly quantifiable statements, I show that a significant amount of general knowledge can be gleaned based on how we talk about the world. I provide experimental results both for the direct extraction and strengthening of such knowledge, and for the automatic acquisition of supporting resources for this task. In addition, I draw attention to the relationship between automatically acquired background knowledge and natural language generic sentences. Humans use generics when they wish to directly assert the same sorts of ?rules of the world? that are of concern to the KA community. And yet, there has been little recognition in applied circles that decades of work from formal linguistic semantics may have a role to play in the representation, and perhaps even the acquisition, of common knowledge.}
     }
     

131. Jonathan Gordon, Benjamin Van Durme, and Lenhart Schubert. 2010. Evaluation of Commonsense Knowledge with Mechanical Turk. In Proceedings of the NAACL HLT 2010 Workshop on Creating Speech and Language Data with Amazon’s Mechanical Turk. [pdf] [bibtex] [abstract]

     @inproceedings{evaluation-of-commonsense-knowledge-with-mechanical-turk,
       author = {Gordon, Jonathan and {Van Durme}, Benjamin and Schubert, Lenhart},
       title = {{Evaluation of Commonsense Knowledge with Mechanical Turk}},
       booktitle = {Proceedings of the NAACL HLT 2010 Workshop on Creating Speech and Language Data with Amazon's Mechanical Turk},
       year = {2010},
       numpages = {4},
       url = {http://www.aclweb.org/anthology/W10-0724}
     }
     

     Efforts to automatically acquire world knowledge from text suffer from the lack of an easy means of evaluating the resulting knowledge. We describe initial experiments using Mechanical Turk to crowdsource evaluation to non-experts for little cost, resulting in a collection of factoids with associated quality judgements. We describe the method of acquiring usable judgements from the public and the impact of such large-scale evaluation on the task of knowledge acquisition.

132. Benjamin Van Durme and Ashwin Lall. 2009. Probabilistic Counting with Randomized Storage. In International Joint Conference on Artificial Intelligence (IJCAI-09). [pdf] [bibtex] [abstract]

     @inproceedings{probabilistic-counting-with-randomized-storage,
       author = {{Van Durme}, Benjamin and Lall, Ashwin},
       title = {{Probabilistic Counting with Randomized Storage}},
       booktitle = {International Joint Conference on Artificial Intelligence (IJCAI-09)},
       year = {2009},
       numpages = {6},
       url = {http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.149.9976&rep=rep1&type=pdf}
     }
     

     Previous work by Talbot and Osborne [2007] explored the use of randomized storage mechanisms in language modeling. These structures trade a small amount of error for significant space savings, enabling the use of larger language models on relatively modest hardware. Going beyond space efficient count storage, here we present the Talbot Osborne Morris Bloom (TOMB) Counter, an extended model for performing space efficient counting over streams of finite length. Theoretical and experimental results are given, showing the promise of approximate counting over large vocabularies in the context of limited space.

133. Benjamin Van Durme and Daniel Gildea. 2009. Topic Models for Corpus-centric Knowledge Generalization. Technical Report 946, Department of Computer Science, University of Rochester, Rochester, NY 14627. [pdf] [bibtex] [abstract]

     @techreport{topic-models-for-corpus-centric-knowledge-generalization,
       author = {{Van Durme}, Benjamin and Gildea, Daniel},
       title = {{Topic Models for Corpus-centric Knowledge Generalization}},
       number = {946},
       institution = {Department of Computer Science, University of Rochester},
       year = {2009},
       address = {Rochester, NY 14627},
       numpages = {11},
       url = {https://pdfs.semanticscholar.org/8aee/8100a94f197e4dd72b509772c56bcfd81b82.pdf}
     }
     

     Many of the previous efforts in generalizing over knowledge extracted from text have relied on the use of manually created word sense hierarchies, such as WordNet. We present initial results on generalizing over textually derived knowledge, through the use of the LDA topic model framework, as the first step towards automatically building corpus specific ontologies.

134. Ting Qian Benjamin Van Durme and Lenhart K. Schubert. 2009. Building a Semantic Lexicon of English Nouns via Bootstrapping. In North American Chapter of the Association for Computational Linguistics - Human Language Technologies (NAACL HLT), Student Research Workshop. [pdf] [bibtex] [abstract]

     @inproceedings{building-a-semantic-lexicon-of-english-nouns-via-bootstrapping,
       author = {{Van Durme}, Ting Qian Benjamin and Schubert, Lenhart K.},
       title = {{Building a Semantic Lexicon of English Nouns via Bootstrapping}},
       booktitle = {North American Chapter of the Association for Computational Linguistics - Human Language Technologies (NAACL HLT), Student Research Workshop},
       year = {2009},
       numpages = {6},
       url = {https://aclanthology.info/pdf/N/N09/N09-3007.pdf}
     }
     

     We describe the use of a weakly supervised bootstrapping algorithm in discovering contrasting semantic categories from a source lexicon with little training data. Our method primarily exploits the patterns in sentential contexts where different categories of words may appear. Experimental results are presented showing that such automatically categorized terms tend to agree with human judgements.

135. Benjamin Van Durme, Phil Michalak, and Lenhart K. Schubert. 2009. Deriving Generalized Knowledge from Corpora using WordNet Abstraction. In The 12th Conference of the European Chapter of the Association for Computational Linguistics (EACL). [pdf] [bibtex] [abstract]

     @inproceedings{deriving-generalized-knowledge-from-corpora-using-wordnet-abstraction,
       author = {{Van Durme}, Benjamin and Michalak, Phil and Schubert, Lenhart K.},
       title = {{Deriving Generalized Knowledge from Corpora using WordNet Abstraction}},
       booktitle = {The 12th Conference of the European Chapter of the Association for Computational Linguistics (EACL)},
       year = {2009},
       numpages = {9},
       url = {http://www.aclweb.org/anthology/E09-1092}
     }
     

     Existing work in the extraction of commonsense knowledge from text has been primarily restricted to factoids that serve as statements about what may possibly obtain in the world. We present an approach to deriving stronger, more general claims by abstracting over large sets of factoids. Our goal is to coalesce the observed nominals for a given predicate argument into a few predominant types, obtained as WordNet synsets. The results can be construed as generically quantified sentences restricting the semantic type of an argument position of a predicate.

136. Benjamin Van Durme, Austin Frank, and T. Florian Jaeger. 2009. Comparing Sources of Corpus Frequency Information. In The 22nd Annual Meeting of the CUNY Conference on Human Sentence Processing (CUNY). [bibtex]

     @inproceedings{comparing-sources-of-corpus-frequency-information,
       author = {{Van Durme}, Benjamin and Frank, Austin and Jaeger, T. Florian},
       title = {{Comparing Sources of Corpus Frequency Information}},
       booktitle = {The 22nd Annual Meeting of the CUNY Conference on Human Sentence Processing (CUNY)},
       year = {2009},
       numpages = {1},
       _url = {missing},
       _abstract = {missing}
     }
     

137. Lenhart K. Schubert and Benjamin Van Durme. 2009. Open Extraction of General Knowledge through Compositional Semantics. In NSF Symposium on Semantic Knowledge Discovery, Organization and Use. [pdf] [bibtex] [abstract]

     @inproceedings{open-extraction-of-general-knowledge-through-compositional-semantics,
       author = {Schubert, Lenhart K. and {Van Durme}, Benjamin},
       title = {{Open Extraction of General Knowledge through Compositional Semantics}},
       booktitle = {NSF Symposium on Semantic Knowledge Discovery, Organization and Use},
       year = {2009},
       numpages = {1},
       url = {https://cs.rochester.edu/u/www/u/schubert/papers/open-extraction-nsf-nyu08.pdf}
     }
     

     Our goal is the accumulation of general knowledge, such as that a room may have windows; people may want to be rid of a dictator; when a person enters a room, it is generally through a door; and when a car crashes, the driver and passengers may well be hurt or killed. The applications we are working on are (1) improved guidance of a parser; (2) support for deep learning by reading; and (3) building a large knowledge base for our Epilog inference engine, to support commonsense reasoning, in particular in a self-motivated, self-aware dialog agent.

138. Benjamin Van Durme and Ashwin Lall. 2009. Streaming Pointwise Mutual Information. In Neural Information Processing Systems Conference (NIPS). [pdf] [bibtex] [abstract]

     @inproceedings{streaming-pointwise-mutual-information,
       author = {{Van Durme}, Benjamin and Lall, Ashwin},
       title = {{Streaming Pointwise Mutual Information}},
       booktitle = {Neural Information Processing Systems Conference (NIPS)},
       year = {2009},
       numpages = {9},
       url = {http://papers.nips.cc/paper/3730-streaming-pointwise-mutual-information.pdf}
     }
     

     Recent work has led to the ability to perform space efficient, approximate counting over large vocabularies in a streaming context. Motivated by the existence of data structures of this type, we explore the computation of associativity scores, otherwise known as pointwise mutual information (PMI), in a streaming context. We give theoretical bounds showing the impracticality of perfect online PMI computation, and detail an algorithm with high expected accuracy. Experiments on news articles show our approach gives high accuracy on real world data.

139. Benjamin Van Durme. 2008. Notes on the Acquisition of Conditional Knowledge. Technical Report 937, Department of Computer Science, University of Rochester, Rochester, NY 14627. [pdf] [bibtex] [abstract]

     @techreport{notes-on-the-acquisition-of-conditional-knowledge,
       author = {{Van Durme}, Benjamin},
       title = {{Notes on the Acquisition of Conditional Knowledge}},
       number = {937},
       institution = {Department of Computer Science, University of Rochester},
       address = {Rochester, NY 14627},
       year = {2008},
       numpages = {54},
       url = {http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.232.3837&rep=rep1&type=pdf}
     }
     

     Research in Information Extraction has been overly focused on the extraction of facts concerning individuals as compared to general knowledge pertaining to classes of entities and events. In addition, preference has been given to simple techniques in order to enable high volume throughput. In what follows we give examples of existing work in the field of knowledge acquisition, then follow with ideas on areas for exploration beyond the current state of the art, specifically with respect to the extraction of conditional knowledge, making use of deeper linguistic analysis than is currently the norm.

140. Benjamin Van Durme and Marius Pasca. 2008. Finding Cars, Goddesses and Enzymes: Parametrizable Acquisition of Labeled Instances for Open-Domain Information Extraction. In Twenty-Third AAAI Conference on Artificial Intelligence (AAAI-08). [pdf] [bibtex] [abstract]

     @inproceedings{finding-cars-goddesses-and-enzymes-parametrizable-acquisition-of-labeled-instances-for-open-domain-information-extraction,
       author = {{Van Durme}, Benjamin and Pasca, Marius},
       title = {{Finding Cars, Goddesses and Enzymes: Parametrizable Acquisition of Labeled Instances for Open-Domain Information Extraction}},
       booktitle = {Twenty-Third AAAI Conference on Artificial Intelligence (AAAI-08)},
       year = {2008},
       numpages = {6},
       url = {http://www.aaai.org/Papers/AAAI/2008/AAAI08-197.pdf}
     }
     

     A method is given for the extraction of large numbers of semantic classes along with their corresponding instances. Based on the recombination of elements clustered through distributional similarity, experimental results show the procedure allows for a parametric trade-off between high precision and expanded recall.

141. Benjamin Van Durme and Lenhart K. Schubert. 2008. Open Knowledge Extraction through Compositional Language Processing. In Semantics in Text Processing. [pdf] [bibtex] [abstract]

     @inproceedings{open-knowledge-extraction-through-compositional-language-processing,
       author = {{Van Durme}, Benjamin and Schubert, Lenhart K.},
       title = {{Open Knowledge Extraction through Compositional Language  Processing}},
       booktitle = {Semantics in Text Processing},
       year = {2008},
       numpages = {16},
       url = {https://aclanthology.info/pdf/W/W08/W08-2219.pdf}
     }
     

     We present results for a system designed to perform Open Knowledge Extraction, based on a tradition of compositional language processing, as applied to a large collection of text derived from the Web. Evaluation through manual assessment shows that well-formed propositions of reasonable quality, representing general world knowledge, given in a logical form potentially usable for inference, may be extracted in high volume from arbitrary input sentences. We compare these results with those obtained in recent work on Open Information Extraction, indicating with some examples the quite different kinds of output obtained by the two approaches. Finally, we observe that portions of the extracted knowledge are comparable to results of recent work on class attribute extraction.

142. Dekang Lin, Shaojun Zhao, Benjamin Van Durme, and Marius Pasca. 2008. Mining Parenthetical Translations from the Web by Word Alignment. In The 46th Annual Meeting of the Association of Computational Linguistics: Human Language Technologies (ACL-08: HLT). [pdf] [bibtex] [abstract]

     @inproceedings{mining-parenthetical-translations-from-the-web-by-word-alignment,
       author = {Lin, Dekang and Zhao, Shaojun and {Van Durme}, Benjamin and Pasca, Marius},
       title = {{Mining Parenthetical Translations from the Web by Word Alignment}},
       booktitle = {The 46th Annual Meeting of the Association of Computational Linguistics: Human Language Technologies (ACL-08: HLT)},
       year = {2008},
       numpages = {9},
       url = {http://www.aclweb.org/anthology/P08-1113}
     }
     

     Documents in languages such as Chinese, Japanese and Korean sometimes annotate terms with their translations in English inside a pair of parentheses. We present a method to extract such translations from a large collection of web documents by building a partially parallel corpus and use a word alignment algorithm to identify the terms being translated. The method is able to generalize across the translations for different terms and can reliably extract translations that occurred only once in the entire web. Our experiment on Chinese web pages produced more than 26 million pairs of translations, which is over two orders of magnitude more than previous results. We show that the addition of the extracted translation pairs as training data provides significant increase in the BLEU score for a statistical machine translation system.

143. Marius Pasca and Benjamin Van Durme. 2008. Weakly-Supervised Acquisition of Open-Domain Classes and Class Attributes from Web Documents and Query Logs. In The 46th Annual Meeting of the Association of Computational Linguistics: Human Language Technologies (ACL-08: HLT). [pdf] [bibtex] [abstract]

     @inproceedings{weakly-supervised-acquisition-of-open-domain-classes-and-class-attributes-from-web-documents-and-query-logs,
       author = {Pasca, Marius and {Van Durme}, Benjamin},
       title = {{Weakly-Supervised Acquisition of Open-Domain Classes and Class Attributes from Web Documents and Query Logs}},
       booktitle = {The 46th Annual Meeting of the Association of Computational Linguistics: Human Language Technologies (ACL-08: HLT)},
       year = {2008},
       numpages = {9},
       url = {http://www.aclweb.org/anthology/P08-1003}
     }
     

     A new approach to large-scale information extraction exploits both Web documents and query logs to acquire thousands of opendomain classes of instances, along with relevant sets of open-domain class attributes at precision levels previously obtained only on small-scale, manually-assembled classes.

144. Benjamin Van Durme, Ting Qian, and Lenhart K. Schubert. 2008. Class-Driven Attribute Extraction. In The 22nd International Conference on Computational Linguistics (COLING). [pdf] [bibtex] [abstract]

     @inproceedings{class-driven-attribute-extraction,
       author = {{Van Durme}, Benjamin and Qian, Ting and Schubert, Lenhart K.},
       title = {{Class-Driven Attribute Extraction}},
       booktitle = {The 22nd International Conference on Computational Linguistics (COLING)},
       year = {2008},
       numpages = {8},
       url = {https://www.aclweb.org/anthology/C/C08/C08-1116.pdf}
     }
     

     We report on the large-scale acquisition of class attributes with and without the use of lists of representative instances, as well as the discovery of unary attributes, such as typically expressed in English through prenominal adjectival modification. Our method employs a system based on compositional language processing, as applied to the British National Corpus. Experimental results suggest that document-based, open class attribute extraction can produce results of comparable quality as those obtained using web query logs, indicating the utility of exploiting explicit occurrences of class labels in text.

145. Marius Pasca, Benjamin Van Durme, and Nikesh Garera. 2007. The Role of Documents vs. Queries in Extracting Class Attributes from Text. In ACM Sixteenth Conference on Information and Knowledge Management (CIKM). [pdf] [bibtex] [abstract]

     @inproceedings{the-role-of-documents-vs.-queries-in-extracting-class-attributes-from-text,
       author = {Pasca, Marius and {Van Durme}, Benjamin and Garera, Nikesh},
       title = {{The Role of Documents vs. Queries in Extracting Class Attributes from Text}},
       booktitle = {ACM Sixteenth Conference on Information and Knowledge Management (CIKM)},
       year = {2007},
       numpages = {9},
       url = {http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.68.1833&rep=rep1&type=pdf}
     }
     

     Challenging the implicit reliance on document collections, this paper discusses the pros and cons of using query logs rather than document collections, as self-contained sources of data in textual information extraction. The differences are quantified as part of a large-scale study on extracting prominent attributes or quantifiable properties of classes (e.g., top speed, price and fuel consumption for CarModel) from unstructured text. In a head-to-head qualitative comparison, a lightweight extraction method produces class attributes that are 45% more accurate on average, when acquired from query logs rather than Web documents.

146. Marius Pasca and Benjamin Van Durme. 2007. What You Seek is What You Get: Extraction of Class Attributes from Query Logs. In International Joint Conference on Artificial Intelligence (IJCAI-07). [pdf] [bibtex] [abstract]

     @inproceedings{what-you-seek-is-what-you-get-extraction-of-class-attributes-from-query-logs,
       author = {Pasca, Marius and {Van Durme}, Benjamin},
       title = {{What You Seek is What You Get: Extraction of Class Attributes from Query Logs}},
       booktitle = {International Joint Conference on Artificial Intelligence (IJCAI-07)},
       year = {2007},
       numpages = {6},
       url = {http://www.aaai.org/Papers/IJCAI/2007/IJCAI07-455.pdf}
     }
     

     Within the larger area of automatic acquisition of knowledge from the Web, we introduce a method for extracting relevant attributes, or quantifiable properties, for various classes of objects. The method extracts attributes such as capital city and President for the class Country, or cost, manufacturer and side effects for the class Drug, without relying on any expensive language resources or complex processing tools. In a departure from previous approaches to large-scale information extraction, we explore the role of Web query logs, rather than Web documents, as an alternative source of class attributes. The quality of the extracted attributes recommends query logs as a valuable, albeit little explored, resource for information extraction.

147. Anna Kupsc, Teruko Mitamura, Benjamin Van Durme, and Eric Nyberg. 2004. Pronominal Anaphora Resolution for Unrestricted Text. In The International Conference on Language Resources and Evaluation (LREC-04). [pdf] [bibtex] [abstract]

     @inproceedings{pronominal-anaphora-resolution-for-unrestricted-text,
       author = {Kupsc, Anna and Mitamura, Teruko and {Van Durme}, Benjamin and Nyberg, Eric},
       title = {{Pronominal Anaphora Resolution for Unrestricted Text}},
       booktitle = {The International Conference on Language Resources and Evaluation (LREC-04)},
       year = {2004},
       numpages = {4},
       url = {https://pdfs.semanticscholar.org/27af/01f3bcd05f33a64f31f456f54760edab3501.pdf}
     }
     

     The paper presents an anaphora resolution algorithm for unrestricted text. In particular, we examine portability of a knowledge-based approach of (Mitamura et al., 2002), proposed for a domain-specific task. We obtain up to 70% accuracy on unrestricted text, which is a significant improvement (almost 20%) over a baseline we set for general text. As the overall results leave much room for improvement, we provide a detailed error analysis and investigate possible enhancements.

148. E. Nyberg, T. Mitamura, J. Callan, J. Carbonell, R. Frederking, K. Collins-Thompson, L. Hiyakumoto, Y. Huang, C. Huttenhower, S. Judy, J. Ko, A. Kupsc, L. Lita, V. Pedro, D. Svoboda, and B. Van Durme. 2003. The JAVELIN Question-Answering System at TREC 2003: A Multi-Strategy Approach with Dynamic Planning. In 12th Text REtrieval Conference (TREC-12). [pdf] [bibtex] [abstract]

     @inproceedings{the-javelin-question-answering-system-at-trec-2003-a-multi-strategy-approach-with-dynamic-planning,
       author = {Nyberg, E. and Mitamura, T. and Callan, J. and Carbonell, J. and Frederking, R. and Collins-Thompson, K. and Hiyakumoto, L. and Huang, Y. and Huttenhower, C. and Judy, S. and Ko, J. and Kupsc, A. and Lita, L. and Pedro, V. and Svoboda, D. and {Van Durme}, B.},
       title = {{The JAVELIN Question-Answering System at TREC 2003: A Multi-Strategy Approach with Dynamic Planning}},
       booktitle = {12th Text REtrieval Conference (TREC-12)},
       year = {2003},
       numpages = {9},
       url = {/papers/The-JAVELIN-Questions-Answering-System-at-TREC-2003.pdf}
     }
     

     The JAVELIN system evaluated at TREC 2003 is an integrated architecture for open-domain question answering. JAVELIN employs a modular approach that addresses individual aspects of the QA task in an abstract manner. The System implements a planner that controls the execution and information flow, as well as a multiple answer seeking strategies used differently depending on the type of question.

149. Benjamin Van Durme, Yifen Huang, Anna Kupsc, and Eric Nyberg. 2003. Towards Light Semantic Processing for Question Answering. In Human Language Technology and North American Chapter of Association of Computational Linguistics (HLT/NAACL-03), Workshop on Text Meaning. [pdf] [bibtex] [abstract]

     @inproceedings{towards-light-semantic-processing-for-question-answering,
       author = {{Van Durme}, Benjamin and Huang, Yifen and Kupsc, Anna and Nyberg, Eric},
       title = {{Towards Light Semantic Processing for Question Answering}},
       booktitle = {Human Language Technology and North American Chapter of Association of Computational Linguistics (HLT/NAACL-03), Workshop on Text Meaning},
       year = {2003},
       numpages = {8},
       url = {https://aclanthology.info/pdf/W/W03/W03-0908.pdf}
     }
     

     The paper presents a lightweight knowledge-based reasoning framework for the JAVELIN open-domain Question Answering (QA) system. We propose a constrained representation of text meaning, along with a flexible unification strategy that matches questions with retrieved passages based on semantic similarities and weighted relations between words.

Graduate Student Advisees

Postdoctoral Fellows Supervised

Teaching

Local Service

Professional Activities

Invited Talks