Difficult relations: Extracting novel facts from text

Creating, populating, updating and maintaining a knowledge resource requires intense human effort. Automatic Information Extraction techniques play a crucial role for this task, but many ongoing production systems still require a large component of human annotation. In this work we investigate how to better take advantage of human annotations by performing active learning on multiple IE tasks concurrently, specifically Relation Extraction and Named Entity Recognition. Our proposed approach adaptively requests annotations for one task or the other depending on the current overall performance of the combined extraction. We show promising results on a small use case extracting relations expressing Adverse Drug Reactions from unannotated sentences.