Sentiment analysis (SA) systems are used in many products and hundreds of languages. Gender and racial biases are well-studied in English SA systems, but understudied in other languages, with few resources for such studies. To remedy this, we build a counterfactual evaluation corpus for gender and racial/migrant bias in four languages. We demonstrate its usefulness by answering a simple but important question that an engineer might need to answer when deploying a system: What biases do systems import from pre-trained models when compared to a baseline with no pre-training? Our evaluation corpus, by virtue of being counterfactual, not only reveals which models have less bias, but also pinpoints changes in model bias behaviour, which enables more targeted mitigation strategies. We release our code and evaluation corpora to facilitate future research.

@inproceedings{goldfarb-tarrant-etal-2023-bias,
  title = "Bias Beyond {E}nglish: Counterfactual Tests for Bias in Sentiment Analysis in Four Languages",
  author = "Goldfarb-Tarrant, Seraphina  and
  Lopez, Adam  and
  Blanco, Roi  and
  Marcheggiani, Diego",
  booktitle = "Findings of the Association for Computational Linguistics: ACL 2023",
  year = "2023",
  url = "https://aclanthology.org/2023.findings-acl.272",
  doi = "10.18653/v1/2023.findings-acl.272",
  pages = "4458--4468",
  abstract = "Sentiment analysis (SA) systems are used in many products and hundreds of languages. Gender and racial biases are well-studied in English SA systems, but understudied in other languages, with few resources for such studies. To remedy this, we build a counterfactual evaluation corpus for gender and racial/migrant bias in four languages. We demonstrate its usefulness by answering a simple but important question that an engineer might need to answer when deploying a system: What biases do systems import from pre-trained models when compared to a baseline with no pre-training? Our evaluation corpus, by virtue of being counterfactual, not only reveals which models have less bias, but also pinpoints changes in model bias behaviour, which enables more targeted mitigation strategies. We release our code and evaluation corpora to facilitate future research.",
}

Semantic role labeling (SRL) is the task of identifying predicates and labeling argument spans with semantic roles. Even though most semantic-role formalisms are built upon constituent syntax, and only syntactic constituents can be labeled as arguments (e.g., FrameNet and PropBank), all the recent work on syntax-aware SRL relies on dependency representations of syntax. In contrast, we show how graph convolutional networks (GCNs) can be used to encode constituent structures and inform an SRL system. Nodes in our SpanGCN correspond to constituents. The computation is done in 3 stages. First, initial node representations are produced by ‘composing’ word representations of the first and last words in the constituent. Second, graph convolutions relying on the constituent tree are performed, yielding syntactically-informed constituent representations. Finally, the constituent representations are ‘decomposed’ back into word representations, which are used as input to the SRL classifier. We evaluate SpanGCN against alternatives, including a model using GCNs over dependency trees, and show its effectiveness on standard English SRL benchmarks CoNLL-2005, CoNLL-2012, and FrameNet.

@inproceedings{marcheggiani-titov-2020-graph,
    title = "Graph Convolutions over Constituent Trees for Syntax-Aware Semantic Role Labeling",
    author = "Marcheggiani, Diego  and
      Titov, Ivan",
    booktitle = "Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)",
    month = nov,
    year = "2020",
    address = "Online",
    publisher = "Association for Computational Linguistics",
    url = "https://www.aclweb.org/anthology/2020.emnlp-main.322",
    pages = "3915--3928",
    abstract = "Semantic role labeling (SRL) is the task of identifying predicates and labeling argument spans with semantic roles. Even though most semantic-role formalisms are built upon constituent syntax, and only syntactic constituents can be labeled as arguments (e.g., FrameNet and PropBank), all the recent work on syntax-aware SRL relies on dependency representations of syntax. In contrast, we show how graph convolutional networks (GCNs) can be used to encode constituent structures and inform an SRL system. Nodes in our SpanGCN correspond to constituents. The computation is done in 3 stages. First, initial node representations are produced by {`}composing{'} word representations of the first and last words in the constituent. Second, graph convolutions relying on the constituent tree are performed, yielding syntactically-informed constituent representations. Finally, the constituent representations are {`}decomposed{'} back into word representations, which are used as input to the SRL classifier. We evaluate SpanGCN against alternatives, including a model using GCNs over dependency trees, and show its effectiveness on standard English SRL benchmarks CoNLL-2005, CoNLL-2012, and FrameNet.",
}

We present a system for answering questions based on the full text of books (BookQA), which first selects book passages given a question at hand, and then uses a memory network to reason and predict an answer. To improve generalization, we pretrain our memory network using artificial questions generated from book sentences. We experiment with the recently published NarrativeQA corpus, on the subset of Who questions, which expect book characters as answers. We experimentally show that BERT-based retrieval and pretraining improve over baseline results significantly. At the same time, we confirm that NarrativeQA is a highly challenging data set, and that there is need for novel research in order to achieve high-precision BookQA results. We analyze some of the bottlenecks of the current approach, and we argue that more research is needed on text representation, retrieval of relevant passages, and reasoning, including commonsense knowledge.

@inproceedings{angelidis-etal-2019-book,
  title = "Book {QA}: Stories of Challenges and Opportunities",
  author = "Angelidis, Stefanos  and
    Frermann, Lea  and
    Marcheggiani, Diego  and
    Blanco, Roi  and
    M{\`a}rquez, Llu{\'\i}s",
  booktitle = "Proceedings of the 2nd Workshop on Machine Reading for Question Answering",
  month = nov,
  year = "2019",
  address = "Hong Kong, China",
  publisher = "Association for Computational Linguistics",
  url = "https://www.aclweb.org/anthology/D19-5811",
  doi = "10.18653/v1/D19-5811",
  pages = "78--85",
}

Information about individuals can help to better understand what they say, particularly in social media where texts are short. Current approaches to modelling social media users pay attention to their social connections, but exploit this information in a static way, treating all connections uniformly. This ignores the fact, well known in sociolinguistics, that an individual may be part of several communities which are not equally relevant in all communicative situations. We present a model based on Graph Attention Networks that captures this observation. It dynamically explores the social graph of a user, computes a user representation given the most relevant connections for a target task, and combines it with linguistic information to make a prediction. We apply our model to three different tasks, evaluate it against alternative models, and analyse the results extensively, showing that it significantly outperforms other current methods.

@inproceedings{del-tredici-etal-2019-shall,
  title = "You Shall Know a User by the Company It Keeps: Dynamic Representations for Social Media Users in {NLP}",
  author = "Del Tredici, Marco  and
    Marcheggiani, Diego  and
    Schulte im Walde, Sabine  and
    Fern{\'a}ndez, Raquel",
  booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)",
  month = nov,
  year = "2019",
  address = "Hong Kong, China",
  publisher = "Association for Computational Linguistics",
  url = "https://www.aclweb.org/anthology/D19-1477",
  doi = "10.18653/v1/D19-1477",
  pages = "4701--4711",
}

Most previous work on neural text generation from graph-structured data relies on standard sequence-to-sequence methods. These approaches linearise the input graph to be fed to a recurrent neural network. In this paper, we propose an alternative encoder based on graph convolutional networks that directly exploits the input structure. We report results on two graph-to-sequence datasets that empirically show the benefits of explicitly encoding the input graph structure.

@InProceedings{W18-6501,
  author =  "Marcheggiani, Diego and Perez-Beltrachini, Laura",
  title =   "Deep Graph Convolutional Encoders for Structured Data to Text Generation",
  booktitle =   "Proceedings of the 11th International Conference on Natural Language Generation",
  year =  "2018",
  publisher =   "Association for Computational Linguistics",
  pages =   "1--9",
  location =  "Tilburg University, The Netherlands",
  url =   "http://aclweb.org/anthology/W18-6501"
}

Semantic representations have long been argued as potentially useful for enforcing meaning preservation and improving generalization performance of machine translation methods. In this work, we are the first to incorporate information about predicate-argument structure of source sentences (namely, semantic-role representations) into neural machine translation. We use Graph Convolutional Networks (GCNs) to inject a semantic bias into sentence encoders and achieve improvements in BLEU scores over the linguistic-agnostic and syntax-aware versions on the English--German language pair.

@InProceedings{marcheggiani-bastings-titov:2018:NAACL,
  author    = {Marcheggiani, Diego  and  Bastings, Jasmijn  and  Titov, Ivan},
  title     = {Exploiting Semantics in Neural Machine Translation with Graph Convolutional Networks},
  booktitle = {Proceedings of the the 16th Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT 2018)},
  month     = {June},
  year      = {2018},
  address   = {New Orleans, US},
  publisher = {Association for Computational Linguistics}
}

Semantic role labeling (SRL) is the task of identifying the predicate-argument structure of a sentence. It is typically regarded as an important step in the standard natural language processing pipeline, providing information to downstream tasks such as information extraction and question answering. As the semantic representations are closely related to syntactic ones, we exploit syntactic information in our model. We propose a version of graph convolutional networks (GCNs), a recent class of multilayer neural networks operating on graphs, suited to modeling syntactic dependency graphs. GCNs over syntactic dependency trees are used as sentence encoders, producing latent feature representations of words in a sentence and capturing information relevant to predicting the semantic representations. We observe that GCN layers are complementary to LSTM ones: when we stack both GCN and LSTM layers, we obtain a substantial improvement over an already state-of-the-art LSTM SRL model, resulting in the best reported scores on the standard benchmark (CoNLL-2009) both for Chinese and English.

@InProceedings{marcheggiani-titov:2017:EMNLP2017,
  author    = {Marcheggiani, Diego  and  Titov, Ivan},
  title     = {Encoding Sentences with Graph Convolutional Networks for Semantic Role Labeling},
  booktitle = {Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing},
  month     = {September},
  year      = {2017},
  address   = {Copenhagen, Denmark},
  publisher = {Association for Computational Linguistics},
  pages     = {1507--1516},
  url       = {https://www.aclweb.org/anthology/D17-1159}
}

We present a simple and effective approach to incorporating syntactic structure into neural attention-based encoder-decoder models for machine translation. We rely on graph-convolutional networks (GCNs), a recent class of neural networks developed for modeling graph-structured data. Our GCNs use predicted syntactic dependency trees of source sentences to produce representations of words (i.e. hidden states of the encoder) that are sensitive to their syntactic neighborhoods. GCNs take word representations as input and produce word representations as output, so they can easily be incorporated as layers into standard encoders (e.g., on top of bidirectional RNNs or convolutional neural networks). We evaluate their effectiveness with English-German and English-Czech translation experiments for different types of encoders and observe substantial improvements over their syntax-agnostic versions in all the considered setups.

@InProceedings{bastings-EtAl:2017:EMNLP2017,
  author    = {Bastings, Jasmijn  and  Titov, Ivan  and  Aziz, Wilker  and  Marcheggiani, Diego  and  Simaan, Khalil},
  title     = {Graph Convolutional Encoders for Syntax-aware Neural Machine Translation},
  booktitle = {Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing},
  month     = {September},
  year      = {2017},
  address   = {Copenhagen, Denmark},
  publisher = {Association for Computational Linguistics},
  pages     = {1947--1957},
  url       = {https://www.aclweb.org/anthology/D17-1209}
}

We introduce a simple and accurate neural model for dependency-based semantic role labeling. Our model predicts predicate-argument dependencies relying on states of a bidirectional LSTM encoder. The semantic role labeler achieves respectable performance on English even without any kind of syntactic information and only using local inference. However, when automatically predicted part-of-speech tags are provided as input, it substantially outperforms all previous local models and approaches the best reported results on the CoNLL-2009 dataset. Syntactic parsers are unreliable on out-of-domain data, so standard (i.e. syntactically-informed) SRL models are hindered when tested in this setting. Our syntax-agnostic model appears more robust, resulting in the best reported results on the standard out-of-domain test set.

@InProceedings{marcheggiani-frolov-titov:2017:CoNLL,
  author    = {Marcheggiani, Diego  and  Frolov, Anton  and  Titov, Ivan},
  title     = {A Simple and Accurate Syntax-Agnostic Neural Model for Dependency-based Semantic Role Labeling},
  booktitle = {Proceedings of the 21st Conference on Computational Natural Language Learning (CoNLL 2017)},
  month     = {August},
  year      = {2017},
  address   = {Vancouver, Canada},
  publisher = {Association for Computational Linguistics},
  pages     = {411--420},
  url       = {http://aclweb.org/anthology/K17-1041}
}

In the last five years there has been a flurry of work on information extraction from clinical documents, i.e., on algorithms capable of extracting, from the informal and unstructured texts that are generated during everyday clinical practice, mentions of concepts relevant to such practice. Many of these research works are about methods based on supervised learning, i.e., methods for training an information extraction system from manually annotated examples. While a lot of work has been devoted to devising learning methods that generate more and more accurate information extractors, no work has been devoted to investigating the effect of the quality of training data on the learning process for the clinical domain. Low quality in training data often derives from the fact that the person who has annotated the data is different from the one against whose judgment the automatically annotated data must be evaluated. In this paper we test the impact of such data quality issues on the accuracy of information extraction systems as applied to the clinical domain. We do this by comparing the accuracy deriving from training data annotated by the authoritative coder (i.e., the one who has also annotated the test data, and by whose judgment we must abide), with the accuracy deriving from training data annotated by a different coder, equally expert in the subject matter. The results indicate that, although the disagreement between the two coders (as measured on the training set) is substantial, the difference is (surprisingly enough) not always statistically significant. While the dataset used in the present work originated in a clinical context, the issues we study in this work are of more general interest.

@article{Marcheggiani:2017:ELT:3139489.3106235,
 author = {Marcheggiani, Diego and Sebastiani, Fabrizio},
 title = {On the Effects of Low-Quality Training Data on Information Extraction from Clinical Reports},
 journal = {ACM Journal of Data and Information Quality},
 issue_date = {September 2017},
 volume = {9},
 number = {1},
 month = sep,
 year = {2017},
 issn = {1936-1955},
 pages = {1:1--1:25},
 articleno = {1},
 numpages = {25},
 url = {http://doi.acm.org/10.1145/3106235},
 doi = {10.1145/3106235},
 acmid = {3106235},
 publisher = {ACM},
 address = {New York, NY, USA},
 keywords = {Information extraction, annotation quality, clinical narratives, machine learning, medical reports, radiology reports},
} 

We present a method for unsupervised open-domain relation discovery. In contrast to previous (mostly generative and agglomerative clustering) approaches, our model relies on rich contextual features and makes minimal independence assumptions. The model is composed of two parts: a feature-rich relation extractor, which predicts a semantic relation between two entities, and a factorization model, which reconstructs arguments (i.e., the entities) relying on the predicted relation. The two components are estimated jointly so as to minimize errors in recovering arguments. We study factorization models inspired by previous work in relation factorization and selectional preference modeling. Our models substantially outperform the generative and agglomerative-clustering counterparts and achieve state-of-the-art performance.

@article{DBLP:journals/tacl/MarcheggianiT16,
  author    = {Diego Marcheggiani and
               Ivan Titov},
  title     = {Discrete-State Variational Autoencoders for Joint Discovery and Factorization
               of Relations},
  journal   = {{TACL}},
  volume    = {4},
  pages     = {231--244},
  year      = {2016},
  url       = {https://tacl2013.cs.columbia.edu/ojs/index.php/tacl/article/view/761},
}

@inproceedings{marcheggianiKT16,
  author = {Diego Marcheggiani, Ehsan Khoddam, Ivan Titov},
  title = {Unsupervised Induction of Shallow Semantic Representations with Feature­ Rich Models},
  booktitle = {Book of Abstracts of the 9th International Conference on Construction Grammar},
  month = {October},
  year = {2016},
  address = {Juiz de Fora, Brazil},
  url = {http://www.ufjf.br/iccg9/files/2015/02/ICCG9_Book_of_Abstracts_v1.pdf}
}

Microblogging is a model of content sharing in which the temporal locality of posts with respect to important events, either of foreseeable or unforeseeable nature, makes applications of real-time filtering of great practical interest. We propose the use of Entity Linking (EL) in order to improve the retrieval effectiveness, by enriching the representation of microblog posts and filtering queries. EL is the process of recognizing in an unstructured text the mention of relevant entities described in a knowledge base. EL of short pieces of text is a difficult task, but it is also a scenario in which the information EL adds to the text can have a substantial impact on the retrieval process. We implement a start-of-the-art filtering method, based on the best systems from the TREC Microblog track real-time adhoc retrieval and filtering tasks, and extend it with a Wikipedia-based EL method. Results show that the use of EL significantly improves over non-EL based versions of the filtering methods.

@inproceedings{berardiCEM15,
  author    = {Giacomo Berardi and
               Diego Ceccarelli and
               Andrea Esuli and
               Diego Marcheggiani},
  title     = {On the impact of entity linking in microblog real-time filtering},
  booktitle = {Proceedings of the 30th Annual {ACM} Symposium on Applied Computing,
               Salamanca, Spain, April 13-17, 2015},
  pages     = {1066--1071},
  year      = {2015},
}

In this paper we present some preliminary results on the generation of word embeddings for the Italian language. We compare two popular word representation models, word2vec and GloVe, and train them on two datasets with different stylistic properties. We test the generated word embeddings on a word analogy test derived from the one originally proposed for word2vec, adapted to capture some of the linguistic aspects that are specific of Italian. Results show that the tested models are able to create syntactically and semantically meaningful word embeddings despite the higher morphological complexity of Italian with respect to English. Moreover, we have found that the stylistic properties of the training dataset plays a relevant role in the type of information captured by the produced vectors.

@inproceedings{berardiEM15,
  author    = {Giacomo Berardi and
               Andrea Esuli and
               Diego Marcheggiani},
  title     = {Word Embeddings Go to Italy: {A} Comparison of Models and Training
               Datasets},
  booktitle = {Proceedings of the 6th Italian Information Retrieval Workshop, Cagliari,
               Italy, May 25-26, 2015.},
  year      = {2015},
  url       = {http://ceur-ws.org/Vol-1404/paper_11.pdf}
  
}

We present the Trip-MAML dataset, a Multi-Lingual dataset of hotel reviews that have been manually annotated at the sentence-level with Multi-Aspect sentiment labels. This dataset has been built as an extension of an existent English-only dataset, adding documents written in Italian and Spanish. We detail the dataset construction process, covering the data gathering, selection, and annotation. We present inter-annotator agreement figures and baseline experimental results, comparing the three languages. Trip-MAML is a multi-lingual dataset for aspect-oriented opinion mining that enables researchers (i) to face the problem on languages other than English and (ii) to the experiment the application of cross-lingual learning methods to the task.

@inproceedings{ZafraBEMMF15,
  author    = {Salud M. Jim{\'{e}}nez Zafra and
               Giacomo Berardi and
               Andrea Esuli and
               Diego Marcheggiani and
               Maria Teresa Mart{\'{\i}}n{-}Valdivia and
               Alejandro Moreo Fern{\'{a}}ndez},
  title     = {A Multi-lingual Annotated Dataset for Aspect-Oriented Opinion Mining},
  booktitle = {Proceedings of the 2015 Conference on Empirical Methods in Natural
               Language Processing, {EMNLP} 2015, Lisbon, Portugal, September 17-21,
               2015},
  pages     = {2533--2538},
  year      = {2015},
  url       = {http://aclweb.org/anthology/D/D15/D15-1302.pdf}
}

Active learning (AL) consists of asking human annotators to annotate automatically selected data that are assumed to bring the most benefit in the creation of a classifier. AL allows to learn accurate systems with much less annotated data than what is required by pure supervised learning algorithms, hence limiting the tedious effort of annotating a large collection of data. We experimentally investigate the behavior of several AL strategies for sequence labeling tasks (in a partially-labeled scenario) tailored on Partially-Labeled Conditional Random Fields, on four sequence labeling tasks: phrase chunking, part-of-speech tagging, named-entity recognition, and bio-entity recognition.

@inproceedings{MarcheggianiA14,
  author    = {Diego Marcheggiani and
               Thierry Arti{\`{e}}res},
  title     = {An Experimental Comparison of Active Learning Strategies for Partially
               Labeled Sequences},
  booktitle = {Proceedings of the 2014 Conference on Empirical Methods in Natural
               Language Processing, {EMNLP} 2014, October 25-29, 2014, Doha, Qatar,
               {A} meeting of SIGDAT, a Special Interest Group of the {ACL}},
  pages     = {898--906},
  year      = {2014}
}

A common feature of many online review sites is the use of an overall rating that summarizes the opinions expressed in a review. Unfortunately, these document-level ratings do not provide any information about the opinions contained in the review that concern a specific aspect (e.g., cleanliness) of the product being reviewed (e.g., a hotel). In this paper we study the finer-grained problem of aspect-oriented opinion mining at the sentence level, which consists of predicting, for all sentences in the review, whether the sentence expresses a positive, neutral, or negative opinion (or no opinion at all) about a specific aspect of the product. For this task we propose a set of increasingly powerful models based on conditional random fields (CRFs), including a hierarchical multi-label CRFs scheme that jointly models the overall opinion expressed in the review and the set of aspect-specific opinions expressed in each of its sentences. We evaluate the proposed models against a dataset of hotel reviews (which we here make publicly available) in which the set of aspects and the opinions expressed concerning them are manually annotated at the sentence level. We find that both hierarchical and multi-label factors lead to improved predictions of aspect-oriented opinions.

@inproceedings{DBLP:conf/ecir/MarcheggianiTE014,
  author    = {Diego Marcheggiani and
               Oscar T{\"{a}}ckstr{\"{o}}m and
               Andrea Esuli and
               Fabrizio Sebastiani},
  title     = {Hierarchical Multi-label Conditional Random Fields for Aspect-Oriented
               Opinion Mining},
  booktitle = {Advances in Information Retrieval - 36th European Conference on {IR}
               Research, {ECIR} 2014, Amsterdam, The Netherlands, April 13-16, 2014.
               Proceedings},
  pages     = {273--285},
  year      = {2014}
}

We discuss the problem of performing information extraction from free-text radiology reports via supervised learning. In this task, segments of text (not necessarily coinciding with entire sentences, and possibly crossing sentence boundaries) need to be annotated with tags representing concepts of interest in the radiological domain. In this paper we present two novel approaches to IE for radiology reports: (i) a cascaded, two-stage method based on pipelining two taggers generated via the well known linear-chain conditional random fields (LC-CRFs) learner and (ii) a confidence-weighted ensemble method that combines standard LC-CRFs and the proposed two-stage method. We also report on the use of “positional features”, a novel type of feature intended to aid in the automatic annotation of texts in which the instances of a given concept may be hypothesized to systematically occur in specific areas of the text. We present experiments on a dataset of mammography reports in which the proposed ensemble is shown to outperform a traditional, single-stage CRFs system in two different, applicatively interesting scenarios.

@article{EsuliMS13,
  author    = {Andrea Esuli and
               Diego Marcheggiani and
               Fabrizio Sebastiani},
  title     = {An enhanced CRFs-based system for information extraction from radiology
               reports},
  journal   = {Journal of Biomedical Informatics},
  volume    = {46},
  number    = {3},
  pages     = {425--435},
  year      = {2013},
}

We demonstrate a metadata enrichment system for the Europeana digital library. The system allows different institutions which provide to Europeana pointers (in the form of metadata records - MRs) to their content to enrich their MRs by classifying them under a classification scheme of their choice, and to extract/highlight entities of significant interest within the MRs themselves. The use of a supervised learning metaphor allows each content provider (CP) to generate classifiers and extractors tailored to the CP’s specific needs, thus allowing the tool to be effectively available to the multitude (2000+) of Europeana CPs.

@inproceedings{BerardiEGMS12,
  author    = {Giacomo Berardi and
               Andrea Esuli and
               Sergiu Gordea and
               Diego Marcheggiani and
               Fabrizio Sebastiani},
  title     = {Metadata Enrichment Services for the Europeana Digital Library},
  booktitle = {Theory and Practice of Digital Libraries - Second International Conference,
               {TPDL} 2012, Paphos, Cyprus, September 23-27, 2012. Proceedings},
  pages     = {508--511},
  year      = {2012}
}

Our approach to the microblog filtering task is based on learning a relevance classifier from an initial training set of relevant and non relevant tweets, generated by using a simple retrieval method. The classifier is then retrained using the (simulated) user feedback collected during the training process, in order to improve its accuracy as the filtering process goes on. In the official runs the system scored low effectiveness values, suffering a strong imbalance toward recall.

@inproceedings{DBLP:conf/trec/BerardiEM12,
  author    = {Giacomo Berardi and
               Andrea Esuli and
               Diego Marcheggiani},
  title     = {ISTI@TREC Microblog Track 2012: Real-Time Filtering Through Supervised
               Learning},
  booktitle = {Proceedings of The Twenty-First Text REtrieval Conference, {TREC}
               2012, Gaithersburg, Maryland, USA, November 6-9, 2012},
  year      = {2012},
  url       = {http://trec.nist.gov/pubs/trec21/papers/NEMIS_ISTI_CNR.microblog.final.pdf},
}

In the first year of the TREC Micro Blog track, our participation has focused on building from scratch an IR system based on the Whoosh IR library. Though the design of our system (CipCipPy) is pretty standard it includes three ad-hoc solutions for the track (i) a dedicated indexing function for hashtags that automatically recognizes the distinct words composing an hashtag, (ii) expansion of tweets based on the title of any referred Web page, and (iii) a tweet ranking function that ranks tweets in results by their content quality, which is compared against a reference corpus of Reuters news. In this preliminary paper we describe all the components of our system, and the efficacy scored by our runs. The CipCipPy system is available under a GPL license.

@inproceedings{BerardiEMS11,
  author    = {Giacomo Berardi and
               Andrea Esuli and
               Diego Marcheggiani and
               Fabrizio Sebastiani},
  title     = {ISTI@TREC Microblog Track 2011 Exploring the Use of Hashtag Segmentation
               and Text Quality Ranking},
  booktitle = {Proceedings of The Twentieth Text REtrieval Conference, ({TREC})},
  year      = {2011}
}

Given a classifier trained on relatively few training examples, active learning (AL) consists in ranking a set of unlabeled examples in terms of how informative they would be, if manually labeled, for retraining a (hopefully) better classifier. An important text learning task in which AL is potentially useful is information extraction (IE), namely, the task of identifying within a text the expressions that instantiate a given concept. We contend that, unlike in other text learning tasks, IE is unique in that it does not make sense to rank individual items (i.e., word occurrences) for annotation, and that the minimal unit of text that is presented to the annotator should be an entire sentence. In this paper we propose a range of active learning strategies for IE that are based on ranking individual sentences, and experimentally compare them on a standard dataset for named entity extraction.

@inproceedings{esuliMS10,
  author = {Andrea Esuli and
               Diego Marcheggiani and
               Fabrizio Sebastiani},
  title     = {Sentence-Based Active Learning Strategies for Information Extraction},
  booktitle = {Proceedings of the First Italian Information Retrieval
               Workshop, ({IIR})},
  pages     = {41--45},
  year      = {2010} 
}

We describe a boosting-based supervised learning approach to the Multi-Way Classification of Semantic Relations between Pairs of Nominals task 8 of SemEval-2. Participants were asked to determine which relation, from a set of nine relations plus “Other”, exists between two nominals, and also to determine the roles of the two nominals in the relation. Our participation has focused, rather than on the choice of a rich set of features, on the classification model adopted to determine the correct assignment of relation and roles.

@inproceedings{EsuliM010,
   author    = {Andrea Esuli and
             Diego Marcheggiani and
             Fabrizio Sebastiani},
   title     = {ISTI@SemEval-2 Task 8 Boosting-Based Multiway Relation Classification},
   booktitle = {Proceedings of the 5th International Workshop on Semantic Evaluation,
             SemEval@ACL 2010, Uppsala University, Uppsala, Sweden, July 15-16,
             2010},
   pages     = {218--221},
   year      = {2010},
   url       = {http://aclweb.org/anthology/S/S10/S10-1048.pdf}
}