Studying and developing platforms for community-led scientific surveys and annotations

Abstract

A core research activity in many sciences concerns the gathering of data through questionnaire-based surveys. On the other hand, annotation projects which require input by field specialists become invaluable in various areas of research by creating evaluation data for automated methods. Surveys and annotation projects share similarities, since they both depend on participants who are prompted to answer a set of questions, optionally referring to a particular artifact such as text segments. Both tasks are hindered by their dependence on volunteers, who are reluctant to participate due to the time required to be spent on them. Additionally, both tasks may require participants of a particular background, thus burdening the research conductors with the weight of finding suitable ones. Despite their similarity, these tasks have been treated separately so far and different tools have been developed to facilitate each one. In this thesis, we present a platform which finds solutions for both tasks and in addition improves the procedures from both the organizer’s perspective and the participant’s one. This is achieved through the incorporation of rewarding mechanisms which make the participation in these types of tasks more interesting and competitive and through the creation of an expert recommendation system that facilitates and reduces the time needed for finding participants that are domain experts.