Browsing by Subject "crowdsourcing"

Sort by: Order: Results:

Now showing items 1-12 of 12
  • Kettunen, Pyry; Rönneberg, Mikko (2022)
    Proceedings of International Conference on Location Based Services
  • Hotti, Helmiina (Helsingin yliopisto, 2023)
    Diagrams are a mode of communication that offers challenges for its computational processing. The challenges arise from the multimodal nature of diagrams. This means that diagrams combine several types of expressive resources to achieve their communicative purposes, such as textual elements, connective elements such as arrows and lines, and illustrations. Humans interpret diagrams by judging how these different expressive resources work together to reach the communicative goals set for the diagram. In order to do that, humans make inferences of the diagram layout and the implicit relations that exist between different parts of the diagram. In order to build computational methods for diagram understanding, large amounts of data annotated with these implicit relations is required. Traditionally, these types of discourse structure annotations have been annotated by experts, due to the difficulty of the task and the requirement that the annotator is familiar with the theoretical framework used for describing discourse relations. The chosen theory for modeling discourse relations in diagrams is Rhetorical Structure Theory, originally developed for modeling textual coherence but applicable to multimodal data as well. This thesis explores the possibility to gather discourse relation annotations for multimodal diagram data with crowdsourcing; employing naive workers on crowdsourcing platforms to complete annotation tasks for a monetary reward. Adapting the task of discourse relation annotation to be feasible for naive workers has been proven challenging by past research concerned with only textual data, and the multimodality of the data adds to the complexity of the task. This thesis presents a novel method for gathering multimodal discourse relation annotations using crowdsourcing and methods of natural language processing. Two approaches are explored: adopting an insertive annotation task where the workers are asked to describe the relationship between two diagram elements in their own words and adopting a multiple-choice task, converting the formal definitions of Rhetorical Structure Theory to understandable phrases to annotate with. Natural language processing is used in the first approach to validate the language and structure of the crowdsourced descriptions. The results of the first approach highlight the difficulty of the task: the workers show tendencies of relying heavily on example descriptions shown in the task instructions and difficulty of grasping the differences of the more fine-grained relations. The multiple-choice approach seems more promising, with annotation agreement with expert annotators higher than in previous research concerned with discourse relations in textual data. The manual inspection of the annotated diagrams show that the disagreement of the crowdworkers and expert annotators is often justifiable; both annotations represent a valid interpretation of the discourse relation. This highlights one of the main challenges of the task, which is the ambiguity of some of the relations. Future work is encouraged to consider this by adopting an approach that is less concerned with a pre-defined set of relations and more interested in how the different discourse relations are actually perceived.
  • Rönneberg, Mikko (Unigrafia Oy, 2022)
    FGI Publications
    The production and use of geographic information have become easier and more social. The interactivity of maps has fundamentally changed, not only because the touch-based interfaces are easier to use, but also because maps offer possibilities to interact with others. Map applications allow citizens to contribute but also share content to others. This contribution and sharing done by regular people is referred to as crowdsourcing. Map applications that utilise crowdsourcing face specific issues regarding the creation process, the usefulness and the crowdsourcing. These issues, however, have not been studied comprehensively and lack real world examples. This dissertation is the initial step to fill this gap by studying map applications that utilise crowdsourcing. These map applications are described using the design science research approach. Three issues relevant for the map application studied are: 1) the creation process, 2) utility requirements and usability heuristics, and 3) crowdsourcing approach. These issues are studied by using the design science research approach to produce theoretical and empirical knowledge of three map applications utilising crowdsourcing. The aim is to use this knowledge to form a design science research based approach suitable for creating map applications utilising crowdsourcing. The results regarding the creation process indicate that following a specific approach will help in creating crowdsourced map applications. This dissertation provides a customised design science research approach for creating crowdsourced map applications. Furthermore, prescriptive knowledge that provides real world examples crowdsourced map applications is provided. The results concerning the usefulness of map applications utilising crowdsourcing indicate that there are specific utility and usability requirements to be accounted for. This dissertation provides key utility requirements and usability heuristics for crowdsourced map applications. In general, a map interface for exploring and sharing content is needed. The map interface should be simple, citizens should be supported and interaction should be intuitive. The results concerning the crowdsourcing approach of map applications indicate that there is a need for specifying how citizens are involved in the process. This dissertation provides key requirements of the crowdsourcing approach of these types of map applications. The community driven crowdsourcing approach should be supported by official content and an engagement approach based on gamified and social elements to motivate content sharing. Privacy of citizens should be preserved by applying the privacy by design approach throughout the creation process. Privacy-preserving map applications utilising community-driven crowdsourcing, in which citizens can be engaged with gamification and social elements to explore and share content can be created by following the designs science research based approach presented in this dissertation.
  • Volodina, Elena; Alfter, David; Lindström Tiedemann, Therese (2022)
    In this study, we investigate theoretical and practical issues connected to dif-ferentiating between core and peripheral vocabulary at different levels of lin-guistic proficiency using statistical approaches combined with crowdsourcing. We also investigate whether crowdsourcing second language learners’ rank-ings can be used for assigning levels to unseen vocabulary. The study is per-formed on Swedish single-word items. The four hypotheses we examine are: (1) there is core vocabulary for each proficiency level, but this is only true until CEFR level B2 (upper-intermedi-ate); (2) core vocabulary shows more systematicity in its behavior and usage, whereas peripheral items have more idiosyncratic behavior; (3) given that we have truly core items (aka anchor items) for each level, we can place any new unseen item in relation to the identified core items by using a series of comparative judgment tasks, this way assigning a “target” level for a pre-viously unseen item; and (4) non-experts will perform on par with experts in a comparative judgment setting. The hypotheses have been largely con-firmed: In relation to (1) and (2), our results show that there seems to be some systematicity in core vocabulary for early to mid-levels (A1-B1) while we find less systematicity for higher levels (B2-C1). In relation to (3), we sug-gest crowdsourcing word rankings using comparative judgment with known anchor words as a method to assign a “target” level to unseen words. With regard to (4), we confirm the previous findings that non-experts, in our case language learners, can be effectively used for the linguistic annotation tasks in a comparative judgment setting.
  • Goldman, Jean-Philippe; Scherrer, Yves; Glikman, Julie; Avanzi, Mathieu; Benzitoun, Christophe; Boula de Mareüil, Philippe (European Language Resources Association (ELRA), 2019)
    We present the crowdsourcing platform Donnez Votre Français à la Science (DFS, or “Give your French to Science”), which aims to collect linguistic data and document language use, with a special focus on regional variation in European French. The activities not only gather data that is useful for scientific studies, but they also provide feedback to the general public; this is important in order to reward participants, to encourage them to follow future surveys, and to foster interaction with the scientific community. The two main activities described here are 1) a linguistic survey on lexical variation with immediate feedback and 2) a speaker geolocalisation system; i.e., a quiz that guesses the linguistic origin of the participant by comparing their answers with previously gathered linguistic data. For the geolocalisation activity, we set up a simulation framework to optimise predictions. Three classification algorithms are compared: the first one uses clustering and shibboleth detection, whereas the other two rely on feature elimination techniques with Support Vector Machines and Maximum Entropy models as underlying base classifiers. The best-performing system uses a selection of 17 questions and reaches a localisation accuracy of 66%, extending the prediction from the one-best area (one among 109 base areas) to its first-order and second-order neighbouring areas.
  • Davis, Keith III (Helsingin yliopisto, 2020)
    We study the use of data collected via electroencephalography (EEG) to classify stimuli presented to subjects using a variety of mathematical approaches. We report an experiment with three objectives: 1) To train individual classifiers that reliably infer the class labels of visual stimuli using EEG data collected from subjects; 2) To demonstrate brainsourcing, a technique to combine brain responses from a group of human contributors each performing a recognition task to determine classes of stimuli; 3) To explore collaborative filtering techniques applied to data produced by individual classifiers to predict subject responses for stimuli in which data is unavailable or otherwise missing. We reveal that all individual classifier models perform better than a random baseline, while a brainsourcing model using data from as few as four participants achieves performance superior to any individual classifier. We also show that matrix factorization applied to classifier outputs as a collaborative filtering approach achieves predictive results that perform better than random. Although the technique is fairly sensitive to the sparsity of the dataset, it nonetheless demonstrates a viable proof-of-concept and warrants further investigation.
  • Laaksonen, Maija; Sajanti, Eeva; Sormunen, Jani J.; Penttinen, Ritva; Hanninen, Jari; Ruohomaki, Kai; Saaksjarvi, Ilari; Vesterinen, Eero J.; Vuorinen, Ilppo; Hytonen, Jukka; Klemola, Tero (2017)
    A national crowdsourcing-based tick collection campaign was organized in 2015 with the objective of producing novel data on tick distribution and tick-borne pathogens in Finland. Nearly 20 000 Ixodes ticks were collected. The collected material revealed the nationwide distribution of I. persulcatus for the first time and a shift northwards in the distribution of I. ricinus in Finland. A subset of 2038 tick samples containing both species was screened for Borrelia burgdorferi sensu lato (the prevalence was 14.2% for I. ricinus and 19.8% for I. persulcatus), B. miyamotoi (0.2% and 0.4%, respectively) and tick-borne encephalitis virus (TBEV; 0.2% and 3.0%, respectively). We also report new risk areas for TBEV in Finland and, for the first time, the presence of B. miyamotoi in ticks from mainland Finland. Most importantly, our study demonstrates the overwhelming power of citizen science in accomplishing a collection effort that would have been impossible with the scientific community alone.
  • Pirttinen, Nea; Hellas, Arto; Leinonen, Juho (Association for Computing Machinery, 2023)
    ACM International Conference Proceeding Series
    Learnersourcing is an emerging phenomenon in computing education research and practice. In learnersourcing, a crowd of students participates in the creation of course resources such as exercises, written materials, educational videos, and so on. In computing education research, learnersourcing has been studied especially for the creation of multiple-choice questions and programming exercises, where prior work has suggested that learnersourcing can have multiple benefits for teachers and students alike. One result in prior studies is that when students create learnersourced content, the created content covers much of the learning objectives of the course. The present work expands on this stream of work by studying the use of a learnersourcing system in the context of teaching SQL. We study to what extent learnersourced SQL exercises cover course topics, and to what extent students complete learnersourced exercises. Our results continue the parade of previous learnersourcing studies, empirically demonstrating that learnersourced content covers instructor-specified course topics and that students indeed actively work on the learnersourced exercises. We discuss the impact of these results on teaching with learnersourcing, highlight possible explanations for our observations, and outline directions for future research on learnersourcing.
  • Pirttinen, Nea (Helsingin yliopisto, 2020)
    Crowdsourcing has been used in computer science education to alleviate the teachers’ workload in creating course content, and as a learning and revision method for students through its use in educational systems. Tools that utilize crowdsourcing can act as a great way for students to further familiarize themselves with the course concepts, all while creating new content for their peers and future course iterations. In this study, student-created programming assignments from the second week of an introductory Java programming course are examined alongside the peer reviews these assignments received. The quality of the assignments and the peer reviews is inspected, for example, through comparing the peer reviews with expert reviews using inter-rater reliability. The purpose of this study is to inspect what kinds of programming assignments novice students create, and whether the same novice students can act as reliable reviewers. While it is not possible to draw definite conclusions from the results of this study due to limitations concerning the usability of the tool, the results seem to indicate that novice students are able to recognise differences in programming assignment quality, especially with sufficient guidance and well thought-out instructions.
  • Pesonen, Anu-Katriina; Lipsanen, Jari; Halonen, Risto; Elovainio, Marko; Sandman, Nils; Makelä, Juha-Matti; Antila, Minea; Bechard, Deni; Ollila, Hanna M.; Kuula, Liisa (2020)
    We used crowdsourcing (CS) to examine how COVID-19 lockdown affects the content of dreams and nightmares. The CS took place on the sixth week of the lockdown. Over the course of 1 week, 4,275 respondents (mean age 43, SD = 14 years) assessed their sleep, and 811 reported their dream content. Overall, respondents slept substantially more (54.2%) but reported an average increase of awakenings (28.6%) and nightmares (26%) from the pre-pandemic situation. We transcribed the content of the dreams into word lists and performed unsupervised computational network and cluster analysis of word associations, which suggested 33 dream clusters including 20 bad dream clusters, of which 55% were pandemic-specific (e.g., Disease Management, Disregard of Distancing, Elderly in Trouble). The dream-association networks were more accentuated for those who reported an increase in perceived stress. This CS survey on dream-association networks and pandemic stress introduces novel, collectively shared COVID-19 bad dream contents
  • Haarasilta, Teemu (Helsingfors universitet, 2013)
    The concept of social media includes a wide range of online, word-of-mouth forums including blogs, company sponsored discussion boards, chat rooms, consumer product or service ratings websites, and social networking sites. Social media is top of the agenda for many business executives today. Decision makers and consultants try to identify ways in which firms can make profitable use of applications such as YouTube, Facebook and Twitter. One of the main goals of an industrial company is to execute its new product development process in a way where new technological opportunities can be identified and commercialized before its competitors. In this study it was investigated how wood processing companies, log house builders and hardware stores currently exploit social media in their commercial efforts such as marketing and sales. Further it was also examined do wood processing companies, log house builders and hardware stores use social media in their new product development (NPD) process. The empirical part of the study was carried out by interviewing a small number of companies and conducting a desk research of the social media activity of a larger number of companies. Most of the companies studied were present in social media. The type and the intensity of social media presence varied depending on the type of the company. E.g. hardware stores used social media merely for marketing and sales related purposes whereas in wood processing companies the focus was on communication and PR. Besides the normal customer feedback systems there was no evidence that the companies of this study would actively use social media in collecting end user driven ideas for R&D purposes. Based on the theoretical background, framework of the study, desk research and company interviews a model for collecting feedback and product development ideas from end users is created. Furthermore some ideas for more effective usage of social media regarding the topic of the study are offered as well.
  • Dobat, Andres; Deckers, Pieterjan; Heeren, Stijn; Lewis, Michael; Thomas, Suzanne Elizabeth; Wessman, Anna Pia Frederike (2020)
    Hobby metal detecting is a controversial subject. Legal and policy approaches differ widely across national and regional contexts, and the attitudes of archaeologists and heritage professionals towards detectorists are often polarized and based on ethical or emotive arguments. We, the European Public Finds Recording Network (EPFRN), have implemented collaborative approaches towards detectorist communities in our respective contexts (Denmark, England and Wales, Finland, Flanders, and the Netherlands). Although our motivations are affected by our national circumstances, we base our work on an agreed set of goals, practices, and visions. This article presents the EPFRN's vision statement and provides insight into its underlying thoughts. We hope to create a debate on how to develop best practice approaches that acknowledge the inherent challenges of hobby metal detecting while realizing its potential.