Abstract
Data quality is key in the success of a citizen science project. Valid datasets serve as evidence for scientific research. Numerous projects have highlighted the ways in which participatory data collection can cause data quality issues due to human day-to-day practices and biases. Also, these projects have used and reported a myriad of techniques to improve data quality in different contexts. Yet, there is a lack of systematic analyses of these experiences to guide the design and of digital citizen science projects. We mapped 35 data quality issues of 16 digital citizen science projects and proposed a taxonomy with 64 mechanisms to address data quality issues before, during and after the data collection in digital citizen science projects. This taxonomy is built upon the analysis of literature reports (N = 144), two urban experiments (participants = 280), and expert interviews (N = 11). Thus, we contribute to advance the development of systematic methods to improve the data quality in digital citizen science projects.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Rotman, D., Hammock, J., Preece, J., Hansen, D., Boston, C., et al.: Motivations affecting initial and long-term participation in citizen science projects in three countries. iConference (2014)
Metso, L., Happonen, A., Rissanen, M., et al.: Data openness based data sharing concept for future electric car maintenance services. In: Smart Innovation, Systems and Technologies, pp. 429–436 (2020). https://doi.org/10.1007/978-3-030-57745-2_36
Bonney, R., Cooper, C.B., Dickinson, J., Kelling, S., et al.: Citizen science: a developing tool for expanding science knowledge and scientific literacy. Bioscience 59(11), 977–984 (2009)
Hand, E.: People power: networks of human minds are taking citizen science to a new level. Nature 466(7307), 685–688 (2010)
Dickinson, J.L., Shirk, J., Bonter, D., Bonney, R., Crain, R.L., Martin, J., Phillips, T., Purcell, K.: The current state of citizen science as a tool for ecological research and public engagement. Front. Ecol. Environ. 10(6), 291–297 (2012)
Dickinson, J.L., Zuckerberg, B., Bonter, D.N.: Citizen science as an ecological research tool: challenges and challenges and benefits. Ann. Rev. Ecol. Syst. 41(1), 149–172 (2010). https://doi.org/10.1146/annurev-ecolsys-102209-144636
Bonney, R., Shirk, J.L., Phillips, T.B., Wiggins, A., Ballard, H.L., Miller-Rushing, A.J., Parrish, J.K.: Next steps for citizen science. Science 343(6178), 1436–1437 (2014)
Palacin-Silva, M.V., Knutas, A., Ferrario, M.A., Porras, J., Ikonen, J., Chea, C.: The role of gamification in participatory environmental sensing: a study in the wild. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, pp. 1–13 (2018)
Balestrini, M., Diez, T., Kresin, F.: From participatory sensing to making sense. ECSA GA’2015 Conference (2015)
Newman, G., Wiggins, A., Crall, A., Graham, E., Newman, S., Crowston, K.: The future of citizen science: emerging technologies and shifting paradigms. Front. Ecol. Environ. 10(6), 298–304 (2012)
Guo, B., Yu, Z., Zhou, X., Zhang, D.: From participatory sensing to mobile crowd sensing. In: 2014 IEEE PerCom Conference Workshop, 2014, pp. 593–598. IEEE (2014)
Christin, D., Reinhardt, A., Kanhere, S.S., Hollick, M.: A survey on privacy in mobile participatory sensing applications. J. Syst. Softw. 84(11), 1928–1946 (2011)
Krontiris, I., Langheinrich, M., Shilton, K.: Trust and privacy in mobile experience sharing: future challenges and avenues for research. IEEE Commun. Mag. 7;52(8), 50–5 (2014)
Jahkola, O., Happonen, A., Knutas, A., Ikonen, J.: What should application developers understand about mobile phone position data. In: CompSysTech’17, ACM, pp. 171–178 (2017)
Foody, G., Fritz, S., Fonte, C.C., Bastin, L., Olteanu-Raimond, A.M., et al.: Mapping and the citizen sensor. Mapping Citizen Sens., 1–2 (2017)
Loss, S.R., Loss, S.S., Will, T., Marra, P.P.: Linking place-based citizen science with large-scale conservation research: a case study of bird-building collisions and the role of professional scientists. Biol. Conserv. 184, 439–445 (2015)
Heggen, S.: Participatory sensing: repurposing a scientific tool for stem education. Interactions 20(1), 18–21 (2013)
Zaman, J., De Meuter, W.: DisCoPar: distributed components for participatory campaigning. In: 2015 IEEE PerCom Conference Workshop, pp. 160–165. IEEE (2015)
Palacin, V., Gilbert, S., Orchard, S., Eaton, A., Ferrario, M.A., Happonen, A.: Drivers of Participation in Digital Citizen Science: Case Studies on Järviwiki and Safecast. Citizen Sci. Theory Prac. 5(1, 22), pp. 1–20 (2020). https://doi.org/10.5334/cstp.290
Jennett, C., Cox, A.L.: Digital citizen science and the motivations of volunteers. In: The Wiley Handbook of Human Computer Interaction, vol.2, pp. 831-841 (2018)
Orchard, S.: Growing citizen science for conservation to support diverse project objectives and the motivations of volunteers. Pac. Conserv. Biol. 25(4), 342–344 (2019)
Kortelainen, H., Happonen, A., Hanski, J.: From Asset Provider to Knowledge Company-Transformation in the Digital Era, In Lecture Notes in Mechanical Engineering (2019). ISSN: 2195-4356, pp. 333-341. https://doi.org/10.1007/978-3-319-95711-1_33
Kinnunen, S.-K., Happonen, A., Marttonen-Arola, S., Kärri, T.: Traditional and extended fleets in literature and practice: definition and untapped potential. Int. J. Strat. Eng. Asset Manage. 3(3), 239–261 (2019)
Kosmala, M., Wiggins, A., Swanson, A., Simmons, B.: Assessing data quality in citizen science. Front. Ecol. Environ. 14(10), 551–560 (2016)
Hunter, J., Alabri, A., Ingen, C.: Assessing the quality and trustworthiness of citizen science data. Concurr. Comput. Prac. Exper. 25(4), 454–466 (2013)
Budde, M., Schankin, A., Hoffmann, J., Danz, M., Riedel, T., Beigl, M.: Participatory sensing or participatory nonsense? Mitigating the effect of human error on data quality in citizen science. ACM 1(39), 1–23 (2017)
Bonney, R., Cooper, C.B., Dickinson, J., Kelling, S., Phillips, T., Rosenberg, K.V., Shirk, J.: Citizen science: a developing tool for expanding science knowledge and scientific literacy. Bioscience 59(11), 977–984 (2009)
Even, A., Shankaranarayanan, G.: Utility-driven assessment of data quality. ACM SIGMIS Database 38(2), 75–93 (2007)
Kahn, B.K., Strong, D.M., Wang, R.Y.: Information quality benchmarks: product and service performance. Commun. ACM 45(4), 184–192 (2002)
Metso, L., Happonen, A., Ojanen, V., Rissanen, M., Kärri, T.: Business model design elements for electric car service based on digital data enabled sharing platform. In: Cambridge International Manufacturing Symposium, 26–27 September 2019, Cambridge, UK, p. 6 (2019)
Kärri, T., Marttonen-Arola, S., Kinnunen, S-K., et al.: Fleet-based industrial data symbiosis. In: S4Fleet—Service Solutions for Fleet Management, DIMECC Publications Series No. 19, 2017. ISBN: 978-952-238-199-6, eISSN: 2342-2696, pp. 124–169 (2017)
Pipino, L.L., Lee, Y.W., Wang, R.Y.: Data quality assessment. Commun. ACM 45(4), 211–218 (2002)
Sabrina, T., Murshed, M., Iqbal, A.: Anonymization Techniques for Preserving Data Quality, p. 11 (2016)
Wand, Y., Wang, R.Y.: Anchoring data quality dimensions in ontological foundations. Commun. ACM 39(11), 86–95 (1996)
Strong, D.M., Lee, Y.W., Wang, R.Y.: Data quality in context. Commun. ACM 40(5), 103–110 (1997)
Cappiello, C., Francalanci, C., Pernici, B.: Time-related factors of data quality in multichannel information systems. J. Manag. Inf. Syst. 20(3), 71–92 (2003)
Heinrich, B., Helfert, M.: Analyzing Data Quality Investments in CRM-A Model-Based Approach (2003).
Kaiser, M., Klier, M., Heinrich, B.: How to Measure Data Quality? A Metric-Based Approach (2007)
Wixom, B.H., Watson, H.J.: An empirical investigation of the factors affecting data warehousing success. In: MIS, vol. quarterly, pp. 17–41 (2001)
Bobrowski, M., Marré, M., Yankelevich, D.: A Software Engineering View of Data Quality (1998)
Wiggins, A., He, Y.: Community-Based Data Validation Practices in Citizen Science, vol. 2, pp. 1548–1559 (2016)
Welvaert, M., Caley, P.: Citizen surveillance for environmental monitoring: combining the efforts of citizen science and crowdsourcing in a quantitative data framework, vol. 5, no. 1. SprinerPlus (2016)
Lukyanenko, R., Parsons, J., Wiersma, Y.F.: Emerging problems of data quality in citizen science. Conserv. Biol. 30(3), 447–449 (2016)
Vaddepalli, K.: Improving Data Quality in Citizen Science. Master’s thesis, LUT University, Lappeenranta (2019). [Online]. Available: http://urn.fi/URN:NBN:fi-fe2019090226408. Accessed from 13 May 2020
Palacin, V., Ginnane, S., Ferrario, M.A., Happonen, A., Wolff, A., Piutunen, S., Kupiainen, N.: SENSEI: harnessing community wisdom for local environmental monitoring in Finland. CHI Conf. 1–8. https://doi.org/10.1145/3290607.3299047
“Home | Doit Europe”, Doit-europe.net, 2020. [Online]. Available: https://www.doit-europe.net. Accessed from 15 May 2022
Vaddepalli, K., Palacin, V., Porras, J., Happonen, A.: Connecting Digital Citizen Science Data Quality Issue to Solution Mechanism Table (2020). [Online]. https://doi.org/10.5281/zenodo.3829498
Buytaert, W., Zulka?i, Z., Grainger, S., Acosta, L., Alemie, T.C., et al.: Citizen science in hydrology and water resources: opportunities for knowledge generation, ecosystem service management, and sustainable development. Front. Earth Sci. 2, 26 (2014)
Alabri, A., Hunter, J.: Enhancing the Quality and Trust of Citizen Science Data, vol. 12 (2010)
Wiggins, A., Newman, G., Stevenson, R.D., Crowston, K.: Mechanisms for Data Quality and Validation in Citizen Science, vol. 12 (2011)
Sullivan, B.L., Wood, C.L., Iliff, M.J., Bonney, R.E., Fink, D., Kelling, S.: eBird: a citizen-based bird observation network in the biological sciences. Biol. Cons. 142(10), 2282–2292 (2009)
Sutcliffe, A.: Designing for user engagement: aesthetic and attractive user interfaces. Synth. Lect. 2(1) (2009)
Sutcliffe, A., Namoune, A.: Getting the Message Across: Visual Attention, Aesthetic Design and What Users Remember, pp. 11–20 (2008)
Guo, B., Wang, Z., Yu, Z., Wang, Y., Yen, N.Y., Huang, R., Zhou, X.: Mobile crowd sensing and computing: The review of an emerging human-powered sensing paradigm. ACM Comput. Surv. (CSUR) 48(1), 7 (2015)
Maisonneuve, N., Stevens, M., Ochab, B.: Participatory noise pollution monitoring using mobile phones. Inf. Pol. 15(1), 2–51 (2010)
Santti, U., Happonen, A., Auvinen, H.: Digitalization boosted recycling: gamification as an inspiration for young adults to do enhanced waste sorting. In: AIP Conference Proceedings, vol. 2233, no. 1, p. 12 (2020)
Crowston, K., Prestopnik, v: Motivation and Data Quality in a Citizen Science Game: A Design Science Evaluation. In: 2013 46th HICSS, pp. 450–459 (2013)
Ren, J., Zhang, Y., Zhang, K., Shen, X.S.: SACRM: social aware crowdsourcing with reputation management in mobile sensing. Comput. Commun. 65, 55–65 (2015)
Arazy. O., Nov, O., Anderson, D.: Scientists@ home: what drives the quantity and quality of online citizen science participation? PloS 9(4), p. 90375, 11 (2014)
Slaughter, S.A., Harter, D.E., Krishnan, M.S.: Evaluating the cost of software quality. Commun. ACM 41(8), 67–73 (1998)
Kelling, S., Hochachka, W.M., Fink, D., Riedewald, M., Caruana, R., et al.: Data-intensive science: a new paradigm for biodiversity studies. Bioscience 59(7), 613–620 (2009)
Roman, L.A., Scharenbroch, B.C., Östberg, J.P., Mueller, L.S., et al.: Data quality in citizen science urban tree inventories. Urban Forestry and Urban 22, 124–135 (2017)
Bennin, K.E., Keung, J., Monden, A., Kamei, Y., et al.: Investigating the Effects of Balanced Training and Testing Datasets on Effort-Aware Fault Prediction Models, vol. 1, p. 6 (2016)
Mashhadi, A.J., Capra, L.: Quality Control for Real-Time Ubiquitous Crowdsourcing, vol. 9, pp. 5–8 (2011)
Graves, M., Constabaris, A., Brickley, D.: Foaf: connecting people on the Semantic web. Catalog. Classif. Quart. 43(3–4), 191–202 (2007)
Jaimes, L.G., Vergara-Laurens, I.J., Raij, A.: A survey of incentive techniques for mobile crowd sensing. IEEE Internet Things J. 2(5), 370–380 (2015)
Kim, S., Mankoff, J., Paulos, E.: Sensr: Evaluating a Flexible Framework for Authoring Mobile Data-Collection Tools for Citizen Science, vol. 2, pp. 1453–1462 (2013)
Mitra, T., Hutto, C.J., Gilbert, E.: Comparing Person and Process Centric Strategies for Obtaining Quality Data on Amazon Mechanical Turk, vol. 4, pp. 1345–1354 (2015)
Khoshgoftaar, T., Folleco, A., Van, J., Bullard, H.L.: Multiple Imputation of Missing Values in Software Measurement Data (2006)
Loss, S.R., Loss, S.S., Will, T., Marra, P.: Linkingplace-based citizen science with large-scale conservation research: a case study of bird building collisions and the role of professional scientists. Biol. Cons. 184, 439–445 (2015)
Cheng, M., Huang, T., Peng, P.L.H.: Bias reduction for nonparametric and semiparametric regression models. Statistica Sinica 28(4) (2018)
Kananura, R.M., Ekirapa-Kiracho, E., Paina, L., Bumba, A., et al.: Participatory monitoring and evaluation approaches that influence decision-making: lessons from a maternal and newborn study in Eastern Uganda. Health Res. Policy Syst. 15(2), 107 (2017)
Garbarino, J., Mason, C.E.: The power of engaging citizen scientists for scientific progress. J. Microbiol. Biol. Educ. 17(1), 7 (2016)
Bonter, D.N., Cooper, C.B.: Data Validation in Citizen Science: A Case Study from Project FeederWatch, vol. 10, no. 6 (2012)
Irwin, A.: Citizen Science: A Study of People. Psychology Press, Expertise and Sustainable Development (1995)
Höller, J., Tsiatsis, V., Mulligan, C., Karnouskos, S., Avesand, S., Boyle, D.: From Machine to the Internet of Things: Introduction to a New Age of Intelligence (2014)
Askham, N., Cook, D., Doyle, M., Fereday, H., Gibson, M., Landbeck, U., et al.: The six primary dimensions for data quality assessment. DAMA UK Working. Group (2013)
Messenger, J.C., Ho, K.K., Young, C.H., Slattery, L.E., Draoui, J.C., et al.: The national cardiovascular data registry (NCDR) data quality brief: the NCDR data quality program in 2012. J. Am. Coll. Cardiol. 60(16), 1484–1488 (2012)
Heinrich, B., Klier, M.: Metric-based data quality assessment-developing and evaluating a probability-based currency metric. Decis. Support Syst. 72, 82–96 (2015)
Caldiera, V.R.B.G., Rombach, H.D.: The goal question metric approach. Encycl. Softw. Eng., 528–532 (1994)
Acknowledgements
Authors thank European Regional Development Funds and Regional Council of South Karelia for funding MINT project supporting experience collection and also AWARE and CroBoDITT CBC projects funded by the European Union, supporting manuscript finalization.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Vaddepalli, K., Palacin, V., Porras, J., Happonen, A. (2023). Taxonomy of Data Quality Metrics in Digital Citizen Science. In: Nagar, A.K., Singh Jat, D., Mishra, D.K., Joshi, A. (eds) Intelligent Sustainable Systems. Lecture Notes in Networks and Systems, vol 578. Springer, Singapore. https://doi.org/10.1007/978-981-19-7660-5_34
Download citation
DOI: https://doi.org/10.1007/978-981-19-7660-5_34
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-19-7659-9
Online ISBN: 978-981-19-7660-5
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)