Digitalization has brought about significant changes to various areas of society, including the evaluation field. The availability of data sets and the speed of processing them have increased, along with the implementation of recommendations. Concepts such as Artificial Intelligence, Machine Learning, Web Scraping, and Big Data have become increasingly familiar, though their everyday usage in evaluations is still behind their potential.
On 30-31 March 2023, the European Evaluation Society and the Profeedback COST Action CA20112 (led by HETFA) co-organized an online workshop focused on the impact of digitalization on evaluations, and the evaluation of digitalization. More than 60 people attended the two-day workshop, which aimed to explore the various ways digitalization has affected evaluations and how evaluations can better assess the impact of digitalization.
The workshop featured a range of expert speakers who discussed various aspects of digitalization and evaluation. The workshop was moderated by Alexander Kocks, Jaroslav Dvorak and Sven Harten and the speakers included:
- Oto Potluka, who provided an introduction to the topic
- Steve Powell, who presented his abstract “StorySurvey” on capturing changes in causal stories with a time machine
- Oto Potluka, Meier Dominik, and Regula Wolf, who presented on the evaluation of co-creating digital policies by civil society, using a case study from Switzerland
- Bouyousfi Eddine Salah, from Mohammed V University of Rabat, who conducted a systematic review on artificial intelligence and big data-driven evaluation research and practices
- Kai Rompczyk, from DEval – German Institute for Development Evaluation, who explored the technological revolution in the field of evaluation, discussing the changes and limitations of integrating artificial intelligence in a standardized evaluation process
- Francesca Ardizzon, Alessandra Caputo, and Silvia Vignetti from CSIL Milano, who presented on using big data for EU programs evaluation and the challenges and lessons learned from monitoring data from ERDF and CF
- Janos Borst, Thomas Wencker, and Andreas Niekler from DEval – German Institute for Development Evaluation, who discussed using text classification with Bayesian correction for estimating overreporting in the Creditor Reporting System on climate adaptation finance
- Nea-Mari Heinonen from the Ministry for Foreign Affairs of Finland, who shared her experience using data science methods in evaluation from an evaluation manager’s point of view
- Elitsa Uzunova from Univation – Institut für Evaluation e.V., Mariella Seel, and Ludwig Grillich from the Department for Evidence-based Medicine and Evaluation, University for Continuing Education Krems, who presented on evaluation and digitalization in health promotion and prevention in settings.
Overall, the workshop provided valuable insights into the impact of digitalization on evaluations and the evaluation of digitalization. It highlighted the need for evaluators to remain current and adaptable in the face of rapidly changing technologies and emphasized the importance of stakeholder engagement in evaluations. As digitalization continues to shape various areas of society, it is essential for evaluators to understand its impact and adjust their approaches accordingly.
The workshop recording is now available, and those interested can register to watch the full video series at the following link: https://forms.gle/jwYbH3knCcyvyR5k7
For more information about the workshop, visit: https://profeedback.eu/wp-content/uploads/2023/03/EES-Digitalization-and-Evaluation-programme-230215.pdf