The event was officially opened by Gábor Balás, Chair of the PROFEEDBACK network and Director of the HÉTFA Research Institute, alongside Tibor Navracsics, Minister for Public Administration and Regional Development of Hungary. The opening and keynote speeches were delivered by experts from international organisations including the OECD, World Bank, and the International Labour Organization (ILO). The presentations addressed the intersections of employment, education, and youth policy, and highlighted the role of evaluation in driving systemic transformation.

The two-day conference brought together more than 100 participants and featured 20 poster presentations and 8 thematic sessions. Several colleagues from the HÉTFA Research Institute contributed to the parallel sessions: Luca Koltai, Eszter Szőnyi, and Kinga Tóth presented evaluation experiences from programmes targeting older NEET youth (Not in Education, Employment, or Training), while Bálint Herczeg discussed the impact of university programmes on student dropout rates through Hungarian case studies. In the poster session, Luca Szöőr-Fülöp and Katalin Bördős shared good practices for supporting the labour market integration of young people.

A dedicated policy roundtable focused on the role of evaluation in EU-level decision-making, bringing together experts and institutional representatives.

As the conference concluded, a clear message emerged: there remains a strong need for continued collaboration to strengthen the culture of evaluation across Europe. One of the closing reflections summed it up well: effective policymaking relies on thoughtful, independent, and reflexive evaluation.

Presentations, videos, and posters from the conference will soon be available on PROFEEDBACK platforms.

The training was led by internationally recognised experts: Marijana Sumpor (Institute of Economics, Zagreb), Sari Rannanpää (Finnish Institute for Health and Welfare), and Kalterina Shulla (Technische Universität Dresden), who shared their extensive experience in the field of policy evaluation.

The course brought together 12 selected young researchers and practitioners from various countries, who spent three intensive working days developing case-based evaluation designs, interpreting complex interventions, and applying what they learned to real-world policy scenarios.

To conclude the week, participants presented their own evaluation designs during the poster session of the 9th PROFEEDBACK Conference, thus linking theoretical learning with ongoing policy dialogue and practice.

The training materials and selection of slides will soon be made publicly available on the official PROFEEDBACK website.

📍 Further information and updates are available here:
🔗 https://profeedback.eu/training-school-4

]]> https://hetfa.eu/2025/03/evaluation-report-on-making-a-museum-activity-released/ Thu, 20 Mar 2025 08:56:58 +0000 https://hetfa.eu/?p=11582 We are pleased to announce the publication of an evaluation report prepared by HÉTFA Research Institute, which presents the results of the impact evaluation of the ‘Making a Museum’ activity. This initiative is part of the broader ‘Save Schindler’s Ark – implement testimony, iWALK and permanent hybrid digital and place-based learning in the unique place where the events occurred’[1] project, which focuses on preserving Schindler’s Ark and implementing innovative testimony-based learning experiences through iWALK and hybrid digital and place-based education.

The primary objective of the evaluation was to assess the impact of the learning process throughout the implementation of the ‘Making a Museum’ activity. The report examines all components of this initiative, including place-based learning, digital learning through video testimonies, and the collaborative co-creation process. Special attention was given to evaluating students’ development in terms of attitudes, critical thinking skills, empathy, behavior, and knowledge acquisition.

About the Project and the ‘Making a Museum’ Activity

The ‘Save Schindler’s Ark’ project aims to transform Schindler’s Ark into a Museum of Survivors. Schindler’s Ark, known for saving 1,200 Jews on Schindler’s List as depicted in Stephen Spielberg’s film, was originally part of one of the largest wool factories in Europe. The Low-Beer family, who owned the factory since 1854, fled the Nazis in 1938. In 2018, they repurchased the factory ruins and partnered with the local community to establish the Arks Foundation, which is leading the museum’s development and testimony-based educational activities.

The ‘Making a Museum’ activity engaged secondary school students from various European countries in a co-creation process, allowing them to contribute ideas to the museum’s development and testimony-based activities. This collaborative learning initiative employed student-centered methods such as site-based learning and digital learning, integrating testimony-based educational materials. Beyond enriching the museum’s content, the project aimed to enhance students’ knowledge and develop key skills such as cooperation, communication, and critical thinking, while fostering empathy and engagement with historical memory.

Evaluation Methodology

To ensure a comprehensive assessment of the activity’s impact, the evaluation employed a mixed-method approach combining qualitative and quantitative research. The primary methods included surveys conducted with participating students and interviews with teachers who guided and observed them throughout the project. This methodology was carefully designed to align with the objectives of the project, ensuring meaningful insights into the educational outcomes and experiences of those involved.

The findings of this evaluation highlight the significant educational benefits of the ‘Making a Museum’ activity and its role in fostering a deeper understanding of historical testimony and memory among students. We are excited about the continued progress of the ‘Save Schindler’s Ark’ project and its contribution to Holocaust education and remembrance.

Access the full evaluation report for more details through this link.

[1] The project itself is funded under the CERV programme of the European Union, under the project number 101089907.

• Analysis of strategic coherence and alignment
• Evaluation of the effectiveness of products
• Examination of the institutional framework.

Based on the result, proposals were outlined for the 2021-2027 programming period.

The executive summary of the report is available here (in English).

We examined which applied methods served the strategic goals most efficiently. We also evaluated the financial and technical progress and results of the interventions, the efficiency of the use of grants, loans and combined financial instruments, as well as the territorial aspects. The impacts of the Covid-19 pandemic on the progress of the projects were also taken into consideration. Considering that the target values of indicators are set to 2023, this evaluation should be regarded as an interim report.

The target of 100% coverage of residential and business demand sites is expected to be almost fully achieved by the end of 2023. The time delays in projects were caused on the one hand by low design and construction capacity, on the other hand caused by design difficulties due to the use of a common electric stakes, and inaccuracies in the address list. The lack of liquidity on beneficiary side due to delays should be also taken into account. Therefore, broadband developments have generated demand in the Hungarian telecommunication sector that the sector was not prepared for when the projects were launched.

Network developments in state/municipal and educational institutions have covered the whole country, however the indicator on average bandwidth available in these institutions is limited in its ability to show the impact of these developments due to the large variance. Therefore, the required bandwidth can vary heavily between different types of institutions.

The take-up of funds from the loan programme to support broadband development has been very low and payments have often been delayed. The conditions of the scheme were not favourable enough for the beneficiaries and the administrative burden increased by the involvement of MFB in the procedure.

In the case of the examined constructions, more sources went to the regions where broadband coverage was the lowest and where the number of service locations to be covered was the largest, so the allocation of resources for broadband development was appropriate.

Our experts propose to differentiate by are and target groups when it comes to broadband deployment. It is also recommended to extend the project implementation period to 36 months and to further simplify the rules of the common pole line. Regarding the residential end-points, it is proposed to differentiate between the targets to be achieved in public institutions. It is necessary to review the system of IT public procurement, to improve the organisation of public procurement and to reduce the cost of procedures. Assessing the need for repayable aid should also be taken into account, in line with the simplification of submission procedure.

The full evaluation report (in Hungarian – with Executive Summary in English ) on the assessment of broadband developments in Hungary can be found here.

In the framework of the Action we are organising a conference titled Evaluation of public policy responses to ’black swans’. The event will take place in person in Budapest on April 28th 2022. The Conference brings together researchers and policymakers from European states, universities, research institutes and consultancies. Its aim is to identify the most significant challenges posed by black swan events from the evaluation community’s perspective. The conference aims to attract papers and posters that contribute to providing theoretical concepts and quantitative/qualitative methodological solutions to forward the participants’ repertoire of evaluating policies borne from black swan events.

More information on the conference is available here. Abstracts shall be submitted until 28 February 2022 and sent to profeedback@hetfa.hu. Registration is available here.

Our Action called PROFEEDBACK aims to foster the networking of the policy evaluation community at EU-level, raise awareness on the importance of evaluation policy research and improve its impact on policy-making. The Platform, following a bottom-up and open approach, will gather researchers and professionals from various scientific fields and sectors to present and evaluate theories, topics, tools and methods of policy evaluation. Results of the Europe-wide assessment of good practices will provide direct and high-quality inputs for national and EU bodies responsible for policy evaluation.

Policy evaluation is a key tool in understanding, developing and modernising EU policies, thus there is a growing demand for EU-wide and high quality evaluation services. The main challenge is that currently there is no bottom-up platform for European researchers and professionals working in policy evaluation. They have limited possibilities to discuss common problems, assess country specific practices and share their knowledge in a mutually beneficial and effective way.

PROFEEDBACK aims to tackle this challenge with an innovative, complex, cross-sectoral and multidisciplinary approach by:

• analysing the theoretical and methodological questions of evaluation policy research putting special focus on Cohesion Policy,
• enriching scientific research of policy evaluation and addressing new challenges,
• harmonizing fragmented approaches to set the research agenda and to get a common understanding,
• contributing to the capacity building of the policy evaluation community,
• involving national and EU policy-makers and the civil society to contribute to evidence-based policy-making,
• framing a Code of Conduct for the policy evaluation community.

More information is available here on PROFEEDBACK.

The aim of the analysis was to assess the impact of measures to support social entrepreneurship. The experts focused on the measures implemented with EU funding in the 2007-2013 programming period and their results and impacts. They also paid particular attention to the evaluation of the progress and results of the 2014-2020 period.

The evaluation assessed the contribution of the funding instruments to the objectives of increasing employment and job creation. The evaluation also assessed the impact of aid on competitiveness, growth, growth potential improvement, investment performance and sustainability. The experts also examined the evolution of corporate governance practices and company balance sheets.

In summary, social enterprises that are supported have great difficulty in remaining competitive in a competitive market while focusing on their social objectives. They face ongoing challenges in building their entrepreneurial footprint, which not only requires a permanent human capital commitment, but also financial investment or significant liquid reserves to sustain and continue operations. The experts suggested the need for continued support for social enterprises, the creation of a legal environment with a clear definition of social enterprises and the development of a certification scheme. The development of a common methodology for measuring the social impact of investments and a long-term strategy to support social entrepreneurship would be beneficial. The development and long-term maintenance of a support framework that builds on each other should also be considered. The experts suggest that a permanent wage subsidy should be available to support the employment objective of social enterprises. Some social enterprises provide transit services for re-integration into the open labour market and funding for this should be secured in the future.

The report entitled “Evaluation of measures supporting social enterprises in Hungary” is available here in Hungarian. The executive summary is available in English as well.

Following the introduction of the district system in 2013, the range of tasks performed by the district seat towns became wider, which affected both the quality of life and the competitiveness of these cities. The District Seat Town Monitor 2020, compiled by HETFA experts, therefore examines the quality of life of Hungarian district seat towns. It reviews the period of 2012-2018 that can be described as an advancement phase with regard to the development of urban networks. The Monitor also explored the strengths and weaknesses of seat towns, and the impact that development projects have made on the life of the locals.

The analysis examines the quality of life of cities along eight dimensions covering their social and economic situation, infrastructure and the range and quality of opportunities and services available. The study focuses on the 2010s – therefore excludes the effects of the pandemic – presenting on the difference between the quality of life before and after the introduction of the district system.

HÉTFA experts ranked cities on the basis of their quality of life, categorizing settlements into five groups: towns of outstanding, high, good, medium or lowest quality of life. In addition to Szentendre, Gödöllő, Paks and Dunakeszi, we can find several county seats at the forefront of the ranking, such as Székesfehérvár, Győr and Veszprém. According to the results, the quality of life offered by district seat towns can be quite different in some large regions of Hungary. In the wider area near Budapest, as well as in Northern Transdanubia and on the shores of Lake Balaton, we can find towns of mostly outstanding and high rankings, proving that a strong network of medium-size cities has developed in the recent years. The overall picture is extremely mixed in the southern parts of the country – in addition to many cities with a good and medium quality of life, we can also find district seats that received the outstanding and the lowest ranking according to the Monitor. Finally, the eastern third of the country has a majority of district seat towns of the medium and the lowest category.

You can access the entire report in Hungarian on this link.

Macro effects of development policy

The study focuses on the joint macroeconomic effects of the programs financed by the Structural and Cohesion Funds for the programming period 2007-2013. The analysis distinguishes the short-term demand and the long term supply effect occurred as a result of subsidies (manifested for instance in productivity change and capacity expansion).
Our results show that the GDP In 2015 was 4.9% higher than it would have been without the programs launched in 2007. Developments increased the level of employment by 2.8% in eight years. This impact was significant especially in the case of low-skilled employees. As a consequence of development projects their employment grew by 5.2%.
A significant part of the growth stems from the capacity-expanding and productivity-increasing effects of the development, thus it sustains after the termination of development programs. As a result of the programs, the level of GDP in 2015 grew by 2.1%, excluding periodic demand-stimulating effect. Long term employment grew with 0.8% as a whole, and 1.9% in case of employment of low skilled workers.

The strength of these impacts depends mainly on the effect of EU funds on export and import. It was also important to separate direct sectoral effect of subsidies and the induced demand for goods of other sectors. The largest beneficiary was the community services sector (public administration, health, education and social care) – it received 38% of the grants. Approximately 45% of the development funds were spent on services of the construction industry.
During the impact assessment we have analyzed several scenarios, using different assumptions on the substitution between skilled labor, unskilled labour and capital.
Our results show that the impact of development resources corresponds to the growth of the economy: if the economy is prosperous, the impact of development resources is stronger.
It is an important methodological strength of our study that we could use aggregated databases of subcontractors’ invoices to asses the demand generated by developments. The study focused on the sector-specific structure of suppliers to estimate macroeconomic impacts, the calculations also revealed some result on the local composition of the supplier side that opens the opportunity for a territorial analysis in the future.

Small and medium-sized enterprise assessment of EU development programs

Another study of HÉTFA examines how small and medium-sized enterprises evaluate EU development resources and their impact. Our analysis is based on a thorough entrepreneurial survey based on a relatively large sample. The most important results:
• Companies used the funds to realize previously planned development projects. Companies financed their part of contribution to the projects from accumulated profits.
• EU funds mostly contributed to capacity expansion, technological development and sales growth. The impact was reported to be small in case of human factors like increase of employment or improving employees’ qualifications.
• In the course of the implementation, administrative burdens were the most problematic issue, while market building and co-operation with suppliers did not mean a major challenge.
• Medium-sized businesses had the most positive view on the grant system.
• Larger companies with higher-productivity and applicants with practice on application for EU funds are more likely to plan application for development grants in the future.
• Those who were more active and successful in development programs seems to be less motivated to to sell abroad in the future.
• In the construction industry, more companies applied for grants and they were more successful than in other sectors. Beside this, three quarters of the construction companies worked on a project the was ordered because the customer got EU co-financed development subsidy.

Documents are available in Hungarian here.