25 min read
Effectiveness of social simulations and serious games.
Although the majority of our daily activities at the Centre for Systems Solutions is connected with developing serious games and social simulations, we do not consider them as the main goal of our organization. Such activities are solely the means, not the end we are trying to achieve. We believe that using social simulations enables participants to get a real first-hand experience of tackling difficult problems. This creates a new understanding of interdependencies, relations, and complexity behind systems that they are part of in real life. But is it really the case? Does using social simulations actually matter and may lead to tangible changes in participants’ lives or in their everyday work? To check if our tools positively affect participants and their understanding of complex reality, we regularly conduct an evaluation of the participants’ results. Addressing users’ comments and suggestions helps us improve our social simulations and learn more about their effectiveness as educational or awareness-raising tools. So far, the Centre for Systems Solutions conducted two major evaluations on large groups of participants.
The first one constituted the part of the project ‘Green games. Libraries and non-governmental organizations involve youth in the game for nature’. The objective of the project was to increase the level of knowledge among young people about biodiversity, climate change, and the economic value of ecosystems. To reach the aim, we invited librarians and workers of non-governmental organizations in Poland and equipped them with tools (climate change-related board and online games) and know-how to prepare for and lead a game-based workshop with youth. Further, to check if playing the games enabled youth to gain a better understanding of environmental issues and inspired them to learn more about such topics, we asked them to answer pre- and post-game questionnaires.
We were able to get 3372 responses from young people between the ages of 15 and 19. We found out that after playing the game, 84% of the participants reported an increase in general knowledge about environmental issues. Out of this, 46.1% responders considered this increase as substantial. Similar results were obtained in reference to the questions about biodiversity and climate change-related knowledge acquisition. In the case of biodiversity awareness, 84.8% participants reported some knowledge increase, out of which 50.5% assessed it as substantial. In the case of climate change awareness, the corresponding values were 78.8% and 43%.
Furthermore, we asked participants how they evaluated their increase in knowledge, understanding, and interest in environmental issues. Only 11.17% of the respondents felt that the workshop gave them no new insights or sparked no interest in these topics. This means that 88.83% of participants learned something new or got inspired to learn more about environmental issues after playing the game.
Finally, we compared the number of participants who responded that they had had no previous interest in environmental issues and felt that this topic is not worth any attention to the number of respondents that held the same opinion after playing the game. It turned out that out of 226 people with the highly negative attitude toward this topic, 38 were inspired to find out more information about this field after playing the game.
The second evaluation was connected with the project ‘New Earth – Catholic Church educators increase the climate sensibility among students and believers.’ The project’s objective was to embed climate change-related issues in a durable and accessible tool that would be used by Catholic educators. We developed thus Laudato Si – a pro-ecological board game inspired by the second encyclical of Pope Francis, and a set of additional educational materials. Further, we compared questionnaire responses of school students that took part in Laudato Si-based lessons (1398 participants) against the replies obtained from the control group (806 students).
We found out that in the control group (who had no contact with the game and educational materials), only 62% of respondents believed that human activity affects climate change. 37% of interviewees were skeptical of this connection, had no opinion or denied climate change in general. Meanwhile, in the group exposed to the game, 77% responders were convinced that climate change is real and that it is affected by people.
What is more, participants who had the opportunity to play Laudato Si as a part of the lesson, felt that climate change is a much more serious matter than the students from the control group.
An important aspect of the research was to find out whether young people feel that their own actions can have an impact on the environment and climate change. According to their responses, 64% of the Laudato Si players believed that their actions matter, with only 34% doubting that an individual can do anything to control the negative environmental impacts. Meanwhile, in the control group, only 54% of the people believed that they have influence, and 37% were convinced that they, on their own, cannot do anything.
These results show that social simulations do have an influence on participants. We can clearly see that they affect the knowledge of participants and their understanding of environmental issues. This makes us optimistic about the positive consequences of using social simulations and gives us the motivation to improve our products to be more effective and impactful. Still, we understand that further research is needed to assess the long-term effects of changes that we observed in participants’ responses. This would require involving a large number of participants and more elaborate means of measuring the effects of social simulations. We believe that the results of such research would enable us to make our workshops even more effective. As was mentioned at the beginning, we continuously put our effort to create better tools to reach our goal.