Modern teaching methods in action in statistical classes

2.1 Problem-based learning

The term “problem-based learning” (PBL) is used to cover a wide variety of pedagogical methods and is a teaching approach that gained huge popularity all over the world at every level of education and every field of study due to its versatility and easy adaptability to many educational scenes. Despite the various definitions used to describe the notion of PBL, what is common in all of them is to define PBL as such a pedagogical method that is based on complex, real-life problems and is used to promote learning through hands-on experiences of the students exploring real problems, and through that, developing not only problem-solving but collaborative, critical thinking and communication skills of them as well [12, 13, 14, 15]. Problembased learning is a teaching approach that can be incorporated and tailored to any learning situation and can be used as the primary teaching method other than the traditional teaching principles of presenting methods and theoretical concepts to students directly PBL is a student-centred teaching method, engages students in solving problems themselves where learning is organized around a problem and where the role of the teacher is less of an instructor but more of a facilitator [16, 17]. Some argue that the essence of PBL is not even finding a solution for the problems presented but the process of learning itself.

The origin of PBL ranges back to roughly six decades ago, to the 1950s and 1960s, to McMaster University in Hamilton, Ontario, Canada [15]. Even though the first mentions of PBL are connected to the McMaster University’s medical programs in Canada, many other PBL-approaches developed simultaneously all around the world in the second part of the twentieth century, e.g. in the US (Michigan State University), in the Netherlands (Maastricht University) or Australia (Newcastle University). The main driver behind the development of the new curricula and teaching methodology was to respond to students’ inadequate clinical performance [15]. What if, students even though are having excellent theoretical knowledge, but are lacking hands-on experience and therefore are not able to use their knowledge in clinical practice when faced with complex medical challenges?

The McMaster medical school PBL model [13, 15, 16] is regarded to be the “pure” version of PBL, where the implementation of this educational approach generally happens without including lectures or tutorial sessions, but where students are usually work in a small group on solving a problem together. However, there exist many different hybrid models of PBL with lectures and tutorial sessions supporting student learning embedded in the models.

Certain learning principles need to be considered when designing a course along with the PBL teaching method, which have been summarized in three main approaches, and which also provide a link to another learning method, project-based learning (PBL) which will be introduced in a later part of this paper:

Learning approach: a central principle for developing PBL based teaching programs is that learning must be organized around a problem that provides context to the learning process. This principle also stands for project-based learning, where the starting point of creating a project is also having a problem.

The implementation of PBL is not unprecedented in statistics education, as this approach provides a better view of linking real-world phenomena with the methodology and statistical analyses to the students. Papers investigated the implementation of a PBL-based teaching or tutoring approach in statistics courses at undergraduate [19, 20, 21] and postgraduate level [22, 23] in different fields, such as mathematics [22], health sciences [19, 23], medical studies [24] or psychology [20]. Evidence suggested that the implementation of a fully integrated PBL teaching approach or introducing elements, such as directive tutor guidance not only results in students performing relatively better compared to others receiving traditional education but can contribute to the better understanding and linking of statistical concepts with other areas of their studies. PBL in statistical education also enhances students’ motivation towards learning statistics and promotes self-directed learning and yields in students memorizing the materials in the long run as well [19, 20, 21, 22, 23, 24].

PBL is undeniably a more interactive pedagogical approach compared to traditional educational settings and helps to develop the key learning skills in any area greatly, and as well can promote the further development of such interpersonal skills like communication, time management, teamwork, collaboration and identification of key problems. Another benefit of PBL is that this method has the potential for increasing learners’ confidence through supporting their professional development. Students learning in PBL can feel that they are treated as equal partners by the teacher and not as inferior to the teacher, thanks to the facilitator role of the teachers, where they do not dominate the discourse but complement and support the learning process. PBL requires a different approach from teachers, who have been mostly trained in the traditional educational setting and as well requires significant preparation especially when introducing PBL. PBL is therefore rather time-consuming to set up and so is to facilitate, as the teachers need to be equally present for each learning group and dedicate equal attention to each of them. Being a facilitator in PBL is not only challenging for the teacher because of time-management concerns but also because teachers need to face the challenge of stepping back from their traditional teaching role and take a more collaborative stance in the entire learning process. Several times problem-solving needs technical equipment to be at hand as well, which might mean some structural changes in the environment of teaching (e.g. the availability of a computer room).

2.2 Project-based learning

Project-based learning (PjBL) is an instructional form and pedagogical approach sharing many similarities with the previously introduced problem-based learning (PBL), even, in many cases is used as a synonym for PBL methodology (hence the use of the same abbreviation, PBL for both in many papers Here we use different abbreviations to make the distinction between the two methodologies clear), because of being equally student-centred [13, 25, 26, 27]. The origins of PjBL date back far earlier (to the late 19 th century) than those of PBL and they are developed independently from each other. John Dewey, a philosopher, and educator from the University of Chicago is regarded as the founder of this educational approach, who argued that students could better learn and engage with study materials if they can simulate the actual activities connected to their areas, therefore by doing what the actual professionals do in their chosen areas [13, 28].

Project-based learning is not only a pedagogical method, but an entire learning environment is not focusing only on how to develop student skills and how to deliver study materials, but also on community actions and the use of technology to promote better learning and collaboration of students and other actors in the learning process [13, 28]. Krajcik and Blumenfeld (2005) summarized the five main features of a project-based learning environment are:

The first feature of a PjBL classroom of having a drivingquestion is very similar to the problem-based learning approach. In PBL students most commonly receive a problem from the teacher which is usually of a lower scale (e.g. a problem that can be discussed in a shorter time, as in the frame of one lecture or just a few lectures). However, in PjBL students are encouraged to ask questions which are driven by their curiosity and entire research projects are built around such an inquiry [26].

As it could be seen from the above description of both PBL and PjBL, we can see that both are student-centred and problem-based teaching methods. However, there are some considerable differences between the two approaches. The first difference between the two methods is that while PBL is equally as problem based as PjBL, the latter requires a solution for the projects, as the aim of the entire research project is the creation of some sort of an artefact at the end In PBL the focus is less on the outcome of the problem-solving but the learning process itself; providing a solution is not a necessary element of PBL and neither is the production of some sort of an outcome (a presentation or a report). Second, the role of the teacher is even though a supportive role in both approaches, in PBL teachers act as facilitators in the problem-solving process, guiding students through the process. In PjBL teachers are supervising the process but are not giving direct guidance to the students. Third, the way of actual teaching in the two approaches differs as well. While in PBL students are expected to work out their learning requirements, in PjBL students receive some degree of tutoring in the form of actual teaching during the lifespan of the project. This means that in PjBL students are delivered the materials in a more direct form by the tutors during their education than in PBL teaching. Fourth, the timing of when to apply these approaches can differ. The aim of PBL is often to discover a topic together by assuming that students already have some degree of propositional knowledge (but it is not necessary to have covered the entire subject), while the application of PjBL can happen later in teaching programs, after assuming students already possess the amount of knowledge required to tackle a research project. The last, fifth difference is that PBL is based on the assumption that learning can happen by crossing interdisciplinary boundaries, while the same is the aim of the PjBL project: the main goal of PjBL projects is to bring together distinct study areas at the end of a course or training project. Accordingly, even though many refer to PBL and PjBL to be synonymous, what differentiates between these two approaches is that while PBL is an overall educational strategy applicable at all levels and phases of study programs, PjBL is rather an instructional tool or teaching technique applicable at a given area in the curriculum [25, 27].

Many recent studies [29, 30, 31, 32, 33, 34, 35] show that implementing a project-based view either for an entire statistics course, a module in a course or as an assessment tool can result in the improvement of student engagement and admittedly make them learn more compared to traditional education. Students seemed to favour projects to be included in statistics because they could learn better in such an environment that simulated future work and could obtain skills that are valued at the labour market as well. Modern statistical education, therefore, is an area that can benefit substantially from a project-based learning approach by empowering students not only with analytical knowledge but with a different way of thinking and a wide range of intangible skills that will be valuable in their future professional advancement.

2.3 Thinking-based learning

Thinking-based learning (TBL) is a teaching method to improve the quality of students’ thinking, enhance their content understanding and learning that improves the quality of their lives, including their work after they leave school [36]. TBL is one of the most suitable tools in the 21st century that infuses the teaching of critical and creative thinking into content instruction [37]. TBL aims at developing more intricate thinking skills far beyond pure memorizing. It promotes skilful thinking, which refers to the use of proper thinking skills for specific tasks [38]. Skilful thinking is where the thinker assesses what is required to accomplish a thinking task and consciously applies the thinking skills and mental behaviours in a strategic combination to produce thoughtful products of high quality. Educators should infuse instruction in skilful thinking into content instruction what means not only teaching students the thinking skill but also teaching them to use the appropriate forms of thinking skilfully [37]. Why is it worth proving skilful thinking with TBL and incorporating this teaching method into higher education? First, TBL provides highly detailed, step-by-step procedures and guides by which it can be used efficiently in higher education [39]. Second, there is empirical evidence that these practices can both improve students thinking and make university courses more interesting for them.

Skilful thinking has three components and all of them must be considered to apply the TBL method successfully: thinking skills, habits of mind (or thinking dispositions) and metacognition. Thinking skills refers to generating ideas, clarifying ideas, and assessing the reasonableness of ideas [40]. Critical thinking skills refers to being able to analyze complex problems, investigate questions, evaluate different points of view or sources of information, and draw appropriate conclusions based on evidence and reasoning [41]. Critical thinking characterized as a purposeful mode of thinking about content, a subject, or a problem, which involves different components [42]. Thinking critically plays an important role in innovations. The recognition that a new approach is required to solve the problem is based on knowledge and understanding of the situation. Moreover, there is judgment involved in determining the broad range and form of possible solutions to problems or how to proceed [43]. These skills can be developed by several kinds of thinking processes: critical thinking activities, activities requiring evaluation, excursions into analysis, classification projects, requests for prioritizing tasks and items, engagement in problem-solving, thinking about options, creating an argument, or developing creative ideas [44].

Habits of Mind (HOM) are related to guiding those procedures to perform broad and productive thinking. This concept implies paying attention to the context to select useful and appropriate thinking skills to complete a complex task [38]. Costa and Kallic (2009) isolated 16 different parts of HOM: persisting; managing impulsivity; listening with understanding and empathy; thinking flexibly; thinking about your thinking; striving for accuracy; questioning and posing problems; applying past knowledge to new situations; thinking and communicating with clarity and precision; gathering data through all senses; creating, imagining and innovating; responding with wonderment and awe; taking responsible risks; finding humour; thinking interdependently; remaining open continuous learning [45].

Metacognition is related to the awareness of one’s thinking to improve learning. It refers to the use of thinking skills and habits of mind, based on the person’s assessment of the task [38]. Metacognition is often simply defined as "thinking about thinking." However, defining metacognition is not that simple and there are several terms currently used to describe the same basic phenomenon (self-regulation, executive control) or an aspect of that phenomenon (meta-memory), and these terms are often used interchangeably in the literature [46].

The components of skilful thinking are strongly related to Bloom’s taxonomy. This framework, originating from a dissatisfaction with the outcomes of students’ learning in higher education, is one of the best approaches for school education. The framework has six objectives: knowledge, comprehension, application, analysis, synthesis, and evaluation. The final four objectives collectively defined as “higherorder thinking” [47]. Considering Bloom’s taxonomy and the factors of skilful thinking, Swartz and McGuiness have created eight principles that should be followed to effective TBL application in classes:

Teach explicit thinking strategies in the classroom.

The TBL method has already been successfully applied in primary and secondary schools, but the method is also receiving more and more attention in higher education [38, 42, 44]. There are countless opportunities for using TBL methods and combining them with other teaching and learning methods through real-life examples when we develop statistical thinking and reasoning in Statistics courses. In Section 3 we show an example of it.

2.4 Flipped classroom

Flipped classroom (FC) or inverted classroom is a pedagogical approach where “students engage in the relatively passive content learning before class (typically by viewing lectures online at their leisure and ideally reflecting on content), allowing for in-class time to be dedicated to the active learning of content (e.g. problemsolving, assignments, group work, quizzes, etc.) with a peer mentor and instructor support available” [48]. This approach has emerged from primary and secondary education [49] and became popular in post-secondary education about a decade ago [50]. The role of online class models such as flipped classrooms has been highly important due to the COVID-19 crisis [51, 52].

There are several advantages of using the flipped classroom approach. It allows students to learn at their own pace and have a deeper understanding of educational material. The prior learning frees up class time which can be utilized for active learning (e.g. discussions, group work, quizzes etc.) [53]. If students are actively involved in the learning process, it leads to higher motivation, confidence, and critical thinking skills [54]. The self-paced learning at home and the learning activities in classes can also result in higher levels of satisfaction [55] and engagement [56]. Based on a meta-analysis [48] it is highlighted that student’s performance was higher in a flipped classroom than in a traditional classroom in most of the examined studies. The paper of Gundlach et al. (2015) showed that traditional students scored higher on average on the exams compared to flipped students but Gundlach et al. explain that by less face-to-face class time in the flipped classroom compared to the traditional one or by the instructor’s higher experience on the traditional lecture [57].

Several activities can be applied in flipped classes. O’Flaherty and Phillips (2015) highlighted that a flipped class has pre-class (asynchronous) and face-to-face (synchronous) activities. They mention among pre-class (asynchronous) activities podcasts/vodcasts, screencasts, annotated notes, videos, use of pre-readings, interactive videos, case-based presentations, or simulations. Among synchronous activities case-based presentations, team-based discussions, panel discussions, expertled discussions, role-plays and student presentations, discussions and debates are mentioned [53]. However, we must note that during the pandemic due to the social distancing, synchronous activities have been held not face-to-face but via video conferencing platforms; some of them have built-in online voting systems which can be used for synchronous activities.

Concerning the class sizes, flipped classrooms can be used in small or large classes too. Farmus et al. (2020) found that the size of the class does not moderate the flipped classroom’s effect on student performance [48]. Nielsen et al. (2018) pointed out in a large class that flipped classrooms improve student performance [58], but there are studies with small sizes [55, 59] where the positive effect of flipped classrooms on student performance is observed. Furthermore, if we are considering the size of flipped content, we can state that flipping works well for an entire course but partially (e.g. flipping only the most challenging content of the course) too [60].

Flipped classrooms are applied in many different fields, but there have already been several examples in the field of statistics education. These studies [55, 57, 58, 59, 61, 62, 63] examined the effect of the flipped classroom on student performance by using a control group which is a traditional classroom. Most of the studies found that a flipped classroom improves performance. However, Gundlach et al (2015) pointed out that the traditional classroom had an advantage over the flipped classroom, which is explained by less face-to-face class time in the flipped classroom compared to the traditional one or by the instructor’s higher experience on the traditional lecture [57]. Besides student performance, increased satisfaction [55] and higher course evaluation scores [58, 59] are mentioned as a positive consequence of flipped classrooms in statistical applications.

Beyond the advantages, there are some challenges to face during a flipped classroom application. Preparing for a flipped classroom requires extra work from instructors for creating pre-recorded classes and planning in-class activities [48] and it also requires substantial IT support both from the side of students and educational institutions as well [56]. However, those burdens can be eased if not the entire course but only the most important parts are flipped [60]. There is also a danger from the field of students if they do not watch videos at home, and it can result in inefficient conversations in classes [56]. Nevertheless, if someone is aware of the challenges of a flipped classroom and prepares for them (e.g. by using additional training the educators or by using exciting elements – such as visualization tools – in pre-class materials) the application of a flipped classroom could be successful.

2.5 Gamification

In recent years gamification has been studied a lot, since the number of companies and sectors who implement this method for instance to encourage workers to work better and more or to learn things, is rising continuously. Several definitions exist to explain this concept. One of these is the definition of Nick Pelling who said that the essence of gamification is that the elements and mechanisms specific to (computer) games are used by users in non-playful environments [64] Helgason stated, it is “the adoption of game technology and game design methods outside of the games industry” [65]. Another definition says that gamification is “the process of using game thinking and game mechanics to solve problems and engage users” [66]. Based on Bret Terill’s thoughts, gamification is when we are “taking game mechanics and applying them to other web properties to increase engagement” [67] which is one of the earliest definitions for gamification [68]. In these definitions there are many overlaps. However, everybody agrees that the word gamification covers “game-based mechanics, aesthetics and game thinking to engage people, motivate action, promote learning, and solve problems” [69] in non-game situations. Gamification means not only trying to bring the motivational power of digital games into education but also using the game as a pedagogical model, as a general form of learning specific to the children. The essence of gamification is not only to play computer games in school, but to use the game as a learning model for more effective learning, especially for continuous motivation [70, 71].

We must face a new generation in schools which is named NetCoGame generation by Fromann. We can communicate with them almost exclusively through the language of gamification and playful mechanisms. [72]. According to Lee and Hammer “understanding the role of gamification in education means understanding under what circumstances game elements can drive learning behaviour” [73]. Regarding this, gamification can be one possible good solution to fulfil the needs of this new generation [74]. Because besides applying the potential of ICT tools, gamification is working with a strategy that covers a huge part of the teaching-learning process by using the motivational elements of playing (scoring, levelling, competition). These ensure the expectations of Generation Z and the positive experiences of the learning process and a longer-term commitment to the acquisition of knowledge [74, 75]. So to say, the key strategy of gamification lies in providing rewards and competing after the completion of the desired tasks. Hamari et al. after their literature review on empirical studies of gamification categorized the tested motivation affordances into ten categories which are the following: points, leaderboards, achievements or badges, levels, story or theme, clear goals, feedback, rewards, progress, and challenge [76].

The reason behind the widespread usage of this method is rooted in its lot of advantages. Kapp says that it increases the engagement of students [69] for example, by telling an exciting storyline. He also mentions as a fact, that it helps education to get closer to realworld contexts and problems. It is the result of that the process of problemsolving during playing the games that develop skills that can be used in the real world to solve complex problems. Another advantage of this method is that it applies and practices learning within a meaningful and authentic context. Furthermore, it promotes the practice, teamwork, and the community of learners in the case of games which requires collaboration to win [77]. Squire mentions that there are examples for gamification, like “Water Challenge” from Minecraft or “Supercharged” from MIT, which places students within systems where they can safely manipulate or explore different kinds of functions [78]. Reference [79] adds two more advantages to this list. One of these is the creation of enthusiasm. It says that gamification can foster enthusiasm in the case of subjects that students struggle with like math or programming. These games also provide constant feedback for the players by using not only points or coins but also leaderboards or dashboards.

Naturally, this method also has its disadvantages besides the advantages. A disadvantage of gamification is that when the rewarding stops, users will likely not continue playing unless they find another motivation to keep gaming [75]. Also, “some research has found using badges and leaderboards may lead to negative outcomes for intrinsic motivation and class satisfaction compared to a non-gamified course” [81]. Another disadvantage is that if the game is not well-structured and a player fails a lot, it can create demotivation or fear of playing. In addition to these, the earlier mentioned motivation systems only work for naturally competitive students, but the others who are not can lose interest very fast. Students also can be distracted from the learning objectives. For example, a poorly designed game can lead to disengagement or it can also cause problems when the game is not implemented into the curriculum effectively [77]. Based on the opinion of Ford gamification can be very costly. For example, if we want to implement complex gamification systems into education, we need to buy or develop expensive software, or the educator is required to go to training [79]. Computers and internet connection needs also can be mentioned here [77]. However, such equipment is part of our everyday life. Many times game logistics also mean a big obstacle because setting up a complex game requires a lot of prior planning and logistics. These are not always worth the effort due to the other disadvantages, like decreased student attention which is caused by the fast pace of the game and the immediate feedbacks. Students tend to expect these from all parts of their education, and lack of it can lead to uninterest [79]. Gamification games can also lead to addiction or overstimulation because competitive and perfectionist students tend to play too much to pass a level or get forward on the leaderboard. An additional disadvantage mentioned by Bellotti et al. is that it can blur the boundaries between virtual life and reality [77].

Fortunately, many of these disadvantages can be prevented by good planning and implementation and with monitoring of students’ playing habits. There are many good examples of gamification software or voting systems that are already used in education and can support or supplement the curriculum but in different ways. In this section we would like to demonstrate some of these without attempting to be comprehensive.

Kahoot is a multiple-choice quiz software that makes teachers able to test the knowledge of students or make them compete against each other during the classes. The questions of the quiz should be programmed in advance by the teacher. Students need computers with Internet connection to log into kahoot.it webpage or any smart device with downloaded Kahoot app on them. When the quiz begins, students can read the questions and the possible answers on the board, while they can vote for the correct answer via their computer or smart device within a certain period (e.g. 20 seconds). It is not only important to choose the correct answer but also speed counts. The faster gets the more points. The students can see the top 3 after every answer and at the end of the quiz there is a final ranking based on the number of good answers and the speed. Kahoot is a good example to make testing more fun and to encourage students to study and be ready to get tested anytime.

VoxVote is an application, is very similar to the previously mentioned Kahoot. However, it can be used not only for quizzes but also to make pool voting or realtime surveys and it is free for educational institutions. Students or respondents can join the quiz or the pool in the same way as described in the case of Kahoot. In the case of open questions, it is suitable for creating realtime word clouds from the answers given. Right after the questions are answered, the distribution of answers can be seen, and the results can be exported into an Excel sheet.

Entrepoly is a serious game which aims to develop entrepreneurial skills with the help of gamification. “Serious games are considered to be the tools of gamification”. The content of Entrepoly can be customized by the teacher which makes it easily adaptable. Students can create their avatars and collect golds by completing playful tasks, while teachers can monitor their progress continuously [68].

Minecraft is a popular, widely used, and well-known game amongst the young generation. In this world everything is built from pixels and children are free to use their creativity and to collaborate with others. The Education Edition of the game offers lesson plans and downloadable worlds to help teachers to integrate the game design into the classroom. In addition to this, there is a Minecraft professional learning community where educators can connect to others to share lesson plans or other resources.

Archy Learning, a gamified e-learning platform is a place where teachers can put YouTube links and classroom notes into a learning pathway. It uses gamification within its class quizzes, educational video games, mixed media exams and awarded certificates after completing courses. This is a gamified classroom which can be used from everywhere.

Exploristica, which won the International Statistical Literacy Project Best Cooperative Project Award in 2015, teaches important statistics and probability concepts with the help of games and gamification. This project is a cooperation of the Portuguese Central Statistical Office, the Portuguese Statistical Society and Lifetime Lab.

These examples demonstrate how many ways gamification can be used and implemented in education. However, there are also many other kinds of apps which are built on gamification, like flash card apps, language learning apps, applications which teach how to code or even gamified apps which help to learn to play instruments. The list of possibilities is endless and constantly growing.

3.1 Analysing Civic problems

The PCS project has developed teaching materials, use authentic large-scale data in such topics as income inequality, migration, ageing society, sustainable development goals, or human development, often presented in innovative ways. Lesson plans, teacher and student versions of the materials, datasets, review of dynamical visualization tools, conceptual framework, syllabuses, workshop materials, publications, and other various collections are available at the project website in English, and selected materials are available in German, Portuguese, and Hungarian languages as well at http://iase-web.org/islp/pcs/. Study materials and syllabuses are all heavily infused with modern techniques, such as the use of gamification elements, problem and projectbased learning approaches, and as well emphasis on the use of visualization and modern data analysis tools in the classroom, together with encouraging the use of technology and smart devices, again, a method unconventional in traditional education.

PCS materials at the University of Szeged have been implemented in various groups concerning the size of the group (from small groups of 20–30 students to large classes of 500+ students), their study level (from high school to master students) and study area (business and law education). At first, the implementation of PCS project study materials served the purpose of testing the materials at various statistics-related courses in business education [82]. The developed PCS teaching materials follow a rather practical and problem-based approach and build the presentation of statistical tools around very relevant current social and economic problems. The introduction of such teaching materials both in introductory statistics courses and as an entirely new optional data analysis course at the bachelor level resulted in major success from the students’ side.

To provide an example of how PCS teaching materials work, let us look at one lesson focusing on the topic of Ageing society. A version of the Ageing Society lesson has two blocks and contains several questions and activities. The first block concentrates on basic terms: age distributions, the use, and the interpretation of (online) population pyramids. The second block focuses on the ageing society. The class can begin with a warmup question such as “which country has the oldest population?”. Then we continue in a group discussion. Students think about the meaning, causes, consequences of ageing society and present their idea. The next question is how we can measure/examine/visualize the ageing society, causes and the consequences. In the first round, students give ideas about potential indicators that can be used and describe why they have selected the indicators, what was their process of thinking. After these presentations we consider the indicators, used by official statistics to measure ageing society. We focus on the Old age dependency ratio and the Ageing index.

After the discussion phase students use statistical data sources and visualization tools to analyze the problem. For instance, students can analyze both national and international levels the ageing. We can use for instance the Gapminder, online population pyramids, international and national statistical data sources to compare the age distributions of different territories in the same year or compare pyramids of territory between different years to do comparison and interpretation. We can compare not only the pyramids but the indicators also. For instance, ask the students to calculate and compare the Old age dependency ratio, and the Ageing index for the years 2000, 2020, 2100 in the case of a country. Seeing the result, we ask the students to formulate (critical) questions from methodological, social, and economic points of view.

Students reported that they enjoyed problem-based learning materials and felt more motivated to participate in the discourse during classes. They also appreciated the facilitator role of the instructors and felt that they could better discover certain topics this way. However, they often had difficulties in linking knowledge elements from different areas as this have not been the general practice in their study programs and as well understanding the importance of the discussed topic and the relationship between civic engagement and the discussed topic resulted in being challenging for them Although students faced some challenges in working with the PCS materials, their overall experiences and opinion remained predominantly positive, reporting that these materials made the otherwise compulsory and less-enjoyable introductory statistics topics a bit more palatable and likeable.

At the same time, educators faced challenges in implementing PCS materials in their courses as well. Compared to traditional statistics classes, when PCS materials were introduced, teachers needed to take an interdisciplinary view as well, preparing in advance from the social context and the relationship between statistics, civic engagement, and the social content of the lessons, therefore preparing for such classes resulted in being rather labour-intensive, requiring significant extra preparations at the beginning. Educators also faced the challenge of how to find the right balance between the statistical methods, the use of digital tools and the social aspects of the discussed topic and as well how to emphasise the importance of each of these elements in class, how to engage students equally in the elements of PCS teaching materials (digital tools, statistical methods, social issues).

3.2 Project work at master level

In this part we are providing an example for introducing a project-based learning methodology at a master level Statistics course in marketing education for firstyear students. This course is a 14 weeks-length course and takes 4 hours weekly. Statistics is a compulsory statistics course for firstyear marketing Msc students where one focus of the course is on the practical application of PPDAC (problem-plan-data-analysis, conclusions) statistical-inquiry cycle, and on developing students statistical thinking. The application of hierarchical cluster analysis, factor analysis in marketing-oriented research are new topics of this class.

Project-based learning usually assumes that students already possess a substantial amount of prior knowledge [25, 27]: it is no different from this time either. Students usually join this course after having at least two semesters of introductory statistics courses at bachelor level, as an antecedent to this course, at which they have covered the basics of descriptive statistics and inferential statistics, already being familiar with the notion of population, sampling, estimates or hypothesis testing, relationship analysis (correlation analysis, bivariate regression, crosstabs analysis).

The entire course is built around a research project that the students are conducting in groups of 3 to 5 persons, and which concludes in the creation of a presentation which constitutes 50% of their assessment in the course (alongside with a test which mainly assesses their practical knowledge about the cluster and factor analysis). Accordingly, by the time students engage in their research projects, they are (supposedly) already equipped with ample statistical and analytical knowledge and skills. In their project, which ranges through the entire semester, students form teams of 4–5 persons and carry out their primary research to solve university service-related problems for the community of the university. An important feature of the PjBL approach in the case of this course is summoning university stakeholders to join the discourse: the problems students are seeking to solve are always coming from a service provider. The different service providers of the University of Szeged (the Career Office, the University Book Shop, the University Gift Shop, the Alma Mater Office etc.) are supplying the students with their current problems (e.g. the number of students participating in volunteering programs announced by the Alma Mater Office is very low or the sales of the Gift Shop are not improving despite an at that time recent introduction of a new online store or what the customers think about the Coffee Shop’s supply of goods or what the future of the copy saloon), for which they are expecting solutions and recommendations from the students. The introduction of university stakeholders puts the research project into a real professional setting where marketing students need to conduct their research in close contact with real actors at the university. Their “job” as part of this course is therefore not a simulation but is a real research where the results of their projects will have actual implications. If students can come up with an excellent solution for one of the problems, that is of great value to the university undoubtedly. However if they do not deliver an excellent solution, they do not get punished, as if they would get in real life. This project aims to make students learn the use of statistical analysis methods by doing, to challenge students in a professional setting, and to get a taste of what an actual research will look like in their future as marketing professionals, but at the same time in a supportive environment to foster learning.

Service providers are invited to the first class and present their general problems, to which they need an answer. At the beginning of the course students need to form teams and choose a university stakeholder. When the teams are formed and are matched with their service provider partners, they are required to meet with them regularly to know the client, to understand the problem deeply. Based on the meetings, students are needed to formulate their research question, a driving question which will orientate their work onwards, what kind of data and data collection are needed, what is the target group, schedule of the research, etc. and they present a research plan to the educators and the representatives of the clients during the 4 th week. Each team and stakeholders participate at every session where the research plans are presented and are highly encouraged to pose questions and make constructive comments on each other’s work. Based on the feedback students can modify their plan and ask for a meeting with the clients and the educators. If the plan is accepted students carry out their measurement tool (e.g. a questionnaire) and present it alongside the data collection plan and survey design on the 8 th week. According to the feedbacks students can modify their tools. In these research projects students collect primary data from the university community. After that according to the survey design students collect data during the 8–13 th weeks. The class is small; therefore, the maximum number of groups is usually around 5. Thus data collection is open for one week or one and a half weeks, and only one data collection run at the same time. The order of the data collection is the readiness level of the group works. After the data collection students manage and analyze the data.

The resulting artefacts of the group projects are presentations in the exam period providing the university stakeholders with solutions and suggestions for their problems. Teams can present anytime between week 14 th and 20 th when the group is finished their work. These presentations are based on the statistical analysis of the collected primary data and a large emphasis is put on presenting the results not in a scientific but practical manner to the peers so that even non-statisticians could understand the results and recommendations. After the final presentation in front of the educators and clients, who evaluate the project together, students get feedback concerning their work both from the lecturers and the stakeholders. Lecturers evaluate the methodological accuracy of the statements in the presentation and as well of their work during the entire process, e.g. if they noticed any statistical misconduct in the process, while client representatives evaluate their work from a customer’s point of view, i.e. whether they received a solution for their problem or not and as well evaluating how easy collaboration was with the teams and have had any problems if they managed to resolve those.

Their research question and research plan are discussed during one of the classes -which are held parallel with the group projects throughout the semester to teach new analytical methods to the students, just as how some tutoring can complement PjBL teaching [25] and is fine-tuned if needed. University peers can express their opinion not only on the survey but also on how the collaboration is going so far, whether they could understand each other with the student team or not, providing students with feedback concerning their work, which supports the learning procedure and helps students identify their weaknesses even during the project.

This teaching method is challenging both for educators, service providers and students. In this process the role of the educators differs from the frontal teaching methods: supervise the whole research process and differentiate based on the problems and progress of the different groups. Furthermore, educators have to know each other, cooperate with the clients also, and have to understand the viewpoint and the requirements of the partner also. For instance, students have to present their results correctly but in an understandable way for non-statisticians as well (how and how deep we present the used method, results, etc.), statisticians also have to learn and show it during the class. It is also a good teaching and learning technique to bring the business and the academic side closer together. Several times the service providers’ issues do not require the use of factor analysis or cluster analysis. In the project work there are no requirements for the statistical techniques, and groups have to tailor it to the problem and the research design. This is the reason that the course is moving on two parallel paths and the evaluation has two parts: the project and the tutoring sessions. During the tutoring sessions, the application of certain statistical analyses is taught to students in a more traditional frontal form of education too. Nevertheless, at these sessions, attention is still paid to initiating collaboration by fostering a problem-based approach, establishing activities on real datasets and questions, and trying to solve them during the tutoring sessions.

This methodology has been applied at the Statistics course for quite a few years now and student feedback has been positive overall: students enjoy participating in large-scale research projects which have some real implications and the collaboration with university stakeholders creates the feeling of a professional environment. Students report the peers treating them as equals, which gives them some degree of confidence and makes them more engaged and motivated in working on the project as they are seeking approval from the university stakeholders. There is a healthy amount of rivalry among the teams (even though there is no competition aspect of evaluating the projects) and this competition is also driving them for further excellence, making them perform better at classes (which is also attributed to better engagement). They admit that this form of statistical education requires significantly more effort from their side; they need to prepare for the course more as if they were just solving directed sample exercises during class. However that feeling of “realness” in the entire research project is making them more engaged in this form of education than in a traditional class, making the project-based learning in the Mathematical Statistics course a successful teaching method.

3.3 Analysing visualizations

Data are often communicated through graphs and charts because they allow the skilled user to quickly recognize and extract important features of the data set under analysis [83]. Nowadays, visualizations are surrounding us everywhere. Thinking about the advertisements and news on the television or thinking about social media where countless information sources and personal opinions are producing a large quantity of data, and providing new data on how people feel, almost about anything. Science also uses infographics in the attempt to reach people and simplify communication, bridging different knowledge backgrounds and cultures [84]. In the case of visualizations, it is crucial that what we want to show, what the prepared visualization shows, and what the viewers perceive and understand have to be the same. Visualization makers must consider the goal of the visualization, about what, how and for whom they want to visualize.

Data visualization is also regarded as a critical skill for today’s data-centric jobs. As such it is frequently found in job postings and the demand for people with this skill increasing exponentially at about 1500% in the past decade [85]. Users also need to have the appropriate skills, which can be called visual literacy to understand these figures and get essential information from them. That is why students also need to be visually literate just as they need to be verbally literate, but visual literacy is often overlooked in undergraduate education [86]. However, interpreting figures does not only provide new information sources but greatly improves the learning process and the way of thinking. This kind of learning as part of what can be called multimedia learning fosters deep learning [87].

At the bachelor level in our introductory statistics courses, we seriously take care of our students’ visual literacy. Using simple and complex figures from real-life examples we try to integrate the thinking-based learning (TBL) method into traditional curricula and encourage students’ critical thinking. In the first part of this lesson, the educator presents the theoretical background, short history, trends, and challenges of data visualization. This part includes the meaning, types and examples of data visualization, the formal requirements of graphs and charts, and the general expectations for figures. This is the basic theoretical knowledge that students need to understand and apply in later parts of the lesson. In the second part, various kinds of real-life figures are presented and analysed that students may have already seen in media (e.g. television news or newspapers) before. These figures are often rather subjective, seemingly simple but not correct from a statistical point of view because these have been made for some special purposes (e.g. advertisements on TV or newspaper, social media news, etc.) Also, these figures do not meet the general requirements of statistical figures that students learned in the first part of the course (e.g. missing formal requirement elements; inadequate proportions; misleading title or captions, or graph types selected incorrectly). Instead of asking a simple closed question, like: What do you think, is this figure correct or not, figures are analysed, and students are asked: “What do you think about this figure? What is illustrated on the chart? What is the message of this figure? What is the purpose of this chart? What about the formal requirements that we have learnt? Do they satisfy them?” The teacher lets the students express their own opinions in the class. This work can be managed as individual or group work also. Meanwhile, the educator collects the different opinions on the board. At this step, students are asked to form critical questions. This is the discourse, the so-called active thinking, in which phase students try to explain their thoughts, even defend them if they see it is necessary. Metacognition is when we encourage students to try to think with the mind of the creator of the figure.

Let us consider an example of it. We want to analyze Hungarians’ satisfaction with personal relationships. To do it, we use Quality of life visualization tools at the Eurostat website (Fig. 1).

OPEN IN VIEWER

If we are not in a computer lab, we can show and examine this chart only, but in a computer lab we suggest using this online tool to examine the question, because in this case meta information and descriptions are also available from the application to answer critical questions. This example can emphasize the importance of using metainformation, descriptions.

We ask students to study this chart in small groups based on the listed questions. What kind of critical questions and comments arise in the students’ minds?

We selected Hungary only. What is the reason for seeing Bulgaria, Austria, Ireland, Malta, Slovenia? Whether these countries have the smallest and highest value of data? Because it is not evident if we compare the positions of these countries to the min. and max. notes’ positions.

After the analysis, we can ask students to give similar examples on different visualizations, different misused figures they may have seen in everyday life if they can. This step represents the application of the acquired knowledge and provides some sort of feedback to the professor about that the students really understood what he tried to teach them. At the end of the class the educator summarizes the essence of the lesson and draws attention to that figures must always be examined critically and with an objective eye. In addition, the list of the questions is also repeated that must always come to mind whenever they see a figure: to whom, why, how, and for what purpose the illustration was prepared. These questions also can be understood as a kind of habit of mind (HOM) that need to master to think critically. This trio essentially corresponds to Curcio’s [88] different levels of interpretations about figures: Reading the data reading between the data, and reading beyond the data.

In this section, we introduce two examples where official statistics takes place in the curriculum and flipped classroom approach was applied as well.

3.5 Voting systems in action

In addition to the earlier mentioned various teaching methods, we also implemented gamification, especially voting systems in our classes. We used this method in introductory statistics classes in two different ways before the COVID-19 pandemic. One of these was using quizzes at the end of the seminars to see how much attention the students paid to the educational material of the seminar. We used the earlier described quiz software, Kahoot. Our aim with implementing Kahoot was to make students come to the classes, take an active role, pay attention to the material, and enjoy themselves. As a motivation the students who got any of the first three places out of the classes of 30 people could earn additional points which had been counted to their final exam at the end of the semester. There were occasions when we gave one point to everybody who had been there and finished the test. Our goal was not only to motivate them to study and to come to classes but also to show them realworld tasks during these tests and reflect on those areas in which they should improve to get excellent results. We tried to avoid the disadvantages of gamification. That is why we did not have a test at the end of every seminar. Students never knew when we were going to test their knowledge, which made them focused and ready all the time since they did not have many chances to collect additional points. Based on our experience this was a very good idea to implement Kahoot and quizzes into the statistics seminars, and students also enjoyed it a lot. We reach the goals we aimed at with the use of this gamification tool. However, we can say that at least in the case of statistics education it can be only a complementary tool, but a whole course cannot be built on gamification this way.

The other application we used in statistics teaching was VoxVote which is also a quiz app, but we used it differently from Kahoot. VoxVote voting was applied in statistics lectures, not seminars. When using the app we had different goals. At the beginning of the semester we used it as an icebreaker. We asked the students in open questions about what they expect about the course and what comes to their minds when they think about statistics. Later VoxVote was used for educational and attitude-forming purposes. Through examples of provoking questions and graphs we tried to show them the misuse of statistics and emphasize how important it is to understand and be able to interpret graphs and data tables. We also wanted to highlight that the media and internet sometimes try to distort the data and the related information. To do so, we used examples from Gapminder Test 2017. Our aim with this was to hand over students the message that people tend to think that global phenomena like poverty, education or gender inequality are getting worse and worse. However, we never think about the fact, that on the one hand all the data from around the world are available for us now, which was not true in the previous decades. It means that it is hard to compare the situation to earlier ages. On the other hand media has a distorting effect on our thinking and the way how the messages it broadcasts are formulated matters a lot. If we are aware of these, we should not accept everything we hear as a fact but using critical thinking instead because poverty is lower than before, education is improving all around the world and we have never been this close to gender equality. Although during the lectures we did not utilize the gamification functions of this application we considered the usage of VoxVote successful. We reached our aims of ice-breaking and attitude formation and we could involve students more in the classes.