Programming with Scratch in primary school,indicators related to effectiveness of education process and analysis of these indicators in terms of various variables

Abstract

Since programming processes involve different thinking skills and different fields of knowledge, it is especially important for children to acquire 21st-century skills. Even though the programming education activities are being intensively applied, it can be said that there is a gap in quantitative researches supporting the effort to reveal the direct or indirect effectiveness of the learning–teaching processes for the programming education. This study, which was done to fill this gap, aims to examine the degree to which students learn programming concepts (PC) and to identify effective variables in that process with a developed curriculum for gifted students studying in the second–third–fourth grade in primary schools. For this purpose, a 15-week application was carried out and each student developed an individual project. In the study, a criterion list, observation forms and peer evaluations were used based on PC to examine projects and learning process. The scores obtained from these tools were used to examine the application of each participant, to comment on the effective variables and the adequacy of the teaching process. The evidence from this study intimates that female participants obtained higher scores than male ones in programming education. Those scores are higher in 9 and 10 age group of students than others. Those who haven’t had Internet access, who have never used computer or have had access to Internet as well as who haven’t had any computer courses had lower scores than others. The upshot of this is that previous computer technology experiences of students may have affected the scores obtained programming education process.

Keywords

Primary education programming education in K12 project-based learning evaluation Scratch

Introduction

Programming processes include different thinking skills and knowledge areas (Yildiz Durak, 2018a; Yildiz Durak and Güyer, 2019; Yildiz Durak, Karaoglan Yılmaz, and Yılmaz, 2019). For this reason, since it has those skills in its nature, it helps children to develop 21st-century skills such as ‘communication skills, creativity and intellectual curiosity, critical and systematic thinking, interpersonal and collaboration skills, defining, formulating and resolving a problem, self-orientation’ (Lau and Yuen, 2011; Partnership for 21st Century Skills, 2007). To gain 21st-century information and communication technologies (ICT) qualifications in education, programming education has started to be seen as important for students of middle school, even primary and nursery school (Fessakis et al., 2013; Kalelioğlu and Gülbahar, 2014). Especially in recent times, political importance of activities such as ‘Hour of Code’ (Microsoft, 2014) in the United States and ‘Code Week EU’ in Europe and the intense interest of children are considered as indicators of importance of early education programming.

The literature on why it is important to give programming education to children are generally discussed in terms of pedagogic (CSTA, 2010; DiSessa, 2001; Fessakis et al., 2013; Lau and Yuen, 2011; Lee, 2011) and economic dimensions (CSTA, 2010). From a pedagogical point of view, the focus is on the effects of software development processes on various thinking skills and learning competence (DiSessa, 2001). Software development processes are considered important for the development of high-level thinking skills as well as algorithmic problem-solving skills in children (Fessakis et al., 2013). On the other hand, in software development processes, children have the opportunity to solve a problem with their role as ‘problem solver’, to express their thoughts while solving problems and to explain their own thinking process, to follow the results and to get instant feedback (Clements and Nastasi, 1999). In addition, children gain and develop high-level thinking skills as well as problem-solving skills in software development processes (Clements and Nastasi, 1999; Liao and Bright, 1991).

If the importance of giving education programming to children is considered in terms of economic aspect, it can be stated that development in the field of technology is important for the development of societies in the 21st century. Developments in computer programming, which is considered as the basis of technology science, will provide the development of various economic sectors (CSTA, 2010). Computer programming education at an early age allows children to become aware of what they can do and to take an active role in the development of 21st-century societies.

In this study, programming activities were employed in the education of gifted students. There have been justified reasons of including gifted students as the participants of this study. First of all, upon bearing in mind that gifted students are able to authentically form the knowledge by themselves (VanTassel-Baska, 1988), we encouraged them to design individual project assignments through Scratch to ease their learning process. It was noted that the learning environment in the education of gifted students must rests on discovering more complex and real-world problems, creating innovative outcomes and presenting the opportunities in which they can develop their thinking skills (Poftak, 1998; Renzulli and Reis, 1997; Tomlinson and Reis, 2004). It is believed that programming activities are effective means of providing learning conditions. After all, the nature of programming education displays similarities with that of educational environment needed by gifted students (Yildiz Durak and Güyer, 2018; Yildiz Durak and Güyer, 2019). Scratch, on the other hand, can help gifted students grasp the abstract knowledge related to programming easily, thereby contributing such thinking skills as problem solving and reasoning in gifted students (Siegle, 2009).

In this study that dwells on the programming education in gifted students, Scratch programming platform was employed. The complexity of traditional programming languages makes it difficult for children to learn, and it can therefore be argued that an easy and understandable programming environment that prioritizes visuality, such as Scratch, may facilitate programming learning (Shin and Park, 2014).

There are various policies and practices in programming education. Evaluation of the effectiveness of programming education process should also be considered. Evaluation of products of projects and observation forms can be considered as a holistic approach that reflects the students’ learning process and products in all aspects. In addition, it can be said that the quantitative findings supporting the effectiveness of the learning–teaching process in programming education processes are lacking. To overcome this deficiency, in this study, the basic programming concepts (PC) were used and findings related to the results of the application process of a curriculum in primary school level and the products produced were included. Scratch is used as a tool to develop programming projects. In this study, 26 students developed 26 projects at the end of 15 weeks. This study aims to examine the degree to which students learn PC and to identify effective variables in that process with a developed curriculum for gifted students studying in the second–third–fourth grade in primary school. For this purpose, answers of the following questions were sought.

What are the scores that students get during the programming education process?

What is the predictive power of various variables on the scores of students in the programming education process (observation, peer evaluation, PC, designing for usability (DU), code organization (CO))?

Conceptual framework

Programming teaching process

It is utmost importance to include activities that develop high thinking skills of students at K-12 level for the purpose that they can gain necessary knowledge and skills for the future. Programming teaching directed to earlier periods of life of the students contributes them to develop their high-level thinking skills (Yildiz Durak and Güyer, 2018). In addition to this, programming is a skill encompassing different skills, and problem-solving and algorithmic thinking skills are the most important ones (Frost et al., 2009). Problem-solving skill can be considered as the one that forms the preliminary conditions of other skills. Chu and Hwang (2010) suggest that programming process is a kind of problem-solving process, whereas algorithmic thinking shelters the determination of a problem, dividing it into smaller units to be solved easily and listing the steps needed for solving it.

As Wang et al. (2016) remind us, programming process is to design a way-out in the event of a problem and to model it, by removing the unnecessary details in the designed solution. Following this, the modelled and abstracted procedures that are evolved into more functional are coded as order ranges. Several programming platforms designed for children don’t include all properties of an ordinary programming language, but they intend to teach the philosophy and concepts of programming and how it works. Furthermore, they enable children to perceive that programming is an entertaining process (Yildiz Durak, 2018b).

Teaching PC and skills, on the other hand, is attracting widespread interest due to the significance (Yang et al., 2015). The primary purpose of programming education is to teach use of programming language, thereby developing their problem-solving and programming skills (Hwang et al., 2008). Feldhusen (1997) highlights the importance of providing complex and multidimensional tasks with gifted students to develop their high-level thinking, problem-solving and creative thinking skills. Programming is a convenient learning task so as to develop thinking skills of gifted students. Kirk et al. (2000) note that gifted students can form more complex and structured knowledge frameworks based on their learnings when compared to average students. Kubilius and Lee (2004) also suggest that gifted ones can learn the basic computer concepts by themselves. However, it is of great importance to revise the learning process of programming education and to give scaffolding.

Scratch as a programming teaching tool

Block-based programming languages are generally preferred due to their simplified interface and convenience during code writing process (Yildiz Durak, 2018c). Scratch comes into prominence among block-based programming systems (Chen et al., 2017; Durak, 2016; Grover and Pea, 2013). There has been growing interest in the role of Scratch in the studies dwelling on teaching programming in the early years since it brings positive outcomes in programming education of children (Koorsse et al., 2015; Sáez-López et al., 2016).

In the studied conducted by Yildiz Durak (2018c), Chang (2014) and Portelance (2015), the positive contributions of block-based programming were listed as following: visual properties, no error during code writing and easy detection of errors, preventing the syntax complexity of programming language, easy to debug, designing projects through multimedia items, reducing cognitive load, easy-to-use interface, diminishing the difficulty experienced in programming tasks, changing the negative attitude towards programming, convenience for the readiness of the target group as well as simplifying the process for novice users. A print screen is presented in Figure 1.

Figure 1.

Screenshot of Scratch 2 offline editor (online editor: http://scratch.mit.edu/).

Scratch project was launched by Lifelong Kindergarten found in Massachusetts Institute of Technology (MIT) in 2003. Scratch website was allowed to ordinary users in 2007. Scratch intends to back up the programming education of children between 8 and 16 years. Studies, on the other hand, suggest that Scratch can also be employed in different age groups (Kafai and Burke, 2014). It aims to design an active online community who can design, share, discuss and combine the projects (Resnick et al., 2009). The convenience of finding explanations about the project in online Scratch editor and being able to add tags make the following similar tasks easier for others.

Gifted students in programming education

Upon reading the existing literature, a key problem with much of the literature on gifted students is that few studies have dwelled on gifted students’ experiences (Henfield et al., 2008). Few researchers have addressed the issue of employing programming education activities in the education of gifted students. As noted by Henfield et al. (2008), gifted students represent a huge potential that can afford the need of qualified work force when educated well.

Gifted students are considered as those who have potential to display high level skills (National Association for Gifted Children (NAGC), 2014). As Gagne (2004) reminds us, utmost attention must be paid to personal needs of the individuals who display high-level performance or become successful in some tasks, several of which are general cognitive talent, special academic talent, creativity, leadership, psychomotor skills, visual/spatial talent (Davis et al., 2013; National Society for the Gifted & Talented (NSGT), 2014) and their potentials must be tried to be developed. With respect to this, special educational programmes designed for gifted students must be implemented (Gagne, 2004). It is, therefore, important to provide gifted students with high-level programming tasks. Additionally, the educational activities must have the necessary quality in terms of competencies and needs of gifted students (Callahan and Hertberg-Davis, 2012; NAGC, 2014; Department of Education and Training (NSW), 2004; Renzulli and Reis, 1997). Within this context, interdisciplinary point of view is needed in the programming education in gifted students.

Evaluating the effectiveness of programming education

A successful programming project process encompasses system-based thinking, critical thinking, problem-solving, aesthetic, writing and storytelling, multimedia design and programming skills (Salen, 2007). When the existing literature examined, however, it was discovered that the outcomes are given more attention instead of figuring out how children learn programming tools. At this precise point, programming processes must be examined to support learning despite the popularity of programming activities at K-12 level and potential of developing high-level skills (Hayes and Games, 2008).

Different means have been used to examine programming processes in the existing literature. One of these is the implementation of game-based learning activities in programming process. Within this context, Koorsse et al. (2015) developed ‘Programming Assistance Tools’ instrument, stating that we need a mechanism in which we can observe the process, determine performance criteria and provide instant feedback to develop programming competence and skills

Denner et al. (2012) noted that the activities conducted and the outcomes produced during programming education are directed to developing and reflecting computational thinking skills. For this reason, Denner et al. (2012) examined Scratch programming projects of learners, developing project evaluation criteria based on PC.

In programming education, on the other hand, active involvement of students in the process and achieving sustainability in activities are vital (Yildiz Durak, 2018b). Otherwise, it is inevitable for students to experience some difficulties in programming session which is a complex one. Peer feedback is of great importance to deal with these difficulties. Therefore, performance criteria must be accompanied by peer feedback in the evaluation of programming session.

According to Akpinar and Altun (2014), students are supposed to find a solution for a problem while performing programming assignments. They must be able to express this solution through computing systems. The so-called outcome may contribute to development of language of students. However, the activities in which students divide problem into sub-problems and design a generalizable solution can contribute to analysis skills of students. To examine the effectiveness of programming process, it is fundamental to examine the outcomes achieved during and after the programming process. Programming involves a skill developmental process.

Methods

Relational screening model was used in the study. This model deals with an existing situation as it exists without attempting to change and influence and is a research approach that tries to determine the degree and direction of changes between variables in the study (Büyüköztürk, 2009; Fraenkel and Wallen, 2006).

Study group and its properties

In the study, the study group consists of 26 students studying second–third–fourth grade of a primary school in Altındağ, Ankara. These students were given sufficient score (130) in the tests aimed at diagnosing the gifted students by Altındağ Counseling and Research Center and were decided to receive support education for the gifted ones by Ministry of National Education (MoNE).

Of the participants, 53.8% are female and 46.2% are male. Distribution of the participants in terms of grade levels is as follows: 38.5% are second grader, 46.2% are third grader and 15.4% are fourth grader. About half (42.3%) of the study group is 9 years old; 57.7% of the participants have two siblings. The highest monthly income levels are ‘3001–5000 Turkish lira (TL)’ and ‘5001–7000 TL’ with rates of 34.6%.

The entire study group has computer at home. When Internet access status is considered, it is seen that 53.8% of the participants do not have Internet access at home. When the frequency of daily Internet usage of the participants was examined, it was determined that 57.7% of the participants do not use Internet. On the other hand, it was found that the participants have access to computers and the Internet (53.9%) mostly at school and that they have learned to use computer and the Internet (26.9%) at school; 61.5% of respondents answered yes to the question of whether a computer course was taken before.

Data collection tools

In this study, four different data collection tools were used. In the second term of the 2014–2015 academic year, these tools were applied face-to-face in class for 15 weeks. The information on data collection tools and sample items is presented in Table 1.

Table 1.

Information on data collection tools and sample items.

Questions	Reference	Samples from contents of data collection instruments
Personal information, computer and Internet usage status survey (QPS1–QPS16)	Developed by researcher	1-Gender a. Male b. Female 2-Grade:…. 3-Age………
Observation form (QO1–QO14)	Developed by researcher	5. Students suggest different ways of problem solving. Not applicable 0 1 2 3 4 5 11. Student can solve the problems encountered while writing a program Not applicable 0 1 2 3 4 5 13. Students ask their peers unless they can solve the problem encountered while writing a program Not applicable 0 1 2 3 4 5
Project evaluation criteria based on programming concepts (QP1–QP22)	Denner et al. (2012)	Criteria Use of programming concepts in scratch 1. Algorithmic order Have the coding blocks been sequenced according to an algorithmic order in order to run the designed programme properly? Coding 0/1 2. Repeating/reiterating Have the coding blocks been used as “continuously” or “…. times” to design reiterating structure? Coding 0–3 3. Variables Have the variables been designed in such a way that they can be formed in Scratch and can be used in the programme later? Coding 0-3 4. Conditioning structures “Have conditioning structures such as “If….”, “If…. continuously” and “If……., If not…….” been used to check the actions? Coding 0–3 Code designing/organization 14. External blocks Are there any code blocks which cannot be launched when the programme starts? Coding 0/1 15. Character names (changing the default value) Have the default character names been changed according to the project? Coding 0/1 16. Variable names Have the meaningful names been given to variables formed? Coding 0/1 Usability of the design 17. functionality Is it launched when the project is started (the green flag is clicked)? Coding 0–3 18. Purpose Is there any well-defined target in the project? Coding 0/2 19. Privatization of character Is there any character defined previously, or is it privatized for the character? If so, what level has this privatization been realized? Coding 0–3 20. Privatization of steps Is there any stage defined previously, or is it privatized for the purpose? If so, what level has this privatization been realized? Coding 0–3
Student peer evaluation form	Developed by researcher

Personal information, computer and Internet usage status survey

A questionnaire consisting of seven questions to collect demographic information such as gender, age, grade, education status of mother and father, income status and number of siblings and nine questions to determine computer and Internet usage status was used.

Observation form

In the context of 14 questions determined through the observation form, the behaviours and reactions of the learners in the learning environment were graded. The rating was done in the range of ‘not valid, 0, 1, 2, 3’. Three field experts were consulted for the reliability of the prepared observation form.

Project evaluation criteria based on PC

Denner et al. (2012) used the criteria that were formed on the basis of PC to examine the degree to which PC were reflected in students’ projects. These criteria are structured in three main categories (PC, CO and design for usability) and 22 subcategories. Scores of existence, absence of the situation (0/1) or scores between 0–2 and 0–3 according to the expression specified in each project subcategories were requested. Twenty-six projects developed by primary school students were examined and graded according to the criteria by the researcher and a field expert.

Student peer evaluation form

During the semester, learners evaluated the learning processes and projects of their friends and gave oneself and their friends symbol scores as follows. Symbol scoring was conducted for achieving the convenience for the age levels of the students, by consulting experts. Scoring can be listed as ‘⋆ One of my favourites (3 points), ♥ Love it (2 points), • Not sure (1 points), × Don’t love it (0 points)’.

Practice

The practice period includes 23 February 2015 to 1 June 2015, spring term of 2014–2015 education year. In addition to presenting the basic concepts of programming, Scratch programme used as a material was shown and the online platform and scratch.mit.edu platform were introduced.

The practice was 2 hours in a week. The steps of the practice are displayed in Figure 2.

Figure 2.

Research period.

Students’ projects were shared in MIT Scratch home page (http://scratch.mit.edu) following the practice. What is more, students were asked to evaluate other projects separately. Later on, the projects were examined by experts one by one.

Coding and analysis of data

Descriptive statistics (percentage, mean, median, standard deviation) and logistic regression analysis were used to analyse the data. In these analyses, the level of significance was defined as 0.05. SPSS 18.0 software was used in the analyses.

While calculating the student scores, the 14-item observation list for each student was scored by the researcher for 15 weeks. At the end of the semester, the learners presented their projects and after the presentation, each learner rated their project and their friends’ projects from 0 to 3. In addition, two field experts encoded learning projects with ‘Project Evaluation Criteria Based on Programming Concepts’. A result score was calculated from the scores obtained after all these procedures.

As seen in Figure 3, the maximum (max) and minimum (min) scores for each test and the total score (p) of each student were calculated to obtain a result score from the scores of three different ways. The result score was determined by the following formula:

Result = 1 - [(max - p)/(max - min)]

Figure 3.

Assessment and evaluation procedures.

Break points were determined in the grouping of the dependent variables (high, moderate and low) that did not show normal distribution. The results obtained from the scores taken during the programming teaching process are divided into groups as below 0.333 as 0; between 0.333 and 0.666 as 1 and above 0.666 as 2.

The logistic regression analysis (Çokluk et al., 2010) was used in cases where the dependent variables did not show normal distribution and dependent variable was considered as a categorical variable with three levels The independent variables (x_ki ) were considered to be effective in measuring the comprehension of the concepts of programming and discussed in the study:

x1: Gender

x2: Age

x3: Grade level

x4: Internet access status

x5: Duration of computer use

x6: Duration of Internet use

x7: Daily duration of Internet use

x8: Taking a computer course before

‘Result’ (formed with PC, DU, CO, peer evaluation, observation scores) dependent variable, yi, was coded as

‘0-Low’,

‘1-Moderate’,

‘2-High’

Role of the researchers

The researcher conducting this study teaches in some courses as an ICT teacher in the primary school where the practice was implemented. The basic roles of the researcher were guiding the practice, observing and collecting the data. The data were obtained through observations and field notes. The researcher took a role as a participant observer to depict the teaching process, including software development process of gifted students in primary school grades 2, 3 and 4 (analyse, design, coding and test), via Scratch in detail.

Findings

The findings are presented in the order in which the research questions are given and in response to these questions.

What are the scores that students get during the programming education process?

The first sub-problem of the study was determined as ‘What are the scores that students get during the programming education process?’ Scores related to this sub-problem is presented in Table 2.

Table 2.

The scores that students get during the programming education.

Properties	Options	Scores (mean)
Properties	Options	Criteria-PC	Criteria-CO	Criteria-DU	Peer evaluation	Observation score
Gender	Female	17.21	2.79	13.36	53.79	13.12
Gender	Male	15.92	2.75	13.33	52.25	12.30
Age (years)	7	14.25	2.65	12.00	52.50	9.83
	8	15.38	2.58	12.25	49.75	11.81
	9	18.00	2.64	14.27	55.00	14.32
	10	16.62	3.00	13.35	55.67	13.31
Grade	2nd	13.80	2.80	11.80	49.70	10.55
	3rd	19.00	2.75	14.42	54.75	14.62
	4th	16.50	2.75	14.00	56.50	12.60
Internet access status	Yes	17.75	2.79	13.75	54.67	14.18
Internet access status	No	15.64	2.75	13.00	51.71	11.51
Duration of computer use (years)	I’ve never used	13.29	2.71	11.43	49.43	10.15
	1	17.11	2.89	13.56	54.56	12.73
	2	21.00	2.67	14.67	51.67	14.56
	3	15.00	3.00	13.50	50.50	13.33
	4 and above	18.40	2.60	14.80	57.40	15.11
Duration of Internet use (years)	I’ve never used	13.13	2.75	11.38	49.75	10.31
	1	18.11	2.89	14.22	54.33	13.22
	2	21.00	2.50	13.50	52.50	13.83
	3	15.00	3.00	13.50	50.50	13.33
	4 and above	18.40	2.60	14.80	57.40	15.11
Daily duration of Internet use (hours)	I do not use	15.60	2.80	12.60	51.67	11.48
	1–3	18.33	2.67	14.33	55.00	14.98
	4–6	16.83	2.67	13.50	53.50	13.54
	7–9	21.00	3.00	17.00	59.50	16.43
Taking a computer course before	Yes	18.38	2.75	14.44	55.13	13.73
Taking a computer course before	No	13.80	2.80	11.60	49.80	11.15
	Total	16.62	2.77	13.35	53.08	12.74

PC: programming concepts, DU: designing for usability, CO: code organization.

In Table 2, the scores taken by the female students during the programming education process were higher than the males in all three sub-dimensions of the criteria determined based on peer evaluation, observation score and PC. When all the scores taken are evaluated in terms of age variables, it is seen that students in 9 and 10 age group get higher scores than others. A similar situation is observed in terms of the grade variable, and the scores of the second-grade students in the programming process are lower than the students in other grades.

Students who had no access to the Internet at home had lower scores than those with Internet access. It is seen that the scores of students who have never used computer and Internet before and who did not take computer course before are lower than those of other students. It can be said that the previous experience of students on information technologies is effective in the scores that they get in the programming education process.

Total scores according to the observation items are presented in Figure 4.

Figure 4.

Total scores according to observation items. Note: O1. Student can grasp the activity at first sight. O2. Students reaction time in grasping the activity changes depending on grade level. O3. The time given to students for the activities is sufficient. O4. Students have difficulty in grasping the problems. O5. Students suggest different ways of problem solving. O6. Students can select the most suitable means presented for the solving of the problems. O7. Students can make a plan for solving a problem. O8. Students themselves can write programmes for the problems they define. O9. Student can evaluate the outcome of the programme in terms of error/accuracy. O10. Student can check the accuracy of programme outcome. O11. Student can easily solve the problems encountered while writing programme. O12. Students ask their teachers unless they can solve the problem encountered while writing a program. O13. Students ask their peers unless they can solve the problem encountered while writing a program. O14. Students ask their users in the online platform unless they can solve the problem encountered while writing a program.

According to Figure 4, the item ‘010. Student can check the accuracy of programme outcome” has the highest mean score (M = 15.88). This finding shows that students can check their mistakes simultaneously in the classroom. The observation items with the lowest score are ‘O1. Student can grasp the activity at first sight’ and O2 ‘Students’ reaction time in grasping the activity changes depending on grade level’ (M = 4.12). These items show that students have difficulty in programming activities despite observed rarely. Additionally, it was seen that grasping time may sometimes change depending on grade level.

What is the predictive power of demographic variables on the scores of students in the programming education process (observation, peer evaluation, PC, DU, CO)?

The second sub-problem of the study was defined as ‘What is the predictive power of demographic variables on the scores of students in the programming education process (observation, peer evaluation, PC, DU, CO)?’ Logistic regression analysis was performed to find the answer to this sub-problem. The results of the analysis are presented in Tables 3 to 6.

Table 3.

Initial model.

Step		−2LL	Coefficient
Step		−2LL	Constant
Step 0	1	36.044	.000

2LL: 2Log Likelihood.

Table 3 presents a history of iterations for Block 0 or the initial model, which is called the initial block.

As seen in Table 3, the value of −2Log likelihood (−2LL) starts with 36.044. This value is found to be high if the −2LL value corresponding to perfect fit is considered to be zero. In Table 4, Omnibus test result of logistic regression model coefficients is presented.

Table 4.

Omnibus test for model coefficients.

Step		χ ²	SD	p
1	Step	22.139	8	.005
	Block	22.139	8	.005
	Model	22.139	8	.005

SD: standard deviation.

Table 4 shows that p ≤ .05 value related to model χ ² value is significant and there is a relationship between the combinations of predicted variable and the predictive variable, and this relationship between the predicted and the predictive variable is supported.

When the predictive variables were analysed according to Table 5, the result of the Hosmer and Lemeshow test that evaluated the compliance of logistic regression model was not significant (p > 0.05). The fact that this value is not significant shows that the model has acceptable compliance and that the model data compliance is adequate.

Table 5.

Hosmer and Lemeshow test.

Step	χ ²	SD	p
1	3.706	7	.813

SD: standard deviation.

When the classification obtained as a result of logistic regression model is examined, it is seen that 50% (13) of the students are moderate level and the others are high level according to the grouping of the ‘result’ scores formed for programming education. There are no students grouped as low level.

In the model, 11 of 13 students with moderate scores were calculated incorrectly, and the correct classification rate was 84.6%. Of the 13 students with high scores in the programming education process, 10 were correctly classified and the correct classification rate was 76.9%. The correct classification rate for the intended model is 80.8%.

According to the data in Table 6, the model is not constant significant (p > 0.05). In this case, it cannot be said that any variable other than the predictive variables taken in the model explains the scores in programming to be high, medium or low. Variables of gender, grade, Internet access status, duration of computer use, duration of Internet use and daily duration of Internet use were not significant in the model (p > 0.05). According to the model, variables of age and taking a computer course before were significant predictors (p < 0.05). For the age variable, the Exp (ß) value, which is the odds ratio of the predictor variable, is 9.291. Accordingly, older students (10 years) have nine times higher probability of getting higher scores in the programming education process than the younger age group (7 years). The Exp (ß) value, which is the odds ratio of the predictor variable, of variable of taking a computer course before is 4.129. Accordingly, the students who took a computer course before are four times more likely to get high scores in the programming education process than the students who did not take a course before. As a result, it can be argued that if the participant is in the largest age group and has taken a computer course before, the scores of this participant from programming education process will increase.

Table 6.

Logistic regression results related to demographic characteristics of students on predicting scores taken during programming process.

Variables	β	Standard error	Wald	SD	p	Exp (ß)
Gender	−.222	.740	.000	1	.997	.000
Grade	.129	1.818	1.005	1	.243	.138
Age	−1.235	2.409	98.263	1	.000	9.291
Internet access status	−.134	1.967	2.005	1	.106	.875
Duration of computer use	−1.243	.373	.000	1	.999	.000
Duration of Internet use	1.518	1.373	.000	1	.999	.102
Daily duration of Internet use	2.394	.680	.000	1	.998	.194
Taking a computer course before	−3.851	.642	40.046	1	.000	4.129
Constant coefficient	1.531	.380	.000	1	.998	7.249

SD: standard deviation.

Conclusions and discussion

In this study, the degree to which students studying second–third–fourth grade in a primary school are able to learn the concepts of programming, the process of teaching the concepts of programming and eight variables (gender, grade, age, Internet access status, duration of computer use, duration of Internet use, daily duration of Internet use and taking a computer course before) that are thought to be predictive in this process was examined.

In the programming education process, females had higher scores than males. In the programming education process, females had higher scores than males. In the existing literature, it has been continuously emphasized that programming self-efficacy, programming achievement level and programming competencies are higher in males rather than females (Byrne and Lyons, 2001; Crews and Butterfield, 2003; Nourbakhsh et al., 2004; Román-González et al., 2017; Werner et al., 2012). Additionally, attitude towards programming, self-efficacy, motivation and different anxiety levels are regarded as the reasons of this situation in the studies which dwell on gender differences (Cegielski and Hall, 2006; Wiedenbeck, 2005). Contrary to popular belief, the finding which conflict with the current literature can be examined in detail. Future studies on the current topic are therefore recommended to elucidate the programming competencies of average and gifted students.

Scores obtained programming education process are higher in students aged 9 and 10 than others. Similarly, the scores of second-grade students in the programming process were lower than the students in other classes. Since education level and age differentiate most cognitive skill depending on the change in ICT use status, it is believed that this difference will also affect programming performance and problem-solving skills directly or indirectly (Askar and Davenport, 2009; Yildiz Durak and Saritepeci, 2018). Depending on this, education level and age are expected to be an effective variable on performance in programming process which includes multi-skills. The reasons of this can be examined through qualitative studies in detail.

It is seen that the students who do not have Internet access at home, who have never used computer and Internet before and who did not take computer course before had less scores than other students. It has been concluded that previous experience of students on information technologies has an effect on the scores they received during the programming education process. Baytak and Land (2011) found that the method (project-based teaching method) has an effect on the learning of PC in their study on programming education with a similar study group.

There have been several studies suggesting that the performances of the individuals in computer-based environments are related to programming skills (Boechler et al., 2014). Several variables including ICT use frequency, ICT use experience, attitude towards ICT use, academic performance, problem-solving skills have been found in relation with programming skills (Werner et al., 2012). However, challenging tasks in programming education are convenient for developing thinking skills of gifted students (Durak, 2016). Kirk et al. (2000) note that gifted students can form more complex and structured knowledge frameworks based on their learnings when compared to average students. Kubilius and Lee (2004), on the other hand, stress that gifted ones can learn the basic computer concepts by themselves. However, it is of great importance to revise the learning process of programming education and to give scaffolding.

Therefore, it can be said that the common point of prior experience in information technologies and active learning–based methods, which are deemed necessary for the effectiveness of the education process, reveals the importance of active applied learning of the learner. Additionally, the programming learning process of gifted students by means of individualistic assignments must be backed up. Gifted students must be provided with Scaffolding when necessary.

As a result of the logistic regression model, it is seen that there is no low-level students in the classification of the scores related to programming education, half was in the moderate level and the rest was in high level. From this point of view, it can be concluded that the process is effective in terms of instructional and students’ learning the programming processes. In some studies conducted in the literature on this subject, it was stated that it is possible to make comments about whether the learners have learned basic PC by examining simple learner projects and that these comments support the results of the applied success tests (Denner et al., 2012; Maloney et al., 2008). It is also emphasized that the application with multiple data sources, which also examines the process for classroom observations, is a requirement to reflect the effectiveness of the process (Wilson et al., 2011).

According to the results of the analysis, it is concluded that the model data compliance is sufficient. Accordingly, it can be stated that the model can be highly effective in predicting the level of learning in programming education process. Variables of ‘age’ and ‘taking a computer course before’ significantly contribute to low, moderate and high scores of students in terms of all measurement tools.

Footnotes

Acknowledgements

This study is derived from the PhD thesis titled “Design and development of an instructional program for teaching programming process to gifted students”, conducted by Hatice Durak in Gazi University, Institute of Educational Sciences, Department of Computer Education and Instructional Technology, under the supervision of Professor Tolga Güyer. This study is the expanded version of the oral paper presented at the ITTES-2017.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

References

Akpınar

Altun

(2014) The need for programming education in information society schools [Bilgi toplumu okullarında programlama eğitimi gereksinimi]. Elementary Education Online 13(1): 1–4.

Askar

Davenport

(2009) An investigation of factors related to self-efficacy for Java programming among engineering students. TOJET: The Turkish Online Journal of Educational Technology 8(1). Available at: http://files.eric.ed.gov/fulltext/ED503900.pdf (accessed 10 November 2018).

Baytak

Land

(2011) An investigation of the artifacts and process of constructing computers games about environmental science in a fifth grade classroom. Educational Technology Research and Development 59(6): 765–782.

Boechler

Dragon

Wasniewski

(2014) Digital literacy concepts and definitions: Implications for educational assessment and practice. International Journal of Digital Literacy and Digital Competence (IJDLDC) 5(4): 1–18.

Büyüköztürk

(2009) Data Analysis Manual. Ankara: Pegem Academy Publications.

Byrne

Lyons

(2001) The effect of student attributes on success in programming. ACM SIGCSE Bulletin 33(3): 49–52.

Callahan

Hertberg-Davis

(Eds.) (2012) Fundamentals of Gifted Education: Considering Multiple Perspectives. Abingdon: Routledge.

Cegielski

Hall

(2006) What makes a good programmer? Communications of the ACM 49(10): 73–75.

Chang

(2014) Effects of using Alice and Scratch in an introductory programming course for corrective instruction. Journal of Educational Computing Research 51(2): 185–204.

10.

Chen

Shen

Barth-Cohen

, et al. (2017) Assessing elementary students’ computational thinking in everyday reasoning and robotics programming. Computers & Education 109: 162–175.

11.

Chu

Hwang

(2010) Development of a project-based cooperative learning environment for computer programming courses. International Journal of Innovation and Learning 8(3): 256–266.

12.

Clements

Nastasi

(1999) Metacognition, learning, and educational computer environments. Information technology in childhood education annual 1999(1): 3–36.

13.

Çokluk

Şekercioğlu

ve Büyüköztürk

(2010) Multivariate Statistics for Social Sciences: SPSS and Lisrel Applications. Ankara: Pegem Academy Publications.

14.

Crews

Butterfield

(2003) Gender differences in beginning programming: An empirical study on improving performance parity. Campus-Wide Information Systems 20(5): 186–192.

15.

CSTA (2010) Running on empty: The failure to teach K–12 computer science in the digital age. Available at: http://runningonempty.acm.org/fullreport2.pdf (accessed 10 November 2016).

16.

Davis

Rimm

Siegle

(2013) Education of the Gifted and Talented. Pearson: New International Edition.

17.

Denner

Werner

Ortiz

(2012) Computer games created by middle school girls: Can they be used to measure understanding of computer science concepts? Computers & Education 58(1): 240–249.

18.

DiSessa

(2001) Changing Minds: Computers, Learning, and Literacy. Cambridge: MIT Press.

19.

Durak

(2016) Design and development of an instructional program for teaching programming process to gifted students. Unpublished Doctoral Dissertation, Gazi University, Ankara.

20.

Feldhusen

(1997) Secondary services, opportunities, and activities for talented youth. In: Colangelo

Davis

(eds) Handbook of Gifted Education, 2nd ed. Boston: Allyn & Bacon, pp. 189–197.

21.

Fessakis

Gouli

Mavroudi

(2013) Problem solving by 5–6 years old kindergarten children in a computer programming environment: a case study. Computers & Education 63: 87–97.

22.

Fraenkel

Wallen

(2006) How to Design and Evaluate Research in Education. 6th ed. New York: McGraw-Hill Book Company.

23.

Frost

Verno

Burkhart

, et al. (2009) A model curriculum for K–12 computer science level I objectives and outlines. Computer Science Teachers Association. Available at: https://cis.tchs.info/L1-Objectives.pdf (accessed 10 November 2018).

24.

Gagne

(2004) Transforming gifts into talents: the DMGT as a developmental theory 1. High Ability Studies 15(2): 119–147.

25.

Grover

Pea

(2013) Computational thinking in K–12: a review of the state of the field. Educational Researcher 42(1): 38–43.

26.

Hayes

Games

(2008) Making computer games and design thinking: a review of current software and strategies. Games and Culture 3(3-4): 309–332.

27.

Henfield

Moore

III JL

Wood

(2008) Inside and outside gifted education programming: hidden challenges for African American students. Exceptional Children 74(4): 433–450.

28.

Hwang

Wang

Hwang

, et al. (2008) A web-based programming learning environment to support cognitive development. Interacting with Computers 20(6): 524–534.

29.

Kafai

Burke

(2014) Connected Code: Why Children Need to Learn Programming. Cambridge: MIT Press.

30.

Kalelioğlu

Gülbahar

(2014) The effects of teaching programming via scratch on problem solving skills: a discussion from learners’ perspective. Informatics in Education 13(1): 33–50.

31.

Kirk

Gallagher

Anastasiow

(2000) Educating Exceptional Children. 9th ed. Boston: Houghton-Mifflin.

32.

Koorsse

Cilliers

Calitz

(2015) Programming assistance tools to support the learning of IT programming in South African secondary schools. Computers & Education 82: 162–178.

33.

Kubilius

Lee

(2004) Parent perceptions of the effects of the Saturday Enrichment Program on gifted students’ talent development. Roeper Review 26(3): 156–165.

34.

Lau

Yuen

(2011) Modelling programming performance: beyond the influence of learner characteristics. Computers & Education 57(1): 1202–1213.

35.

Lee

(2011) Scratch: multimedia programming environment for young gifted learners. Gifted Child Today 34(2): 26–31.

36.

Liao

YKC

Bright

(1991) Effects of computer programming on cognitive outcomes: a meta-analysis. Journal of Educational Computing Research 7(3): 251–266.

37.

Maloney

Peppler

Kafai

, et al. (2008) Programming by choice: Urban youth learning programming with scratch. In Proceedings of SIGCSE '08. New York, NY: ACM Press, pp. 367–371.

38.

Microsoft (2014) Hour of code. Available at: http://www.microsoft.com/about/corporatecitizenship/en-us/youthspark/youthsparkhub/hourofcode/ (accessed 10 November 2016).

39.

National Association for Gifted Children-NAGC (2014) Computing in the national curriculum- A guide for primary teachers 2014. Available at: http://www.computingatschool.org.uk/data/uploads/CASPrimaryComputing.pdf (accessed 10 November 2016).

40.

National Society for the Gifted & Talented-NSGT (2014) Giftedness Defined. Available at: http://www.nsgt.org/giftedness-defined/#4 (accessed 10 November 2016).

41.

New South Wales (NSW)-Department of Education and Training (2004) Policy and implementation strategies for the education of gifted and talented students. Available at: http://www.curriculumsupport.education.nsw.gov.au/policies/gats/assets/pdf/polimp.pdf (accessed 10 November 2016).

42.

Nourbakhsh

Hamner

Crowley

, et al. (2004, April) Formal measures of learning in a secondary school mobile robotics course. In International Conference on Robotics and Automation, New Orleans, LA, pp. 1831–1836. IEEE.

43.

Partnership for 21st Century Skills (P21) (2007) Partnership for 21st Century Skills. Available at: http://www.p21.org/about-us/p21-framework/60 (accessed 10 November 2016).

44.

Poftak

(1998) Technology and gifted education: a talk with Carol Wilson. Technology & Learning 19(4): 14–14.

45.

Portelance

(2015) Code and tell: an exploration of peer interviews and computational thinking with ScratchJr in the early childhood classroom. Doctoral dissertation, Tufts University, Massachusetts.

46.

Renzulli

Reis

(1997) The Schoolwide Enrichment Model: A How-to Guide for Educational Excellence. Mansfield, CT: Creative Learning Press.

47.

Resnick

Maloney

Monroy-Hernández

, et al. (2009) Scratch: programming for all. Communications of the ACM 52(11): 60–67.

48.

Román-González

Pérez-González

Jiménez-Fernández

(2017) Which cognitive abilities underlie computational thinking? Criterion validity of the Computational Thinking Test. Computers in Human Behavior 72: 678–691.

49.

Sáez-López

Román-González

Vázquez-Cano

(2016) Visual programming languages integrated across the curriculum in elementary school: a two year case study using “Scratch” in five schools. Computers & Education 97: 129–141.

50.

Salen

(2007) Gaming literacies: a game design study in action. Journal of Educational Multimedia and Hypermedia 16(3): 301–322.

51.

Shin

Park

(2014) A Study on the effect affecting problem solving Ability of primary students through the Scratch programming. Available at: http://onlinepresent.org/proceedings/vol59_2014/27.pdf (accessed 10 November 2016).

52.

Siegle

(2009) Developing student programming and problem-solving skills with Visual Basic. Gifted Child Today 32(4): 24–29.

53.

Tomlinson

Reis

(Eds.) (2004) Differentiation for Gifted and Talented Students. Thousand Oaks: Corwin Press.

54.

VanTassel-Baska

(1988) Curriculum design issues in developing a curriculum for the gifted. In: VanTassel-Baska

Feldhusen

Seeley

Wheatly

Silverman

Foster

(eds) Comprehensive Curriculum for Gifted Learners. Boston: Allyn & Bacon, pp. 53–76.

55.

Wang

Huang

Hwang

(2016) Comparison of the effects of project-based computer programming activities between mathematics-gifted students and average students. Journal of Computers in Education 3(1): 33–45.

56.

Werner

Denner

Campe

, et al. (2012, February) The fairy performance assessment: measuring computational thinking in middle school. In Proceedings of the 43rd ACM technical symposium on computer Science Education (SIGCSE '12), pp. 215–220. New York, NY: ACM.

57.

Wiedenbeck

(2005, October) Factors affecting the success of non-majors in learning to program. In: Proceedings of the first international workshop on computing education research, pp. 13–24. New York, NY: ACM.

58.

Wilson

Connolly

Hainey

, et al. (2011) Evaluation of introducing programming to younger school children using a computer game construction application. In: Proceedings of 5th European conference on games-based learning (ECGBL), Athens, Greece, 20–21 October 2011.

59.

Yang

Hwang

Yang

, et al. (2015) A two-tier test-based approach to improving students’ computer-programming skills in a web-based learning environment. Journal of Educational Technology & Society 18(1): 198–210.

60.

Yildiz Durak

(2018a) Digital story design activities used for teaching programming effect on learning of programming concepts, programming self-efficacy, and participation and analysis of student experiences. Journal of Computer Assisted Learning 34(6): 740–752.

61.

Yildiz Durak

(2018b) Flipped learning readiness in teaching programming in middle schools: modelling its relation to various variables. Journal of Computer Assisted Learning 34(6): 939–959.

62.

Yildiz Durak

(2018c) The effects of using different tools in programming teaching of secondary school students on engagement, computational thinking and reflective thinking skills for problem solving. Technology, Knowledge and Learning 1–17. DOI: 10.1007/s10758-018-9391-y.

63.

Yildiz Durak

Güyer

(2018) Design and development of an instructional program for teaching programming processes to gifted students using scratch. In: Cannaday

(Ed.) Curriculum development for gifted education programs. Hershey: IGI Global, pp. 61–99.

64.

Yildiz Durak

Güyer

(2019) An investigation of the opinions of gifted primary school students’ in the programming training processes. Ankara University Journal of Faculty of Educational Sciences 52(1): 107–137. DOI: 10.30964/auebfd.466922.

65.

Yildiz Durak

Karaoglan Yılmaz

Yılmaz

(2019) Computational Thinking, Programming Self-Efficacy, Problem Solving and Experiences in the Programming Process Conducted with Robotic Activities. Contemporary Educational Technology 10(2), 173–197.

66.

Yildiz-Durak

Saritepeci

(2018) Analysis of the relation between computational thinking skills and various variables with the structural equation model. Computers & Education 116: 191–202.