Examining the Learning Progression of Undergraduate Students’ Scientific Imagination: A Measurement Perspective

Imagination and imagery

Imagery is the mental ability to imagine a scene, image, or item beyond existing reality (Cheng, 1993). Imagery is perceived as a basic form of cognition and has been identified as playing a crucial role in many mental activities, such as direction recognition, memory, and problem solving (Han et al., 2010; Riquelme, 2002). Imagery possesses three characteristics: (1) the ability to generate imagery is a human universal; it can help with an incomplete formation of an item or scene in reality; (2) it can be manipulated freely and be fictive; for instance, one can picture an elephant as small as a mouse; and (3) it can be merged, as when one thinks of an image of a bus with a facial expression and wearing glasses. Thus, imagery can be not only fictive but also a combination of different ideas. Thoughts are processes that execute or transmit messages mentally or cognitively. To sum up, imagery is a mental process in which individuals think using imagery: the phenomenon of cognition through visual processing (Cheng, 1993).

Cognitive neuroscience studies (e.g., Farah et al., 1988; Kosslyn, 1994; Kosslyn & Koenig, 1992; Levine et al., 1985) suggest that imagery incorporates two subsystems that illustrate the brain’s different ways of encoding and processing visual messages: object imagery and spatial imagery. Object imagery denotes individual’s representations of items in their minds considering their actual form, size, shape, color, and brightness, while spatial imagery involves individual’s cognition of the spatial relations between parts of an object, the spatial location and movement of the object, and other complex spatial representations of transitions (cited from Blajenkova et al., 2006).

Imagination plays a vital role in the process of creation, from identifying the problem, through thinking of solutions, evaluating, collecting data, and editing (image/media/meaning), to presenting the final product. Scientific imagination is a mental activity that involves generating new imagery by connecting life experience with scientific principles (Ho et al., 2013) and includes the behavior of forming, or being able to form, mental imagery (Chen, 2013; Efland, 2004). Riquelme’s (2002) study, which focused on creative thinking and the identification of ambiguous images, reported an experiment with 47 managers who were attending an MBA course. The study found that people who engaged in creative imagery were faster at finding changes in ambiguous images and at explaining the relationships among images. Past research has also found that different types of imagery address different aspects of creativity; for instance, visual imagery is related to artistic creativity, and spatial imagery to scientific creativity (Kozhevnikov et al., 2013). Furthermore, the available empirical evidence now supports the view that mental imagery can occur in any sensory modality, though degrees of vividness may widely vary (Spence & Deroy, 2013). In other words, imagery is closely related to imagination.

Imagination and age (grade level)

Imagination begins in infancy and continues to develop until adulthood; individuals’ childhood experiences with imagination often influence their imagination as adults (Gündoğan et al., 2013; Root-Bernstein & Root- Bernstein, 2006). The functioning of imagination is very complex and can be influenced by a series of various factors. For example, life experience serves as a basis for imagination. The world that children experience is different from that of adults. In addition, the relationship between children and their environment, be it complex, simple, or traditional, will stimulate their creative processes, which are very different from those of adults. Furthermore, children and adults have different interests in objects and events. Hence, due to differing life experiences and development of cognitive tools, the imagination of an adult is completely different from that of a child (Eckhoff & Urbach, 2008; Vygotsky, 1930/2004).

Ren et al.’s (2012) study explored the development of students’ imagination from grade 4 to grade 12. They found that imagination increased with the students’ progression in school (from grade 4 to grade 11) but decreased in grade 12. Mun et al. (2015) investigated the scientific imagination performance of Korean students in grades 4 to 8 and found that elementary students performed better than junior high school students. They found that from grades 4 to 6, students’ scientific imagination did not differ; in grades 6 and 7, their scientific imagination started to decrease; by grades 7 and 8, their performance had declined significantly. In other words, the developmental trajectory of scientific imagination varies at different levels of education. Generally speaking, grades 4–6 and grades 7–9 are periods of rapid development. When children enter puberty, their dramatic physical and mental development lays foundations for their growth of imagination. Furthermore, at different education levels, students tend to express their imagination through different media: younger students tend to express their thoughts through drawing or illustrations, whereas older students are more likely to use language for this purpose. Wang et al. (2017b) suggested that age (grade level) might not be an important factor determining scientific imagination among elementary school students, and advised that in the future, more samples be collected from high schools, and universities, along with qualitative data (e.g., thinking aloud, observation, and interviews) to verify a more comprehensive LP for scientific imagination at different ages (grade levels).

On the other hand, given the fact that divergent thinking (DT) is the foundation of creativity and imagination, Shah and Gustafsson (2021) conducted a study on British primary school students’ creative thinking using verbal and figural creative thinking tasks, noting that students’ DT, on average, increased with age (grade level) for verbal originality and verbal elaboration. However, a meta-analysis of 16 papers from 1970 to 2018 exploring the effect of age on DT indicated that there may not be a linear relationship between age and DT (Fusi et al., 2021); namely, one’s divergent thinking does not necessarily increase or decrease by age. The mixed results from previous studies have suggested a need to further clarify the association between age (grade level) and divergent thinking. Additionally, past studies dealing with this issue were mainly conducted among primary and middle school students. Hence, exploring the varying development and expressions of imagination among undergraduate students in different grade levels was one of the objectives of the present study.

Imagination and gender

Previous research has not produced clear conclusions about gender differences in creative behaviors (Alfonso-Benlliure & Santos, 2016; Baer & Kaufman, 2008). The issue has received even less attention in imagination research. Using electroencephalograms, Razumnikova’s (2004) study investigated differences in brainwave activities between genders as participants engaged in creative problem solving. The study found neuroscientific evidence that gender differences existed in the process of successful divergent thinking. Studies on gender differences in creativity often showed contradictory results (Baer & Kaufman, 2008). Some research found that girls had higher creativity scores than boys (e.g., Wang et al., 2014b; Dudek et al., 1993), some found the opposite (e.g., Chan et al., 2001), and some found no gender differences (Hargreaves, 1977). In addition, some studies have suggested that gender differences in creativity may be influenced by the environment (McVey, 2004; cited from Baer & Kaufman, 2008) or may simply reflect differences in assessment tools (Hardy & Gibson, 2017).

With particular regard to scientific imagination, Wang et al. (2015) found no difference between boys’ and girls’ performance in elementary school. Mun et al. (2015) noted that girls in Korean elementary schools and junior high schools performed better than boys on measures of scientific imagination. A study carried out by Hsu et al. (2012) found that students’ visual imagination varied according to gender and level of expertise in the mental factors that facilitate imagination. Though these factors themselves did not differ by gender, males’ experience with autonomy and respect from others generated positive emotions that facilitated imagination. In another study investigating the influence of psychological and environmental factors on teachers’ imagination in designing lessons, the female teachers reported teaching resources, intrinsic motivation, positive emotion, and learning by doing had more significant impact on their imagination in lesson designs than school environment (Hsu et al., 2013). Thus, individuals having different stations in life may exhibit gender differences in imagination, which is an issue worthy of investigation.

Imagination and academic background

Individuals with different learning backgrounds may portray varying imagination performance due to their different learning experiences. Wu et al. (2016) examined the creative behaviors of students from the College of Business Management, College of Engineering, College of Liberal Arts, and College of Design and found students from the College of Design had better creative performance than those from the other colleges. The difference might arise from the fact that students at College of Design had more creative experiences through their assignments and more exposure to creative books or creativity-induced environments. Pan (2016) investigated the creativity and remote associations of students in the College of Arts showed that students in the department of performing arts excelled on tests of remote associations than students in the department of design, who outperformed in divergent thinking.

In sum, students from different departments or colleges possess varying ways of thinking and life experiences; thus, they may differ in their creative thinking or imagination performance. However, previous research has rarely compared the imaginative performance of students from different departments or colleges. This study aimed to investigate further the relationship between students’ academic background and their imagination.

Developmental progress variables

The development progress variables are the variables or constructs corresponding to the understanding level of the student over the duration of the LPs, which are assessed as a means of gaging progress. To clarify students’ understanding level in one or more categories, hierarchies of conceptual understanding or construct maps created by Wilson (2005, 2009) was also employed. A group of experts who specialized in creativity, imagination, and educational testing and assessment were invited to panel discussions on the component hierarchies of LPs of scientific imagination for undergraduate students, developed and validated by Wang et al. (2017b).

Item design

Item design governs the match between classroom instruction and the various types of assessments (Wilson, 2009). After experts’ panel discussions, it was decided not to modify either the components of that LP or the content of the SIT-Verbal. As the SIT-Verbal was designed according to the situations that students will encounter at school, it can be applied to assess scientific imagination among students of different age groups or grade levels, including college students (see Table 2).

OPEN IN VIEWER

Table 2.

Contents of the Revised Version of the Scientific Imagination Test-Verbal (SIT-Verbal).

Mission 1 The planet LABIDO has rich natural resources and minerals that can help to solve crises faced by the earth. However, LABIDO is in big trouble now. You have been chosen as one of the spacemen to undertake an adventure to help LABIDO. Before your journey, you need to accomplish one mission. Then you can go to LABIDO and complete the second mission with the other spacemen. Situation: “There is a school on LABIDO. Many trees are planted at the school. There are always lots of falling leaves near the trees….” Please answer the following questions. 1) Question 1: What problems might this situation bring about? (Try to think about as many problems as you can.) 2) Question 2: Based on Question 1, how might these problems influence daily life? (Try to think about as many influences as you can think.) 3) Question 3: How many solutions can you think of? (Try to think about as many solutions as you can.)

Mission 2 Based on the same situation, please draw one “new invention” that you think can efficiently solve the problem described in the above situation. You have to explain the materials you will need for your new invention and specify the functions of this invention. Please also give your invention a name. Time limit for these two missions: 15 minutes.

Note. From “Validation of learning progression in scientific imagination using data from Taiwanese and Finnish elementary school students,” by Wang et al. (2017a, p. 76).

Outcome space

An outcome space is the set of categorical outcomes into which test-takers’ performances are categorized for all items associated with a progress variable (Wilson, 2009). In the present study, published scoring guides (Wang et al., 2017b) were adopted by raters to assess students’ responses. Among the three raters, two held master’s degree in education and one was an assistant professor whose expertise was educational psychology, creativity & imagination, and educational assessment (e.g., Ho et al., 2013; Wang et al., 2011, 2014a, 2014b). After all the raters reached their consensus on the scoring criteria and rubrics, a set of 10 questionnaires was given to each of the raters to allow for the calculation of inter-rater reliability. The reliability values ranged from 0.20 to 0.83, indicating considerate stability among raters (Miles & Huberman, 1994).

Measurement model

The BAS uses Rasch models to analyze data. Rasch models are commonly used to represent participants’ responses in terms of probabilities, which may overcome some limitations of psychological testing associated with classical test theory (CTT). The next step was to validate the LP of scientific imagination using the BAS, a measurement model using Rasch techniques, to analyze data in the outcome space. According to the Standards for Educational and Psychological Testing (American Educational Research Association, American Psychological Association, & National Council on Measurement in Education [AERA, APA, & NCME, 1999, 2014), the Rasch techniques can be used to evaluate an assessment tool by providing multiple sources of validity evidence. In the previous study, we validated the SIT-Verbal using a sample of elementary school students (Wang et al., 2015). As the cross-sample validation is also necessary for enhancing the quality of the adopted test—the SIT-Verbal, the current study revalidated the SIT-Verbal with a sample of undergraduate students by evaluating its content validity, structural validity, generalizability, substantive validity, interpretability, and external validity (Messick, 1994, 1995a, 1995b; Wolfe & Smith, 2007).

Considering that each item of the SIT-Verbal has four response categories, the Rasch partial credit model (Rasch PCM; Masters, 1982) was employed to analyze multiple ordered responses to the test. The PCM is a unidimensional model consists of two primary parameters: the person’s (n) ability (θ_n) and the item’s (i) difficulty (δ_i). When person n responds to item i, the probability of that person having the correct answer to that item is given as follows:

P nix = exp ∑ j = 0 x [ θ n − ( δ i + τ ij ) ] ∑ k m i exp ∑ j = 0 k [ θ n − ( δ i + τ ij ) ] , x = 0 , 1 , . . . , m i

(1)

where P_nix is the probability of person n scoring x on item i. Additionally, θ_n, which is the latent trait level of person n, refers to the construct that is the target of measurement (e.g., scientific imagination), and δ_j is the overall difficulty of item i (e.g., difficulty or threshold value). τ _ij is an additional step parameter for scoring j rather than j−1 on item i.

Scientific imagination test-verbal

The SIT-Verbal, a context-based measurement tool designed by Wang et al. (2015) to measure fifth and sixth graders’ scientific imagination in the present study, covers four domains: brainstorming, association, transformation/elaboration, and conceptualization/organization/formation (see Table 2). The test items involved a space mission to another planet in which test-takers encountered “a lot of falling leaves” at a school on the planet. In the open-ended test, students were asked to accomplish two tasks. After successfully completing Task 1 (answering the first three questions), students continued to carry out Task 2, in which they needed to draw an inventive idea to solve the problem identified in Task 1. The transformation/elaboration and conceptualization/organization/formation components were assessed in Task 2. Among the test items, six of them addressed the four components of scientific imagination. Students were given 15 minutes to complete both tasks and their responses ranked from “category 0” (the lowest level) to “category 3” (the highest level).

The revised object-spatial imagery questionnaire

To provide external validity evidence of the SIT-Verbal, the Object–Spatial Imagery Questionnaire (OSIQ; Blajenkova et al., 2006; Wang et al., 2017b) was revised to assess individual differences in visual imagery preferences and experiences. As the original questionnaire questions were written in English, Wang et al. (2017b) selected the suitable items for measuring imagery, translated them into Chinese, and asked two experts specializing in English literature and language to review and modify items. Furthermore, two experts with expertise in educational psychology, educational testing and assessment were invited to examine content validity and the construct of imagery. The Revised OSIQ consisted of a total of eight items in two sub-scales: an object imagery scale (OIS) and a spatial imagery scale (SIS). The OIS assessed test-takers’ preferences for representing and processing colorful high-resolution pictorial images of individual objects (e.g., my images are very vivid and photographic). The SIS evaluated test-takers’ preferences for representing and processing schematic images, spatial relations amongst objects, and spatial transformations (e.g., I can easily imagine and mentally rotate three-dimensional geometric figures).

The unidimensionality of the Revised OSIQ was scrutinized through the Rasch partial credit model (Rasch PCM). The results demonstrated acceptable infit and outfit mean squares (MNSQs; range: 0.6–1.4) in all items. No substantial differential item functioning (DIF) for gender was found. The person separation reliability for mental imagery was 0.73. The participants were asked to respond to all questionnaire items and rated each item on a 5-point scale, where 0 = totally disagree and 4 = totally agree.