Abstract
Spoken English Competency (SEC) remains a critical weakness in English Proficiency (EP) of Chinese college students, particularly when compared with their English competency in writing and reading. Despite years of English learning since elementary school, Chinese college students often struggle to master SEC. Although previous studies have explored Chinese students’ SEC from various perspectives, there is still a significant knowledge gap, especially from students’ perspectives. This study aims to analyze students’ Perceptions and Needs of Spoken English (PNSE) and their Self-Assessment of Spoken English Competency (SA-SEC) with the language output evaluation indicators (including three dimensions: complexity, accuracy, and fluency [CAF]). The study analyzed 2,677 (male = 1,340, female = 1,337) online questionnaires from college students across four English proficiency distinct groups. There are four identified aspects of PNSE (i.e., oral importance, difficulty, purpose, and improvement willingness). Similarly, SA-SEC was analyzed with three dimensions of CAF, namely complexity, accuracy, fluency, and nominal variables like pause type. The findings revealed that gender and EP significantly influenced PNSE and SA-SEC, and correlations were observed between paired variables of PNSE and dimensions of CAF in the SA-SEC section. Notably, complexity was identified as the weakest dimension of CAF for most students. Future studies are imperative to further explore the intricate relationship between PNSE and SA-SEC, and how factors such as pause type influence students’ spoken English performance.
Plain Language Summary
Background: Chinese college students have weaker oral English skills than their other English skills. Few studies have investigated their perceptions and needs for their oral English learning, including their self-assessments. Purpose: This study aims to analyze students’ perceptions and needs of Spoken English (PNSE) and their Self-Assessment of Spoken English Competency (SA-SEC) with the language output evaluation indicators (including three dimensions: complexity, accuracy, and fluency [CAF]). Methods: An online questionnaire was used to collect data, which had mainly sections related to PNSE and SA-SEC. Statistical methods, including correlation analyses, independent samples t-tests, linear regression, ANOVA, and Chi-square tests, were used to analyze the data. Results: The study found that gender and English proficiency could influence students’ PNSE and SA-SEC. Students thought the most common pause type is “en∼” when speaking English, and complexity was their poorest dimension of CAF. Implications: The results have significant implications for Spoken English learning and teaching. Specifically, the findings can encourage students to evaluate their CAF actively and know more about their strengths and weaknesses, and also help teachers understand students’ attitudes towards oral English learning and their weaknesses, thus adjusting teaching methods to improve students’ SEC. Limitations: The study focuses on spoken English from students’ perspectives, and future studies are needed to incorporate teachers’ perspectives and other important PNSE and SA-SEC variables.
Keywords
Introduction
English has become a compulsory course for college students since the reinstatement of the National College Entrance Examination in 1977 in China (X. X. Li, 2019). College English education has undergone several reforms with vicissitudes of history. In the 1980s, the focus was predominantly on students’ English reading and writing competencies, with limited spoken English training in many Chinese colleges. In the late 1990s, educators realized the limitations of the traditional English teaching modes, which often left students struggling with conversation in English. This led to a wave of reforms aimed at bolstering students’ spoken English competency (SEC) (Shao, 1999; Wang & Li, 1995; R. W. Zhang, 1996; Zou, 1996). Since the turn of the century, most colleges have started oral English courses, resulting in a noticeable improvement in students’ SEC. However, students’ SEC is still weaker than reading and writing competencies (Feng & Gong, 2007; Feng & Wu, 2005; Y. L. Li & Liu, 2003; Liu, 2008; Tang, 2005; Yang, 2002). In response, English listening and speaking courses have been integrated into junior and high school curricula. Despite these efforts, students often encounter significant challenges when it comes to speaking English fluently outside the classroom. Many students perceive oral English as a burden, leading to reluctance or even avoidance of speaking English aloud. Therefore, investigating students’ attitudes towards oral English becomes extremely urgent.
Literature Review
EFL (English as a Foreign Language) Learners’ Attitudes Towards Oral English
Learners’ attitudes are crucial in shaping learning effects. Positive attitudes can significantly enhance the effectiveness of the learning process. Scholars, such as Zeidner and Bensoussan (1988), have investigated the factors that influence EFL learners’ attitudes, responses, and anxiety toward oral tests. In their study, 170 EFL learners participated, revealing a preference for written tests over oral ones. These learners perceived written tests as more pleasant and less anxiety-inducing compared to oral tests. Lee and Winke (2018) also investigated the responses of young EFL learners in a speaking test. Their findings indicated that learners tended to fixate long on and frequently glance at the countdown timer, often coinciding with hesitation phenomena (e.g., hemming pauses and silences). Moreover, some scholars are interested in EFL learner’s oral production process. They found cognitive ability, work memory, and L2 (second language learning) knowledge act as key roles during utterance. Learners’ perception of oral English provides a telling insight into their attitudes. For instance, Kuo (2011) found that learners’ perceptions can contribute to their communicative intelligibility, grammatical accuracy, and classroom interaction. Therefore, understanding learners’ perceptions is beneficial to both teaching and learning effectiveness.
Learners’ Self-Assessment
Self-assessment serves as a valuable measurement tool that is widely believed to foster learners’ self-regulatory learning and autonomy (Dann, 2002; Paris & Paris, 2001). With a growing trend of a conceptual shift from teacher-centered to learner-centered instructions, numerous studies have demonstrated the positive effects of self-assessment on students’ English performance, as well as their confidence in learning English (Babaii, 2016; Butler & Lee, 2010; Oscarson, 1989). Babaii’s (2016) investigation into EFL learners’ speaking self-assessment revealed a generally positive evaluation of this process. Therefore, learners effectively utilize self-assessment as a tool to facilitate their second language learning. For instance, by using specific criteria for their self-assessment, learners can identify where their weaknesses are in their studies and what they should improve, thereby optimizing their learning outcomes.
CAF—Dimensions of Language Productive Evaluation
Language researchers have identified three key dimensions of language production, including complexity, accuracy, and fluency (CAF). Their objective is to establish appropriate and meaningful indicators of CAF to measure test-takers’ language output proficiency. Hunt (1970) (as in our knowledge) first introduced the concept of a T-unit, or minimal terminal unit, essentially an independent clause, to measure CAF. Larsen-Freeman (1978, 1983) and Larsen-Freeman and Strom (1977) concluded that error-free T-units could serve as a valid measurement index. Harrington (1986) further employed the T-unit approach to analyze learners’ spoken Japanese as a second language (L2). Since the 1980s, researchers have used the average length of error-free T-units to measure the accuracy of CAF about L2 speaking or writing. Notably, Wolfe-Quintero et al. (1998) conducted a comprehensive study and found that T-unit length, error-free T-unit length, and clause length were the most effective indexes for assessing fluency. On the other hand, the number of error-free T-units, the ratio of error-free T-units to total T-units, and errors per T-unit were determined as the best measures for accuracy. Finally, for complexity, the following were identified as the most suitable metrics: clauses per T-unit, the number of dependent clauses per T-unit, or the number of dependent clauses per T-unit.
Although they have traditionally been employed in the assessment of EFL writing, complexity and accuracy are now used to evaluate EFL speaking with growing popularity. Researchers believe that CAF can be critical dimensions for gauging EFL learners’ spoken proficiency. However, it is important to note that indicators of CAF for speaking may differ from those for writing. Housen and Kuiken (2009) emphasized that the use of the ratio of subordinate clauses to total clauses as a measure of structural complexity, and type–token ratios for lexical complexity to measure the complexity of CAF in L2 speaking. Kobayashi et al. (2022) further identified six key features in the assessment of L2 spoken language progress, including causative adverbial subordinators, independent clause coordination, emphatics, nouns, prepositional phrases, and present tense. Thus, Kobayashi’s 6 features can be considered as indicators of structural complexity and lexical complexity. As for accuracy, it is often measured by calculating the ratio of error-free T-units or using more specific metrics such as verb tense accuracy or word collocation accuracy. Fluency has been frequently measured by examining the speed (e.g., the average number of syllables or words produced per second), as well as pause and repair behaviors of L2 speakers. By incorporating these measures, a more comprehensive evaluation of EFL speaking proficiency can be achieved.
Chinese scholars have used theories of language production, trade-off, and cognition to measure the CAF of EFL speaking. For example, esteemed linguists W. Z. Zhang and Wu (2001) discussed indexes of fluency and pointed out that the ratio of reported necessary events to total necessary events could serve as a metric for evaluating the consistency of speaking contents, complementing traditional measures like speech speed. Zhang (2000) developed a model of fluency tailed for learners’ SEC, which included four dimensions (time, content, utterance, and expression), based on an empirical study with 12 indicators on 12 English major students for 28 weeks. Jiang and Dai (2018) used 11 indexes to measure the accuracy of test-takers’ speaking, including the total number of words, the total number of clauses (C), the total number of T-units, the total number of verb phrases, the number of error-free T-units, the number of error-free clauses, the number of errors, errors per 100 words, the ratio of error-free C-units, the ratio of error-free T-units to total T-units, and the ratio of correct verb phrases. Xie and Zhou (2018) broadened the scope, incorporating four dimensions (i.e., discourse volume, accuracy, complexity, and narrative structure) to measure testers’ oral narrative proficiency. Among these, discourse volume was gauged by the total word production, accuracy by the proportion of grammatically correct clauses, and complexity by the ratio of clauses to analysis of speech units (AS-units). In their study, clauses included independent, parallel, and subordinate ones. Narrative structure, specifically, focused on the ratio of narrative clauses to AS-units. J. X. Xing (2019) also used AS-unit to measure complexity and introduced lexical indexes to evaluate lexical diversity and lexical complexity. Lexical diversity was measured by D values, and lexical complexity was measured by lambda values.
L2 oral learning and assessment have been an increasing research hotspot for global scholars. For example, Fulcher et al. (2022) did an extensive study about language testing and assessment, especially about performance-based assessment of speaking. Additionally, numerous scholars have examined the impact of task types on L2 learners’ oral output (Dimova, 2022). However, our study focuses on students’ self-assessment of their SEC on CAF.
Research Questions
Although previous researchers have done plenty of work on CAF, college students in China are still confused about indicators for evaluating their SEC. Most students participate in the College English Test Band 4 or 6, which primarily evaluates their EFL reading and writing proficiency in China. However, few students take Band 4 or 6 of the Oral English Test, leaving a significant gap in their understanding of spoken English evaluation. Currently, there is limited research investigating students’ perceptions of their spoken English learning, their needs for spoken English, and their self-assessments. Moreover, little is known about which dimension of CAF students consider as their weakness. To bridge these gaps, our study aims to explore college students’ perceptions, needs, and self-assessment of spoken English in China, and we hypothesize that: (1) English proficiency and gender can affect students’ perception and needs; (2) There exist close relationships among three dimensions of CAF in student’s self-assessment; (3) Gender has significant effects on pause types and weakness dimensions of spoken English.
Methods
Questionnaire Design
Based on previous studies about CAF measurement, this study carefully selected parameters frequently used by most Chinese scholars. After selecting these indexes of CAF, the authors sought inputs from English teachers to ensure the appropriateness of the chosen indicators. As a result, 7 indexes were finalized for the study. Complexity was measured using “lexical diversity” and “syntactic complexity” derived from Skehan (2009) and Skehan and Foster (2012), as well as “syntactic diversity” from Yuan and Ellis (2003). Accuracy was gauged through “correct clause ratio” and “correct verb form” derived from Foster et al. (2000) and Foster and Wigglesworth (2016). For fluency, “flow length” from Prefontaine and Kormos (2015) and “speech speed” from Ellis (2009) were employed. Table 1 presents the selected indexes of CAF in our study.
CAF Indexes in the Study.
The questionnaire, meticulously designed by the authors, was circulated among teachers for their feedback and suggestions. This questionnaire comprises three sections, encompassing a total of 28 items. The first section delves into students’ Perceptions and Needs of Spoken English (PNSE), exploring their perspectives on oral importance, difficulty, purpose, and improvement willingness, across items 1 to 8. The second one focuses on students’ SA-SEC, covering items 9 to 25. The third section collects demographic information, including items 26 to 28. At the beginning or end of the questionnaire, some terms and measurements are introduced, like CAF definition or guidelines on how to divide syllables and measure fluency. This ensures that respondents have a comprehensive understanding of the questionnaire and can provide informed responses.
In section 1, items 1, 2, 6, and 7 specifically address students’ perception of oral difficulty, item 5 refers to oral purpose, items 3 and 4 delve into oral importance, and item 8 focuses on speaking improvement willingness. Students respond to most items in section 1 on a 5-point Likert-type scale, ranging from 1 (‘totally disagree’) to 5 (‘totally agree’). Additionally, for items 5 and 7, multiple choices are provided to allow students to specify the purpose of learning spoken English and the type of speaking difficulty they encounter.
In the second section, item 9 is about lexical diversity, and item 11 focuses on syntactic diversity. Items 14 and 15 specifically address the accuracy of clauses and verb tense. Items 18 and 20 are about fluency indexes (e.g., flow length and speech speed). For items 10, 12, 16, and 19, students are prompted to provide numerical or letter ratings to the CAF level of example characters. Items 13, 17, and 21 are about students’ self-assessments on CAF, and item 23 provides a comprehensive score reflecting their SA-SEC. Item 22 investigates pause types that commonly occur in students’ English speaking. Item 24 asks students to rate their proficiency on a three-level scale (i.e., low, intermediate, and high). Item 25 explores students’ perceived weaknesses in three dimensions of CAF. In section 2, almost all items are set with a 3-point scale for rating. Students rate each dimension of CAF in a range of 2 to 10 points, which are categorized into three levels: 2 to 5 points for level one, 6 to 7 points for level two, and 8 to 10 points for level three.
Item 23 evaluates students’ SA-SEC of CAF once they have become familiar with relevant indexes and measurements. The scale for this item is as follows: 1 = “6 to 15 points,” 2 = “18 to 21 points,” 3 = “24 to 30 points” (each dimension is rated on a scale of 2 to 10 points, resulting in a total score range of 6 to 30 points for CAF). Similarly, item 24 employs a similar rating on students’ proficiency with 1 = “low level,” 2 = “intermediate level,” and 3 = “high level.”
Data Collection
The online SA-SEC questionnaire was primarily completed by most freshmen, with contributions from some sophomores and graduate students. All participants agreed to take part in the study and were assisted by their teachers. A total of 3,420 questionnaires were collected online. However, 405 questionnaires were considered invalid because of unusually short completion time, and 339 were excluded because the Z-score (a standard score in statistics) fell outside the acceptable range of −2 to 2. The remaining valid questionnaires were categorized into four groups, including 1,252 freshmen A (630 male and 622 female), 1,134 freshmen B (552 male and 582 female), 224 sophomores (135 male and 89 female), and 67 graduate students (male = 23, female = 44). Table 2 shows information on the four groups. In China, English proficiency is typically assessed through various competency tests (i.e., College English Proficiency Test, College English Test Band 4 to 6, and English for Postgraduate Admission Examination). Based on these assessments, the English proficiency of freshmen A is superior to that of freshmen B but inferior to graduate students. Sophomores exhibited the lowest SEC among the four groups. Therefore, four groups are designated as groups 1 to 4 in ascending order of English proficiency (e.g., sophomores who did not pass College English Test Band 4 to 6 were named group 1).
Information on Four Groups of Participants.
In section 1, four variables capture students’ PNSE, including “Oral Difficulties (OD)” (involving items 1, 2, and 6; the resulting value is the average value of these three items), “Oral Importance (OI)” (referring to items 3 to 4; the average value of both items), “Oral Purpose (OP)” (involving item 5; higher values mean stronger purpose), and “Speaking Improvement Willingness (SIW)” (referring to item 8; higher value of the variable indicates stronger willingness).
In section 2 on SA-SEC, several variables have been defined. For example, complexity, represented by C-CAF, is determined by the average response values of items 9, 11, and 13. Similarly, variables of accuracy (A-CAF) and fluency (F-CAF) are also defined. Additionally, there are nominal variables, like pause type (PY), SEC level (EL), and weakness part (WP), providing categorical information. With regard to these variables, a higher value indicates stronger student competency in a particular domain. Item 20 is about students rating their audio recordings by Ellis’s (2009) method, which has been introduced in detail at the bottom of the questionnaire. For this item, the values below 225 are coded as 1, those falling between 225 and 290 are coded as 2, and values exceeding 290 are coded as 3. The speech speed is calculated by dividing the total number of syllables in the words spoken by the number of seconds taken to complete the task, and then multiplying by 60. For instance, if a person speaks 90 words in 1 min with an average syllable count of 2.5, then his speech speed would be calculated as 90 × 2.5/60 × 60 = 225. The study uses the average speech speed derived from student responses.
Results
The Relationships Between English Proficiency and PNSE Variables
To investigate if EP has effects on student’s PNSE, our hypotheses are given as follows:
H0: There are no correlations between English Proficiency (EP) and the four variables (i.e., OI, OD, OP, and SIW in section 1) of PNSE.
H1: There are significant correlations between EP and the four variables.
To test the hypotheses, Spearman’s rho correlation analysis was employed. The results are presented in Table 3.
Correlation Values Between EP and Four Variables.
Note. OI = oral importance; OD = oral difficulty; OP = oral purpose; SIW = speaking improvement willingness; EP = English proficiency.
Correlation is significant at the 0.05 level (2-tailed).
Correlation is significant at the 0.01 level (2-tailed).
Based on Table 3, we found that: (a) there were no correlations between English proficiency and oral English importance (p > .05); (b) there was a significant correlation between English proficiency and oral difficulties, and the correlation coefficient was negative (Rs = −.171, p < .01); (c) there was a significant correlation between English proficiency and oral purpose (Rs = .089, p < .01); and (d) a significant correlation was also found between English proficiency and students’ speaking improvement willingness (Rs = .046, p < .05).
(2) H0: There are no significant differences across genders in the four variables.
H1: There are significant differences across genders in the four variables.
To test the hypotheses, the Independent-Samples t-test was conducted and the results are shown in Table 4.
Comparisons Between Genders for the Four PNSE Variables.
Note. OI = oral importance; OD = oral difficulty; OP = oral purpose; SIW = speaking improvement willingness; EP = English proficiency.
According to the t-test analyses in Table 4, we found that there were significant differences across genders in oral importance, oral purpose, and speaking improvement willingness at a confidence level of 95% (α = .05), but no statistical differences between males and females were identified for the variable OD. Figure 1 shows mean differences between males and females for the above-mentioned four variables. The biggest mean difference was found for the variable OP.
Mean performance of males and females for the four PNSE variables.
(3) H0: There are no significant correlations between paired variables of PNSE
H1: There are significant correlations between paired variables of PNSE.
For these hypotheses, Spearman’s rho correlation analysis was conducted again as shown in Table 5.
The Correlation Values Between Paired Variables of PNSE.
Note. OI = oral importance; OD = oral difficulty; OP = oral purpose; SIW = speaking improvement willingness; EP = English proficiency.
Correlation is significant at the 0.01 level (2-tailed).
We observed that OI and the other three variables had positive correlations (Table 5; OD: Rs = .374, p < .01; OP: Rs = .056, p = .004; SIW: Rs = .071, p < .01). OD showed a negative correlation with OP (Rs = −.067, p < .001). There were no significant correlations between OD and SIW. Conversely, OP showed a significant correlation with SIW (Rs = .214, p < .001).
The Close Relationships Among Three Dimensions of CAF
Our hypotheses are as follows:
H0: There are no significant correlations between paired variables of CAF.
H1: There are significant correlations between paired variables of CAF.
For these hypotheses, Pearson’s correlation analyses were conducted as presented in Table 6.
The Correlations Between Paired Variables of CAF.
Note. C-CAF = complexity; A-CAF = accuracy; F-CAF = fluency.
Correlation is significant at the 0.01 level (2-tailed).
Significant correlations were observed between paired variables of CAF, as indicated in Table 6 (all p < .001). The correlations were positive and relatively strong (r = .510 between C-CAF and A-CAF; r = .374 between C-CAF and F-CAF; r = .405 between A-CAF and F-CAF).
To delve deeper into the relationships among CAF variables, regression analyses were conducted. Linear equations were derived by using three variables C-CAF, A-CAF, and F-CAF to examine how one of the variables might be influenced the remaining two. The results are shown in Table 7.
The Linear Equations Derived from Three Variables of CAF.
Note. C-CAF = complexity; A-CAF = accuracy; F-CAF = fluency.
As Table 7 presents, in the SA-SEC section, linear relationships were identified among the three variables of CAF. For example, it was observed that the variable C-CAF was influenced by both A-CAF and F-CAF, but the constant was small (a = .199), compared to the other two constants. Additionally, A-CAF was also associated with C-CAF and F-CAF (e.g., if both C-CAF and F-CAF increase by 1, A-CAF will increase by 0.689).
Gender and English Proficiency Discrepancies on Dimensions of CAF
We also aim to assess whether gender and EP have impacts on SA-SEC, and our hypotheses are as follows:
(1) H0: There are no significant differences between males and females in SA-SEC.
H1: There are significant differences between males and females in SA-SEC.
To test the hypotheses, the Independent-Sample t-test was conducted. In all comparisons, the significance values (i.e., sig (2-tailed, p)) were greater than 0.05(α = .05). Therefore, no significant differences were found between males and females in students’ self-assessment of SEC.
(2) H0: English proficiency has no significant effects on three dimensions of CAF.
H1: English proficiency has significant effects on three dimensions of CAF.
To test the hypotheses, ANOVA analyses and Post Hoc tests were conducted. The results are presented in Tables 8 and 9.
ANOVA Results for the Three CAF Dimensions Across Groups.
Note. C-CAF = complexity; A-CAF = accuracy; F-CAF = fluency.
Mean Differences Between Groups on Three Dimensions of CAF by Post Hoc Tests.
Note. C-CAF = complexity; A-CAF = accuracy; F-CAF = fluency; MD = mean difference.
The mean difference is significant at the p < .05 level.
Based on Tables 8 and 9, we had several findings: (1) Students’ EP had a notable impact on their SA-SEC across three dimensions; (2) Significant mean differences were identified among groups on the three dimensions; (3) Group 1 (sophomores) rated their SEC and their scores were lower than all others groups; (4) Group 3 (freshman A) rated their SEC and presented higher scores than the other three groups; (5) Group 2 (freshmen B) rated their SEC with higher scores than group 4 (graduate students); and (6) Group 4 rated their SEC only higher than group 1, as shown in Figure 2 (presenting mean differences among groups across three dimensions).
Mean performance of groups across three dimensions of CAF.
Gender Effects on Pause Type and Weaknesses of CAF
In section 2, there are some nominal variables, such as pause type (PY), SEC level (EL), and weakness part (WP). To investigate potential gender discrepancies, the hypotheses are as follows:
H0: Gender does not significantly influence students’ evaluations of pause type (PY), SEC Level (EL), and weakness part (WP)
H1: Gender significantly influences students’ evaluation of PY, EL, and WP.
For these hypotheses, the crosstab analysis and Chi-square tests were conducted. The results are presented in Tables 10 and 11.
The Crosstabs of Gender and Pause Type (PY).
Chi-Square Tests on PY, EL, and WP Across Genders.
Note. PY = pause type; EL = spoken English competency level; WP = weakness part.
Confidential level α = .05.
From Tables 10 and 11, it was observed that gender had significant effects on pause type (PY) with a p-value of .046. However, no significant gender differences were observed for SEC level (EL) or Weakness Part (WP).
From Figure 3 (providing a visual representation of the gender effects on PY), it can be seen that more than half of the female students frequently used “en∼” in their oral English, while 48.8% of male students exhibited a similar pattern. Taking “en∼” as a pause is the most prevalent pause type among students, with “keeping silence” ranking as the second most common, albeit significantly less frequent.
Gender discrepancy on pause types.
We further conducted descriptive statistical analyses on the variable WP. As shown in Figure 4, 44.2% of the students identified complexity as their weakest dimension in CAF, 22.7% perceived accuracy as their weakest aspect, and 33.1% attributed it to fluency.
The frequency and percentages for students’ WP in CAF.
Discussion
The study developed a questionnaire, titled “College Student Self-Assessment Spoken English Competency (SA-SEC),” and conducted statistical analyses based on 2,677 valid responses. In the first section focusing on students’ Perceptions and Needs for Spoken English (PNSE), four variables were established: Oral Importance (OI), Oral Difficulties (OD), Oral Purpose (OP), and Speaking Improvement Willingness (SIW). These variables were utilized to analyze students’ perceptions and needs for oral English. The study participants were categorized into four groups according to their grades and English proficiency. After several English proficiency tests, it was determined that graduate students’ English proficiency exhibited the best among the four groups, followed by freshmen A, freshmen B, and sophomores. Based on these variables, we discovered several interesting findings. (1) Conducting Spearman’s rho correlation analyses, we found that students’ English proficiency significantly influenced their needs and perceptions towards oral English. Specifically, those with higher English proficiency demonstrated a stronger willingness to enhance their speaking skills, echoing the finding of D. Xing (2019). Xing’s research revealed that students’ motivation to learn oral English increased as their proficiency in the language improved. Similarly, in our study students pursuing advanced education exhibited a desire to improve their oral English skills, whereas those with poor English writing and reading skills, like sophomore participants, showed less enthusiasm for improving their oral English. This observation aligns with the study conducted by Zarate (2022), who investigated how subjective factors like emotional expressions influenced the English competency of 150 English major students. Zarate (2022) found that negative behaviors could hinder the promotion of language proficiency, further supporting our findings on the relationship between students’ attitude and their willingness to improve their spoken English skills. There was a significantly negative correlation between students’ English proficiency and oral English difficulty, with a correlation coefficient of r = −.171. This indicated that students with lower English proficiency tended to consider oral English more challenging. Most students recognized the importance of oral English, as supported by the mean value of 3.8, with a mode of 4.5. (2) The Independent-Sample Test revealed that gender also played a role in students’ PNSE, specifically in terms of OI (oral importance), OP (oral purpose), and SIW (speaking improvement willingness), but not OD (oral difficulties). A noteworthy observation was that female students exhibited stronger needs and preferences in OI, OP, and SIW compared to male students. However, there were no significant gender differences in the perception of OD. Figure 1 graphically illustrates these gender-based discrepancies across the four variables. These discrepancies may be attributed to the fact that female students tend to enjoy oral English practices more and consider it more important in their future lives. Therefore, female students demonstrated a stronger willingness to improve their spoken English skills. (3) There were significant correlations between paired variables of PNSE. For example, a positive correlation was found between OI and OD. Students who value oral English highly in their social interaction tend to have a stronger purpose for learning and a greater willingness to improve their skills. This is consistent with Lin’s (2018) finding that particular sociocultural contextual factors can cause a stronger willingness to speak English. However, OP exhibited negative correlations with OD. This observation suggests that students with a stronger purpose for learning tend to perceive oral English to be less difficult compared to others. This finding mirrors the actual situation in students’ English studies. Nevertheless, no significant correlations were observed between OD and SIW. In most students’ opinions, oral English difficulty does not significantly affect their willingness to improve their speaking skills. Conversely, many teachers assume that, if they consider oral English particularly challenging, students will have a stronger willingness to improve it. This assumption does not align with the findings mentioned above, highlighting a notable disconnect between teachers’ and students’ perceptions. It would be worthwhile to conduct further studies to explore teachers’ assessments of students’ SEC and compare them with those of students.
In section 2, students completed online questionnaires titled SA-SEC (Self-Assessment of Spoken English Competency) to evaluate their spoken English proficiency. The questionnaire included variables based on three key dimensions of spoken English evaluation: complexity (C-CAF), accuracy (A-CAF), and fluency (F-CAF). Additionally, some nominal variables were introduced, like PY (pause type, reflecting common pauses during students’ English speaking), EL (self-assessment level, indicating students’ perception of their spoken English level), and WP (weakness part, the weakest aspect of students’ CAF). To analyze these variables, this study employed various statistical analyses, including Pearson’s correlation, linear regression, ANOVA analyses, and Chi-Square tests.
The first finding revealed that each dimension of CAF was influenced by the other two dimensions, and significant correlations were found among the variables. Furthermore, linear regression relationships were identified among these variables, offering insights into the intricate interplay between different aspects of spoken English proficiency. Linear equations derived from the data in Table 7 indicated that enhancing accuracy and fluency could lead to an improvement in the complexity of students’ oral English. Notably, accuracy appears to improve at a faster rate compared to the other two dimensions. Specifically, when complexity and fluency increase by 1 unit, accuracy can improve by 0.689 units. Complexity and fluency can increase by 0.645 units and 0.474 units, respectively, when the remaining two dimensions increase by 1 unit. This suggests that the three dimensions of CAF—complexity, accuracy, and fluency—interact synergistically in the process of oral English learning. This contrasts with the findings of Ahmadian and Tavakoli (2015) that complexity and accuracy had a competitive relationship with fluency (meaning that when the values of complexity or accuracy increased, the values of fluency would decrease), but accuracy and grammatical complexity could be improved concurrently. The inconsistency may stem from various factors, including differences in participant nationality, sample size, and the nature of assessments—students’ self-assessments versus those of teachers. However, our finding is in line with that of Vercellotti (2017), who observed that the three dimensions of CAF have no competitive relationships. Vercellotti (2017) conducted a longitudinal study involving 66 EFL learners over 9 months and collected a comprehensive set of CAF data to explore the interrelationships among dimensions of CAF. The strength of this study lies in its inclusion of a dynamic learning phase, which provides a more nuanced understanding compared to Ahmadian and Tavakoli’s (2015) study. Taken together, these studies highlight the intricate interplay between complexity, accuracy, and fluency in oral English learning, and suggest that a balanced approach to enhancing these dimensions may be most effective in improving overall proficiency.
The second finding in section 2 is that students’ English proficiency has impacts on the three dimensions of CAF: complexity, accuracy, and fluency. Our analyses revealed that there were significant differences among various proficiency groups across the three variables. Notably, students with lower proficiency demonstrated correspondingly lower mean values for CAF. For example, sophomores, who comprised group 1 with the lowest proficiency level, exhibited the lowest mean scores on all three variables. Intriguingly, graduate students (group 4), despite possessing the best English proficiency among the four groups, did not demonstrate the highest CAF mean values. In fact, their CAF scores were only higher than those of group 1 but lower than those of groups 2 and 3. This observation can be attributed to the fact that college students typically take oral English courses, which is a compulsory course during their first college year. Once they pass College English Test-4 or 6 (CET-4 or 6) at the end of their first year, they are free to choose elective courses that may not include oral English. As a result, some sophomores, who fail to pass CET-4 in their first college year, continue with their English course in their second college year. However, even with continued practice in English writing and reading, graduate students (group 4) often find limited opportunities to practice their oral English. Conversely, freshmen, who are still enrolled in oral English courses, have more opportunities to practice and therefore tend to outperform graduate students in CAF self-assessments. Our findings are consistent with Hanzawa (2021), who demonstrated a positive correlation between the L2 learning experience and the oral language output of learners. In Hanzawa’s study, 50 Japanese university students were exposed to L2 learning experiences, and a correlational analysis revealed that both classroom and extracurricular L2 learning experiences contributed to enhancing oral fluency. Similarly, in our study, the limited in-class opportunities for graduate students to engage in oral English practices likely contribute to their lower CAF evaluations compared to freshmen. It is noteworthy that, except for graduate students, all other students rated their CAF levels in alignment with their actual SEC (spoken English competence) scores. This suggests that given appropriate opportunities for practice and exposure, students can accurately assess their progress in developing complexity, accuracy, and fluency in their oral English skills.
The third finding is that no significant gender discrepancies were observed across three variables of CAF by conducting Independent-Sample t-tests. This finding is in line with the finding of O’Loughlin (2002). In his study, eight female and eight male test-takers participated in a practice IELTS (International English Language Testing System) interview. Interviewers (one male and one female) evaluated test-takers on two separate occasions. Utilizing a multi-faced Rasch bias analysis, O’Loughlin determined that gender had no significant impact on the scores awarded to the test-takers during the IELTS interview. Furthermore, our finding is also consistent with the work of Bonk and Van Moere (2004). In their study, the researchers examined L2 speaking scores across genders. They found that gender appeared to have little effect on the oral English scores of college students. This is an important observation as it implies that when assessing or teaching oral English skills, gender should not be a primary consideration. Instead, focus should be placed on individual students’ learning needs, experiences, and progress in developing complexity, accuracy, and fluency in their language use.
The fourth finding revealed a gender-specific influence on pause type in oral English proficiency, while no such effects were observed on the SEC Level (EL) and weakness part (WP). Specifically, male students are more likely to choose “keeping silence” as their pause type compared to female students. Additionally, male students tend to use “repeating words,” and “repairing sentences” more frequently than female students. Figure 3 clearly illustrates that male students used a wide range of pause types, except for “en∼….” It suggests that gender plays a role in the way students manage pauses in their spoken English. It is noteworthy that female students often perceive their English fluency to be superior to that of male students. As a result, they tend to avoid pause types such as “repeating words” or “repairing sentences,” viewing them as indicators of disfluency or stutter. However, these pause types can reflect cognitive fluency in speech production, and be engaged in “unexpected speech output planning” (Segalowitz, 2016). Segalowitz (2016) believed that these pauses often occur when students are searching for words or sentence patterns in their memory, indicating a lack of sufficient oral vocabulary and sentence structures. Consistent with Segalowitz’s theory, our study found that a significant proportion of students, accounting for 50.9%, opted for the “en∼…” option, indicating a limitation in their oral vocabulary. The choice of “Keeping silence” suggests that students are unsure of how to use conjunctions to fill the silence in their oral English. Echoing this, Yan et al. (2021) found that silent pauses or non-juncture pauses of L2 speakers can serve as indicators of fine-grained features. They postulated that silence and no junctions reflect a lower level of automatic processing of lexico-gramma, indicative of the speakers’ efforts to search for appropriate lexical and grammatical elements in their minds. Furthermore, they observed that the speakers with higher proficiency tended to pause at syntactic junctures such as clause boundaries. In our study, approximately 23.6% of students frequently resorted to silence when speaking English, suggesting that they perceived themselves as lower-proficiency speakers. Conducting further studies that delve into pause length and its related factors will be meaningful.
The fifth finding revealed that most students perceived complexity to be their weakest dimension of CAF. This observation is consistent with the finding of Yu and Kongjit (2022), who attributed Chinese learners’ lower spoken English proficiency to a lack of lexical complexity. It seems that most students believed that they would be able to speak English more fluently if they could overcome issues related to complexity or accuracy. This perception is likely influenced by their ability to read English texts much more smoothly. In students’ minds, enhancing complexity in their English speaking can lead to a rapid improvement in their SEC. Consequently, complexity is viewed as a crucial dimension of CAF to evaluate oral language output competency. In many previous studies, like Housen and Kuiken’s, the complexity of CAF was categorized into two main types, including cognitive complexity and lingual complexity. Lingual complexity is mostly measured by lexical diversity and syntactic complexity (Skehan & Foster, 2012; Housen & Kuiken, 2009, Vercellotti, 2017). However, cognitive complexity involves more intricate processes and appears to have little correlation with students’ perceptions in our study. Nevertheless, students can indeed encounter significant challenges with cognitive fluency, a concept introduced by Segalowitz (2010) and explored in numerous studies (e.g., Kahng, 2014, 2018). These studies emphasize that cognitive fluency involves the processing of target language clauses and lexical chuck in memory before articulation. Interestingly, our study identified associations of cognitive fluency with the complexity dimension of CAF related to SEC. This may explain why students frequently use “en∼…” and “keeping silence” as pauses when speaking English. As for why most students consider complexity as their weakest aspect of CAF, it seems that students tend to believe that syntactic complexity and lexical diversity are particularly challenging and demanding. Their perception is that these require more time and effort to master, leading to complexity being identified as the poorest dimension of CAF. This finding suggests that educational interventions need to focus on enhancing students’ understanding and application of syntactic and lexical complexity to improve their overall oral English proficiency.
Conclusion
The Findings of the Study
The study explored college students’ perceptions and needs for spoken English, along with their self-assessment of their spoken English competency. Firstly, the aim was to investigate the effects of English proficiency and gender on four variables of PNSE, namely, oral importance, oral difficulty, oral purpose, and speaking improvement willingness. Secondly, we examined disparities in English proficiency and gender in three dimensions of CAF, as well as the linear correlations of CAF in the SA-SEC section. Additionally, we analyzed students’ pause types and areas of weaknesses in their oral English.
Firstly, regarding PNSE, it was found that students’ English proficiency significantly shaped their perceptions and needs toward oral English. There are significant differences among the four groups across variables of PNSE. Secondly, our findings revealed that gender had an impact on variables of OI, OP, and SIW within PNSE, but not on OD. Notably, a majority of students perceived SEC to be the most challenging one among the five competencies of listening, speaking, reading, writing, and translation. Female students exhibited a stronger willingness to improve than their male counterparts. Thirdly, significant correlations were identified between paired variables of PNSE, except for OP and OD. For example, OP had impacts on OI and SIW. Our study offers insights into why female students often demonstrate superior oral English skills compared to male students. These findings can assist teachers in gaining a deeper understanding of students’ attitudes towards oral English learning, enabling them to devise effective strategies to alleviate student anxiety and enhance students’ willingness to engage in spoken language learning.
Regarding SA-SEC aspects, it was found that: (1) there were significant and linear correlations between paired variables of CAF. Notably, accuracy appeared to be influenced more strongly by the other two dimensions of CAF (i.e., complexity and fluency), rather than the reverse; (2) English proficiency had a significant impact on students’ self-assessment across three dimensions of CAF. Specifically, higher EP levels led to higher mean values of CAF variables among all test groups, except for graduate students; (3) gender had significant effects on students’ pause types, but did not significantly affect their SEC level or WP (weakness part); and (4) complexity emerged as the weakest dimension of CAF for the majority of students.
In summary, Chinese college students often perceive their competency in spoken English to be inferior to their reading, writing, and other English skills. They also believe that enhancing the complexity of oral English is particularly challenging. Our findings can assist students in gaining a deeper understanding of how to evaluate their CAF strengths and weaknesses. Furthermore, our results can inform teachers about students’ areas of weaknesses, enabling them to adjust teaching methods and strategies to effectively improve students’ SEC.
The Limitations of the Study
This study is subject to several limitations, which future researchers need to notice. Firstly, the data collected through online questionnaires focused on the students’ perspectives of SEC, lacking the input of teachers’ comments. Consequently, the results could be somewhat subjective, relying solely on students’ self-assessed SEC levels.
Secondly, this study explored gender and group differences in variables of PNSE and SA-SEC but did not delve into whether there were significant relationships between these two sections. Future studies could explore the potential connections to gain a more comprehensive understanding.
Thirdly, the study identified common pause types exhibited by students when speaking in English, but the underlying reasons for these pauses remain elusive. Future studies could explore the factors influencing these pauses, gaining a deeper understanding of this linguistic phenomenon.
Lastly, while pinpointing out student’s weaknesses in oral English, this study did not delve deeply into the factors influencing these weaknesses. Further study can investigate these influencing factors, offering insights into how to address and improve students’ SEC more effectively.
Footnotes
Acknowledgements
The authors would like to acknowledge co-workers for suggestions on questionnaire design and corrections. The authors are grateful to some teachers for their help in calling on students to fill out questionnaires. The authors also thank students who filled out the questionnaire online.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The study has been approved by Education and Teaching project “The Study of the Construction of Oral CAF Evaluation System Based on English Corpus” (No. XGH21053) from Shaanxi Higher Education Society, by the Teaching Reform project “The Study of the Optimization and Evaluation of Oral CAF Indexes Based on English Corpus” (No. JY2103201), by the Experimental Technique project “Research on the Function Innovation of Oral Test Based on Digital Speech Technology” (No. SY20220217) from Northwest A&F University, and by Shaanxi Social Science Project “Construction and dissemination of Shaanxi’s new image from the perspective of cultural soft power” (No. 2018N14) from Shaanxi Philosophy and Social Science Planning Office, and by the Produce-Learn-Research Project “Immersive Oral English Experimental Teaching Reform Research” (No. 220501867114542) from the Department of Higher Education of the Ministry of Education of China.
Data Availability Statement
The data that support the findings of this study are available from the corresponding author upon reasonable request.