“Sometimes I Have to Figure Things Out by Talking About Them”: Conceptualizing Think-Aloud Protocols as Spaces for Reflexivity During Open Textbook Construction

A Textbook Production Study

Along with textbook evaluation and use, textbook production has been identified as one of the pillars of materials development research (Harwood, 2022), though its intricacies continue to be under-investigated (Yıldız & Harwood, 2024a). This observation holds true for commercial coursebooks (Yıldız & Harwood, 2024b) and is particularly applicable in the case of open textbooks (Atkinson & Corbitt, 2022a, 2023a, 2023b, 2024), open educational resources that can be freely used, shared, and sometimes adapted and rereleased due to their copyright permissions. Product-oriented studies have tended to predominate over process-oriented ones in the textbook literature (Harwood, 2022), meaning that textbook authors’ perspectives on the construction of a key pedagogical genre used to educate millions around the world remain underexplored (Norton & Buchanan, 2022).

Concurrent verbalization (think aloud) is a standard tool in writing process research (Hyland, 2022, p. 113), but it has not been widely employed in textbook production studies despite its ability to expose project particulars and resultant appeals for its use (see Harwood, 2017, 2021, 2025; Yıldız & Harwood, 2024a, 2024b). In reply to these calls, this study concentrates on the various functions that concurrent verbalization played during the construction of an open writing textbook, and it underscores the method’s potential for documenting reflexivity (reflective practice) in writers’ operations. Researchers have begun to explore concurrent verbalization’s roles in applied sports and exercise psychology (see A. E. Whitehead et al., 2025); however, specific functions of the data collection method have yet to be examined in the pedagogic materials development literature. Hence, the goal of this paper is to take a fresh look at a data gathering method that has found prominence in writing research to determine how it may operate in process-tracing investigations of textbook production. Spotlighting the functionality of concurrent verbalization may prove helpful to researchers who are curious about tracking textbook writing sessions and writers’ operations in fine detail; educators who seek to undertake open textbook development may likewise find value in this study’s findings. In the coming paragraphs, we delve further into concurrent verbalization’s use in materials development research before detailing the part it played during the construction of an open writing textbook.

The Challenges Associated with Investigating Materials Writing Processes

Textbook writers’ approaches to their craft vary, and the materials development literature has addressed the challenge of documenting them precisely. Atkinson (2020, 2024) observed, for instance, that accounts of textbook production based solely on authors’ recollections may lack specifics because they rely on memory retrieval rather than on data collected simultaneous with textbook writing sessions, a factor that might leave readers wondering about possible omissions. Shu et al. (2023) added that relying heavily on retrospective data in textbook development studies could lead to misleading results as participants may alter their descriptions of project particulars post hoc. Project data that are “speculative” in nature also bear consideration, remarked Samuda (2005, p. 235), since even well-established materials developers might not act as they presume to during writing sessions. To illustrate, many of the professional coursebook authors in Johnson’s (2003) research articulated a plan for building an English language teaching (ELT) task but then followed a different approach when writing it. While not focused on materials development, other writing process studies have highlighted similar discrepancies—for example, in the areas of convenience editing (Willey & Tanimoto, 2015) and journal article writing (Rymer, 1988)—and this factor led Harwood (2018, 2019) to employ concurrent verbalization to investigate the real-time rather than reported practices followed by essay proofreaders.

During a concurrent verbalization session, a participant engages in an activity whilst conveying their thoughts in vocal form to produce a think-aloud protocol (TAP) that is recorded and analyzed. This method can track materials development processes in ultrafine detail since it coincides with writing activity, but its implementation requires careful thought as environmental and situational factors may affect the character and amount of data gathered. For instance, in Johnson’s (2003) investigation, which compared the processes used by eight teachers and eight specialist task designers as they thought aloud while creating a pedagogic task according to a design brief, the participants were asked to write in a laboratory environment under researcher-observed conditions, a configuration that may have impacted the flavor of data gathered. Johnson (p. 35) noted that this research design was used to simplify and speed data collection, though he conceded that collecting concurrent verbalization data from authors in their ordinary working environments as they produce materials for their own projects holds great promise in terms of face validity. According to Kellogg (2018), many experienced writers devise ritualistic ways of working that ready their capacities for composing text, and though not focused on pedagogic materials development, Berkenkotter’s (1983) study of distinguished author Donald M. Murray’s revising practices illustrates in stark terms what can happen when investigators ask participants to write in situations they find alien. Berkenkotter tried to collect think-aloud data from Murray in an unfamiliar room as he attempted to respond to a researcher-supplied writing prompt that specified a particular rhetorical situation, but the author (Murray, 1983) revealed that he found the condition atypical and face-threatening; as a result, he only produced two lines of text during the hour-long concurrent verbalization session. When investigating how experts (highly skilled participants) wrote textbooks for English language teaching, Atkinson (2013) sought to mitigate this type of situation by incorporating naturalistic features into her research design. Specifically, she asked two authors to compose textbook material for their own projects at times and in spaces they selected while she was present to record their concurrent verbalizations and requested that they self-record TAPs during any additional textbook writing sessions that occurred outside of those occasions. Textbook Writer 1 (TW1), one of the participants in the study, did not record any of his own TAPs because he only wrote chapters when the researcher was present. Nevertheless, during a post-concurrent-verbalization interview, he said that he found thinking aloud while writing his textbook a natural activity since he ordinarily read back the material he created to determine whether it was worded correctly and pitched at an appropriate level for learners (pp. 389, 400). In contrast, Textbook Writer 2 (TW2), the other participant, provided four self-recorded TAPs and expressed during a post-concurrent-verbalization interview that although she initially found thinking aloud in the researcher’s absence “artificial,” she grew accustomed to the procedure as data collection proceeded during the longitudinal study (Atkinson, 2013, p. 780). The points addressed in this paragraph illustrate the need for careful calibration of research designs to account for factors that may impact data collection during writing process studies.

Concurrent Verbalization’s Capacity to Reveal the Finer Aspects of Materials Development Work

Concurrent verbalization offers the opportunity to capture writing process data substantive in extent and detail, and investigations into ELT materials development expertise have revealed some of the ways participants use TAPs as spaces for thought during writing episodes. In Johnson’s (2003) research, for example, the code “Philosophise” was applied to TAP extracts in which participants made comments about materials development itself rather than the tasks they designed during concurrent verbalization (p. 148). Interestingly, Johnson (p. 108) postulated that the participants may not have philosophized about materials development had he used a naturalistic research design to collect TAP data—namely, without having researchers on hand to monitor concurrent verbalization (pp. 186-187)—and that instances of philosophizing revealed little else beyond participants trying to be helpful in the presence of investigators. Johnson (p. 108) also commented that the specialist task designers (the professional coursebook writers) who participated in the study expressed pleasure at the prospect of research that at long last focused on materials development processes and that they likely philosophized as a result. Though these conclusions seem plausible, Atkinson’s (2013) research demonstrated that participant TW2 commented on materials design regardless of the investigator’s presence. Indeed, TW2 analyzed her own materials development behavior during one self-recorded think-aloud session when she remarked that writing an ELT textbook for commercial publication is “a little bit like plastering…it’s putting [on] layers…you start off with a…kind of basic layer and then just…Patch it…change it…and improve on it so many times” (Atkinson, 2013, p. 883). Concurrent verbalization provided TW2 with a medium for metacognitive musing about her work, and she duly touched upon her self-knowledge and awareness of how she operated during the session. TW2 also used the self-recorded TAPs to communicate in soliloquy fashion with the researcher regarding textbook development issues. In one instance, as TW2 worked on an origami activity for the textbook meant to introduce learners to the imperative mood, she commented that she would complete the activity and then ask the researcher to usability test it (Atkinson, 2013, p. 909). During a subsequent self-recorded TAP, the participant mentioned that although she had intended to begin work on a new textbook unit that day, she changed her mind because she still was not satisfied with the origami-themed activities for the chapter she had been writing, and she made reference to the researcher during concurrent verbalization: “I was intending…to go on to unit four…but in the few days that have passed since you were here Dawn I can’t get rid of this origami from my head everywhere I go it’s following me” (Atkinson, 2013, p. 911; see also Atkinson, 2020, p. 497). In these instances, TW2 used self-recorded TAPs in a journal-like manner to note the challenging but engrossing nature of designing functional textbook content for others to use.

Concurrent verbalization can also provide insight into the considerable time and effort that go into the production of quality textbooks and the cognitively taxing nature of such work. Indeed, in the case of TW2, a participant in Atkinson’s (2013) investigation of ELT textbook writing expertise, the method brought into focus the effects of sustained deliberate practice during writing episodes. Deliberate practice is the composite outcome of maintaining full concentration whilst engaging in effortful, domain-pertinent tasks coupled with the intrinsic motivation to repeatedly undertake such activity and refine performance in light of feedback (Ericsson et al., 2018). At the end of one researcher-recorded concurrent verbalization session during which TW2 shuffled the order of chapters and activities in her textbook, she expressed her intention to conclude by commenting “me brain hurts” (Atkinson, 2013, p. 859), presumably a side effect of the intense concentration the work demanded. After composing another chapter during a TAP, the participant likewise signaled her intention to end the session due to fatigue: “I’m actually getting a bit tired now…my brain’s going a bit wobbly,” she said (Atkinson, 2013, p. 717). When the author drafted another chapter, she concluded the concurrent verbalization session in a similar way: “I’m wilting a bit,” TW2 said (Atkinson, 2013, p. 748). Kellogg (2018) noted that such exhaustion is one of the aftereffects of intense periods of deliberate practice during which determined writers seek to refine their skills as they produce quality text, and it is why some engage in deliberate practice for limited stretches of time each day. As the transcript excerpts in this paragraph illustrate, concurrent verbalization can reveal the fine details of materials writing episodes.

Methodological Issues Associated with Concurrent Verbalization

Though concurrent verbalization is capable of exposing the particulars of materials development, the veridicality of TAP data has periodically been called into question. One of the primary concerns is that the method cannot exhaustively document every thought a participant has whilst engaged in a task (Bowles, 2019). This seems an unrealistic expectation, however, as some thoughts “will be hidden from the thinkers themselves” because they occur on an unconscious level (Johnson et al., 2008, p. 158). That concurrent verbalization may offer a partial view of the full range of a participant’s thoughts should not detract from the worth of what the method can document, argued Ericsson and Simon (1980), especially since investigators can use methodological triangulation to supplement data gathered via TAPs. Hyland (2016) and Li et al. (2023) also made the point that concurrent verbal protocols offer insight into thoughts and associated actions that are difficult to capture through other means, and various studies have demonstrated concurrent verbalization’s capacity to document participants’ processes in greater detail than other data collection methods, including observation (Aitken et al., 2011), interviews (Abdel Latif, 2019; A. E. Whitehead et al., 2015), and stimulated recall (Dempsey, 2010; Whyte et al., 2010). In short, the benefit of using concurrent verbalization is that it makes underlying processes audible and thus more apparent than they might be otherwise. Furthermore, it can be used in situ to provide contextualized views of participants’ practices, in contrast to laboratory observations or one-off interviews that may separate participants from their own realities and limit their opportunities to comment and share perspectives on their actions (Stodter & Whitehead, 2024). The use of recording equipment during a TAP or other type of data gathering session may also spark concerns associated with the observer’s paradox; however, the participants in Gordon’s (2012) study spoke freely about sensitive family matters despite audio recording their interactions, providing evidence that the presence of recording equipment does not inevitably alter the veridicality of data collected.

Reactivity is also a concern with concurrent verbalization since it may be presupposed that asking participants to speak their thoughts as they engage in activities fundamentally changes the nature of their ordinary processes and compromises the integrity of data gathered. Regarding this point, Ericsson and Simon (1993) defined three levels of TAP verbalization: when a participant is asked to (a) give voice to their inner speech (level one); (b) vocalize what they consciously attend to, including sensory information that would not typically take on verbal form (level two); and (c) express their thoughts aloud and give reasons for their actions (level three). Bowles (2010) labelled the first type of verbal reporting procedure as non-metacognitive and the latter two as metacognitive. Meta-analyses of studies employing concurrent verbalization have shown negligible reactivity effects, except in cases when participants were asked to analyze or supply motivations for their operations (Ericsson & Simon, 1993; Fox et al., 2011; Smagorinsky, 1989), actions that shifted their attentional focus from the activity at hand (Ericsson & Moxley, 2019). Nevertheless, thinking aloud can increase the time spent performing an activity (Ericsson, 2018). Yang et al. (2020) also concluded that reactivity effects may be greater in instances when participants are asked to think aloud while writing in a second language, and Charness (2021) postulated that varying degrees of reactivity on problem-solving tasks may be attributed to individual differences among participants, although this point has not been confirmed.

In broad terms, reactivity may be described as having negative or positive results. Negative effects impede or interfere with a participant’s task performance during concurrent verbalization, whereas positive effects facilitate it (Double & Birney, 2019). TW1, a participant in Atkinson’s (2013) study, seemed to make reference to a positive reactivity effect when he remarked during a post-TAP interview that thinking aloud spurred his textbook writing activity: “it’s been interesting [thinking aloud] and…it’s actually forced me to do it [write]…It’s imposed some discipline on me” (pp. 588-589). TW2, the other participant, likewise commented during a post-TAP interview that thinking aloud encouraged her to sustain writing activity: “it [the think aloud]…was rather funny…because I probably wouldn’t have sat working…for that long. I’m a terrible putterer when I’m writing…I’m very evasive” (Atkinson, 2013, pp. 729–730). Although investigators aim to mitigate the influence of reactivity effects due to concerns they may lead to inaccurate data and thus do not oftentimes elicit what Ericsson and Simon (1993) classified as level-three verbalizations, Harwood (2018) used metacognitive TAPs along with other data gathering methods to investigate what proofreaders did when reviewing a student essay and why. Sports researchers have also encouraged such verbalizations to promote reflective practice amongst coaches and players (see, e.g., McGreary et al., 2021; A. E. Whitehead et al., 2016). To illustrate, Stephenson et al. (2020) used the level-three think-aloud technique alongside other data collection methods to explore how a soccer coach reflected upon his actions whilst coaching and found that it provided him with a “meta-cognitive vantage point” from which to view his work at a distance while still being “physically immersed in the practical context” (p. 15). Though the researchers detected benefits when using the TAP procedure, including the participant’s perceived sense of self-awareness relative to his situated coaching behaviors and thoughts, they also found that he felt anxious about thinking aloud at the study’s inception and indicated that the TAP procedure sometimes prevented him from concentrating on the entire coaching environment since he was focusing on individual incidents and reflecting in the midst of them. These findings speak to the positive and negative reactivity effects associated with level-three concurrent verbalization.

Reactivity points notwithstanding, participants may still comment upon their own actions and thoughts about them during TAPs, even when they are involved in research that seeks to gather level- one verbalizations. Participant TW2 in Atkinson’s (2013) study indeed commented upon her processes and thoughts and gave reasons for them as she crafted coursebook chapters during concurrent verbalization sessions, although level-one verbalization was encouraged in the study. Rather than simply regarding these remarks as evidence of think-aloud reactivity, they might be reconceived as markers of metacognitive awareness—that is, TW2’s “knowledge and recognition of how [she] thought and worked” (Atkinson, 2020, p. 477)—and, in this way, concurrent verbalization may be viewed as a vehicle to encourage reflexivity, an individual’s ability “to examine their own feelings, reactions, and motives…and how these influence what they do or think in a situation” (Cambridge Dictionary, n.d., Social Science section). To take a case from the writing process literature that illustrates the benefits of such reflexivity, the eminent natural scientist in Rymer’s (1988) study who thought aloud while composing a journal article acknowledged that it made him more cognizant of his writing behaviors, and he was inspired to reflect as a consequence. This reflexivity was a beneficial outcome of the research, at least for the participant, since he was interested in optimizing his writing efficiency and effectiveness (Rymer, 1988). The soccer coach in Stephenson et al.’s (2020) study also wished “to learn from [his] situated experiences” and participated as a result (p. 12). The conceptual shift away from reactivity seems important to consider, particularly in the case of writing research, since writing may be construed as a means of learning (Emig, 1977; Graham, 2020; Klein & Boscolo, 2016) and since skilled writing performance demands metacognitive awareness (Hacker, 2023), and thus reflexivity, given that authors reason and confer with themselves about word choices and effective rhetorical strategies whilst keeping communicative goals in mind (Kaufman, 2013). And even after they have initially committed words to the page, they oftentimes revisit and critique their efforts (Olson, 2023). To dissuade participants from thinking aloud about such important self-negotiation during writing process research for fear of reactivity would seem to obscure its influence on writing work and inject artificiality into what might otherwise be an author’s standard means of operating.

This Study’s Focus

The points discussed thus far came into view when this paper’s authors used concurrent verbalization along with interviews to chronicle the production of Mindful Technical Writing: An Introduction to the Fundamentals (Atkinson & Corbitt, 2021), an open textbook designed for corequisite courses that emphasizes the growth of academic and workplace writing skills in concert with effective study skills. Corequisites offer dedicated learning supports for underprepared students attending university in the United States. By implementing concurrent verbalization as their primary means of data collection, the authors sought to track the behaviors of neophyte open textbook writers in detail, as they also indicated when discussing textbook genesis, data collection, and data analysis in Atkinson and Corbitt (2022a, 2023a, 2023b, 2024), but in the process discovered that TAPs can serve as spaces for reflexivity, and herein lies the niche for the current article. Atkinson and Corbitt (2022a, 2023a, 2023b, 2024) addressed the utility of concurrent verbalization to textbook development research; this paper instead concentrates on the functionality of TAPs, particularly the participant-recorded variety, to research focused on textbook production. Specifically, it addresses the following research question: What roles do think-aloud protocols serve during the development of an open writing textbook? Process-tracing studies of open textbook construction are rare (for exceptions, see Atkinson & Corbitt, 2022a, 2023a, 2023b, 2024), as are investigations that chart textbook writing episodes more generally (Harwood, 2021), and this study offers a magnified view of two novice (inexperienced) textbook writers’ efforts via data collected contemporaneous with an unfolding materials design project. It reinforces the usefulness of concurrent verbalization to coursebook research and highlights its potential for capturing reflexivity during textbook creation. In particular, concurrent verbalization exposed the open textbook authors’ actual writing behaviors, logged instances of their metacognitive awareness, and documented their project reflections, as qualitative content analysis of their datasets revealed. Textbooks influence much of what happens in courses, and the burgeoning presence of open textbooks in education warrants research into their creation to chart how they are written in practice and how authors conceive of them as learning and teaching artefacts.

Though readers may initially question the authors’ dual roles as both textbook writers and researchers, this situation was the result of convenience sampling: the authors’ inaugural coursebook writing project provided a chance to delve into how an open textbook was written in practice by initiates to the domain. Yıldız and Harwood (2024a) identify self-study as a methodological approach that can be used in textbook production research, and other writers, including Clare and Wilson (2022), DeCarlo (2023), Finlayson (2020), Hadfield (2014), Hilchey (2021), Jhangiani et al. (2016), Sternberg and Hayes (2018), and West (2019), have explored their own approaches to pedagogic materials development, helping to establish a foundation for this study, although their accounts were reflective in nature or relied on data collected through retrospective rather than concurrent means. Atkinson and Corbitt (2022a, 2023a, 2023b, 2024) made the point that detailed investigations of open textbook development can inform the efforts of aspiring authors, and this study reinforces the idea by concentrating on how concurrent verbalization can be used flexibly and may serve to stimulate writers’ reflexivity and learning during composing sessions. It also offers implications for materials development researchers and practitioners by envisioning TAPs as spaces ripe with opportunities to view reflective practice in action; for expertise researchers, it demonstrates the benefit of collecting TAP data multiple times over the course of prolonged writing projects.

Background on the Participants

The participants, referred to here as Textbook Writer Dawn (TWD) and Textbook Writer Stacey (TWS), began the research project as textbook production novices, although they had developed lesson materials throughout their teaching careers, as they also indicated in Atkinson and Corbitt (2022a, 2023b, 2024). Having earned a master’s degree in journalism and a PhD in applied linguistics, TWD had taught writing, English for academic purposes, and English as a foreign language courses at the university level for 15 years. TWS had taught university-level writing and sociology courses for 11 years; her master’s degree was in technical communication, and she had also worked in that area for 15 years. These details evidence the interdisciplinary nature of the participants’ collaborative efforts.

Despite being neophytes to the domain of open textbook construction, TWS and TWD entered into the coursebook project familiar with writing course outcomes and with an understanding of corequisite learners and teachers’ needs after having taught and observed corequisite writing classes, and this knowledge informed their textbook design principles (vide Atkinson & Corbitt, 2024) and situated their writing efforts. They had also used numerous commercial textbooks throughout their teaching careers and were interested in learning more about textbook development; indeed, TWD had investigated commercial textbook production (Atkinson, 2013, 2020, 2021, 2024, 2025), use (Atkinson & Smith Risser, 2023), and evaluation (Johnson et al., 2008). This mix of factors provided a contextual overlay when TWS and TWD embarked on open textbook research and writing work, meaning they started with an awareness of their materials evaluation, use, adaptation, and production practices of the kind expressed by Dyck (2017). They were hence poised to examine their approaches to textbook construction through a probing, critical, and reflexive lens.

Data Collection and Analysis

Prior to commencing data collection, TWS and TWD sought approval to proceed from the University of Montana’s Institutional Review Board (IRB) and gave their verbal informed consent to participate in, conduct, and publish the research. On May 10, 2019, they were granted IRB approval under the exempt review category (IRB #102-19); afterwards, they read about concurrent verbalization and reviewed transcribed TAP data from another project to prepare for think-aloud sessions.

Use of concurrent verbalization can result in richly detailed accounts of materials writing processes, as studies such as Atkinson (2020, 2021, 2024, 2025), Johnson (2003), Salisbury (2005), and Samuda (2005) have shown, but the method also has the capacity to generate an expansive amount of data that may take considerable time to transcribe and analyze (Bowles, 2019). It can also increase time spent on domain tasks during data collection periods (Ericsson, 2018). For these reasons, TWS and TWD decided to think aloud whilst constructing four chapters (two per participant) written in multiple-day installments rather than the entire 43 contained in the open textbook. They compiled a scope and sequence to blueprint chapter plans and recorded TAPs near the beginning and end of the coursebook project’s duration. To inspire textbook progress while thinking aloud during the first TAP chapter, they concentrated on topics they routinely covered in writing classes—integrating graphic elements into documents (TWS) and demonstrating academic integrity (TWD)—and concluded concurrent verbalization with lengthier chapters that required more time to complete—finding secondary sources (TWS) and writing in a persuasive manner (TWD). Hence, the sampling of chapters for concurrent verbalization was purposive (Yin, 2016, p. 93). TWS and TWD triangulated the think-aloud data by using interviews to inquire about chapter plans, impressions, and writing practices and preferences. Since some authors conceive of and rework text internally outside of physical writing sessions (Kellogg, 2018), interviews were also used to augment the TAP data and provide an added “dimension” to the study of open textbook production (Saldaña, 2011, p. 76). Although TWS and TWD had intended to collect interview data before and after the four chapters written during concurrent verbalization, the disruptive effects of COVID-19 prevented them from carrying out their final interviews. In total, they conducted four pre- and two post-concurrent-verbalization interviews.

TWS and TWD elected to audio record their own TAPs to ease data collection and facilitate timely completion of chapters, an aspect of the research design that demonstrates the versatility of concurrent verbalization as a data gathering technique (see also Bowles, 2019). Other investigations, including Atkinson (2013), Berkenkotter (1983), Salisbury (2005), and Willey and Tanimoto (2015), have similarly used participant-recorded concurrent verbalization to track writing processes. In this study, it enabled capture of writing sessions that occurred over late night and early morning periods and spanned hours in between. It also allowed TWS and TWD to collect data in their homes and offices, locations where they normally undertook writing work, so they might be encouraged to proceed with the writing project as they would any other.

Writing studies generally preference data collected under naturalistic conditions (Hyland, 2022, p. 98), and by selecting which textbook chapters to write while thinking aloud, composing at times and in spaces they chose, and self-recording TAP sessions, TWS and TWD aimed for a research design with naturalistic parameters to counteract what Murray (1983) experienced as the negative reactivity effects of concurrent verbalization upon a participant’s typical writing processes. Even so, they entered into the research project fully aware that their vocalizations were being recorded, as the participants in Gordon’s (2012) study of family discourse also did, and were therefore cognizant that they were opening up their writing practices to personal and public scrutiny to learn more about them. The opportunity to advance understanding of how textbooks are written in practice offset any apprehension they had about such exposure.

To “reexperience” and begin analysis on the dataset, TWS and TWD transcribed it themselves, as Saldaña and Omasta (2018, p. 115) recommended. The months-long open textbook project had removed the authors from the immediacy of individual data collection sessions after they concluded since their attention turned to future chapter production, but the immersive experience of transcription enabled TWS and TWD to become reacquainted with the dataset (see also Cheek & Øby, 2023, p. 158). The concurrent verbalization sessions, semi-structured interviews, and subsequent transcription yielded 897 double-spaced pages of data, which can be accessed in Figshare at https://doi.org/10.6084/m9.figshare.18131150 (Atkinson & Corbitt, 2022b).

TWS and TWD entered into their textbook writing and research project with some underlying knowledge of the materials development and concurrent verbalization literature, which informed their understanding as they applied qualitative content analysis to the dataset in an inductive fashion. Selecting a data collection method that optimally addresses the research question at hand is vital for ensuring trustworthiness when using this form of analysis, according to Elo et al. (2014), and applying qualitative content analysis to TAP data to address the functionality of concurrent verbalization aligned with that objective. Outside coders could not be employed due to project funding limits, so TWS and TWD coded the data themselves; Hadfield (2014) and Salisbury (2005) likewise functioned as both participants and investigators in their materials development research, helping to establish a basis for these tandem roles in the current study. TWS and TWD began by reading through the entire dataset twice before bracketing the data by meaning unit and applying descriptive labels to the contents. Parsing the transcripts by meaning unit meant that codes could be applied to short stretches of text with variable word counts that held manifest (surface-level) or latent (subtextual) meaning regarding open textbook production. Coding was done by hand to develop a close sense of the data’s meanings, and analysis proceeded in phases, beginning with manifest-level codes and proceeding on to latent-level ones as TWS and TWD gained further familiarity with the data, although these planes sometimes overlapped during recursive coding cycles, as Vaismoradi and Snelgrove (2019) described. Coding inspired thoughts about coalescing themes and outliers in the dataset, which, in turn, sparked residual reflexive thoughts about textbook writing experiences. To capture their insights about the developing themes, TWS and TWD made analytic jottings, as Miles et al. (2020) advised. To illustrate, TWD wrote the following jotting as coding pointed to the theme of concurrent verbalization functions in the dataset: “‘talking it out’ helps TWS refine her ideas for chapter → TAPs as a conceptual space to work out plans for content creation (refers to page 31 in TWS’s 2^nd TAP).” Here is another one of TWD’s jottings:

TAP serves as a planning space for textbook development: for emerging ideas and reflections on plans (refers to page 394 in TWD’s 2^nd TAP). It also serves a reflective diary function (e.g., pages 249-253, 347, 388, 390, 429 in TWD’s 2^nd TAP): i.e., a way of looking under the bonnet of a chapter’s design, as TWD demonstrates when discussing the rationales for textbook materials decisions.

TWS and TWD recognized the importance of agreeability checks in helping to build confidence in analytic procedures. Hence, they met two different times to come to agreement on codes, themes, and areas of analytic prominence in the data: after they finished the initial TAPs, pre- and post-concurrent-verbalization interviews, and accompanying transcripts, and again after they finished writing the textbook and transcribing the second set of TAPs and interviews so they could revisit and update the initial codes and themes based on those that surfaced during later data analysis. When meeting, TWS and TWD focused on sections of data rather than the entire dataset given its considerable size, and their discussion systematized directions for further analysis. As they progressed toward higher levels of abstraction during analysis and made headway on delineating salient themes in the data, they composed analytic memos, as Yin (2016, p. 195) recommended, to record their ideas regarding the data’s meanings. One of TWD’s memos can be found in Appendix A; it speaks to the presence of metacognitive comments in TAPs. These analytic procedures exposed a number of concurrent verbalization functions during textbook writing episodes, as indicated here. When appropriate, the findings are mentioned alongside relevant literature to contextualize their coverage, as Flick (2018, p. 155) suggested.

Concurrent Verbalization Chronicled Actual Behaviors During Textbook Writing Sessions

Concurrent verbalization has the capacity to track writing practices in real time and is therefore valuable for detailing actual rather than reported behaviors during open textbook construction. In other words, it can document what authors do when working as opposed to what they believe they do, as Samuda (2005) articulated, and the current study illustrated the legitimacy of this point. For instance, during her first pre-concurrent-verbalization interview, TWS articulated that she was “a plan while writing person,” though she also said she typically began writing with “a general outline in mind.” In the interview, she also conveyed that her outline for a chapter titled “Integrating Graphic Elements” consisted of six major sections she would incorporate. As TWS subsequently thought aloud while constructing the chapter, she revealed the considerable amount of detail present in her plan for these sections—including defined headings—and prefaced her description of the sections with the following comment: “as I think about this process [of chapter creation] one thing that matters the most to me is that I understand the framework that I want to try and use.” During her second pre-concurrent- verbalization interview, TWS emphasized the importance of “scoping” (a code that referred to establishing chapter and section parameters) to her actual writing procedure and conveyed how thinking aloud while writing helped her discover the degree of specificity she needed in chapter plans to facilitate chapter production. She commented that when producing the “Integrating Graphic Elements” chapter

I had a plan (planning) together and because…it was…a longer chapter (length) than maybe I thought it would be (changing plans) when I got through the…concurrent verbalization I came to the realization (realizing)…that there were detailed subsections (structure)…I needed to include (covering what I need to) that I hadn’t really thought about as I was thinking and writing. So really what I…came away from that with (self-assessment) was the planning (planning) should involve…the scoping (scoping) a lot more than anything else.

Here, ellipses mark the omission of irrelevant content (e.g., word repetitions), while the parenthetical text in this first extended transcript excerpt exemplify how the data were coded. Associated coding definitions can be found in Appendix B.

Reflective of TWS’s realization regarding scoping, she said she would plan a future chapter, titled “Identifying Secondary Sources,” in some level of detail before writing it. This cascading account of the particulars TWS built into her chapter schematics conveys that she in fact planned prior to writing, despite her initial contention that she did not; many of the professional coursebook writers in Johnson’s (2003) study likewise operated differently than they anticipated as they thought aloud whilst building a task for English language teaching. TWS’s data highlighted the usefulness of concurrent verbalization in verifying writing behaviors during composition sessions, but they also evidenced how an author might develop enhanced insight into their own processes through the act of thinking aloud while writing.

TAPs Recorded Instances of Metacognitive Awareness During Textbook Production

Concurrent verbalization can document authors’ actual behaviors during writing episodes, as the previous paragraph explained, and it consequently captured instances of TWS and TWD self-monitoring while composing chapters. During one such instance, as TWD wrote a chapter focused on academic integrity and established how the textbook might be used in classes, she expressed how she would use page design elements to encourage students to respond to case studies about cheating behaviors and prepare group presentations. As TWD deliberated over whether to incorporate blank text boxes into that section as a mechanism for signaling that students should make notes prior to the presentations, she said, “Sometimes I have to figure things out by talking about them,” a statement that evidenced her conscious awareness of thinking aloud while writing. Frustrated with her lack of eloquence as she defined the word “argument” while thinking aloud and composing the “Writing to Persuade” chapter, TWD again demonstrated that awareness by saying, “this is so silly sometimes when you’re trying to say something that’s so simple and you can’t get it out” before she went on to refine the text. When TWS wrote the “Integrating Graphic Elements” chapter during her first TAP and tied its theme to ethical source use, she contemplated how she might approach "academic integrity…in every single chapter” by relating it to current events and then commented, “I just wanted to get that out of my head.” TWS similarly expressed during her second TAP that “talking it out” helped her establish content boundaries for the “Identifying Secondary Sources” chapter and a related one titled “Selecting Secondary Sources.” These examples demonstrate that TWD and TWS used concurrent verbalization to work through the issues they encountered during textbook development, but they also reflect the authors’ cognizance of data collection. Some of the participants in Johnson’s (2003) pedagogic task design research similarly philosophized about their work in their TAPs, as did TW2, one of the expert participants in Atkinson’s (2013) study of commercial textbook production. Instead of underscoring the reactivity effects of concurrent verbalization, these moments might be seen to reflect the participants’ metacognitive awareness as writers. As further justification for this conceptual shift, TWD’s dataset demonstrated that despite the presence of recording equipment during TAPs, as well as her talking while writing, concurrent verbalization did not constrain chapter completion. That is because she was able to switch between internal and external articulatory functions to encourage text production and refinement, a practice that reflected her ordinary means of working. To delve further into this point, TWD said during an interview that she read over text she produced repeatedly when writing to get a sense of where she was in terms of project objectives and establish momentum for subsequent text production and revision (Atkinson & Corbitt, 2023b)—evidencing her metacognitive awareness of her writing processes—and she proceeded in this manner when writing chapters during concurrent verbalization. The concentration required during chapter production, a constituent component of deliberate practice (Ericsson et al., 2018), nonetheless could test TWD’s capability to simultaneously review text already written, revise it, and produce new text. Hence, while composing a chapter titled “Writing to Persuade” during concurrent verbalization, she both read existing text about thesis statements and research questions out loud during her TAP and conveyed that she would “have to read [it] again silently,” that is under her breath, to proceed with chapter development. TWD thus monitored her actions whilst writing and adjusted as necessary to encourage progress during an intensive and lengthy writing project. TWS similarly signaled the taxing effects of deliberate practice upon her writing efforts—a circumstance also noted by Kellogg (2018) and participant TW2 in Atkinson’s (2013) research—when she said “I’m starting to get a little bit loose in the brain” and “I’m starting to fade” as she composed the “Integrating Graphic Elements” chapter during concurrent verbalization. With this knowledge, TWS ended the TAP segments shortly thereafter, although she indicated she would later revisit and refine the chapter’s material. To bring the threads of this paragraph together, the open textbook authors displayed metacognitive awareness during textbook production, but this factor did not necessarily compromise the integrity of data collected nor indicate they were unduly reactive, and TWD and TWS proceeded accordingly during intensive periods of deliberate practice.

Self-recorded concurrent verbalization also captured instances of TWD and TWS acclimating to and embracing the process of thinking aloud during open textbook production and offered another glimpse of their metacognitive awareness. On these occasions, the authors spoke openly about their thoughts, reinforcing the idea that concurrent verbalization can function as a running record—or a type of journal, as participant TW2 in Atkinson’s (2013) study demonstrated—that captures the exacting detail of textbook writing episodes. When TWS recorded her first concurrent verbalization session, for instance, she was clearly concerned about being accountable to the think-aloud process (coded as such in the data), and she thus commented on getting accustomed to vocalizing her ideas while composing the “Integrating Graphic Elements” chapter. As TWS discussed the content of a section called “Problem or Opportunity” and thought about a possible case study to prompt students’ incident report deliverables, she said,

I feel like I should apologize because…I haven’t quite nailed down exactly what I wanna use here but then again I don’t feel like I should have…written it and then go back you know and describe…the process so…I’m really thinking out loud right now…I didn’t know [if] I could do it.

This transcript excerpt features TWS speaking in monologue fashion about her ability to verbalize concurrently, and later during the same TAP, she again referenced the behavior when deliberating about the role of checklists in the chapter:

for purposes of…this chapter it might make sense to build into this section these checklists…in terms of a rubric…for a student to…be able to measure their own success in…integrating graphic elements…into their documents….and that’s what I think I’m gonna develop here…I’m…literally thinking in real time.

Both examples evidence concurrent verbalization’s capacity to chronicle participants’ thoughts and accompanying behaviors as they occur. Further along into the think-aloud session, as TWS continued to ponder the checklist matter as it related to the deliverable, she came to the conclusion that she would incorporate a rubric and example into the chapter and commented, “Yep I definitely added another session onto this recording…I’ve actually developed…some good…insights that I hadn’t seen coming in this process and…have some action items here.” In the interview following TWS’s first concurrent verbalization session, the author noted the value of gaining an enhanced metacognitive perspective by virtue of thinking aloud while writing:

I enjoyed every…bit of that [concurrent verbalization] it was something…I’ve never done before and I was not overly worried about it…I’m very comfortable with the fact that…you do it and the data comes out of it…and I found…there’s a…fair amount of discovery in that process for me in that I don’t have hang ups about listening to my own voice in a room by myself.

The data extracts in this paragraph showcase the detail present in TWS’s dataset. They also evidence concurrent verbalization’s ability to capture and even spark reflexivity during data collection as participants turn their exploratory and considered gaze inwards to examine their actions, motivations, and associated feelings with respect to materials development.

Concurrent Verbalization Served a Reflective Diary Function During Textbook Development

In addition to the functions of concurrent verbalization noted thus far, TWD’s dataset displayed that the author sometimes used TAPs in a reflective diary-like manner to detail her progress on the textbook and explain why she did what she did in chapters, again evidencing reflexivity. The TAPs consequently offered her an opportunity to unpack actions and decisions taken while writing past chapters as a way to commence further work on the textbook. A case in point: when TWD began thinking aloud whilst composing the “Writing to Persuade” chapter, she acknowledged the “need to…reflect a little bit on what’s happened…with other chapters,” which were written in between concurrent verbalization sessions. Over the course of 27 transcribed pages, TWD then overviewed how she fashioned several chapters and the rationales for her decisions before she took a break to rest. When she recommenced work on the chapter several hours later, she discussed her feelings about open textbook development: “this…project has just been…very engrossing…I really do like this whole process…you can insert…your teaching…approaches…into the textbook…giving students a fresh way to look at things. Or a fresh presentation of information.” TWD then said, “I’m starting on this ‘Writing to Persuade’ chapter from scratch,” and she indicated that her plan was to “talk about what arguments are and what…the components of arguments are…before [going] into how to write a researched argument paper.” Afterwards, she crafted the chapter’s introduction, which she used to “outline…the rest of the chapter,” as she indicated was her standard practice (see also Atkinson & Corbitt, 2023b, p. 771). Thereafter, whenever she was away from the “Writing to Persuade” chapter for more than a brief break or refined chapter material during the TAP that she had already written off recording, she followed a similar pattern of reflecting on what she had done before recommencing writing activity, generally by reviewing what she had previously written to initiate succeeding efforts. Repeated occurrences of this sequence led to the full development of the chapter over the course of 458 transcribed pages of data. Juxtaposing this study with Johnson’s (2003) research, in which participants thought aloud in a laboratory setting with a researcher present while developing one pedagogic task in response to a design brief, TWD’s scenario exhibited the value of collecting self-recorded concurrent verbalization data over prolonged time spans during multiple materials writing episodes to expose how an author utilized the think-aloud technique to reflect on previous work in order to pave the way for subsequent writing efforts.

Discussion and Implications for Researchers

Atkinson (2020, 2024) and Atkinson and Corbitt (2022a, 2023a, 2023b, 2024) commented on concurrent verbalization’s ability to track materials writing processes in great detail; this study reinforced the strength of that conclusion and spoke to the functionality of TAPs within the context of an open textbook project. In particular, the investigation sought to answer the following research question: What roles do think-aloud protocols serve during the development of an open writing textbook? Validating Samuda’s (2005) assertion that concurrent verbalization can define what participants do while writing materials rather than what they assume they do, the data gathering technique helped establish that TWS planned her chapters in some detail before writing them, for example. And corroborating Stephenson et al.’s (2020) contention that concurrent verbalization offers participants a venue for reflexive thought about their operations as they are engaged in them, TWS also gained an enhanced sense of her own planning processes as a result of thinking aloud while writing and reflecting on that activity. TWD likewise reflected on her operations, at times during concurrent verbalization, and those instances informed and spurred her further efforts on the textbook. In summary, concurrent verbalization cataloged TWS and TWD’s actual writing behaviors, instances of their metacognitive awareness, and reflections on their ensuing work, and it offered a view of reflexivity markers in the novice textbook authors’ datasets. Such reflexivity is valuable to open textbook production as writers seek to heighten the effectualness of their books for students and teachers outside their immediate frames of reference. They may self-monitor, for instance, as they balance their own preferences for textbook content and page design with the needs and desires of other users, as TWD illustrated while composing a chapter focused on academic integrity.

This study provides a look at open textbook composition as viewed through the medium of concurrent verbalization, and it may demystify the method for others interested in conducting process-tracing investigations of coursebook production—a valuable but seldom-used approach in materials development research (Harwood, 2017). The open textbook authors recorded their own TAPs and selected which chapters to write during concurrent verbalization, demonstrating that the method can be used in a flexible manner to address research questions. Furthermore, participants can customize their concurrent verbalization processes to encourage progress whilst thinking aloud during an extended writing project, such as when TWD exhibited internal and external articulatory behaviors during TAPs. These points exemplify that concurrent verbalization procedures can be tailored to fit research objectives and contextual circumstances, expanding the catalogue of methodological techniques available to researchers (see also Bowles, 2019). For individuals interested in learning more about how open (and commercial) textbooks are developed in practice, this observation presents a veritable wealth of possibilities, particularly in light of the fact that textbook production remains perennially under-researched (Yıldız & Harwood, 2024b) and textbook writers’ voices are “woefully underrepresented” in the materials development literature (Norton & Buchanan, 2022, p. xix).

From an expertise perspective, the research design used in this study afforded an intimate view of the demands that deliberate practice can place on practitioners and how they may respond—and engage in reflexive thought—over successive data collection sessions. Implementing a naturalistic approach to data collection meant that TWD and TWS selected which chapters of their textbook to write during concurrent verbalization, decided the times and locations for their writing sessions, and self-recorded their TAPs; they also gathered data during multiple think-aloud and interview sessions. The momentum for data collection and chapter development consequently ran in parallel and accelerated as the textbook project progressed. Nevertheless, the rigors of deliberate practice still weighed upon the novice textbook authors’ capacities to proceed with writing work, and TWD and TWS devised their own techniques to meet the challenge, as this paper conveys. TWD used internal and external articulatory functions to counterbalance the cognitive demands of several simultaneous operations—thinking aloud, reviewing text, revising text, and drafting chapter material—while TWS concluded concurrent verbalization and writing work when fatigue posed a barrier to chapter progression. TWD and TWS self-monitored their progress and vocalized about their strategies to maintain it, evidencing reflexivity in their datasets, but the authors also deployed, self-evaluated, and refined the strategies over succeeding periods of data collection and reflected on their actions, demonstrating the merit of gathering data over multiple stages of materials development. Unlike Johnson’s (2003) research, which employed concurrent verbalization to log participants’ writing behaviors during a single task design session, these researchers perceived value in documenting materials writers’ practices over time as they worked on an open textbook project to ascertain how they balanced deliberate practice with project progression and reflected on their endeavors in the process.

TWS and TWD acclimated to writing while thinking aloud during data collection, as their TAPs evidenced, and though some readers might view this circumstance as proof of concurrent verbalization’s reactivity effects, in reality, participants may similarly need to adjust to other methods of data collection, such as focus group participation, as Johnson et al. (2008) also observed. Getting accustomed to data collection procedures might thus be construed as a typical aspect of a participant’s involvement in research rather than an unusual circumstance that threatens the integrity of data, and in this study, self-recorded concurrent verbalization allowed the authors to adjust to the think-aloud process unencumbered by another individual’s presence. Moreover, although the participants self-monitored as they adjusted to concurrent verbalization, this factor did not adversely affect their ability to think aloud in order to generate data about their operations while producing textbook content. The open textbook authors’ comments regarding acclimation ultimately offered a window into how they conceived of their own behaviors during data collection, as they associated thoughts with actions during reflexive moments.

To extend the points made thus far, self-recorded TAPs need not be thought of as echo chambers in which authors express their thoughts aloud in stilted, performative style; such practice might actually negate the opportunities for growth that Rymer (1988) and Stephenson et al. (2020) found concurrent verbalization can offer participants. In the spirit of Emig (1977), Graham (2020), and Klein and Boscolo (2016) who framed writing as a means of learning, we posit that self-recorded concurrent verbalization within a naturalistic research design can offer participants a platform to engage reflexivity as they compose text for their own projects, and achieve learning in the process. Moreover, conceptualizing think-aloud protocols as spaces for reflexivity positions participants as agentive contributors to the research process: thoughtful, reflective individuals whose motivations, challenges, and discoveries feed into thoughts expressed during concurrent verbalization. The current study revealed that a host of predicted and unexpected considerations came into play during the real-time construction of open textbook chapters—ones that would ultimately affect the textbook’s composition and users’ experience with its content—and TWS and TWD’s self-recorded concurrent verbalization sessions yielded insight into their handling of these matters and, further, how they conceived of them in reflexive terms. With textbook users in mind, the authors forged ways forward with their project as they thought aloud while writing, and they made metacognitive remarks regarding their realizations (i.e., learning) as they examined these instances in an afferent manner. By ascribing meaning to the occasions, they thereby engaged in the type of reflexive analysis that Saldaña (2018) has contended is vital to qualitative research.

Implications for Materials Development Practitioners

For educators interested in undertaking open textbook production, this study offers several practical implications that may be of use. Firstly, the investigation opens a window onto writing sessions to expose the concerted thought and effort that textbook development demands, and it addresses the value of reflexivity to such projects. Similar to Yıldız and Harwood (2024a) who sought to demystify local textbook production through their research, it may therefore help to inform other open textbook writers’ efforts. The study also illustrates how self-recorded concurrent verbalization can chronicle reflexivity during writing episodes; fellow open textbook writers might also use TAPs to document their writing practices as a way to look behind the pages that comprise their chapters. Such highly detailed records of writing sessions may ultimately enable materials development practitioners to gain valuable perspective and learn from their work as they seek to refine their skills, as did the participants who thought aloud in Rymer (1988) and Stephenson et al.’s (2020) studies. In addition, educators who wish to involve their students in open textbook production could ask the learners to self-record concurrent verbalization sessions as they write textbook content and then review the TAPs to encourage the metacognitive awareness essential to the growth of proficient writing skills (Hacker, 2023). Students could also use TAPs as needs-analysis tools (see also A. E. Whitehead et al., 2025) by reviewing one another’s think-aloud protocols and comparing their writing practices, which may help them identify and experiment with other approaches in future writing projects (vide Birch et al., 2022). Taking a cue from A. Whitehead’s (2023) work in the area of sports coaching, teachers might participate in this exercise by sharing their own TAPs as a means of mentoring through “critical friend support” (p. 123).

Limitations and Recommendations for Further Textbook Production Research

The findings presented here coalesce around the roles that think-aloud protocols served during the production of an open writing textbook, though we recognize that other researchers may have come to different conclusions about the dataset considering the interpretive character of qualitative data analysis (Saldaña, 2016). We selected certain data extracts over others to illustrate the study’s findings with respect to the research aim and note that the presentation of data is therefore limited in scope and reflects our interpretation of the evidence (see also Saldaña, 2025, p. 4). Atkinson and Corbitt (2022a, 2023a, 2023b, 2024) discuss additional findings from the dataset. Given that textbook production continues to be under-investigated (Yıldız & Harwood, 2024b), these acknowledgements point to the value of conducting additional empirical inquiry to further probe the roles that TAPs play during extended materials writing projects, including those focused on open textbook production.

Differing research objectives may have also taken the investigation in other directions. For instance, this paper did not consider open textbook production within the larger discussion of open pedagogy and its potential for transforming education, nor did it address generative artificial intelligence with respect to open textbook development, although both of those areas offer avenues for future research. The present account instead focuses on the functionality of concurrent verbalization in a study of open textbook production and brings in the perspectives of two novice textbook authors to help redress what Norton and Buchanan (2022) have observed as the scarcity of such voices in the materials development literature.

Concurrent verbalization helped track writing sessions in detail, but it also brought reflexivity markers into view—an observation that served as the impetus for this article. Future process-tracing studies of textbook production may likewise highlight instances of reflexivity as the participants examine their own actions, motivations, and feelings while thinking aloud, offering the potential to better understand how pedagogic materials are produced in practice from their authors’ perspectives.