Abstract
This paper presents a comprehensive methodological framework for evaluating blended learning in vocational contexts through multi-stakeholder qualitative inquiry. Drawing on an evaluative case study of vocational English training in Saudi Arabia’s oil and gas industry, the study demonstrates how systematically gathering perspectives from learners at different stages (current students and post-academic trainees) alongside their instructors (teachers and technical trainers) generates developmental insights into learning progression and skill transfer through a multi-cohort cross-sectional design. Grounded in Social Constructivism and Situated Learning theories, the research utilised semi-structured interviews, focus groups, and email-based qualitative questionnaires, analysed through systematic coding and theme development. The research produced a five-dimensional evaluation framework encompassing modality integration, skills development, learning environment, technology effectiveness, and implementation effectiveness. This framework fills critical methodological gaps by moving beyond single-source evaluations and purely quantitative assessment towards comprehensive multi-perspective inquiry. The study discusses methodological considerations including insider researcher positioning, trustworthiness strategies, and ethical dimensions specific to workplace-based educational research. The framework offers transferable guidance for researchers and practitioners seeking to evaluate technology-enhanced vocational education programmes comprehensively while honouring the experiential knowledge of multiple stakeholder groups. The paper’s primary contribution is methodological, grounded in and demonstrated through original empirical inquiry: it shows how theoretical frameworks can guide systematic multi-stakeholder evaluation whilst maintaining responsiveness to contextual particularities.
Keywords
1. Introduction
Educational evaluation faces an ongoing methodological challenge: how to assess programme effectiveness in ways that are simultaneously theoretically grounded, contextually sensitive, and practically actionable (Saunders, 2011; Weiss, 1998). This challenge intensifies in vocational education contexts, where effectiveness ultimately depends on workplace readiness — a complex outcome shaped by interactions between pedagogical approaches, learner experiences, and authentic professional demands (Billett, 2011). Whilst quantitative metrics offer measurement precision, they often overlook the nuanced processes through which stakeholders experience and make sense of educational innovations (Darling-Hammond et al., 2020). This paper addresses this methodological gap through an evaluative case study of vocational English training in Saudi Arabia’s oil and gas industry, presenting a theory-grounded qualitative framework that demonstrates how Social Constructivism and Situated Learning can guide systematic multi-stakeholder inquiry to generate insights that complement quantitative evaluation whilst providing actionable guidance for programme improvement.
The research responds to calls for evaluation approaches that position stakeholder experience as primary evidence of effectiveness (Weiss, 1998) while maintaining methodological rigour through systematic evidence gathering, analysis, and trustworthiness strategies. Unlike traditional evaluation models that provide generic structures applicable across contexts (Kirkpatrick & Kirkpatrick, 2006), this study’s framework offers specificity to blended learning for vocational purposes while remaining transferable to similar contexts through clear documentation of methodological decisions and theoretical grounding.
1.1. The Evaluation Challenge in Vocational Blended Learning
Vocational education serves a fundamentally different purpose than general academic education: preparing learners for specific professional roles through development of immediately applicable competencies (Guile & Unwin, 2019). This practical orientation creates unique evaluation challenges. Traditional assessment approaches focusing on test scores or completion rates may indicate academic achievement without revealing whether learners can actually apply knowledge in workplace contexts (Orr, 2019). Similarly, satisfaction surveys capture affective responses without illuminating the processes through which learning occurs or fails to occur.
These challenges intensify in blended learning contexts, where instruction across modalities creates additional complexity (Garrison & Vaughan, 2008). Learners, teachers, and technical trainers each experience the model differently, hold distinct knowledge about its effectiveness, and can identify different opportunities for enhancement.
Existing evaluation research in blended learning has tended towards two problematic extremes. Experimental and quasi-experimental designs comparing learning outcomes across modalities (Means et al., 2010, 2014) provide statistical evidence but cannot reveal how and why particular approaches work or fail in specific contexts. Conversely, satisfaction surveys and single-source qualitative studies capture stakeholder perspectives but may lack the theoretical grounding and methodological rigour necessary to produce transferable insights (Harrison et al., 2017).
1.2. Towards Theory-Grounded Multi-Stakeholder Evaluation
This paper argues for evaluation that is simultaneously theory-grounded, multi-stakeholder, process-oriented, and context-sensitive. The framework operationalises these principles through qualitative case study methodology (Bassey, 1999; Stake, 1995; Yin, 2018), with Social Constructivism and Situated Learning providing conceptual foundations that direct attention towards social knowledge construction and authentic workplace connections (Lave & Wenger, 1991; Vygotsky, 1978). This approach generated a five-dimensional evaluation framework that extends beyond the specific case studied, providing structure for understanding blended learning effectiveness whilst remaining adaptable to varied contexts.
1.3. Research Questions
Two empirical research questions guided this inquiry:
1.4. Paper Structure
Section 2 reviews relevant literature on case study evaluation and technology-enhanced learning. Section 3 details the research design. Section 4 presents the five-dimensional framework. Section 5 discusses findings in relation to both research questions, addressing framework universality, practical implications, and future research directions.
2. Literature Review: Positioning the Methodological Approach
2.1. Qualitative Case Study in Educational Evaluation
Case study methodology has long been recognised as particularly suited to educational research, enabling in-depth investigation of complex social phenomena in natural settings while providing rich, holistic descriptions that capture nuances often overlooked by other approaches (Bassey, 1999; Merriam, 1998; Stake, 1995). Bassey’s (1999) definition proves especially relevant for evaluative purposes: evaluative case studies examine bounded educational phenomena within natural contexts, maintaining ethical respect for participants whilst explicitly aiming to inform practical decision-making rather than only contributing to theoretical knowledge.
The flexibility of case study design allows examination of contextual factors that shape educational processes — organisational culture, workplace practices, infrastructure conditions — which may prove as influential as the educational model itself (Tight, 2017). This contextual sensitivity becomes particularly valuable in vocational settings, where learning occurs within communities of practice that bridge educational and professional environments (Fuller, 2007; Lave & Wenger, 1991).
However, case study research faces persistent criticisms regarding generalisability, researcher bias, and potential lack of rigour (Merriam, 1998; Yin, 2018). These critiques necessitate careful methodological design and transparent documentation of research processes. This study addresses these concerns through multiple strategies detailed in Section 3.
2.2. Multi-Stakeholder Perspectives in Educational Research
Educational programmes involve multiple participant groups whose experiences and perspectives may align, complement, or contradict each other (Cousins & Whitmore, 1998). Single-source evaluations risk providing incomplete or distorted understanding of programme effectiveness by privileging one viewpoint over others. In vocational blended learning contexts, at least four distinct stakeholder groups hold valuable evaluation knowledge: 1. 2. 3. 4.
Multi-stakeholder approaches align with participatory evaluation frameworks (Cousins & Whitmore, 1998) and recognise evaluation as inherently social, focused on “attributing value or worth” to educational activities (Saunders, 2011, p. 3). However, this study’s approach differs from purely participatory models in maintaining researcher control over design and analysis while systematically incorporating stakeholder perspectives as primary evidence. Research on blended environments shows that learners’ inquiry approaches vary substantially with prior experience and disciplinary context (Ellis & Bliuc, 2016), underscoring the need to gather diverse stakeholder accounts. Recent empirical work reinforces this: Sharma et al. (2024) found that learners, faculty, administrators, and industry professionals each contributed qualitatively distinct dimensions of understanding that no single group could have provided independently. A systematic review of 39 international studies confirms the methodological gap, with most research examining only one stakeholder group at a time (Sareen & Mandal, 2024). The same conclusion emerges across sectors: Janes et al. (2023) identified the absence of ‘comprehensive multi-stakeholder, exploratory evaluation’ as a critical gap in pre-registration nursing and midwifery education, and Coe et al. (2025) similarly concluded that educator-only evaluation produced incomplete findings about blended learning outcomes.
2.3. Evaluating Technology-Enhanced Learning: Existing Approaches
Evaluation research in technology-enhanced learning has evolved considerably over recent decades. Early studies often focused on proving whether technology-based approaches were ‘better than’ traditional instruction, employing experimental designs comparing learning outcomes across conditions (Means et al., 2010). Meta-analyses revealed modest positive effects for technology integration while highlighting substantial variation based on implementation quality and contextual factors (Means et al., 2014).
More recent evaluation has shifted towards understanding how and why technology-enhanced learning works (or fails) in specific contexts, recognising that effectiveness depends less on technology itself than on pedagogical integration and alignment with learning goals (Garrison & Vaughan, 2008; Graham, 2013). Data-driven approaches such as lag sequential analysis (Yang & Ogata, 2023) and longitudinal self-regulation profiling (Esnaashari et al., 2025) offer granular insights into learner–technology interaction, complementing — but not replacing — the stakeholder perspectives and contextual meanings that qualitative case study methods provide.
Nevertheless, significant gaps persist in evaluation approaches for vocational contexts. Tempelaar et al. (2021) illustrate how learning analytics applications must link predictive data to actionable interventions, a principle that extends to evaluation design more broadly. Much existing research examines higher education settings where learning objectives centre on knowledge acquisition rather than immediate workplace application (Billett, 2011); vocational training has tended to prioritise generic compliance criteria over sector-specific, experiential dimensions that authentically prepare learners for professional roles (Caner, 2026).
Systematic reviews highlight persistent challenges in blended learning’s online components, including technical difficulties, insufficient interaction, and inadequate self-regulation support (Rasheed et al., 2020). Brezowar et al. (2026) extend this picture by demonstrating how multi-stakeholder co-design can surface contextual conditions shaping hybrid vocational learning in ways that single-perspective evaluation cannot. However, evaluation methodologies have consistently struggled to capture stakeholder experiences comprehensively, particularly in vocational contexts where authentic workplace preparation remains the ultimate measure of success. This study’s framework addresses these gaps through systematic qualitative evaluation that positions multiple stakeholder perspectives as complementary, maintains theoretical grounding, and connects findings to actionable guidance.
2.4. Theoretical Foundations and Their Influence on Research Design
Evaluation research requires theoretical foundations that structure inquiry without predetermining findings (Anfara, 2008). This study employs two complementary theoretical perspectives: Social Constructivism and Situated Learning. Empirical applications of Social Constructivism in blended learning evaluation demonstrate the value of this theoretical lens. Du Preez and West (2022), using the Community of Inquiry framework as a Social Constructivist lens to evaluate student-teachers’ experiences of blended learning, found that whilst cognitive and teaching presences were well-supported within the blended environment, social presence required deliberate design attention — a finding that illustrates how theory-grounded evaluation reveals dimensions of programme effectiveness that atheoretical approaches would overlook.
Social Constructivism (Pritchard & Woollard, 2010; Vygotsky, 1978) emphasises that knowledge is actively constructed through social interaction rather than passively received. In blended learning contexts, this perspective directs evaluative attention towards collaborative knowledge construction, the mediating roles of teachers and peers, and the quality of meaning-negotiation across modalities (Ellis & Bliuc, 2016).
Situated Learning (Brown et al., 1989; Lave & Wenger, 1991) posits that learning occurs through participation in communities of practice, sensitising evaluation to the authenticity of activities relative to workplace demands, learner progression from peripheral to fuller participation, and whether academic knowledge transfers to vocational application. These theories function not as rigid templates but as sensitising frameworks that guide attention towards theoretically significant dimensions while remaining open to unanticipated themes emerging from stakeholder accounts (Blumer, 1954; Braun & Clarke, 2019). Their combined influence on specific methodological decisions — from participant selection to data collection instruments to analytical categories — is traced directly below and operationalised in Section 3.
Social Constructivism and Situated Learning shaped the research design in direct and traceable ways. The constructivist premise that knowledge emerges through social interaction necessitated the multi-stakeholder approach: understanding how blended learning functions requires capturing perspectives from all participants in these social learning processes, not privileging any single viewpoint. It also justified the choice of qualitative methods, since understanding how stakeholders construct meaning across modalities requires access to their subjective experiences through their own accounts — something quantitative measures of test scores or completion rates cannot provide. Situated Learning’s emphasis on progressive participation in communities of practice shaped the multi-cohort cross-sectional design: gathering perspectives from both current students and graduated trainees was theoretically necessary because competence in vocational contexts is only visible when learners engage in authentic workplace activities, not only when they complete academic tasks. Together, these commitments required case study methodology, which enables deep, contextualised examination of social and educational processes within specific bounded settings (Stake, 1995; Yin, 2018). This explicit theory-to-method connection addresses methodological scholarship emphasising coherent alignment between philosophical commitments, theoretical frameworks, and methodological choices (Crotty, 1998).
3. Methods
3.1. Philosophical Underpinnings
The research adopts an interpretivist epistemology, holding that knowledge is constructed within human minds and varies according to individual experience (Moon & Blackman, 2014). This position aligns with the study’s focus on understanding how different stakeholders experience and make sense of blended learning, recognising that multiple valid interpretations may coexist. Ontologically, the research assumes relativism: that reality is constructed through individual and social meaning-making processes rather than existing independently of human interpretation (Crotty, 1998). This stance acknowledges that understanding social phenomena requires examining “the standpoint of individuals who are part of the ongoing action” (Cohen et al., 2018, p. 17). These commitments justify the qualitative, multi-stakeholder, theory-sensitised design adopted in this study, with theory functioning as a sensitising lens rather than a hypothesis-testing framework.
3.2. Case Selection and Bounding
The case was deliberately selected for several reasons: its scale and impact, operating across seven training centres serving approximately 2,000 annual learners; its intentional design as a purpose-built curriculum for blended delivery rather than emergency adaptation; its vocational specificity, with a clear connection between language learning and workplace communication demands in oil and gas; the presence of an evaluation gap, with existing quantitative measures but no qualitative assessment; and a developmental stage sufficiently mature for meaningful evaluation yet still amenable to enhancement. Case boundaries spanned four dimensions: temporal (December 2024 – March 2025), organisational (seven training centres under a single administrative structure), programmatic (a standardised curriculum, assessment framework, and instructional model), and stakeholder (four distinct participant groups). These boundaries maintained focus while acknowledging that educational phenomena cannot be fully isolated from their implementation contexts (Tight, 2017).
3.3. Participant Selection and Recruitment
The study employed purposive sampling (Cohen et al., 2018), selecting participants who could provide information-rich perspectives on the blended learning model. Students (n=14) were required to be currently enrolled at the highest academic level (English Level 6), to have had sufficient exposure to the blended learning model, and to possess adequate English proficiency for meaningful participation in focus group discussion. English Level 6 denotes the most advanced band of the organisation’s six-level in-house English curriculum. Placement at and progression to this level are determined by the organisation’s standardised placement test and end-of-level assessments covering reading, writing, listening, and speaking. Selecting students at this level ensured that participants had progressed through the full blended programme and could reflect on all its components. Teachers (n=7) were selected on the basis of a minimum of two years’ organisational experience, familiarity with both current and previous programme models, and diverse nationalities, ensuring the study drew on multiple cultural perspectives on pedagogy and programme delivery. Trainees (n=8) were required to be recent graduates of the blended learning English programme, to have completed at least one job skills training unit, and to represent a range of technical specialisations, thereby reflecting the diversity of vocational pathways the programme serves. Trainers (n=7) were required to be currently teaching graduates of the blended learning programme and to bring diverse nationalities and technical specialisations to the inquiry. The total sample size (n=36) balanced depth of inquiry with practical data management constraints, aligning with recommendations for qualitative case study research (Creswell & Poth, 2018).
Recruitment occurred through organisational channels, with programme coordinators introducing the research to potential participants. All received detailed information sheets and gave informed consent before involvement; consent procedures, voluntary participation, confidentiality, and pseudonymisation are detailed in the Ethical Dimensions section below.
3.4. Data Collection Methods
Three complementary methods for gathering participant accounts were employed, each selected for its appropriateness to specific stakeholder groups and research objectives:
3.4.1. Semi-Structured Interviews (Teachers and Trainers)
Individual interviews allowed teachers and trainers to reflect deeply on their professional experiences without peer influence that might arise in group settings. Interview guides contained core questions addressing modality support for skill development, integration between modalities, authenticity of learning contexts, implementation challenges, and recommendations for enhancement. Interviews lasted 30–60 minutes, were audio-recorded with permission, and were transcribed verbatim. The semi-structured format allowed flexibility to explore emerging themes while maintaining focus on key evaluation dimensions.
3.4.2. Focus Groups (Students and Trainees)
Group discussions were judged more appropriate for students and trainees based on prior experience suggesting greater comfort in collective settings. Group discussions also enabled observation of how participants built understanding through dialogue — directly demonstrating social constructivist principles under investigation. Each focus group included 4–5 participants, following Morgan’s (1997) guidance for maintaining manageable dynamics. Discussion guides paralleled interview questions, adapted for group context. Sessions lasted 45–75 minutes, were audio-recorded and transcribed. A key advantage of these collaborative sessions was enabling participants to build on, challenge, or confirm each other’s perspectives, generating richer collective understanding than individual accounts alone might provide.
3.4.3. Email-Based Qualitative Questionnaires (All Groups)
For participants unable to attend in-person sessions due to geographic distance or scheduling conflicts, email-based questionnaires provided alternative participation. While lacking the dynamic interaction of interviews or group discussions, questionnaires enabled thoughtful reflection at participants’ own pace. Questions paralleled those in interviews and group sessions, maintaining data comparability across methods. Participants received one week to respond, with reminders sent as needed. All questionnaire participants responded, though some required follow-up for clarification.
It is important to acknowledge that email-based data collection carries an inherent depth risk compared to synchronous methods: without the ability to probe, follow up in real time, or observe non-verbal cues, responses risk being thinner than those generated through interviews or focus groups. Several measures were taken to mitigate this. Questions were carefully scaffolded to prompt narrative reflection rather than brief answers, and participants who provided short or ambiguous responses received targeted follow-up requests for elaboration. Responses averaged approximately two to three substantive paragraphs per question, and the resulting data proved sufficient to illuminate patterns consistent with, and in some cases extending, themes that emerged through the synchronous methods. An unexpected challenge emerged when some student responses showed evidence of Artificial Intelligence (AI) generation (overly formal language, suspiciously comprehensive answers, lack of personal examples). After clarification that authentic personal responses were required, only four students provided genuine second responses. Non-original responses were excluded from analysis, highlighting the need for clearer initial instructions in future email-based inquiry.
3.4.4. Critical Friend Review
All data collection instruments underwent critical friend review by two colleagues — one doctoral graduate in educational technology and one doctoral researcher — before deployment. Reviewers assessed alignment between questions and research objectives, appropriateness for target populations, adequate coverage of key evaluation areas, tool length and question clarity, and potential ambiguities or difficulties. Feedback informed instrument refinement before fieldwork commenced.
3.5. Data Analysis: Reflexive Thematic Analysis
Data analysis employed reflexive thematic analysis (Braun & Clarke, 2006, 2019, 2022), a flexible approach enabling iterative exploration of participant experiences while allowing theoretical frameworks to inform without predetermining emerging themes. The analysis adopted a hybrid deductive-inductive approach, with theory guiding attention towards significant phenomena while remaining open to unanticipated patterns. It is important to clarify how this balance was maintained in practice. Initial coding proceeded as an open, data-driven process: transcripts were read repeatedly without applying theoretical categories, generating codes grounded in participants’ own language and concerns — for example, “motivation challenges,” “vocabulary gaps,” and “awkward transitions between modalities.” Only after this initial stage were the resulting codes reviewed in relation to Social Constructivism and Situated Learning, examining which clusters mapped onto theoretically significant concepts. This sequencing ensured that both frameworks functioned as sensitising concepts in Blumer’s (1954) sense — directing analytical attention rather than imposing predetermined categories. Where participant accounts raised concerns that did not map straightforwardly onto either framework, these were retained as analytically distinct and considered in their own right during theme development.
The six-phase analytical process proceeded as follows:
Phase 1: Familiarisation
Repeated reading of transcripts while noting initial observations and patterns. Audio recordings were reviewed alongside transcripts to capture tone and emphasis lost in written form.
Phase 2: Initial Coding
Systematic generation of codes representing meaningful ideas or concepts within the data. Codes were data-driven (e.g., “technology over-reliance,” “motivation challenges,” “vocabulary gaps”) and theory-informed (e.g., “scaffolding,” “authentic tasks,” “peer interaction”). This dual approach honoured participant perspectives whilst maintaining theoretical relevance.
Phase 3: Theme Development
Codes were organised into broader themes through iterative review, identifying connections and relationships. This phase involved active interpretation, moving beyond surface description towards understanding deeper meanings and implications.
Phase 4: Theme Review
Themes were refined through checking internal coherence (whether theme components fit together) and external distinctiveness (whether themes are genuinely distinct rather than overlapping). Some themes were split when too broad; others were merged when insufficiently distinct.
Phase 5: Theme Definition
Each theme received a clear definition and descriptive name. Theoretical mapping examined relationships to Social Constructivism and Situated Learning, generating five main themes with 2–4 sub-themes each, supported by participant quotes across all stakeholder groups.
Phase 6: Report Production
Findings were written in narrative format using thick description (Harrison et al., 2017), with participant quotes selected to illustrate themes while maintaining anonymity. Throughout, reflexive practice involved critically examining how insider position, contextual familiarity, and theoretical commitments might inform interpretation (Braun & Clarke, 2021); analytical notes documented decisions, alternative interpretations, and the evolving understanding of themes.
Coding Scheme: From Participant-Generated Codes to Dimensions
3.6. Trustworthiness Strategies
Qualitative research requires attention to credibility, dependability, and transferability (Rashid et al., 2019). Several strategies were employed to address these concerns. Triangulation occurred at three levels: method (three complementary data collection approaches), source (four stakeholder groups), and theory (two theoretical frameworks). Member checking involved sharing draft findings with one teacher and one trainer; feedback confirmed accurate representation of their views whilst offering additional nuance that was incorporated into final reporting. Critical friend review extended beyond research instruments to include the final paper draft, reviewed by two research colleagues whose constructive feedback refined the reporting. Methodological transparency was maintained through detailed documentation of all research processes, enabling readers to evaluate logical connections between evidence gathering, analysis, and conclusions (Merriam, 1998). Finally, thick description provided rich contextual detail, enabling readers to assess transferability to their own settings (Creswell & Poth, 2018). Beyond these general strategies, the multi-stakeholder design raised three further trustworthiness considerations specific to reconciling evidence from four distinct groups (Cousins & Whitmore, 1998).
The first concerns the handling of contradictory accounts. Rather than seeking consensus or privileging certain voices, contradictions were treated as meaningful data revealing complexity. When stakeholders disagreed, analysis explored the reasons for divergence — different positions within the system, varied priorities, or distinct aspects of experience — honouring multiple realities rather than forcing artificial resolution. For example, teachers described vocabulary coverage as sufficient, whilst trainees reported gaps in field-specific terminology — a contradiction treated as analytically revealing rather than as flawed data.
The second consideration concerns comparative data volume. Because some stakeholder groups generated more data than others due to group sizes and method suitability, analysis attended to the quality and significance of contributions rather than frequencies alone. When only one stakeholder group raised a concern, context determined whether it represented a unique insight or a marginal observation.
The third consideration concerns representational balance across all four groups. Thematic presentation deliberately drew on participant accounts from all four stakeholder groups for each major finding, ensuring that diverse perspectives remained visible whilst analytical coherence was maintained.
3.7. Insider Researcher Positioning
The researcher’s 19-year history within the training organisation created a complex insider positioning with both advantages and challenges. On the positive side, contextual familiarity enabled recognition of significant data, established rapport facilitated honest participation, and an understanding of organisational culture and history supported meaningful interpretation of local practices. Access to multiple stakeholder groups and credibility with participants through shared experience further strengthened the inquiry. At the same time, this positioning carried risks: potential bias towards organisational perspectives, a tendency to overlook taken-for-granted aspects of the context, power dynamics with participants despite the absence of a direct supervisory relationship, the possibility of socially desirable responses, and the need to maintain appropriate analytical distance.
Several strategies addressed these risks. The researcher’s current position as career counsellor created practical distance from teaching and programme management, reducing participant concerns about professional consequences. Systematic reflexive documentation, theoretical grounding, member checking, and critical friend review (described in the Trustworthiness Strategies section above) provided additional safeguards. Importantly, these strategies produced findings that included critical perspectives on the programme: students identified inconsistency in assessment practices across teachers, trainees reported that technical vocabulary preparation was insufficient for their vocational contexts, and teachers raised concerns about rote learning driven by assessment pace — none of which were suppressed or minimised in the analysis. This positioning ultimately proved advantageous, providing contextual insight necessary for meaningful evaluation while maintaining sufficient distance for critical analysis. As Taylor (2011) argues, insider researchers can leverage familiarity while maintaining analytical rigour through systematic reflexive practice.
3.8. Ethical Dimensions
3.8.1. Informed Consent in Organisational Contexts
Ensuring voluntary participation required careful attention to power dynamics. Despite researcher assurances, participants might have felt pressure to participate or provide favourable accounts due to organisational relationships. Several measures were taken to mitigate this. Participation was explicitly stated as voluntary with no professional consequences for declining, and fieldwork was conducted outside working hours where possible. Anonymisation procedures were clearly explained to all participants, who were also given the option to withdraw within one week of participation. Throughout recruitment and data collection, emphasis was placed on the value of honest feedback, including critical perspectives on the programme.
3.8.2. Confidentiality and Anonymity
The organisation’s unique characteristics created anonymity challenges, as detailed descriptions might enable identification despite the use of pseudonyms. Several protection strategies were employed to address this. All identifying details were removed from participant quotes, and demographic information was aggregated to prevent identification through contextual inference. Organisational permission was secured before fieldwork commenced, and participants were given the opportunity to review how their contributions were represented in the reporting.
3.8.3. Dual Relationships
The researcher’s insider status created dual relationships with some participants, requiring careful boundary maintenance throughout the research process. Research responsibilities were kept separate from professional duties, and professional distance was maintained during data collection. Discussion of research findings in workplace contexts was avoided, and care was taken to ensure that research outcomes could inform organisational improvement without exposing individual participants. These ethical considerations required ongoing attention at every stage of the research process, not merely during initial approval, with regular reflection on ethical implications informing decision-making throughout.
4. Results: The Five-Dimensional Evaluation Framework
The five-dimensional framework presented below emerged through iterative cycles of coding and theme development. Each dimension reflects theoretically significant aspects of blended vocational learning, drawing on Social Constructivism and Situated Learning to varying degrees as indicated in the Theoretical Connection notes within each dimension below. To demonstrate how each dimension emerged from and is grounded in the data, multiple participant accounts are presented for each dimension, drawn from across the four stakeholder groups and the three data collection methods. Each extract is attributed to its participant code and data source, and the coding scheme in Table 1 shows how participant-generated codes were organised into the sub-themes and dimensions reported below, providing a transparent audit trail from raw data to analytical framework.
4.1. Dimension 1: Modality Integration
This dimension examines how face-to-face and self-directed components function individually and together to support learning. Its sub-concerns encompass the effectiveness of classroom instruction for developing skills requiring synchronous interaction, immediate feedback, and social learning; the capacity of online components to enable personalised pacing, flexible access, and independent practice; and how effectively modalities complement each other through intentional sequencing, clear connections, and coherent learning pathways. Key evaluation questions ask whether learners experience smooth transitions between modalities or perceive them as disconnected, whether integration creates synergistic value or components merely coexist, whether activities are strategically allocated to modalities where they can be most effective, and how sequencing supports progressive skill development. The face-to-face element really allows for students to try out the target content in a simulated environment, e.g. role play in class. The self-study content carried out with automated feedback at home allows trainees to go at their own pace and repeat as necessary, while also allowing for more productive skill practice to happen in class time. It is a good balance, and makes good usage of contact time. (Teacher 3, interview)
This account illustrates how an experienced teacher perceives the two modalities as performing distinct but complementary pedagogical functions — self-study providing preparation and input, face-to-face enabling production and application — which is precisely the integrative logic that Social Constructivism predicts when modalities are coherently sequenced. This integrative logic was corroborated by learners themselves. One student described the classroom as a reference point that remained available during independent work: A small example, me and (my colleague) are reviewing an email, and we are disagreeing about punctuation, a comma or something, or capitalising a word, so we can recall the face-to-face class, and we say the teacher said that it should be like that. (Student 5, focus group).
A further account framed the two modalities as mutually reinforcing rather than redundant: “…they [the face-to-face and self-directed components] complement each other for those who seek to learn…” (Student 3, email questionnaire). The convergence of teacher and learner accounts, across two methods, grounds this dimension in more than a single perspective.
4.2. Dimension 2: Vocational Skills Development
This dimension assesses how effectively the model develops specific language competencies required for vocational success. Its sub-concerns span functional workplace communication — the ability to perform job-relevant tasks such as reporting, requesting, and instructing — alongside competence in producing workplace documents including emails, reports, and technical documentation; oral communication skills in workplace discussions, presentations, and collaborative dialogue; and the acquisition of industry-specific technical vocabulary. Key evaluation questions ask which skill areas develop most successfully and which require enhancement, how well skill development aligns with actual workplace demands, whether skills transfer effectively from academic to vocational contexts, and what gaps exist between training outcomes and professional requirements. After the English phase, I had an issue outside with a gaming company and I had to send an email. I wrote like 90% [of] a perfect email to that company... from the start to the end by myself, with my own words and my own grammar and my own choosing of words. (Trainee 7, focus group).
This account demonstrates how skill transfer — a central concern of Situated Learning — can extend well beyond the training context, confirming that effective vocational preparation equips learners for authentic communication demands encountered independently of institutional support. A trainer observing graduates from the receiving end of the programme reinforced this transfer evidence: Trainees with good oral English communication excel better in process units of the PCST (Process Control Systems Training) programme, in comprehending the subject [matter]… [they] exhibit confidence, are assertive, dialogue more frequently with [the] trainer to clear the doubts, ask questions… (Trainer 1, interview).
Pairing a graduate’s self-reported transfer with a trainer’s independent observation strengthens the evidential basis of this dimension.
4.3. Dimension 3: Learning Environment and Social Dynamics
This dimension examines how human elements — teachers, peers, and community — shape learning experiences. Its sub-concerns address teacher roles and contributions — how instructors facilitate learning through scaffolding, guidance, feedback, and authentic contextualisation — alongside peer interaction and collaboration, examining how learners support each other’s development through formal and informal knowledge sharing. Key evaluation questions ask how effectively teachers adapt their roles across modalities, what types of peer interaction prove most valuable, whether the environment supports both individual and collaborative learning, and whether social dynamics are conducive to knowledge construction. In every lesson that takes place in the morning, we have safety meetings whereby a safety message has to be delivered by the trainees themselves... Once a week, one of them is given the opportunity to prepare a presentation, then he will stand in front of the class and present to the rest of the class. It depends on the number — sometimes you find a lot of questions from the other trainees, sometimes none, but it is more of a support — they are supporting their colleagues rather than asking questions. (Trainer 7, interview).
This account illustrates how trainers actively construct social learning communities within job skills settings, assigning structured peer-facing tasks that require trainees to use language for authentic communicative purposes — directly enacting the Social Constructivist principle that knowledge is consolidated through social participation rather than individual study alone. Learners described the reciprocal character of this social learning in their own terms: “When you discuss with other students about your writing, they might see something you haven’t seen, or the opposite, you might help them.” (Student 4, email questionnaire). This learner account complements the trainer’s structural perspective, evidencing peer-to-peer knowledge construction alongside teacher-orchestrated activity.
4.4. Dimension 4: Technology Effectiveness
This dimension assesses how digital tools support or hinder learning processes. Its sub-concerns address the advantages digital tools provide through accessibility, repetition, immediate feedback, and simulation; the barriers created by technical problems, over-reliance, or inappropriate application; and the quality of technology integration — whether tools are purposefully selected and pedagogically embedded. Key evaluation questions ask whether technological tools enhance or replace meaningful learning activities, whether technical reliability issues affect learning continuity, how learners use technology — productively or counterproductively — and what balance exists between technological and human support. There is a function in Blackboard when you need to pronounce the words, and it would tell you if you said it correctly or not. But that is not enough. You are still not communicating with a person. You are not improving your English that much, especially in the speaking part. (Student 11, focus group)
This account reveals a limitation that students themselves identified in automated technology feedback: whilst digital tools can verify pronunciation accuracy, they cannot replicate the interactive, socially embedded communication that genuine language development requires — precisely the tension between technological mediation and authentic skill development that this dimension captures. Set against this limitation, learners also recognised genuine affordances of the digital component for autonomous practice: “Self-directed learning helps me improve English by practising at my own pace. For speaking I use apps [like] Duolingo to improve my pronunciation.” (Student 1, email questionnaire). Read together, the two accounts show the dimension capturing both the advantages of digital tools and the limits of what they accomplish without human interaction.
4.5. Dimension 5: Implementation Effectiveness
This dimension synthesises overall effectiveness while identifying specific challenges and enhancement opportunities. Its sub-concerns encompass stakeholder recognition of model strengths and valuable features; systematic barriers to effectiveness including motivation, time management, quality assurance, and resource constraints; and stakeholder-generated suggestions for improvement. Key evaluation questions ask what aspects of implementation prove most successful, what systemic challenges require institutional response, what refinements could enhance effectiveness, and how stakeholder recommendations converge or diverge. There are 9 different assessments in a short space of 8 weeks. There is a concern that students may be rote learning in order to pass rather than meaningful learning. To overcome this, I challenge trainees to envision themselves in a specific real-life situation and find meaning in how they associate the target language to the task at hand. (Teacher 2, interview).
This account captures a systemic implementation tension that surfaced across multiple teacher accounts: the programme’s pace and assessment density created conditions where surface-level compliance became more manageable than the meaningful engagement with vocational language that the model was designed to produce. The same systemic pressures surfaced in teachers’ accounts of resourcing and motivation: Some students do not have good internet access at home, so they cannot finish online work. I give them extra time during class to complete these tasks. Others need more motivation for self-study, so I give clear instructions and encourage them with rewards or praise. (Teacher 1, email questionnaire).
These accounts locate implementation effectiveness in the interaction between programme design and the institutional and material conditions of delivery.
4.6. Framework Visualisation and Application
Figure 1 illustrates relationships among dimensions, positioning Social Constructivism and Situated Learning as grounding principles, stakeholder groups as sources of evaluation evidence, and the five dimensions as lenses through which blended learning effectiveness is examined. Qualitative framework for evaluating blended learning in vocational education
5. Discussion
5.1. Methodological Contributions
Contribution 1: Multi-Stakeholder Perspectives as Complementary Knowledge Sources
Rather than treating different stakeholder views as competing claims, this study positioned them as complementary perspectives illuminating different aspects of programme effectiveness. Students identified advantages and barriers in technology-mediated learning; teachers articulated how the two modalities function together; trainees demonstrated how academic preparation transferred to authentic vocational communication; trainers illuminated where preparation gaps remain. Together, these perspectives created understanding impossible from single sources, operationalising participatory evaluation principles (Cousins & Whitmore, 1998) whilst challenging traditions that privilege particular voices over others.
Contribution 2: Theory as Sensitising Framework Rather Than Hypothesis-Testing Mechanism
Social Constructivism and Situated Learning functioned not as hypotheses to be tested but as lenses directing attention towards theoretically significant phenomena (Blumer, 1954), enabling discovery of both theory-confirming patterns and theory-problematising tensions. Social Constructivism directed attention towards knowledge construction through social interaction — evident in Dimension 1’s finding that face-to-face instruction serves as a retrievable resource during self-directed work, and Dimension 3’s evidence that incidental social exposure generates understanding deliberate individual study cannot replicate. Situated Learning directed attention towards authentic workplace participation, evident in Dimension 2’s demonstration of independent skill transfer beyond the training context. Social Constructivism’s conception of technology as mediating tool shaped Dimension 4’s finding that automated feedback cannot substitute for interactive social communication. Dimension 5 proved particularly revealing: assessment pace and density favouring surface compliance simultaneously undermined collaborative knowledge construction and authentic vocational preparation — demonstrating that the frameworks illuminate failure conditions as clearly as successful ones. Taken together, this challenges both atheoretical evaluations and overly rigid applications that force data into predetermined categories (Saunders, 2011).
Contribution 3: Developmental Insights Through a Multi-Cohort Cross-Sectional Design
By including learners at different stages of their vocational journey — current students and post-academic trainees — alongside their respective instructors, the study generates developmental insights into learning progression and skill transfer that a single-cohort study could not offer. This does not replicate the rigour of a true longitudinal design, and the constraints of cross-sectional comparison apply: differences between cohorts may reflect factors beyond the programme itself. Nevertheless, the approach offers a practical and ethically manageable pathway to capturing the arc of vocational learning in contexts where extended temporal tracking is not feasible (Merriam, 1998).
Contribution 4: Insider Positioning as Methodological Resource
Rather than treating insider status as a liability requiring compensation, this study demonstrated its productive use as a methodological resource through systematic reflexive practice (Taylor, 2011), challenging persistent assumptions that external evaluators are inherently more objective than insiders.
5.2. Methodological Implications for Theory-Grounded Evaluation
This study demonstrates that explicit theoretical grounding enhances rather than constrains qualitative evaluation (Anfara, 2008). By making Social Constructivist and Situated Learning commitments explicit, the research maintained analytical focus on theoretically significant dimensions whilst avoiding the diffuse, atheoretical approaches that sometimes characterise educational evaluation (Saunders, 2011). The framework shows that theory functions most productively when treated as a sensitising lens rather than a predetermined coding scheme — guiding where to look whilst remaining responsive to contextual particularities stakeholders reveal (Blumer, 1954). Future evaluation research might profitably adopt similar approaches, explicitly mapping theoretical frameworks to methodological decisions and demonstrating how those commitments shape both data gathering and analytical categorisation.
5.3. Practical Implications
For researchers, the study demonstrates rigorous qualitative case study methodology and the productive integration of theory with empirical investigation. For practitioners, the framework offers structured guidance for multi-stakeholder evaluation with clear connections between evidence and actionable findings (Cousins & Whitmore, 1998; Saunders, 2011). For administrators, it provides a systematic mechanism for ensuring diverse stakeholder voices inform improvement decisions (Weiss, 1998).
5.4. Framework Universality and Transferability
The five dimensions possess a logical coherence that holds beyond the Saudi Arabian corporate context in which they emerged. Rather than arbitrary categories derived from a single case, they map onto structural features present in any blended vocational learning context (Stake, 1995): a researcher evaluating a blended nursing, construction trades, or language programme in another setting would find these five analytical concerns consistently present, even if their specific content and weighting differ. The relationship between dimensions — particularly the way implementation conditions (Dimension 5) both shape and are shaped by the other four — gives the framework coherence as an integrated whole rather than a checklist of isolated concerns.
Users should assess alignment with the originating conditions: corporate training, industry-specific language education, and intentionally designed blended learning. Similar contexts will find direct transferability; partially similar contexts should adapt specific components whilst retaining the overall architecture; substantially different contexts can apply the five dimensions as analytical principles whilst rewriting specific evaluation questions for local purposes.
5.5. Limitations
Several limitations warrant acknowledgment. The framework emerged from a well-resourced corporate vocational training environment in Saudi Arabia, which may not represent the full range of vocational settings, though the transferability argument above addresses this. Voluntary participation may have skewed responses towards stronger opinions. The cross-sectional design cannot capture how experiences evolve over time, and email questionnaires generated less rich data than synchronous methods — both mitigated through the triangulation strategies detailed in Section 3.
5.6. Future Research Directions
This framework opens several research directions: testing transferability across varied vocational domains and cultural settings; longitudinal applications providing genuine temporal depth; mixed-methods extensions combining qualitative and quantitative outcome measures; and comparative studies across multiple institutions to identify contextual factors influencing blended learning effectiveness at a systemic level.
6. Conclusion
Educational evaluation ultimately serves practical purposes: informing improvement decisions, ensuring accountability, and advancing understanding of what works for whom under which conditions. This requires evaluation approaches that are simultaneously rigorous in method and responsive to context, theory-grounded yet empirically open, systematic in process yet flexible in application. The framework presented in this study demonstrates one approach to achieving these multiple aims through qualitative case study research that positions diverse stakeholder perspectives as primary evidence, employs established theories as sensitising lenses, and produces findings that illuminate both specific programmes and broader principles.
This paper’s primary contribution is methodological, grounded in original empirical inquiry. The multi-stakeholder inquiry revealed how blended learning effectiveness is experienced across four participant groups (RQ1), whilst the identification of key implementation challenges and opportunities directly addresses RQ2. The five-dimensional framework carries transferable analytical coherence beyond the Saudi Arabian context, offering vocational education researchers and evaluators a principled basis for adaptation. By making methodological reasoning explicit, this study enables others to adapt and extend this approach to their own vocational evaluation contexts, demonstrating that rigorous qualitative evaluation depends on the systematic application of sound principles rather than context-independent formulas.
Footnotes
Acknowledgements
I acknowledge the participants who generously shared their experiences and insights for this research. I thank my doctoral supervisors at Lancaster University, Dr. Philip Moffitt and Dr. Carmen Martinez Vergas, for their guidance throughout this research.
Ethical Considerations
This study received ethical approval from the Faculty of Arts and Social Sciences and Lancaster University Management School’s Research Ethics Committee [EdRes-2024-4443-EdAp-1] prior to fieldwork. Pre-approval from the participating organisation’s management was also obtained, ensuring conformity with the organisation’s research and publishing ethical guidelines. The research adhered to the ethical guidelines of the British Educational Research Association (BERA), with particular attention to the ethical considerations of conducting research within the researcher’s workplace setting. Clear boundaries were maintained between the researcher’s employee and researcher roles, and the dual position was disclosed to all participants.
Consent to Participate
All participants were provided with detailed information sheets explaining the study’s purpose, procedures, potential risks, and benefits. Written informed consent to participate was obtained from all participants before their involvement in the study. Participants were informed of their right to withdraw from the study at any time without consequence and without needing to provide a reason.
Consent for Publication
All participants gave written informed consent for the information they provided to be used in academic articles, publications, and presentations arising from this research. Participants were assured that their personal information would not be included and that all reasonable steps would be taken to protect their anonymity. Accordingly, all quotations and data excerpts reproduced in this article are presented anonymously, with identifying details removed.
Author Contributions
MM designed the study, conducted all data collection and analysis, and wrote the manuscript. MM read and approved the final manuscript.
Funding
The author received no financial support for the research, authorship, and/or publication of this article.
Declaration of Conflicting Interests
The author declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: The author was employed by the organisation under study at the time of research as a career counsellor, which represents a potential non-financial competing interest. However, the author had no direct supervisory relationship with participants and no financial interest in the research outcomes. All fieldwork and analysis procedures were designed to mitigate potential bias through reflexive practice, member checking, and critical friend review as detailed in the manuscript.
Data Availability Statement
The datasets generated and analysed during the current study are not publicly available due to confidentiality agreements with the participating organisation and to protect participant anonymity in a workplace-based educational setting. However, anonymised data excerpts are available from the corresponding author on reasonable request.
Use of Artificial Intelligence
The author used AI-assisted tools (specifically Claude, developed by Anthropic) to support manuscript drafting, structural organisation, and compliance checking during the preparation of this work. All theoretical analysis, interpretation of findings, participant data, and scholarly conclusions are the author’s own. No AI tool was used to generate, fabricate, or substitute participant data. All participant quotes come directly from the original data of the doctoral project.
