Abstract
This review article is to examine the technology-related literature published in major human resource development (HRD) journals from a design-based research perspective. Despite the important role of technology in HRD activities for promoting organizational learning and performance, our understanding of the potential impact of our technology research efforts in the workplace remains limited. To address this deficiency, the authors reviewed technology research studies from five journals using two conceptual frameworks: (a) goal categories of technology research and (b) research phases and activities using a design-based research perspective, to better understand how technology research processes have been localized in organizations. Findings indicate that the major trend of technology research is in describing and conceptualizing what is happening. When using a design-based research framework, a substantial number of articles focused on analysis and reflection aspects of research activities. Implications for the technology research landscape in HRD are discussed.
Introduction
Technology is believed to drive changes in organizational learning and performance. The meaning of technology itself has also been changing over time. Bigelow (1829) defined technology as “principles, processes, and nomenclature of the more conspicuous arts, particularly those which involve applications of science, and which may be considered useful, by promoting the benefit of society, together with the emolument of those who pursue them” (p. 4). Galbraith (1967) considered technology the “systematic application of scientific and other organized knowledge” to solve practical problems (p. 12). A more recent view on technology entails organized human-created systems for attaining specific goals (Volti, 2009). Along with the constant evolvement of technology, new challenges arise that demand new perspectives and approaches embodied by human resource development (HRD) research and practices. Historically, although brief, HRD literature has discussed the roles of technology in the intersection between HRD activities and their intended learning or performance outcomes. For example, Benson and Johnson edited a special issue in Advances in Developing Human Resources (Vol. 4, Issue 4) to explore the role of technology in the digital workplace where individual and organizational learning is the priority. Given technology’s influence on learning and performance improvement, instructional technologies could contribute to the training and development aspect of HRD (Barlett, 2008). Swanson and Holton (2009) conceptualized three tiers of technology: process-oriented technology that articulates job-specific tasks and skills to improve the performance output (i.e., Workforce Technology), application-oriented technology that delivers HRD activities (i.e., Technology Within HRD), and Information Technology (IT) and HRD that is focused on the decentralization of HRD activities in the workforce. Although the value of technology in HRD activities is promising for improving learning and performance outcomes and facilitating organizational changes (Bennett, 2010; Benson, Johnson, & Kuchinke, 2002), integrating technology into HRD practice and research meaningfully and efficiently remains a daunting organizational task. Wang (2012) identified the following issues critical to effective technology and HRD integration: changing HRD roles and skills (e.g., horizontal communications amplified by technology), unintended consequences of technology (e.g., trolling on social media), ethical concerns (e.g., privacy vs. accessibility), and emerging HRD practices in virtual/online settings (e.g., virtual teams, virtual worlds, virtual mentoring, and virtual knowledge management). Bennett (2014) further encapsulated the values and efforts of technology-driven changes in HRD with the concept of technology research, which considers technologies as tools for solving HRD problems while sustaining the use of technologies in organizations (Bennett, 2014). Nevertheless, neither perspective has articulated various attributes inherent in technology-driven organizational processes in HRD activities.
Considering that HRD’s core values should be best demonstrated and judged via processes (Kuchinke, 2003), the lack of process-oriented frameworks for technology research in organizations needs to be addressed. This study first adopted the design-based research (DBR; McKenney & Reeves, 2013) as a technology research process framework. The DBR is introduced as a generic process model for technology research in organizations, which is a distinctive research process for developing solutions to real-world learning and performance problems through empirical investigation (Design-Based Research Collective, 2003). A recent salient example of DBR is that of a group of practitioners and educational researchers in the World Health Organization (WHO) who designed and offered experiential professional training to health care professionals concerning management and distribution of temperature-sensitive pharmaceutical products (Vesper, Kartoglu, Bishara, & Reeves, 2010). Following the DBR approach, this WHO team has demonstrated the iterative cycles of the DBR process and have described their DBR efforts in multiple publications (Kartoglu, Vesper, & Reeves, 2017; Vesper, Herrington, Kartoğlu, & Reeves, 2015; Vesper, Kartoğlu, Herrington, & Reeves, 2016).
Managing technology research remains a topic of immense global interest in various disciplines. Cabrera and colleagues (2001) proposed that technology is one of many components that drive organizational changes; therefore, understanding organizations as multisystems encompassing technological innovation, structure, and culture is important. To optimize the outcomes of technology research (Bennett, 2014), organizational processes are inevitable in facilitating a calibration between new technological features and organizational cultures. Furthermore, additional processes are warranted to maintain a fit between technological affordances and organizational behaviors. In their multisystem view on organizations, a particular emphasis was given to organizational culture when integrating new technological systems, given the dynamics of the person–organization fit (Lau, McLean, Hsu, & Lien, 2017). This implies that technology research should be a constantly ongoing process, from the initial acquisition of buy-in to a long-term commitment to the targeted technology.
Purpose of the Research
Based on the aforementioned HRD process concerns and challenges facing technology research in organizations, this article aims to reveal the process patterns among technology studies to advance HRD practices and research. In particular, we reviewed and analyzed the technology studies published in major HRD journals in the past decade.
A Decade of Inquiry on Technology in HRD
Research in HRD has demonstrated an increasing interest in understanding the role of technology, the processes that technology affects, and the organizational values that HRD activities and technology have co-created. The emphasis on integrating technologies into organizational activities, however, is limited. A majority of work has been focused on the conceptualization of constructs, with preliminary efforts devoted to surveying the field of HRD through the lens of technology.
Based on a review of 1,675 journal articles published in Advances in Developing Human Resources (ADHR), Human Resource Development International (HRDI), Human Resource Development Quarterly (HRDQ), and Academy of Human Resource Development (AHRD) conference proceedings between 2000 and 2006, Githens, Dirani, Gitonga, and Teng (2008) surveyed the presence of technology-related studies in the HRD research landscape. They identified topics addressed, author affiliations, research design, research contexts, and meta-perspectives (e.g., systems, societal community) presented in the studies. Among their recommendations, the authors exercised caution on the need for additional technology-related studies in HRD while suggesting that future research focus on “organization-focused technology research such as technology transfer, virtual teams, knowledge management systems” (p. 210). Although these topics might have hinted at the complexity of interfacing technology with HRD activities, the authors did not specify the role and process of technology research.
In addition to understanding the research landscape of technology in HRD, great strides have been made regarding the strategic roles of technology in HRD. Bennett (2009) proposed that technology’s roles in HRD could be viewed collectively under the umbrella of Virtual HRD (VHRD), which consists of the applications of technology for HRD activities (i.e., training and development, career development, organizational development; Swanson & Holton, 2009), environments that technology creates for innovations and community building (p. 365), and effects of technology in adult learning (McWhorter, Mancuso, & Hurt, 2008). Furthermore, in an increasingly virtualized workforce, VHRD could be a strategic hub to cultivate the value of organizational knowledge management and organizational culture through applications such as intranets (Bennett, 2009, 2010; Hanandi & Grimaldi, 2010). Huang, Han, Park, and Seo (2010) further considered VHRD as a cultivated organizational process to improve organizational-level VHRD activities based on theoretical foundations in pertinent areas and performance analytics.
Technology integration for VHRD was also discussed. How technology and people should interact and interconnect with each other in the context of VHRD (Bennett, 2010; Mancuso, Chlup, & McWhorter, 2010) speaks to technology integration’s purpose at the individual level. By situating VHRD as a subset of Virtual HR practices in organizations, Bennett and Bierema (2010) proposed a structural integration approach for a better interface between technology and HRD activities (p. 640). Specifically, they suggested that VHRD is capable of facilitating organizational processing for HR, VHRD can improve the quality of services to organizational stakeholders, and VHRD has the potential to reorganize organizational hierarchy due to increased access through IT. These value propositions of VHRD are important at a strategic level to garner buy-in for a long-term commitment to VHRD. However, the processes with which, how, and when these multilevel integrations should happen and how the processes should be facilitated remain unclear.
In a more recent iteration of the VHRD conceptualization effort, two modes of technology development were described (Bennett, 2014). The first “adapts technology to support HRD interventions and processes in practice” (p. 271), which considers technologies as utilities to solve HRD problems. The second mode “applies HRD techniques, processes, and models to ensure organizational needs are met strategically during and after design and implementation of new technology” (p. 272). This mode is focused on the long-term sustainability of technology adoption in HRD activities. Bennett and McWhorter (2014) also defined technology development as “the integration of technology with HRD objectives and processes to improve learning capacity and performance” (p. 581). Various factors influencing the integration efforts were also suggested, including informal learning, webbed environments, organizational culture, and media richness (McWhorter, 2014). Although the strategic importance of technology integration is recognized under the conceptual umbrella of technology research, specific approaches and frameworks to initiate and sustain technology research remain elusive in the context of HRD.
DBR and Technology Research in HRD
One of the important missions of research is to accumulate empirically proven knowledge to advance a specific field. However, despite a plethora of peer-reviewed publications, a “research-practice divide” (Kuchinke, 2013, p. 370) persists in many fields including HRD. Ardichvili (2012) argued that HRD research is merely “scratching the surface of practically important problems” (p. 267). In various disciplines such as the health professions education (e.g., Kartoglu et al., 2017); science, technology, engineering, and mathematics (STEM) education (e.g., Ketelhut, Nelson, Clarke, & Dede, 2010); and higher education faculty training (e.g., Shattuck & Anderson, 2013), DBR has gained much attention over the past two decades due to an increasing frustration over the lack of the relevant and sustainable impact of findings of traditional research on actual practice (Ardichvili, 2012; Kuchinke, 2013; Oh, Lee, & Reeves, 2016). In the context of technology research, Reeves (2006) compared a traditional predictive approach with a DBR approach (Figure 1).
Predictive and design-based research approach in technology research.
As seen in Figure 1, in predictive research, the conceptualization of problem and hypothesis frequently comes from an identification of a knowledge gap in the scholarly literature, which drives the remaining research process toward an analytic rather than a design nature (Kuchinke, 2013). Researchers may conduct experiments to test hypotheses, and the goal of the research effort is to contribute to a theory refinement. However, considering the isolated nature of the initial conceptualization of a study and its focus on a primarily theoretical final outcome, the impact of such findings in predictive research is limited for advancing learning capacity, individual and organizational performance, and HRD processes with technology integration.
In fact, researchers promoting DBR argue that the nature of problems to be studied and the knowledge generated in social science generally are different from those of the natural sciences (e.g., physics) and those of a more explanatory science (e.g., anthropology; Oh et al., 2016; Sadler-Smith, 2014). However, we tend to adopt analytical methods from natural science to make predictions of complex learning and performance problems and use qualitative methods from the humanities to describe and interpret such problems (Oh et al., 2016; Sadler-Smith, 2014). In the HRD field, developing and sustaining innovations in an organization by solving real-world learning and performance problems, particularly through and with technology, cannot be effectively achieved without a synergistic interplay between prolonged design efforts and systematic inquiry activities.
Recently, HRD scholars emphasized that the potential of DBR in advancing the field comes from tackling important problems (Ardichvili, 2012; Kuchinke, 2013; Sadler-Smith, 2014). In essence, DBR is a genre of research that aims to achieve dual goals: (a) improving practices and (b) advancing theories based on a tangible learning or performance problem (McKenney & Reeves, 2013; Oh & Reeves, 2010). To achieve these dual goals, DBR has these distinctive characteristics (van den Akker, Gravemeijer, McKenney, & Nieveen, 2006): “interventionist,” “iterative,” “process-oriented,” “utility-oriented,” and “theory-oriented” (p. 5). First, the interventionist nature of DBR stems from the real needs and desire of practitioners to solve the problems they experience; the desired outcome is to fulfill these specific needs with refined interventions. Second, DBR requires an iterative process of design, enactment, analysis, and redesign to test and refine interventions and design principles. Through this iterative process, design researchers investigate the important variables and constraints influencing the effective implementation and sustainability of the interventions in a selected local setting (Tel & Reeves, 2008). At the same time, through each cycle of iteration, researchers refine design principles applicable to similar problems encountered in the fields beyond the local setting. Third, DBR is process-oriented, as design researchers engage in a naturalistic inquiry and make an effort to understand the context of application thoroughly and refine the interventions continuously. Utility orientation is important as the entire DBR effort is “problem-focused, solution-driven” (Short, Bing, & Kehrhahn, 2003, p. 242). The designed intervention must work in the real practice of an organization. The iterative process of design inquiry and collaboration of multidisciplinary efforts increases the optimal utility of interventions in organizations. Finally, an intervention is developed based on initially conjectured theoretical grounds, and the empirical refinement process of interventions also generates the theoretical outcomes of a DBR.
In a similar vein, from a technology research perspective in HRD, Reeves’ (2006) three critical characteristics of DBR can shed light on the overall direction. First, for DBR to bring about innovations in HRD using technology, the research must “address complex problems in real contexts in collaboration with practitioners” (p. 58) found by identifying and defining real learning and performance problems in their organizations. The relevance and importance of identified problems is a significant focal point in the DBR process, which facilitates prolonged collaboration with HRD practitioners. Furthermore, this participatory problem-solving effort provides the crucial impetus for the iterative cycles of creation and refinement of prototypes of technology solutions and contributing to the applicability and sustainability of the final outcome. Second, DBR must incorporate both known and conjectured design principles with technological advances to extract workable solutions to these problems within the organization. Third, design researchers must engage in reflective inquiry to examine and refine their innovative work-in-progress solutions and to refine the new design principles. This last emphasis on empirical effort is not only important for design and implementation of the best solution for an organization but is also critical for scalability of the solutions in a broader setting. In addition to the solution, the newly generated design principles can be applied in organizations encountering similar problems that hinder their practice and outcomes.
Method
We used two conceptual frameworks to guide this review. First, as research is a knowledge generation activity, different types of research generate different kinds of knowledge. We used the goal categories of educational technology research (Table 1) to identify the ultimate intent of and nature of knowledge created from each technology-related publication. This particular framework was adapted because, as an interrelated field with HRD, educational technology practice and study deal with all educational sectors (e.g., corporations, government organizations, higher education institutions, K-12 schools) and various settings (e.g., formal, informal, nonformal learning, etc.). The Association of Educational Communication and Technology most recently defined educational technology as “the study and ethical practice of facilitating learning and improving performance by creating, using, and managing appropriate technological processes and resources” (AECT Definition and Terminology Committee, 2008, p. 1). The framework was initially developed and used by Reeves (1995) based on Solitis’s (1992) work on categorizing dominant research paradigms in educational research literature. Recently, the framework has been slightly updated so that the goal categories more precisely describe the intent of the researcher. For instance, the 1995 category empirical was changed to exploratory/hypothesis testing (EH) to better explain the goal (Reeves & Oh, 2017). Second, to help identify the process patterns of scholarly activities beyond the first framework and to examine the areas and extent of DBR phases and activities in each article, we modified the DBR activities by labeling the four major phases as analysis, development, testing and refinement, and reflection. Then, each research phase can be explained using associated research activities at two levels. For example, the analysis phase includes the following: Analysis 1: Analysis of a gap in research literature or a practical problem and Analysis 2: Analysis of practical problems by researchers and practitioners in collaboration. Analysis 2 is considered as the ideal DBR activity for the phase, and Analysis 1 is considered the initial effort for the phase (Figure 2). DBR provides a new perspective for understanding technology research in HRD. By including an initial level in each research phase, the modified framework allows us to understand the extent of DBR activities currently exhibited in the technology research literature and identify the areas to pursue for future technology research efforts in HRD.
Goal Categories of Educational Technology Research and Examples.
Source. Adapted the content from Reeves and Oh (2017).
Research phases and activities in spectrum using design-based research perspective.
Therefore, the following research questions guide this review:
In this article, we reviewed 70 peer-reviewed journal articles published from 2007 to 2016. The selected articles had technology-related topics and keywords and were published in peer-reviewed journals associated with professional associations and/or were perceived as highly respected in the HRD field. The journals selected are the following: ADHR, HRDI, Human Resource Development Review (HRDR), HRDQ, and European Journal of Training and Development (EJTD, previously published as Journal of European Industrial Training).
In reviewing articles from these five journals, we organized all the published items in an Excel file and conducted a content analysis using the two aforementioned frameworks. First, each author conducted a content analysis separately and compared his or her analyses with the other author’s analysis in face-to-face meetings. The interrater reliability (Cohen’s Kappa) regarding the research goals was .85, and for the research phases and activities in DBR perspective was .86, both of which are considered very good strength of agreement (Landis & Koch, 1977). During the coding comparison meetings, we explicitly discussed any disagreement among our individual analysis and established a shared understanding of the coding categories (Schreier, 2012). An initial disagreement in the goal analysis mostly occurred when assigning theory development/synthesis (TS) and descriptive/interpretivist (DI) goals to a few nonempirical articles. With regard to the second framework, an initial disagreement mostly occurred in assigning Analysis 1 (analysis of a gap in the research literature or a practical problem) and Analysis 2 (analysis of practical problems by researchers and practitioners in collaboration). Through the cyclical process of individual coding, comparison, and negotiation, the researchers have achieved full agreement on their coding.
Results
Overall, 70 articles, an average of seven article per year, regarding technology research were published by selected journals. During the past decade, the highest number of articles were published in 2010 (n = 14) followed by 2014, in which the second highest number of articles were published (n = 13). In both 2008 and 2016, the lowest numbers of articles (n = 3) were published. Among the five journals, ADHR published approximately 34% of the articles. JEIT/EJTD, HRDI, HRDR, and HRDQ published 27%, 23%, 10%, and 6% of technology research, respectively, from 2007 and 2016.
The 70 articles have included 318 keywords in various areas regarding technology. The top three most frequently used keywords were (a) VHRD or Virtual Human Resource Development, (b) Technology, and (c) E-learning. Less frequently used keywords were those related to leadership, HRD, virtual learning, online communities, and social media. Approximately more than 58% of the keywords were listed only once.
Types of Research Goals
With regard to the research goals pursued in these articles, more than 44% of the articles were published with DI goals. Also, more than one third of the articles (39%) had TS goals. Approximately 12% of articles were published with EH goals. The total number of articles published with action/evaluation (AE; 3%), design/development (DD; 1%), and critical/postmodern (CP; 1%) goals were approximately 5% of the overall publications. A sample article for each category is included in Table 1.
Figure 3 presents the types of research goals pursued in articles published by each journal. JEIT/EJTD has published articles with five types of research goals. HRDI has included articles with four types of research goals although 75% of the articles had DI goals. ADHR has printed articles with three types of research goals, and more than 62.5% of the articles had TS goals. Both HRDR and HRDQ have published articles with two types of research goals. In the case of HRDR, published articles mostly had TS goals. Four articles evenly pursued DI and EH goals in HRDQ.
Research goals pursued in articles from each journal (2007-2016).
Technology Studies From the DBR Perspective
When examining research activities using DBR cycles, all the articles had analyses of a gap in the research literature or a practical problem (Analysis 1), yet only four articles (6%) analyzed practical problems experienced by researchers and practitioners in collaboration (Analysis 2). As an example of Analysis 2, Simmonds and Lupi (2010) described not only analyses based on the literature but also the details regarding the context and process of e-mentoring program development and implementation in Starwood Hotels and Resorts. Nine articles (13%) included development of relevant hard or soft technological solutions (Development 1), but only two articles reported existing design principles clearly for developing such technological solutions (Development 2). As an example of Development 2, Short (2013) reported how a customized training solution for a company in a virtual 3D environment was designed based on specific guidelines from research literature. Six articles (9%) discussed the testing of solutions (Testing 1) using qualitative, quantitative, and mixed methods, yet none included iterative cycles of testing and refinement of solutions in practice (Testing 2). All the articles ended with a reflection on the literature and their findings; however, we found no article in which authors devoted their reflections to producing specific design principles and enhancing solution implementations, both of which are regarded as key elements in DBR studies. See Table 2.
Research Phases and Activities Included in Publications.
Examining the types of research activities from DBR perspective, 59 articles (CP, TS, DI, or EH goals) covered two research activities such as analysis of a gap in the research literature or a practical problem (Analysis 1) and a reflection on the literature and its findings (Reflection 1). Seven articles included three research activities. Five (TS or DI goal) involved development of a solution (Development 1), and two (AE or EH goal) contained testing of solutions (Testing 1) in addition to analysis and reflection activities. Finally, four articles contained research activities from all four phases. All four conducted analyses of practical problems done by researchers and practitioners working in collaboration (Analysis 2) and provided rich description regarding the problems and contexts of the problems. Two of them (DI goals) developed solutions (Development 1). And two of them (AE or DD goal) not only built the solutions but also presented specific design principles and technological innovations (Development 2) that informed the solution development. All four articles tested solutions (Testing 1) and provided reflection on literature and research findings (Reflection 1). The researchers in the article with an AE (n = 1) goal based on the literature review developed a new model for an international chain of hotels, implemented the e-mentoring program, and evaluated its benefits (Simmonds & Lupi, 2010). One of the two articles with a DI (n = 2) goal explored learners’ engagement in a Performance Appraisal training course developed based on an experiential learning approach in Second Life virtual world environment (Morse, 2010). The researcher discussed findings and reflections regarding issues such as learning, design, and delivery in that environment. The article with a DD (n = 1) goal reported a case study on designing customized trainings for an organization in a 3D virtual world as a learning system (Short, 2013). Such research effort was made based on the organization’s need to develop a virtual training solution to reduce training travel costs while providing a high-quality professional development experience for the employees.
Discussion
We reviewed 70 technology research studies published between 2007 and 2016 in five major HRD journals. The analysis of research goals showed more than 40% of the articles pursued a descriptive/interpretive goal, and almost 39% of the articles had a synthesis/theory development goal. The major strength of technology research studies in HRD is in describing and conceptualizing what is happening. Although there is not a one-to-one match, our findings echo the proposition that some research goals tend to be related to certain research methods (Reeves & Oh, 2017). For instance, when researchers pursue synthesis/theory development goals, a literature review is often used. The articles having DI goals tend to employ qualitative methods. Research goals drive the methods we select, and the nature of generated knowledge is determined by the research methods chosen. Although researchers can pursue technology research efforts through research studies with DI and synthesis/theory development goals, the theoretical understanding that emerges from those studies can come as a result of asking research questions such as “What do we know about this topic?” or “How is the experience perceived by participants?” While those are important, such inquiry effort may be limited in initial understanding of theoretical grounds for technology research process or experiences from technology interventions. In fact, both DI articles reviewed with a developed technology solution seemed to describe an early technology research effort. Typically, the overall DBR efforts focus more on design and development goals by asking research questions such as “What is the problem and how can we solve it?” (Reeves & Oh, 2017, p. 335). Answering this question requires using a mix of research methods. And theoretical outcomes generated from a DBR project can be “descriptive, explanatory or predictive in nature” and more “prescriptive/normative, providing guidance for future design efforts” (McKenney & Reeves, 2013, p. 98). From a DBR perspective, employing more diverse research efforts will contribute to developing and refining the technology research solutions in organizations and theoretical outcomes of HRD scholarship.
A relatively small number of articles included development and testing and refinement activities. The articles with all four activities included neither iterative cycles of testing and refinement of solutions in practice (Testing 2) nor reflections to produce design principles and enhance solution implementations (Reflection 2). These are important steps for technology research and adoption, not only to refine the solution in the local setting but also to distribute theoretical outcomes for a broader audience. Our review also found that the majority of keywords were used only once among the reviewed studies, which may imply a possible lack of prolonged effort in continuing research programs.
The absence of prolonged technology research engagement in HRD is not uncommon in workplace contexts. The findings of this study, to a large extent, reflect the reality of technology adoption and acceptance in the workplace. Rogers’s (2010) diffusion of innovation (DoI) provides some explanation, which entails multiple stages for technology adoption to initiate, be interrupted, become sustainable, and eventually fade out. In other words, long-term engagement with technology adoption efforts is essential. Among factors influencing the technology adoption processes, Rogers favors the uncertainty aspect of adopting new and somewhat experimental technologies when the outcomes of such adoption cannot be predicted with confidence. Such uncertainty further leads to undercommitted resources from organizations. As a result, the essential lengthy technology research processes are often interrupted.
Considering the sustainability of technology research in organizations, the finding of this review study revealed which technology research processes would be more likely to be adopted by organizations. This pattern can be explained by the contingency theory in HRD (Kuchinke, 2003) that explains why HRD processes are often localized to organizations’ realistic needs; therefore, they deviate from generic process models (i.e., the DBR process in this present study) that are often theoretically sound. In particular, we consider the following two propositions from the contingency theory in HRD to be most relevant to the findings of this review study:
Organizations will decide on investment in HRD based on cost–benefit considerations and with investment in HRD products and services against other forms of building, maintaining and increasing human capital. Given uncertain returns on investment, organizations will invest conservatively in HRD and focus on system maintenance rather than system change. (p. 304)
The above proposition echoes our previous discussion that organizations are less likely to commit resources long term due to a perceived high level of risk associated with technology research investment for HRD purposes. Uncertainty of technology research outcomes seems highly likely to associate with the perceived high level of risk.
Organizations will support multistage HRD core processes, such as instructional systems design or action research, by assessing the need for each stage and implement HRD core process steps based on the expected utility of each step. (p. 305)
The above proposition aligns with the finding of this study that none of the reviewed studies applied the full DBR process. This could be owing to the challenging nature of designing and conducting longitudinal studies that are supposedly ideal for fully implementing the DBR process. Cross-sectional studies reviewed in this study are very much limited in taking full advantage of the DBR process. Our finding suggests that both organizations and individuals directly involved with technology research efforts tend to localize technology research processes whenever they see fit based on organizational resources and needs.
The significance of this research is twofold. First, the findings provide a bird’s eye view of a decade of scholarly efforts on technology research in HRD considering various research goals and a designated process for technology research. Second, situating technology research in the DBR process framework not only helps us understand current trajectories of technology inquiries in HRD but also encourages a holistic approach to manage technology research efforts in organizations.
Moving forward, we would like to encourage HRD researchers to consider DBR as a viable approach in HRD technology research. We acknowledge that there is a perception that DBR takes a longer time and requires greater effort that doctoral students and early career scholars can put forth. However, a DBR project should be understood as a strategic research program that can be divided into smaller chunks of cycles. Each chunk of cycles will be likely to include empirical effort that can serve as an important building block for the overall DBR project. For example, a dissertation and a publication can target a full or partial meaningful chunk of cycles and report that (Oh et al., 2016). In addition, during the DBR process, the longest time could possibly be spent on the first iteration, as it requires the theory-grounded enactment of the first-version solution and its testing. Due to the organizational situation, even if a DBR project needs to be ended only after the two iterative cycles including a refinement and re-implementation/re-testing, the DBR effort will still lead to a more refined solution to the problem and more useful theoretical outcomes. As a scholarly community, we believe that the accumulation of such DBR effort and activities will lead to a vast volume of process and outcome knowledge beneficial to both the research and practice of technology research in HRD.
Finally, in terms of limitation, although the five journals were selected based on their respective representation in the HRD field, we also acknowledge that there are journals beyond these five in which HRD scholars publish their work. In addition, the limitation of this review consists of common cautions in qualitative inquiries.
Conclusion
Technology studies have presented many opportunities to improve the research and practice for the HRD community. The present review offers a different perspective, based on DBR thinking, to illustrate the progress and the accompanying challenges in identifying and advancing the role of technology research in HRD research and practices. Among many possibilities, moving technology research efforts from conceptualization to empirical implementations and refinement is a timely task, considering the maturity of technological applications reviewed by this study. The literature also has accumulated sufficient conceptual foundations for furthering technology research in HRD in the past decade. In the near future, scholarly activities ought to focus more on the “design” and “testing” of technological applications, tools, and artifacts, and their implementation and integration into the workplace for improving individual learning and organization performance. In pursuing such an effort, DBR can play an important role in bringing about meaningful advancement in technology research in both HRD scholarly and practice communities.
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
