Abstract
According to certain thought leaders, the research–practice gap in compensation management is an important issue, but our ability to understand and make final judgments about it is hindered because important pieces of information are missing. Studies are lacking that show whether compensation professionals are out of touch with research findings, the practical value practitioners place on academic research and the reasons why academics have shown little interest in conducting research in compensation management. This article explains why it is difficult to assess the gap’s seriousness in the current state of knowledge.
In 1969, the Conference Board surveyed 302 companies to determine how much they had been influenced by the findings from the behavioral sciences. A total of 241 reported having some degree of interest in the behavioral sciences, of which 70% had a least a moderate to great interest. Seventy-three percent used behavioral science consultants, about 40% participated in research, and 56% installed new management systems. About half of the consultants were drawn from management consulting firms and half from academia. Approximately 3,000 research or application projects were reported, 8% of which dealt with compensation and benefits programs.
The research report also stated that behavioral psychologists had only scratched the surface in offering help on the practical problems of industry. It stated that more could be done if practitioners were more willing to apply the latest thinking from the behavioral sciences. It also cited a gap between behavioral theory and practice that was the prime responsibility of behavioral scientists to close. Scientists were faulted for not taking the necessary steps to implement their theories and research findings in practical ways and for being unable to “speak the language” of business and industry. 1 This was the first mention of a research–practice gap in human resources (HR) that could be located.
The gap issue has been revisited in the past decade, and some experts believe that it merits attention. In recent years, interest can be traced to the introduction of the concept of evidenced-based management.2,3 Evidence-based management is a recent development in management thinking that calls for organizational policies and practices to be based on the best available scientific evidence from peer-reviewed sources, along with professional judgment and ethics. It received attention as thought leaders believed that practitioners were not taking enough advantage of scientific evidence in their work and were relying too much on unproven traditions, familiar skills and information from consultants with services to sell. 4
In recent years, the Academy of Management and the Society for Industrial & Organizational Psychology (SIOP) have devoted considerable resources to evidenced-based management and the research–practice gap in HR, and several scholars have noted a divergence between research and practice in compensation management and stated that many organizations are not following best compensation practices.5,6,7
This article examines the major studies and forces behind the HR research–practice gap issue and recommends actions to resolve it, primarily as it relates to compensation and benefits management.
Knowledge of Research Findings
In 2002, the Society for Human Resource Management (SHRM) surveyed 959 HR managers, directors and vice presidents to determine the extent to which their beliefs about employee behavior were consistent with research findings from the behavioral sciences. Academic researchers developed 35 true/false items to test practitioners’ knowledge in five practice areas.
Respondents answered an average of 57% of the questions correctly, according to the researchers, and showed a wide variation on the extent of agreement on particular items as individuals and as a group. The largest gaps between research findings and practitioner beliefs pertained to some of the most central issues in HR—employee selection, performance management and motivation through goal setting. The practice areas and percent of correct answers are shown below.
Employee training and development: 71%
General employment practices (legal issues, performance appraisals and employee attitudes): 68%
Management practices (motivation, leadership, performance management, employee involvement and HR roles): 63%
Compensation and benefits: 58%
Staffing: 39%
The eight questionnaire items on compensation and benefits are shown below, along with the percentage of correct answers. Correct answers for each item are shown in the appendix for readers who wish to test their knowledge and gage the difficulty level of the survey.
Most employees prefer individual pay plans over team and organizationally based pay plans: 81%
When pay must be reduced or frozen, little can be done to reduce employee dissatisfaction: 72%
Companies with no merit pay plans tend to outperform perform those that have them: 66%
There is a positive relationship between organizational performance and the number of managers paid under organizational pay plans: 62%
New companies have a better chance of succeeding if all employees receive organizationally based financial incentives: 59%
Discussing salary increases during performance reviews hurts morale and future performance: 51%
Most employees prefer variable pay systems over fixed pay systems: 40%
When asked in surveys to directly rate the importance of pay, employees overestimate its importance as compared to their actual decisions about it: 35%
Performance management items in the general employment practices section are shown below. Correct answers for each item are shown in the appendix for readers who wish to test their knowledge and gage the difficulty level of the survey.
Supervisors tend to underrate employees’ performance: 94%
Poor performers are more realistic about their performance than good performers: 88%
Rater errors on employee performance appraisals can be corrected with increased training: 25%
The researchers also reported the following additional findings:
Respondents had an average of 13.8 years of HR experience.
Higher level and Senior Professional in Human Resources–certified professionals scored better on the test than lower level, uncertified professionals.
Respondents were neutral about the usefulness of research findings.
The most frequently used source of information for solving HR problems is other HR professionals in the same organization.
The only periodical that the respondents read more than “sometimes” was SHRM’s HR Magazine, which the typical respondent “usually” read. Less than 1% usually read the three most research-oriented journals.
Since most practitioners did not read research journals regularly, the study authors’ main recommendation was to include more research content in practitioner publications, where according to the researchers, most content was provided by nonresearchers—consultants and HR professionals. 8
Comments
The survey results cannot be taken at face value for several reasons:
Despite their great importance to HR leaders, no coverage was given to employee benefits topics, such as health care cost containment, where research studies exist. 9 One has to question whether the questionnaire tapped into the pressing issues in the world of HR.
In the compensation and benefits section, none of the research findings were based on the results of meta-analytical, summary studies of many individual studies to support the idea that the findings rose to the level of a principle of organization behavior (OB). 10 This raises the question of whether it is reasonable to expect HR practitioners to be aware of research findings that do not have broad support and general applicability. For example, in the compensation and benefits section, for the item pertaining to methods for obtaining employees’ beliefs about the importance of pay, on which the HR practitioners scored the lowest (35%), the right answer was primarily based on one research study done in 1978 with 62 graduate students. 11
The study authors claim that the correct answer to the following questionnaire item is “false”: Rater errors in performance appraisals can be corrected with increased training. Survey respondents “correctly” answered it only 25% of the time. Other experts would disagree with the correct answer. A well-respected compensation textbook, written by compensation scholars George Milkovich and Jerry Newman, stated in 1987 that “although there is some evidence that training is not effective or less important in reducing errors than other factors, the majority of findings are quite supportive of training raters as an effective method to reduce appraisal errors (p. 347). It also stated, “Several researchers have indicated that errors can also be prevented in the planning and actual conduct of appraisal interviews” 12 (p. 349). The 2011 edition of the textbook supports the value of rater training. 13 When experts disagree over what the research says, it is easy to see why HR practitioners can give “wrong” answers.
The study authors did not seem to consider that the experiences of HR practitioners could cause them to hold beliefs that would conflict with research findings. Research studies can be general in nature or apply only to certain groups of employees and organizations, which makes it possible for practical experience to sometimes conflict with research results and cause people to hold different, and valid, views about HR concepts. The results of consultant surveys can also have the same effect. This is probably one reason why practitioners did not score higher on the test.
The study did not obtain questionnaire responses from HR specialists to provide a basis to judge whether it was reasonable to expect the survey respondents, who were generalists (HR managers and above), to know of research findings in HR specialty areas. If specialists demonstrated a low level of awareness of the findings, perhaps it would be unreasonable to expect generalists to be aware of them.
The study authors, who are academic researchers, assumed that the consultants who write for professional journals are unaware of academic research findings, despite the fact that some of the most prominent consultants who make substantial contributions to professional publications (e.g., Edward Lawler, Gerald Ledford and Robert Greene) are members of the same professional association as academic researchers, the SIOP. In addition, large consultancies employ OB psychologists to conduct research and supply content for articles.
The study was conducted in 2002, and it is uncertain whether the stated gaps in knowledge exist today. Some experts, primarily academic researchers, believe that gaps persist, as described below.
In general, this review of the research–practice gap study does not conclusively demonstrate that HR managers and above are out of touch with compensation and benefits research findings that have broad support and general applicability.
Practical Relevance of Research
In 2007, two academic researchers conducted a study to determine if academic researchers were interested in the same HR topics as practitioners and to learn if different interests explained why practitioners were uninterested in research. The thinking was that if practitioners generally felt that research findings did not pertain to their interests, they would see little value in being informed by research and not seek it out.
Published articles in 15 topic areas were considered as indicators of HR interest areas. The researchers performed a content analysis of two HR academic-oriented, research publications (Journal of Applied Psychology and Personnel Psychology) and two professional-oriented publications (Human Resource Management and HR Magazine) over a 20-year period. The sample size was 1,665 academic articles and 2,691 professional articles.
They found that the three dominant interests of HR professionals pertained to compensation and benefits, HR functional issues, such as departmental innovation and effectiveness, and employee training and development. The three dominant interests of HR researchers were OB and motivation, staffing and HR functional issues. In general, they found that HR professionals and researchers did not agree on the importance of most topic areas.
The largest gaps in topics were for compensation and benefits (ranked 1st for professionals, with 386 articles, and 14th for researchers, with 33 articles) and for OB and motivation (ranked 1st for researchers, with 278 articles, and 11th for professionals, with 96 articles). They found that the gap for compensation and benefits was consistent in each of the 20 years and was not affected by outside influences such as economic recessions. This finding was supported by an analysis of calls received by SHRM’s information center for a 6-month period in 2006, during which 49% of the calls pertained to compensation and benefits matters. Fifty-two percent of those calls concerned the FLSA (Fair Labor Standards Act), FMLA (Family and Medical Leave Act), COBRA (Consolidated Omnibus Budget Reconciliation Act) and military leaves.
The study authors believed that the gap in interest was due to the different goals of the two groups. HR professionals appeared to be most interested in the practical aspects of their jobs and the HR researchers were most interested in creating basic, generalizable research. They believed that as long as the goals diverged, the interests of the two groups would also diverge and knowledge gaps would persist. They recommended several steps to narrow the gap, such as forming closer alliances between researchers and professionals in their professional associations and developing a common body of knowledge that HR professionals should possess and obtain through their HR education programs. 14
Comments
Much of the disparity in the number of compensation and benefits articles in research and professional journals is due to the fact that a significant amount of the information needed by practitioners relates to legal matters (e.g., FLSA, FMLA, and COBRA) and recurring survey results (e.g., budgeted salary increases and professional practices), which are being met with articles in professional journals with content supplied by practicing attorneys, consultants and professional associations.
The relatively small number of research studies on compensation and benefits over a 20-year period in two major research journals (33 articles or 2% of the total number of articles) indicates that researchers are not giving this major HR area an adequate amount of research attention.
Lack of Interest in Compensation Management
The lack of academic interest in compensation and benefits has been confirmed in recent years by other academic researchers. In 2000, compensation scholars, Sara Rynes and Barry Gerhart,
15
observed that many OB psychologists apparently had lost interest in the topic of employee compensation. Rynes and Bono
16
stated that
when the present body of research is evaluated in light of the broader environment, it becomes clear that industrial and organizational psychology research has not kept pace with the changes in real world pay systems, with developments in other disciplines, or with concerns of practitioners (p. 22).
The lack of interest exists today.
In 2011, only 24 of the more than 8,000 members in the SIOP had a primary interest in the field of pay and compensation. 17 A review of the association’s 2011 annual conference program found that only a handful of the over 300 sessions dealt with pay or compensation. One compensation scholar ran a posting at the conference to form a community of interest on compensation management, as if were a newly developing field that deserved attention. This low level of interest has probably minimized the size of any research–practice gap.
Science You Can Use
In 2010, because of the interest in evidenced-based management, SIOP-affiliated organizational psychologists attempted to provide relevant organizational psychology science to practitioners through an annual publication, Science You Can Use. One goal was to help close the gap between OB academics and practitioners by bringing them together to collaborate on the publication’s chapters. 18
In 2011, the SIOP executive board concluded that there was not much momentum behind the idea and stopped work on the publication. It concluded, “Regarding the science you can use idea, there is a long line of failed attempts to create practitioner-oriented materials, and we are not optimistic that the evidence-based series would fare any better.” 19 SIOP is working to develop other materials about topics to be distributed by SHRM to its membership. To date, two articles have been prepared by SIOP members who are HR consultants, not academics. 20
Contributions of Consultants
Edward Lawler, who by virtue of his position as the Director the University of Southern California’s Center for Effective Organizations, is in a good position to comment on both academic and practice-related issues. He also has many years of experience as an academic and business consultant and is an expert in the field of employee compensation.
Lawler 21 has noted that the lack of academic research has led to the growth of research studies by consulting firms. He stated that the leading consulting firms have created their own research departments that regularly contribute to research-based articles to practitioner journals and have written books that describe best practices based on their research findings. Recent developments in the field of employee rewards lend support to his observations, as the major new concepts that have dominated the field in the past 10 years—total rewards, work–life balance and employee engagement—have neither been initiated by nor do they reflect significant contributions from academic researchers.
Consultants have a major advantage over academic researchers—business credibility. Their work is oriented more toward HR practices than theory and relates to current organizational issues. Furthermore, it is based on the behavior of actual employees, not college students who are frequently used in academic research studies.
Research tends to give practitioners a general understanding of employee behavior. They also need help in determining if it applies to their particular workforce and how solutions should be implemented for best results. Consultants seem to be in a better position to provide this guidance than academics. They have far greater experience in the business world than do most academic researchers and have field-tested tools and systems to recommend. The dominant position of consulting firms may be one reason why academics show little interest in the field of employee compensation.
Also noteworthy is the fact that professional associations provide practitioners with a considerable amount of useful information related to the prevalence and effectiveness of HR policies and practices.
Final Thoughts
Although the research–practice gap is an important issue for many experts, important pieces of information are missing that hinder our ability to understand and make final judgments about it, as it pertains to compensation and benefits matters. Following are the missing pieces of information and action steps to resolve them.
Academic researchers have not done a convincing job of showing that HR practitioners are out of touch with significant research findings on compensation and benefits matters. To see if a meaningful gap exists, researchers need to test practitioners’ knowledge with questions based on generally accepted meta-analytical research studies that are at an appropriate level of difficulty and reflect the major issues of current interest to HR practitioners.
HR practitioners have not been surveyed in the past 10 years to find out how much value they place on academic research and the reasons why they hold those beliefs. The very real possibility exists that practitioner informational needs are being adequately met by HR consultants and professional associations. Professional associations need to survey their members to be better informed on this topic.
We do not know why OB psychologists have shown little interest in employee compensation management, even though it has great importance in most organizations. The SIOP should survey its members to learn the reasons why. Could it be that they believe there is little more to learn about employee compensation?
Despite practitioners’ apparent lack of knowledge about research findings, academics have not compiled a list for practitioners of meta- analytical OB studies. A list would make it easier for practitioners to access accepted research results, as well as to understand the magnitude of the research–practice gap issue.
Two books are a good starting point for HR practitioners seeking evidenced based HR data. At the end of each of the 18 chapters in his textbook, Organizational Behavior, Fred Luthans 22 summarizes the results from 1 to 4 meta-analytical studies (total of 25) to establish a basis for stating a principle of organizational behavior. Edwin Locke’s Handbook of Principles of Organizational Behavior 23 lists 33 HR principles of organizational behavior that are evidenced based.
There has been much discussion about the HR research–practice gap issue, but not enough research has been done to assess its seriousness and to uncover the root causes to deal with it effectively if it is a problem.
Footnotes
Appendix
Declaration of Conflicting Interests
The author declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author received no financial support for the research, authorship, and/or publication of this article.