Selecting the Right Evidence to Inform Compensation and Benefits Decisions

Using Research

To be sound, research must be conducted using a methodology that conforms to the scientific method. It must result in findings that are both internally and externally valid. Internal validity is achieved through appropriate study design. For example, an often-quoted lab study found that people would throw tennis balls at targets longer if they were not paid to do so. Because the study design was sound, it indicates that a similar result might occur under similar conditions, although a single study is unlikely to stand alone as sufficient evidence. But that study has been cited as evidence that extrinsic rewards can diminish intrinsic rewards (Pink, 2011). To assume a task that may be enjoyable and that is performed for a short period for an insignificant amount of money bears any resemblance to someone working at an undesirable job for many years to support a family is folly. To conclude that the results of a lab study would have a similar outcome in another setting, the two contexts would have to be similar, which in this case they are not. This highlights the danger of applying results from a controlled lab study to field settings, where the same conditions do not exist. In this case, the findings of that single lab study conflict with a substantial amount of field research. That research indicates that there is no causal relationship and that organizations should strive to offer both types of rewards.

Lab research studies can certainly provide valid evidence, even if it cannot be assumed that they are relevant in very dissimilar settings. There are ways to extract useful intelligence from study findings. The academic literature has historically reported on individual studies that have limited generalizability. In an attempt to increase the probable usefulness of those studies, a tool called meta-analysis can be used to aggregate the findings of individual studies. Subjective information (such as individual opinions and anecdotal evidence) can also be added to create what are called systematic reviews (Barends & Rousseau, 2018). By testing hypotheses such as “basing rewards on performance increases motivation to perform well” across a number of studies, knowledge can be gained as to whether the principle might apply in a variety of contexts (Barends & Rousseau, 2018).

Using Personal Expertise

Practitioners accumulate professional knowledge through experience, education, and training. Someone deemed an expert on a topic may legitimately be viewed as a source of information that can be used to inform someone addressing an issue. Recognized experts are often sought as advisors (who can be either internal or external) and their contributions to the literature can be used to inform decision-makers who might not possess the required knowledge. One of the advantages of working in a collaborative culture and in teams is that the aggregated pool of knowledge possessed by colleagues can be accessed.

The literature in a field is a medium for accessing the knowledge of experts who otherwise might not be accessible or affordable. Consulting firms and accomplished practitioners produce articles, white papers, and seminars and speak at professional conferences, all of which may be useful information practitioners can benefit from. Yet there is an inherent bias in the practitioner literature that may mislead. There are articles that will never be written or published. For example:

“This article describes how an incentive plan I designed and implemented resulted in major employee dissatisfaction, increased turnover and diminished performance.”

People do not rush to publish or publicly advertise their mistakes, so the practitioner literature only contains successes. One of the authors consulted with a number of organizations to undo the damage caused by an inappropriate implementation of broadbanding systems. When he asked the people who made the decision to adopt these systems, they cited all of the stories in their practitioner publications that portrayed successes, and no failures were reported.

For these reasons, more articles similar to the following should be considered for publication:

“This research study did not support the hypotheses it set out to test. Yet failure can be a source of valuable learning, so its findings help us understand what is unlikely to work.”

Reviewers and editors of academic journals rarely accept studies for publication that are not “successful,” which means the hypotheses were not supported. This biases that literature.

Most famous inventors, such as Edison, failed repeatedly but managed to use their errors to find the right path. Human nature makes it unlikely we will make the effort to find the attempts by others that did not result in success. For this reason, the counsel provided by “experts” should be scrutinized and not accepted as sound without ensuring that experts’ knowledge is relevant to the organization’s specific context. Experts should also know when an action is likely to fail to produce the desired results and should inform the potential adopter of what experience has shown.

Using Organization Data

Data analytics has become one of the most popular topics addressed in articles, books, seminars, and consulting reports. The emergence of new technology has made it possible to mine data banks and to discover relationships. For example, an organization may test the model it uses for selecting candidates for employment by analyzing the relationship between the factors used and the performance and retention results. Regression analysis and other tools can test the effectiveness of individual practices, such as recruiting at specific schools or the type of selection process used. One of the current issues is pay equity, and it is incumbent on organizations to ensure their practices do not contribute to inequities. Analytical tools can enable comparisons to be made across “similarly situated” incumbents, rather than the meaningless aggregated numbers that include all employees. This intelligence can be used to create algorithms that can guide consistency in adopting programs and practices that have been shown to work.

The use of analytics must however be subjected to human judgment. Facebook and Google both invested large sums into developing algorithms aimed at lessening bias in selection, only to find they resulted in more bias than prior decisions had produced. Algorithms are limited to basing their prescriptions on the data used to create them and any human direction that was provided in choosing the datasets, termed “supervised machine learning.” One of the errors made by these two organizations was feeding primarily male resumes to the machine learning system, which led to male characteristics to be favored. Another concern about algorithms is that they use past and current data, which may be problematic if the future is not going to be similar to the past and present. Finally, algorithms are not influenced by evaluating the desirability of outcomes. An organization adjusting pay for female employees to make relationships more equitable may find it unleashes negative reactions by male employees who have come to accept existing relationships as equitable.

Decision-makers who do not possess the technical knowledge that the data scientists who develop the analytical systems have may not be able to understand how the “black box” formulates recommendations. And the data scientists may not possess the business knowledge necessary to understand the likely impact of things like human nature on the outcomes. A multinational that adopts a global pay system without having considered the impact of the cultural diversity among members of the workforce may find that some participants reject the system as inappropriate (Trompenaars & Greene, 2018). Factoring in human reactions could result in concluding that there is no one system that will be accepted as equitable, competitive, and appropriate by everyone. Knowing that to be the case can lead to appropriate customization across cultural groups.

Considering Stakeholder Perceptions

Developing a pay philosophy that is ideal from the organization’s perspective may not be accepted as ideal by other stakeholders. Some public sector organizations use automatic time-based pay adjustment systems, which simplifies administration and may please employees. But taxpayers who fund the costs may find this inappropriate since behavioral science research suggests this approach does not motivate maximum effort and high levels of performance (Greene, 2019). Automatic inflation of fixed costs may also make it difficult for management to keep costs aligned with revenues. And the rich defined benefit retirement programs that exist in much of the public sector may promote retention, but also create large unfunded liabilities that the organization and/or the taxpayers will have to pay for in the future.

Executive compensation is a contentious issue, particularly in the United States. The current record high stock valuations can result in the creation of considerable wealth for participants in equity-based plans. The shareholders may accept this enrichment as justifiable if they are also benefitting, but employees who are not plan participants may believe the gap between what a few make and what the many make to be unwarranted. And if the performance of an organization is not consistent with the rewards accruing to executives, the shareholders may have mixed emotions about the plans.

Stakeholder perceptions can be rejected as subjective opinions but ignoring them can result in decisions that will be resisted by them. Their perceptions are their reality, whether or not they are warranted. Once all the scientific evidence, professional expertise, and data analytics are applied to inform a decision, the last step should be the consideration of how the concerned parties will react.

Conclusion

Correctly using all the sound and relevant evidence that is available will increase the likelihood that good decisions will be made. Everyone facing surgery wants their doctor to base recommendations on evaluating all the evidence that is likely to lead to the right decision. Decision-makers are increasingly facing complex issues created by turbulent and unpredictable environments. By using all of the available evidence and analytical tools, the likelihood of choosing wisely among alternative compensation and benefits solutions is greatly increased.