Organizational Metrics of Technology Use and the Transparency Paradox

Abstract

Metrics are increasingly used by organizations to communicate and evaluate how employees’ actions align with organizational goals. In practice, metrics provide a mechanism for making some work visible, but in doing so can create a paradox whereby it is more difficult to understand what behaviors are contributing to outcomes. This study focused on the use of TechMetric, a composite score reflecting online behaviors at Beta Corp., a large multinational technology-consulting organization. Ethnographic research captured the communication surrounding the introduction of TechMetric. Our findings showed that the implementation of TechMetric created situations in which the use of metrics communicated a shift in what managers prioritized and ultimately discouraged desired behaviors and undermined optimal work routines. This research makes an important contribution to the study of organizational metrics by illustrating how the measurement of workers’ behaviors can enact distinct tensions. Further, the communicative efforts to navigate those tensions can produce paradoxical outcomes.

Keywords

metrics organizations social media sales processes transparency

Get full access to this article

View all access options for this article.

References

Ananny

Crawford

(2016). Seeing without knowing: Limitations of the transparency ideal and its application to algorithmic accountability. New Media & Society, 20(3), 973–989. https://doi.org/10.1177/1461444816676645

Ashcraft

G. E. K. L.

Trethewey

(2004). Special issue synthesis: Developing tension: An agenda for applied research on the organization of irrationality. Journal of Applied Communication Research, 32(2), 171–181. https://doi.org/10.1080/14795752.2004.10058565

Bader

Kaiser

(2019). Algorithmic decision-making? The user interface and its role for human involvement in decisions supported by artificial intelligence. Organization, 26(5), 655–672. https://doi.org/10.1177/1350508419855714

Beer

(2016). Metric power. Springer.

Bernstein

E. S.

(2012). The transparency paradox: A role for privacy in organizational learning and operational control. Administrative Science Quarterly, 57(2), 181–216. https://doi.org/10.1177/0001839212453028

Christensen

L. T.

Cheney

(2015). Peering into transparency: Challenging ideals, proxies, and organizational practices. Communication Theory, 25(1), 70–90. https://doi.org/10.1111/comt.12052

Christin

(2020). Metrics at work: Journalism and the contested meaning of algorithms. Princeton University Press.

Espeland

W. N.

Sauder

(2007). Rankings and reactivity: How public measures recreate social worlds. American Journal of Sociology, 113(1), 1–40. https://doi.org/10.1086/517897

Espeland

W. N.

Stevens

M. L.

(1998). Commensuration as a social process. Annual Review of Sociology, 24(1), 313-343. https://doi.org/10.1146/annurev.soc.24.1.313

10.

Faraj

Pachidi

Sayegh

(2018). Working and organizing in the age of the learning algorithm. Information and Organization, 28(1), 62–70. https://doi.org/10.1016/j.infoandorg.2018.02.005

11.

Flyverbom

(2015). Sunlight in cyberspace? On transparency as a form of ordering. European Journal of Social Theory, 18(2), 168–184. https://doi.org/10.1177/1368431014555258

12.

Flyverbom

(2019). The digital prism. Cambridge University Press.

13.

Flyverbom

(2022). Overlit: Digital architectures of visibility. Organization Theory, 3(3), 26317877221090314. https://doi.org/10.1177/26317877221090314

14.

Glaser

(1978). Theoretical sensitivity: Advances in the methodology of grounded theory. University of California Press.

15.

Glaser

Strauss

(1967). The discovery of grounded theory: Strategies for qualitative research. Sociology Press.

16.

Hammerness

(2004). Teaching with vision: How one teacher negotiates the tension between high ideals and standardized testing. Teacher Education Quarterly, 31(4), 33–43.

17.

Hansen

H. K.

(2015). Numerical operations, transparency illusions and the datafication of governance. European Journal of Social Theory, 18(2), 203–220. https://doi.org/10.1177/1368431014555260

18.

Hansen

H. K.

Flyverbom

(2015). The politics of transparency and the calibration of knowledge in the digital age. Organization, 22(6), 872–889. https://doi.org/10.1177/1350508414522315

19.

Hodder

(2020). New technology, work and employment in the era of COVID-19: Reflecting on legacies of research. New Technology, Work and Employment, 35(3), 262–275. https://doi.org/10.1111/ntwe.12173

20.

Jean

(2010). Identity management’s misaligned incentives. IEEE Security & Privacy, 8(6), 90–94. https://doi.org/10.1109/MSP.2010.178

21.

Jian

(2007). “Omega is a four-letter word”: Toward a tension-centered model of resistance to information and communication technologies. Communication Monographs, 74(4), 517–540. https://doi.org/10.1080/03637750701716602

22.

Kellogg

K. C.

Kessinger

R. R.

Rothbard

Borpujari

DeFilippis

Leonardi

Polzer

J. T.

(2020). Emerging technologies, work relationships, and the new role of middle managers. Academy of Management Proceedings, 2020(1), 20558. https://doi.org/10.5465/AMBPP.2020.20558symposium

23.

Khanagha

Volberda

Sidhu

Oshri

(2013). Management innovation and adoption of emerging technologies: The case of cloud computing. European Management Review, 10(1), 51–67. https://doi.org/10.1111/emre.12004

24.

Leonardi

P. M.

Treem

J. W.

(2020). Behavioral visibility: A new paradigm for organization studies in the age of digitization, digitalization, and datafication. Organization Studies, 41(12), 1601–1625. https://doi.org/10.1177/0170840620970728

25.

Mazmanian

Beckman

C. M.

(2018). “Making” your numbers: Engendering organizational control through a ritual of quantification. Organization Science, 29(3), 357–379. https://doi.org/10.1287/orsc.2017.1185

26.

Mazmanian

Orlikowski

W. J.

Yates

(2013). The Autonomy paradox: The implications of mobile email devices for knowledge professionals. Organization Science, 24(5), 1337–1357. https://doi.org/10.1287/orsc.1120.0806

27.

Mennicken

Espeland

W. N.

(2019). What’s new with numbers? Sociological approaches to the study of quantification. Annual Review of Sociology, 45(1), 223–245. https://doi.org/10.1146/annurev-soc-073117-041343

28.

Miles

M. B.

Huberman

A. M.

(1984). Qualitative data analysis. Sage.

29.

Mitra

Sambamurthy

Westerman

(2011). Measuring IT performance and communicating value. MIS Quarterly Executive, 10(1), 47–59.

30.

Orlikowski

W. J.

Scott

S. V.

(2013). What happens when evaluation goes online? Exploring apparatuses of valuation in the travel sector. Organization Science, 25(3), 868–891. https://doi.org/10.1287/orsc.2013.0877

31.

Poole

M. S.

Van de Ven

A. H.

(1989). Using paradox to build management and organization theories. Academy of Management Review, 14(4), 562–578. https://doi.org/10.5465/amr.1989.4308389

32.

Putnam

L. L.

Fairhurst

G. T.

Banghart

(2016). Contradictions, dialectics, and paradoxes in organizations: A constitutive approach. Academy of Management Annals, 10(1), 65–171. https://doi.org/10.5465/19416520.2016.1162421

33.

Putnam

L. L.

Myers

K. K.

Gailliard

B. M.

(2014). Examining the tensions in workplace flexibility and exploring options for new directions. Human Relations, 67(4), 413–440. https://doi.org/10.1177/0018726713495704

34.

Ranganathan

Benson

(2020). A numbers game: Quantification of work, auto-gamification, and worker productivity. American Sociological Review, 85(4), 573–609. https://doi.org/10.1177/0003122420936665

35.

Rindova

V. P.

Martins

L. L.

Srinivas

S. B.

Chandler

(2018). The good, the bad, and the ugly of organizational rankings: A multidisciplinary review of the literature and directions for future research. Journal of Management, 44(6), 2175–2208. https://doi.org/10.1177/0149206317741962

36.

Ringel

(2019). Unpacking the transparency-secrecy nexus: Frontstage and backstage behaviour in a political party. Organization Studies, 40(5), 705–723. https://doi.org/10.1177/0170840618759817

37.

Rivera

Qiu

Kumar

Petrucci

(2021). Are traditional performance reviews outdated? An empirical analysis on continuous, real-time feedback in the workplace. Information Systems Research, 32(2), 517–540. https://doi.org/10.1287/isre.2020.0979

38.

Saldaña

(2015). The coding manual for qualitative researchers. Sage.

39.

Sauder

Espeland

W. N.

(2009). The discipline of rankings: Tight coupling and organizational change. American Sociological Review, 74(1), 63–82. https://doi.org/10.1177/000312240907400104

40.

Schafheitle

Weibel

Ebert

Kasper

Schank

Leicht-Deobald

(2020). No stone left unturned? Toward a framework for the impact of datafication technologies on organizational control. Academy of Management Discoveries, 6(3), 455–487. https://doi.org/10.5465/amd.2019.0002

41.

Slager

Gond

J.-P.

Moon

(2012). Standardization as institutional work: The regulatory power of a responsible investment standard. Organization Studies, 33(5-6), 763–790. https://doi.org/10.1177/0170840612443628

42.

Stelmaszak

(2022). Behavioral visibility through analytics: How organizations construct actionable insight. Academy of Management Proceedings, 2022(1). https://doi.org/10.5465/AMBPP.2022.12191abstract

43.

Stohl

Leonardi

P. M.

(2016). Digital Age | Managing opacity: Information visibility and the paradox of transparency in the Digital Age [transparency, digital age]. International Journal of Communication, 10, 123–127.

44.

Strauss

Corbin

(1998). Basics of qualitative research: Procedures and techniques for developing grounded theory. Sage.

45.

Tracy

S. J.

(2004). Dialectic, contradiction, or double bind? Analyzing and theorizing employee reactions to organizational tension. Journal of Applied Communication Research, 32(2), 119–146. https://doi.org/10.1080/0090988042000210025

46.

Van Zoonen

Treem

J. W.

Sivunen

(2022). An analysis of fear factors predicting enterprise social media use in an era of communication visibility. Internet Research, 32(7), 354–375. https://doi.org/10.1108/INTR-05-2021-0341