Perspectives

Washington DC 2001

The morning of Monday 10 September 2001 in Washington DC was simply glorious, with a clear blue sky. The Intranet Focus team of four consultants walked down 19th Street and into the lobby of the International Monetary Fund (IMF) for the start of a three-month long intranet strategy project. We very quickly felt at home, helped by the warmth of the welcome from the IMF project team and the fact that one of the team was a retired long service economist from the IMF who seemed to know everyone and everything about the Fund.

In addition, we had read ‘Inside the IMF’. This was the title of a book authored by Richard Harper in 1998 while working for the Rank Xerox Research Centre in London. This 300-page book, subtitled ‘An Ethnography of Documents, Technology and Organisational Action’, provided us with an immense amount of insight into how information flowed around the Fund and how it was used to further the Fund’s objectives. This remains the only book ever published on the working practices of a major organization.

You may be wondering how I remember the weather on 10 September. It is because the following morning the weather was just as exhilarating, but that day now and forever is known as ‘9/11’.

The term ‘ethnography’ originates from a combination of the Greek words ἔθνoς ethnos (‘folk, people, nation’) and γράφω grapho (‘I write’). It describes a method initially used by social science researchers (cultural anthropologists in particular) to examine the meaning in the lives of a cultural group. Researchers conducting ethnographical studies, or ethnographers, strive to develop ‘thick’ descriptions of everyday life and practice. The challenges in adopting ethnographic research include the amount of time and effort needed to undertake the observations and interviews, the possibility of a bias being introduced by the research team and the analysis of the outcomes.

Over the last few years, computational ethnography has emerged as a powerful means of overcoming, or at least reducing, the challenges of interview-based ethnography. Computational ethnography leverages computer or sensor-based applications to unobtrusively or nearly unobtrusively record end users’ routines in undertaking work tasks. Because data are automatically captured it naturally provides higher objectivity, less intrusion, more inclusiveness (i.e. into spaces and time where/when direct observation by human observers is not possible), and better scalability for data collection, aggregation, and analysis. Among these applications are keyboard and screen-logging software developed to mitigate the loss of sensitive data. However, computational ethnography research needs to be integrated with qualitative research designs such as interviews, diaries and observations. This is termed a mixed-methods research strategy and is an essential tool for enterprise search managers.

There has been a significant amount of discussion in the research community about the practice of ethnographic research and the potential of introducing observer bias into the process. In a review paper (Akemu and Abdelnour, 2018), the authors focus on the extent to which ethnography is relevant and appropriate in digital organizations with a mix of face-to-face and digital (e.g. email and social media). They begin by showing how the overlap of participants’ face-to-face and digitally mediated interactions challenges ethnographic practice. Next, they review the emerging Information Systems (IS) theory on the nature of digital artefacts. The third section of the paper gives two illustrative cases detailing the authors’ experience doing ethnographic fieldwork in digitally mediated settings. The paper concludes with a discussion of field research guidelines and directions for future research. One of the underlying themes of the paper is a consideration of how ethnographic research opens up the possibility of undertaking multisite projects contemporaneously; most ethnographic studies by the nature of the requirement for observation, restricted to a single location. Certainly, there is a possibility of using multiple research teams but this may introduce a variable into the research outcomes which is very difficult to assess.

The authors suggest two ways for organizational ethnographers to be co-present with informants in digitally mediated settings. First, an organization’s digital artefacts and infrastructure may permit an ex post record of informants’ digital interactions. The authors refer to this as closed mode ‘digital as archive’. Second, an organization’s digital artefacts and underlying infrastructure may allow the researcher, at least in principle, to observe how participants interact with and create a digital artefact such that the researcher observes social processes in real time. This is denoted ‘digital as process’. Two case studies are presented which provide a context for the two modes and create a basis for a set of research guidelines.

Abramson et al. (2018) consider the promise of computational ethnography in improving transparency, replicability and validity for realist approaches to ethnographic analysis. They describe the utility of three potentially complementary forms of visualization enabled by modern computation: (1) the use of ethnographic heatmaps (ethnoarrays) to aid in pattern analysis and representation, (2) the combination of participant observation data with techniques used in social network analysis to visualize social relations, and (3) text mining. On the basis of a thorough review of these applications, the authors take the view that computational ethnography is a complementary research tool and needs to be carefully positioned within a research project so that the apparent authority of digital data does not overwhelm the importance of qualitative evidence from observation and interviews.

Text mining

Text analytics first emerged in the late 1990s as ‘text data mining’ or just ‘text mining’. Early approaches would treat a text source as a ‘bag of words’. They evolved to use basic, shallow linguistics to handle variant word forms such as abbreviations, plurals and conjugations as well as multi-word terms known as n-grams. Basic lexical analysis might count frequencies of words and terms in order to carry out elementary functions such as attempting to classify documents by topic. But there was no ability to understand the semantics – the meaning – contained within a given document. A very significant amount of development has taken place over the last two decades in terms of both linguistic analysis using natural language processing and the very significant increase in computational power needed to work through large collections of text. In this paper, the authors (Kobayashi et al., 2018) illustrate how text mining could be of value in research with a very thorough description of the various stages of the text mining process, with the inclusion of some useful example of outcomes.

Text mining is increasingly widely used to explore contextual relationships in documents. The author (Watts, 2018) has developed an interesting view on market dynamics for high-technology opportunities. Using textual data from 13 years of trade journal articles in biotechnology he has constructed a measure which he refers to as ‘language consistency’ and tracks its evolution over the formative years of the biotechnology industry. Consistent with contemporary descriptions of market emergence. The author’s analysis suggests that consistency generally increases over time (i.e. as the industry matures). However, he also shows that many of the drastic drops in language consistency correspond with important market events. In conclusion, the author discusses how this method can be applied more broadly to address contemporary issues in market research such as spotting trends, constructing promotions and tracking market development. I’m not totally convinced by the approach but it is certainly a novel concept and for that reason it gains a place in Perspectives.

Text mining has to take account of the way in which words are being used out-of-context, and a good example is the use of English-language business and finance terms in European languages.

Although it is generally accepted that English is becoming the lingua franca of international business, the details of this process are not well understood. This paper (Anglemark and John, 2018) uses the Google Books corpus to provide both a quantitative and a qualitative investigation of the ways in which specific English business terms are penetrating major European languages. Some English business terms now appear to be firmly established in other languages, and can be classified as lexical borrowings, while the use of other terms is better described as code-switching.

While some of the terms we investigated appear to be integrated to a degree in some target languages (TLs), the English-language terms are still often used in a way that marks them as foreign. This applies especially to the French and Spanish texts. Italian and in particular German show a much greater degree of integration of the terms. This conclusion emerges both from examination of the time-series data and from examination of usage in individual texts. In German and Italian, there are some business terms that are sufficiently integrated to be reasonably classified as lexical borrowings. Specifically, business and investment are, in the author’s judgement, lexical borrowings in both languages, and some specific usages of market in German also appear to be borrowings. But in French and Spanish, the usages they sampled suggest that it is still possible to identify code-switching.

Not surprisingly, the terms they examine have made inroads primarily in specialized senses, including usage as part of compound words or phrases. It is well attested that loanwords are more easily accepted in a TL if they fill a semantic gap; it is also well understood that lexical borrowings are often more narrowly defined in the TL than they are in the source language. In addition, given that we are investigating terminology, it is natural that the specific lexical items we study have a specialized semantic field. However, the study has also shown that in some cases, the terms are used in a less narrow and more general sense, such as the term business in both German and Italian.

The readability of infographics

I think that is quite enough on computational ethnography and computational linguistics, so I will make an abrupt move into the usability of infographics. Arguably, the infographic can trace its origin back to William Playfair (1759–1823), a Scottish economist who introduced the concept of using diagrams to present information. He was certainly the inventor of the pie chart. An infographic is just another in the long line of graphics designed to present information in a way that will be easily absorbed by a reader. An infographic invariably tells a story and so in principle requirement of the design is that the flow of the story is logical. In my experience, the quality of the presentation varies quite considerably, but this paper seems to be the first to assess the usability of the design format.

The authors (Majooni et al., 2018) used eye gaze tracking equipment in a way that is similar to the method used to monitor how users look at web pages. Two different layouts were used for two different stories and readers were asked questions to judge how well they understood the story as soon as they had an opportunity to look at it. The extent of the recall, together with the way in which the readers scanned across and down the infographic, was used to assess the overall level of the quality of the infographic.

The outcome was that the approach that seemed to work best involved a series of left-to-right reading motions, in the same way that we would read a book. My observation is that designers what to show how clever they are, often dotting extraneous information objects around the basic graphic design rather than looking at the design from a user perspective. My main takeaway is that it would be advisable to mark on the initial design for an infographic the eye path than the designer envisages, and then ask questions if there seems to be any significant variance from a ‘reading style’ progress down the infographic.

Application programming interface documentation

The acronym API, or application programming interface, is probably one of the most commonly used acronyms in IT projects. It is a piece of code that, in effect, takes sections of data from a database and presents them in an interface that requires little or no knowledge of the structure of the underlying technology. Typically, a software application vendor will offer an API that a developer can then use to create a customized interface to the software for their organization.

This paper is quite unusual in that it does not wander off into an academic analysis of document structures and usability but maintains a very practical focus. The authors (Meng et al., 2018) address three issues. The first of these concerns the initial questions developers raise when approaching a new API and the initial steps they usually take at the beginning of the learning process. This is followed by a consideration of information resources developers use in order to get the information they need. Special interest in this respect was devoted to the use of documentation provided with an API, but other information resources such as developer forums are taken into account as well. Finally, the quality criteria developers apply to API documentation are discussed. The overall goal was to collect experiences and opinions from developers in order to get a better understanding of their learning strategies, the types of information they look for during learning, the resources used to foster learning and potential obstacles that hinder learning.

The authors ran a series of expert interviews with junior and senior developers to collect feedback arising from their work with APIs and API documentation. In total, 17 interviews were initially conducted and then 112 responses to the survey were analyzed. This feedback was used to identify issues that are relevant from a developer’s point of view and grounded in their practical experience. In other words, the idea of the initial phase of the study was to let the developers talk and then see which issues they bring up. However, on the basis of the data generated by the interview, it was not possible to draw conclusions regarding which issues point to general problems and which are rather individual. They therefore designed a questionnaire as a follow-up to the interviews to test the main findings against a larger population in order to better understand their relative importance and relevance.

Two different pathways seem to emerge. Some developers work top-down in the sense that they try to get a deeper understanding of the system as a whole before turning to individual components. On the other hand, the learning goals of opportunistic developers are more narrowly focused on solving a particular problem and dependent on the specific issues and blockers they encounter while working towards a solution.

Among the conclusions from the study are API documentation needs to serve the information needs of both the systematic and the opportunistic developer persona, for example, by not only providing information on API structure or the rationale behind API design decisions but also enabling developers to start work with code immediately. One possible strategy is to clearly separate code examples from text, for example – as some API documentation sites already do – by placing code examples in a separate column adjacent to the text. A particular challenge arises from the findings with respect to conceptual information that is necessary for any developer to use the API efficiently, such as conceptual background knowledge related to the specific domain covered by the API. Conceptual knowledge like this needs to be presented redundantly and integrated into information resources that developers prefer to use, such as tutorials, code examples and the API reference, to facilitate access for both the systematic and the opportunistic type of developer. Regarding quality criteria developers apply to API documentation, the results show that high-level criteria like completeness and accuracy are very relevant to API documentation as well. Overall, this is an excellent example of academic rigour being applied to a very practical and important subject.

Agility as a strategy

This is my final Perspectives column, so where best to end these journeys through the academic literature? I have decided to comment on a paper about the role of IT in strategic agility. By Year 4 of my career minicomputers were starting to arrive and network speeds were just 330 bits/second over a dial-up connection. PCs began to appear until Year 12 and CD-ROM in Year 14. Ethernet connected my office together in Year 20. I started using Alta Vista in Year 25 and started working on intranets in Year 29. Google arrived the same year. Throughout that time, my colleagues in IT departments were learning as they went along, trying to future-proof the organization and at the same time manage a transition from a legacy system to a new system, knowing that it would become a legacy system itself in no more than a couple of years. That chronology omits any reference to the way in which changes in the way in which business has been conducted over the same period.

Every paper that I have included in Perspectives has reported on a development that was depending on IT infrastructure availability. Although it remains unusual for the senior manager with IT responsibility to be on the management board of an organization, without doubt these managers are now very much involved with the strategic direction of the business.

This paper (Morton et al., 2018) is unusual in a number of respects. The first is that the authors are European rather than coming from a US IT strategy perspective and further than of the authors is the senior director in a German financial services business. The second is that the focus is not on IT strategy itself but how it is going to be delivered at an operational level, based on discussions with 18 IT executives from multinational companies.

From these discussions, the core skills at an operational level emerge as strategic sensitivity, resource fluidity and leadership unity. The advice given by the authors is that IT executives need to strengthen their strategic alliances across the business so that they can be proactive in identifying opportunities, explore external and internal organizational domains to monitor the impact of IT on operations while being aware of what competitors are achieving, effectively communicate and collaborate, and be ready to manage tensions inside the organization. My take on this is that it is all about managing while walking around.

On reflection

As information professionals we have to make sure we understand the context, both organizational and personal, in which information is being sought, assessed, used and shared. The digital click data are only one element of the audit trail. In my experience, people who see little value in academic research often do so on the basis of anecdotal evidence and not from personal experience. It is not easy to access this information (though it is usually possible to buy an individual paper) but the benefits can be quite considerable. In the years ahead and the arrival of technically more sophisticated digital workplace applications, academic research may not provide definitive answers to assessing issues and identifying opportunities, but it can provide a vendor-independent framework for discussion and inspiration. It is also important that we as information practitioners build strong relationships with academia, especially in the information schools, and invite them in as colleagues to help in maintaining a focus on the user experience.