Abstract
If my consulting work is an accurate barometer of technology trends then in 2014 we should see a rise of interest in using text analysis as an emerging trend for outcomes such as automated sentiment analysis. It is emergent in the transactional systems of the organization, such as CRM (Customer Relationship Management), but not yet in mainstream Information Management. This article discusses how this shift is inevitable as the balancing innovation for the dramatic improvements we have seen in delivery and infrastructure – connected by a well-established S-curve of progress. While we have focused on the efficiency of information delivery, our efforts have lagged progress on the effectiveness agenda in information use. Indeed in the move from paper we have lost any competitive edge. Technology to deliver facts has ‘levelled the playing field’ and removed the inherent institutional advantage over the selection of, and access to, information. Technology here, and in other areas, has changed the balance between delivery and use emphasizing delivery – everyone sharing the same facts – but has washed out narrative, annotation, selection and opinion. Technology can be employed to provide differentiation across the vast but inscrutable information sources available to every organization. Our challenge is, with technology, to establish the personalization agenda, and abilities, across the featureless landscape of external information resources.
Keywords
Running away to failure?
The common theme of Malthus and many other futurologists is to view a significant and growing trend – population, communication, consumption, debt – and predict that it begins as a runaway success to then reach a point at which it is out of control and comes to an abrupt end. It is the early steam engine which, without a regulator, goes faster and faster until it explodes. It is also the characteristic hockey stick graph popular amongst the doomsayers.
There are many examples where these runaway events and catastrophes occur, but there are many more where there are either natural regulators or regulators which have been designed into the system. Indeed there are many examples where altogether harmful characteristics are managed and moderated to beneficial effect, for example, the toxic deadly nightshade is managed – and the dose regulated – to become a cardiac medicine.
Information Technology is no different. It can be beneficial or harmful; it can be designed with an ambidextrous approach to efficiency and effectiveness and to size. Historically the IT applications which follow power relationships, such as control and communication, come before the use of the same technology in collaboration and communication. IT departments first delivered accounting and stock control systems before email and the web.
This ability of technology to be ambidextrous and support both ends of any spectrum makes forecasting difficult and binary predictions wrong: size seen as an advantage becomes a liability, as one element of technology becomes a commodity another becomes a source of profit 1 and, as one aspect advances in significance, another will take a back seat. The information professional must shift focus to take advantage of every possibility of technology and be alert to accidental biases introduced by technology. In 2013 we are at the point where there is little to be gained by improving the efficiency of delivery. The opportunity lies in improving effectiveness.
In this article the central topic, text analysis, is set to redress the imbalance of recent technological innovation. I will argue that a new tipping point is emerging which will drive a new era of development and focus for the Information Professional. The prediction is developed from the following observations:
S-curve: shaping the future.
Size: Our triumph of efficiency in content delivery has been at the cost of effectiveness.
Scale: The size is that all advantage of volume and context (which lies with Google and the NSA (US National Security Agency)
Souls: The lone entrepreneur advantage of the emergent Internet and tablet revolution needs to mature to the benefit of the small enterprise.
System: The progress of a hardware led infrastructure revolution has advanced beyond the quality and widespread use of software algorithms.
If my analysis is correct it brings a return to the role of the Information Professional: to provide unique information resources to our organizations that assist their thinking and ensure a beneficial, and for some, profitable outcome. Accidentally, the progress in Information delivery has eroded any competitive advantage and forced our thinking to worry more about being left behind than leading from the front.
The S-Curve – the pattern of innovation
In an earlier article for Business Information Review 2 I outlined the work of David Moschella; the graphics from his work, which shows the bell curves for each generation of technology, is an example of the S-curve in operation (see Figures 1 –3). The characteristic S-curve is a simple representation of the lifecycle of any species in a competitive and progressive ecology – it is the mathematical signature of evolution and has been used to characterize generations of technologies, software productivity, competition of ideas, the popularity of boy bands, classes of pharmaceutical innovation and many others. 3
The S-curve applied to competition between technologies. 4
The impact of the S-curve as the ‘bell curves’ of technology adoption.
The S-curve applied to the introduction of a new technology or application. 5
The doomsayers following Malthus assume a characteristic ‘hockey stick curve’ of unbridled and uncontested over-adaptation followed by abrupt failure which makes for dramatic news headlines. A better, and more common model, is the flattening of progress and the emergence of the new.
Size – technology does not just favour the large
I recently attended a presentation on the next big thing in teaching – the MOOCs (Massive Open Online Course). Sadly I did not share the presenter’s enthusiasm and take a more sanguine view as the current implementation of MOOCs follows the dominance of power in education and the over-adapted support of lectures and assessment. In the current state of development MOOCs miss the social media elements, of collaboration and conversation, so evident in the use of technology elsewhere (Figure 4).
Pictorial Definition MOOC (Mathieu Plourde licensed CC-BY on Flickr).
It is not that MOOCs are doomed; just that the use of technology in teaching has been lopsided – emphasizing the focus on pedagogy and ignoring the need for andragogy, 6 or power over self-determination. Technology as I have discussed many times in ‘Out of the box’ is inherently neutral but often is associated with specific motives and purposes; it is as capable a tool for totalitarian control (‘big brother’ and the outrage at the work of the NSA) as it is a force for social democracy (such as the Arab spring and other activist movements). It is not that technology is good or evil but our thinking is shaped by the uses we choose to make of it. 2
My own experience at university contained a much richer mix of experience and interaction with content than simply broadcast lectures. It comprised a balance of elements which are not easily matched in the current state of online learning platforms where personalization and facilities for intimate self-managed groups is so under-developed.
I was lucky enough to benefit from a university in which most of the learning was in twinned tutorials – in which the end point and the detailed progress assessment was on an individual basis – but with an essay scheme common to my fellow students and university-wide tutorials. This clearly matched the elements of learning with the form of learning (Table 1).
Technology directions in teaching and learning.
The social grouping of my learning was across different scales and offered different technological possibilities:
my understanding and assessment of progress – on an individual basis;
my learning and subject exploration – on a self-appointed team basis;
my syllabus, reading and study – at an institution level through lectures;
my assessment and grading – at an institutional level.
Scale – a new era of effectiveness?
Everyone is entitled to his own opinion, but not to his own facts 8
My objective in introducing the case study on MOOCs is to provide an example where the forces of power and efficiency have over-developed at the expense of personalization and effectiveness. It is a platform on which to ask the same question of our work: have we spent too much time and money on improving efficiency and too little on improving effectiveness?
My response would be … absolutely! As professionals in the (typical) organization we have for the last two decades moved from an era of primitive effectiveness in delivering information to dramatic, perhaps dominant, efficiency. Today we are concluding this arms race in which we make everything available to our organization, a remarkable achievement measured against the partial coverage of the physical libraries of the past. What has happened, though, is that any information advantage and knowledge gradient between organizations has been lost. The endpoint of this heroic and successful effort is to deliver a Knowledge monoculture. 9 Our pride in our work which scores top marks for efficiency in that no-one is disadvantaged nor is challenged by the zero marks for effectiveness in that we do not deliver any unique advantage.
In sport, in life and in business we all obey the rules but where we can we look to gain an advantage however small. In football it is the home advantage, the extra-large, small or sloping pitch that forms part of the tactics of the canny successful manager. In Information delivery the progress of the last 20 years has been to deny any company advantage and remove the natural information gradient present in the paper era and in throwing out the paper we have also eliminated the institutional advantage. Like MOOCs this represents a lopsided use of technology.
There are monocultures in other elements of our businesses. In the move to global supply chains we compete using the same, single sourced components with our sales teams using the same out-of-the-box systems and our research teams using the same standard pieces of equipment. In the last 20 years we have seen the march of efficiency replacing custom IT systems with standard products, handmade laboratory instrumentation replaced by off-the-shelf kit and local culture and working practice replaced by global policies. It is not surprising that price competition, alone, is emerging as the only differentiator of choice.
In an era of monocultures we, Information professionals, have delivered the facts, but what are we now doing to deliver opinion?
Looking outside to look in
What can internal groups do to break the monoculture of knowledge to the advantage of the organization? What can we do in our jobs that will sustain a gradient and a marketable distinctive competence for our organizations?
Souls – tipping technology?
The dynamic balance of technology with efficiency and effectiveness both eliminates and enables opportunity. As we have seen the march of efficiency in information delivery has eliminated the factual advantage, but I would argue it has also enabled the effectiveness of opinion advantage. 10
The evidence again lies in the news and the activities of the young ‘before they were old enough to know better’. 11 My example from this year is Nick D’Aloisio 12 a London schoolboy who in March 2013 sold his smartphone news app to Yahoo for a reported $30 million. His work was to develop a news reader and summarizer, Summly, 13 which integrates technology from SRI International to summarize text from multiple news feeds. Whilst Nick is not exactly the sole inventor (he had guidance and support from his executive parents and Venture Capital funding) nonetheless his product was produced with the resources and budget available to nearly all institutional readers of Business Information Review.
His development, integrating web-based software services, into an app tipped the ‘level’ infrastructure of the Internet to advantage. The entry price for this development was the globally available open RSS feeds and readily available developer’s kit for the iPhone 14 ($40). These were the only resources aside from imagination and luck. Nick’s work signals a new wave of innovation, one which re-builds advantage in the flat commoditized landscape of information.
The IT profession is already familiar with these new waves of technology emerging from, and signalled by, bottom-end innovation. In the 1990s it was the PC enthusiasts who challenged and overcame the professional orthodoxy of the mainframe IT department, now it is the Apps and tablet advocates who are challenging the corporate PC hegemony. 15 As I have argued earlier in Out of the Box progress is made by the fusion of ideas and approaches, the knowledgeable and the naive, the technology and information specialist and the pioneers and the service experts.
System – the design message
Size does not fit all
My critique of the use of technology in teaching is that, by accidental design, progress has been made in an unbalanced way – the power elements of institutional control, summative assessment and one way communication have become dominant over the social elements of student self-determination, formative development and social interaction. History anticipates, and common sense predicts, that there needs to be a correction and a new era of technology developments around the person, focused on learning and open to effective dialogue.
Search marks
The sad triumph of search lies in the statistics of the quality of the queries designed to find our needles in the haystacks of content. We have advanced only slightly beyond Google’s ‘feeling lucky’ invitation but few searches use many terms and fewer still employ the full power of Boolean logic. It is unreasonable and poor design, to expect each and every user to master the intricacies of good search; without help we are losing the content arms race (Figure 5).
No of terms used per query across different information sources. 16
Full marks for opinion
Our salvation is opinion – the selection, summation, annotation and narrative on raw content. This is the antithesis of the domains for the many that technology has, to date, delivered – not the average 510 friends on Facebook, 17 not the ever increasing amount of published content nor the scramble for a niche popularity amongst the plethora of journal titles. In the printed word we seem more to be experts at haymaking in our search for the elusive needle (Figure 6).
Impact factors for 8000 academic journals (log scale).
In Table 1, I presented the idea of the technology directions, the direction of travel for technology for the aspects of learning. Some elements had an advantage of scale; others had the advantage of small in which suitable selected intimate groups made the problem easier. The challenge of design is to maintain the balance between efficiency and effectiveness and we need to ask the question where do we get our ideas for effectiveness?
We can learn from the landmark research pre-web, such as the academic studies which compared the work of Sheridan, Dryden and Shakespeare 18 or traced the development of concepts and ideas in limited corpuses of academic texts. Just like speech recognition 19 which seemingly has been predicted as next year’s breakthrough for the last 20 years so text technology still falls short of the untrained, universal encyclopaedic automated sage of science fiction. Nonetheless, the small breakthroughs, against limited corpuses of information and in restricted domains, continue to be dramatic. It is here in these micro-worlds, that are too difficult and too small for Google, that local initiative will flourish and that a new competitive edge can be formed. Our institutions have and know these small expert communities, our challenge is to support them.
Many of these tools were developed and used in the early research studies but have lacked mainstream adoption and use with our focus on efficiency. Now is the time to re-visit this work starting with any compendium of research, such as Taming Text: How to find, organize, and manipulate it which reveals the wealth of different tools for the analysis of text. 20 The table of contents of this book 21 presents the many facets of technology for text: such as searching, identifying people, clustering text and documents, summarization, classification, categorization and relationship extraction. In a look beyond present capabilities it introduces techniques to identify important content and people and detecting emotions in sentiment analysis.
As a designer and technologist I can see the potential for this technology tuned around my business communities and providing a clear information vision in the blizzard of information delivery. Fifteen years ago when I entered the revolution in the Internet, Web and Search, I had a more balanced view and was more familiar with these techniques. In the meantime my attention has been transfixed in the race for size and the growth in on-line information. Now is the time to re-balance my technological diet and jump the S-curve from the top of delivery to the start of a new era of effectiveness. An emerging era, optimistic with potential and ready for investigation.
A recipe for the Next Year
My Christmas recommendation in 2011 was for an Android tablet, for 2014 it is to explore automated text handling and put to work some of the techniques contained in Taming text. This provides a wealth of ideas and sources for really making use of your information and shading the flood of undifferentiated information into the focused short-list of your opinion. It can provide your institution a unique custom insight.
To complete the recipe you need to add a youthful programmer, season with your experience, mix in the efficiency of 30 years’ text research, sprinkle with the expertise in your organization and deliver by tablet. Every recipe will be unique to your organization and remain out of reach to the mass production methods of Google. ‘Genius is seeing what everyone else sees and thinking what no one else has thought.’
What can we do to add the features to the flat Information landscapes in order to help the geniuses in our organizations?