‘Insight unlocked’: Applying a collective intelligence approach to engage employers in informing local skills improvement planning

Local skills planning and skills supply in vocational education and training to frame ‘the problem’

The limited availability of skills appropriate for local economies, in this case Leicester and Leicestershire, has been well documented by both the Local Enterprise Partnership for Leicester and Leicestershire (LLEP) and local business representation organisations (LLEP 2022). In a recent survey of businesses, East Midlands Chamber, the Chamber of Commerce for the region, identified that of the two-thirds of businesses that attempted to recruit, 82% struggled to fill vacancies (East Midlands Chamber 2022). When asked at what level difficulties were being experienced, respondents reported this existed equally across skilled, professional, manual and clerical roles. Whilst not unique to Leicester and Leicestershire, certain businesses in the area are believed to be more acutely affected by factors such as the departure of European Union citizens who left the labour market and have not been replaced, along with the high number of skilled workers over 50 who have become economically inactive (LLEP 2022). These include businesses in the traditionally strong local manufacturing and logistic sectors, along with rural businesses where access to work is complicated by poorer transport connections.

Vocational education and training seek to respond to these needs, but their effectiveness have been hindered by a range of factors. These include the lack of a consistent taxonomy when it comes to businesses discussing their people needs and the offerings of vocational curricula, and poor communications between the private sector and educators (East Midlands Chamber 2022). Businesses find forward planning for their future human resources difficult. This is particularly the case when looking at smaller firms, which has a disproportionate impact in Leicester and Leicestershire, where 90% of businesses employ fewer than 10 people. These factors make it difficult to both articulate the ‘demands’ of businesses and then to ensure the understanding of this by the right people who can deliver change in the ‘supply’ of vocational education and training (VET) courses. A further difficulty arises with regards to publicly funded vocation education, where in the past there has been a disconnect between national funding priorities and local needs. This disconnect is in part behind the Government’s Skills and Post-16 Education Act 2022. This introduced legislation to create employer-led Local Skills Improvement Plans which included a requirement for Further Education Colleges to respond to the recommendations in those plans (HM Govt 2022).

The importance of ensuring that the people requirements of local businesses are matched by the supply of skills provided by educators is illustrated by the impact of not getting this right. Having a workforce with the relevant skills levels supports competitiveness and gains in productivity, with access to the relevant skills is also often cited by business representation organisations as being one of the largest determinant factors of inward-investment decisions. Conversely, not having the relevant skills levels not only impacts business productivity and growth, but also limits earnings potential and spending power within the local economy.

Recent NOMIS data for current levels of educational attainment in Leicester and Leicestershire (Jan-Dec 2021) shows 38% of the population being qualified at NVQ Level 4 and above. This compares to 43.6% for Great Britain (GB) as a whole. 59.7% are qualified at NVQ Level 3 and above (compared to 61.5% for GB) and 77.7% are qualified at Level 2 and above (compared to 78.1% for GB). Conversely, 8.1% have no qualifications, compared to 6.6% for Great Britain.

It is likely that these lower qualification figures translate into lower average earnings for people living in the city and county when compared to the rest of the country. The same data shows that in 2022 a full-time worker in Leicester and Leicestershire had a gross weekly pay of £607.90, £34.30 lower than the national average of £642.20. This difference is more pronounced when looking solely at female workers (£43.60) than when looking at male workers (£41.40).

Application of open and economics-based statistical data

Recent decades have seen fast-growing use of social and economic data to transform the way that economics and other social sciences operate (Einav and Levin 2014a, 2014b; Kitchin 2014). As discussed by Einav and Levin (2014a) this is largely due to the growing accumulation and accessibility of datasets as well as increased computing power to analyse those datasets. While economic analysis 40 years ago drew inferences from relatively small-scale surveys, we can now access much larger datasets, that in some cases offer universal population coverage. Such data, combined with advanced econometric and statistical techniques, including machine learning, offers the potential for more powerful research insights. Einav and Levin (2014b) give a range of examples of how the availability of data is transforming our understanding of economic policy. For instance, Choi and Varian (2012) demonstrated how Google Trends can predict economic indicators such as car sales, unemployment claims, and consumer confidence. Jun et al. (2018) provide a review of 657 papers which used Google Trends data since 2006, demonstrating the increasing use of prediction in large-scale data analysis.

While data is transforming economic and social science research, there is a challenge of data availability. To illustrate the size of the challenge, Einav and Levin (2014a) documented the proportion of papers published in the flagship journal, the American Economic Review, that asked for a journal exemption to use non-publicly available data. In 2006 only 8% of papers used non-publicly available data. By 2014 this had risen to 46% of papers, of which 26% used administrative data and 20% private data. This dramatic rise in non-publicly available data is alarming. It implies privileged access to data by a small subset of researchers, which at best limits economic insight and at worse can lead to corrupt practices. Christensen and Miguel (2018) summarise the case for a different approach to economic research. One strand of this approach is to encourage open data.

Open data can be defined as data that is non-confidential and non-privacy restricted and made publicly available for wider use with no (or very limited) restrictions on use. Open data offers exciting potential to improve economic and social science research and policy through its greater availability. Given these potential benefits an increasing number of organisations, including national and local government, are making their datasets openly available. As an example, Leicester City Council launched an Open Leicester web portal that provides datasets on a range of topics, such as planning, transport and public safety. This data can be used to inform our understanding of the local economy. For example, the study by Ferm et al. (2021) uses data from five local authorities, including Leicester, to assess the impact of deregulation of planning control. The data used for this paper was obtained by asking local planners. Comprehensive open data would, hopefully, facilitate such research by reducing the costs of data collection.

Janssen et al. (2012) summarize the many benefits of open data, including transparency, accountability, improved decision making, ‘wisdom of the crowd’, and re-use of data. Advantageous though it can be, there are several non-trivial challenges in effective open data. These challenges include concern over privacy and confidentiality (Christensen and Miguel 2018) as well as the quality and usability of data (Einav and Levin 2014b). In terms of quality and usability the key point to keep in mind is that open data is typically not collected and published with the researcher or analyst in mind. It is instead data that has been collected for some administrative purpose and then released to the public. Such data may miss key questions of interest to the researcher. It may also be difficult to combine and analyse data sets published in widely differing formats. Such challenges mean that there are substantive barriers in the use of open data. In short, the existence of open data is no guarantee it will be used. Moreover, organisations may be reluctant to release data that could be of most use. These barriers and more are also summarized by Janssen et al. (2012).

To date it seems clear that the potential for open data has yet to be fully harnessed. This is not only about the availability, but also the quality of data. The most effective form of data collection and analysis involves a two-way process in which the researcher can help shape the data collection (Einav and Levin 2014a). Researcher involvement in data collection maximizes the chances the data will have the desired quality and content to allow powerful research analysis and insight. Such involvement, however, imposes costs on all parties involved. In particular, it involves researcher time to work with the organisation and it means the organisation may need to implement new data collection processes. Having incurred such costs, the researcher and organisation may be reluctant to make the data freely available.

Atapour-Abarghouei et al. (2020) discussed, in the context of cyber-security, this data-sharing paradox. They argue that, because all parties have differing incentives, it is difficult to obtain effective data sharing. The solution ultimately needs to come from all parties ‘co-owning’ the problem. This logic applies not only to the cyber security challenge but other pressing social and economic problems, such as addressing skill needs in a local economy. Effective open data thus requires aligning incentives and enabling an eco-system in which all parties can see the benefits of open data collection and sharing, pointing towards the value of collective intelligence. This is a basic design feature of the Regional Business Intelligence Unit.

Using collective intelligence in economic development

Atlee and Por (2007) defined Collective Intelligence (CI) to include (but not restricted to) human intelligence, as it also relates to the intelligence of ecosystems. Socially, they described it as “the capacity of human communities to evolve towards higher-order complexity and harmony, through such innovation mechanisms as differentiation and integration, competition and collaboration”. This points towards forms of CI which may occur in entrepreneurial and innovative activities.

CI increasingly finds points of convergence with Artificial Intelligence (AI), aiming to inform and supplement human judgement rather than replacing it (Shadbolt and Hampson, 2018). Mignenan (2021) hypothesised CI as the connections between Human, Relational and Intellectual capital. Human collective and Artificial intelligence should be seen as complementary, but the interfaces between them are complex and involve both high-level skills and technology applications, which increasingly need to be shared to inform business practices and decision-making. This process is demonstrated in the case study.

The concept of Collective Intelligence as a field of knowledge stems partly from growing interest in sociated and distributed organising, evident in research from the mid-2000s. For example, in ‘Democratizing Innovation’ (Von Hippel, 2006); by Surowiecki, in “The Wisdom of Crowds (2004); Howe (2008) on crowdsourcing; Sunstein (2006) on Infotopia; through ‘Swarm Intelligence’ (Bonabeau and Meyer, 2001); and in an influential Sloan Management Review article (Bonabeau, 2009). This interest also arose from curiosity about new forms of organising, and their relation to the growing power and access to distributed forms of computing and access to online and digital knowledge resources. There are credible prior works in multiple disciplines on (for example); social cognition (Bandura, 1986); organisational social psychology and sensemaking (Weick 1979, 1995); collective memory (Freud) and social learning (Wenger, 1998).

Mulgan (2018a) brought the topic to professional and public attention with the notion of CI as a ‘Big Mind’. This connected with works on collectivism in sociology (Tarde via Latour, 2015) on group mind and collectives; and with the sociology of networks (Granovetter, 1973); “actor network theory” (Latour, 2015) seeing humans and machines as inseparable, all playing roles in networks. CI forms a convergence between these and other streams of psychological, sociological and philosophical awareness of collective thinking; with the burgeoning fields of both digital access to knowledge resources; and the growth of ‘information commons’ and Open, freely shared resources. The ascent of Artificial Intelligence (AI) and Machine Learning as influential cyberscience movements opened possibilities of intersections between AI and CI (Shadbolt and Hampson, 2018), as well as the role of AI in education (Seldon, 2018).

The relationships between CI, cyberscience and data science have also seen CI become a subdomain of data analytics, with lesser attention paid to the human and social interactions (eg Nguyen and Nguyen, 2018; Reia et al., 2018; Suran et al., 2020). However, CI represents more than a system or a knowledge platform; the interaction with information systems increasingly enhances human and collaborative capabilities in novel ways, whilst the relationships with ‘the information commons’ and the field of Open resources are influential. Creative and Learning Commons have developed for 20 years as means of freely sharing ideas, resources and thinking (Meyer, 2020). Whilst open access to scientific data developed from the 1957 advent of World Data Centres, the Internet made scientific, governmental, organisational and private knowledge publicly available and enabled their widespread use.

Open data is an essential contribution within the range of Open resources and processes (Rae, 2020), which enables their application through, for example, Open-prefixed Government, Science, Innovation and Entrepreneurship. Advocates of Open Data have also progressed to champion CI, including Shadbolt, Mulgan and others. Mulgan (2018b) argued that the convergence of AI and CI represent a fruitful direction for integration of human, collaborative and machine intelligences.

The uncontrolled growth in the availability of data resources meant that analysis by human agency alone is increasingly unrealistic and machine-assistance via AI has become essential. However, there are growing concerns that dependence on digital determinism alone could have adverse consequences for humanity, as well as interest in how the limitations on the effectiveness of individual decision-making can be moderated by shared intelligence systems. In May 2023, the Center for AI Safety issued a statement on AI Risk, signed by many AI scientists and public figures. This called for “Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war” ¹ .

The CI movement was marked by the emergence of research and innovation centres, created to recognise opportunities and develop data- and intelligence-led innovations in this field. The NESTA Centre for Collective Intelligence Design enables uses of CI in social innovation. Open Innovations in Leeds (UK) developed problem-led innovation from Open Data systems. Such organisations and leading practitioners advance innovation, animating communities of collective intelligence in their fields.

Collective Intelligence methods have informed collaborations by groups of business owners to address market disadvantages (eg by Wang and Tan in Canadian telco’s, 2019) and in collective action in small-scale fisheries (Child, 2018). Rouse et al. (2021) charted collective action by micro-entrepreneurs excluded from income protection during COVID lockdown in the UK, highlighting the informational asymmetry encountered by individual entrepreneurs and small networks in confronting data-rich public organisations. There is advanced analysis on path-creating new entrants in the Artificial intelligence industry in the Montreal city-region by mapping analysis of the AI industry policy network, such as by Doloreux and Turkina (2021).

Application of new data analysis and artificial intelligence methods for public policy

Few previous studies address the application of AI methods to local labour market analysis, but interestingly two were found in the Russian context. Belov et al. (2022) demonstrated the use of methods and algorithms in an analytical platform for automated monitoring and analysis of the regional and national labour market in the Russian Federation, and linked analysis of the compliance of the higher education system’s specialist training provision in meeting current labour market needs. The study involved natural language processing methods and Big Data technologies, using semantic analysis, automated monitoring and intellectual analysis on open-source datasets. This study developed and applied the prior work of Belov et al. (2017) and collection of secondary data gathering has continued from 2015. This demonstrated analytical capability, but the input from employers was limited to analysis of job vacancy data and applicant details.

An additional challenge when analysing gaps, barriers or disparities between skills requirements and supply, is the amount of data to be considered. Although the amount of data available in this project is not quite in the realm of Big Data, it is still too large to be analysed manually whilst too unstructured and vague to be easily organised and manipulated by standard statistical/automated techniques. The project used AI, mainly Natural Language Processing (NLP), techniques to help analyse the data at large scale (regional economy, and scalable even further to national level with no extra effort) and provide a tool for quick correlation of the data.

A similar approach was reported previously by Belov et al. (2017 and 2022) but subsequently followed by few studies, concentrated on the Russian job market. In recent years the NLP models based in deep learning have become much more advanced than word2vec used by Belov. We compared the GPT-3 and BERT models, finding the latter the most efficient and reliable for analysis rather than generation of text.

Although no novel AI techniques have been developed, we applied in a novel and much more generalised way the available AI based language models than had been used previously. The results were then fed into the CI process, for use by the Insight Unlocked portal.

Recommendations

The LSIP report made 10 recommendations, detailed in the report. Coordinated work is needed with third-party suppliers to maximise their ability to support the Future Skills Insight Unit. This includes Government ensuring it is asking the ‘right’ questions, determined through meaningful engagement with businesses and providers to provide insights relevant to the needs of a local area and not assumptions made nationally.

There is much that third-party suppliers can offer to a system that makes best use of data to understand skills supply and demand-side issues, however, at present there is little consistency in quality or provision. Current provision is often based on what providers are easily able to provide and not necessarily what would support better outcomes.

It is essential that the ‘right’ questions are asked at the outset and third-party providers are challenged to come forward with solutions to these questions as opposed to their own pre-existing products. While recognising that in many instances these are commercial businesses, they all demonstrated a willingness to support, and with coordinated direction can play an important role in providing the solutions required.

The CISO design was seen by experts at DfEE and in the Local Enterprise Partnerships and Chambers of Commerce networks to be an innovative and effective model. As a result, the CISO model is being adopted by four areas in the East and West Midlands, with further potential for development. This includes CI applications for different clients beyond the LEP and Skills areas. The region includes a major freight airport, international logistics businesses and a Freeport which present future opportunities. The ability to engage employers through gathering regular narrative inputs from them is a distinctive feature, which takes the project beyond the earlier work by Belov et al. (2022). This capability has value in its own right as a longitudinal survey method for use with businesses, and an App is being developed to exploit this.

Create a shared vision

The case for the project was developed through dialogue and shared diagnosis by actors within EMC and DMU of a need for enhanced regional economic and business intelligence data, which would inform better decision-making. This recognised need was explored and shared with others, and during the gestation period for the project proposal developed a scope embracing enterprise policy; economic development; applications of open data; data analytics and science, which then extended to include Artificial Intelligence as the enabling technology. This developed an interdisciplinary project and strengthened the innovative potential.

Build on trust

The relationship between EMC and DMU was not new and there was a productive history of collaboration. Writing the proposal required significant investments of time from both organisations which was facilitated by the KTP Officer in the University who was recently appointed and committed to securing an approved project. The first iteration was reviewed by the KTP Advisor, whose feedback informed a revised proposal with a more detailed project plan and stronger business case. This gained approval in the formal Innovate UK process. This shared experience enhanced the trust and collaboration within the project team and strengthened the relationship with the KTP Advisor.

Hire the best person

Project approval and contracting was followed by a recruitment process for the Associate. This involved both partners fully in specifying the role and, after advertisement, shortlisting and appointing to the role. It was evident in the selection interviews that just one candidate met all the requirements and gained the approval of the full selection panel. He (Harsh) had the advantages of Indian software industry experience and a recent MSc from DMU, but the slight disadvantage of requiring a work visa for the UK as an Indian citizen. The project start was delayed for 2 months whilst this was obtained, but the lesson is very clear: appoint the person best fitted for the role, and give the Industry Partner the final selection decision as they are, in effect, the employer (contractually the University is the employer who deals with the legal and administrative issues).

Effective partnership and collaboration require structured, regular engagement

KTPs have a series of four-monthly formal Local Management Committees. These were supplemented by weekly half-hour Teams-based stakeholder meetings. As the project started during the COVID lockdown, these were essential for updating, communication and co-ordination.

Let the Associate drive the project

The proposal requires detailed project and workplans which are difficult to prepare in advance for a project which is emergent, inter-organisational and interdisciplinary. The plans were accepted, and the Associate and supervisory team confirmed them. But fairly quickly the plans started to change. The Associate brought new ideas and techniques, the external context was evolving during the COVID lockdown with new policy and economic factors bringing new opportunities. Trying to adhere to or revise the original detailed plans would have constrained the project’s effectiveness.

Open collaboration enables innovation

The project did transfer existing knowledge from DMU to EMC via the staff and Associate, but the innovation was based on a much broader knowledge and expertise base. Both organisations and the team members had networks of expertise and contacts with other knowledge-based organisations, including public sector agencies, businesses and not-for-profits. Being able to sample and select experience, data and services from a wide palette resulted in experimentation and the optimal combinations being found. An example was the interaction with Open Innovations, a data innovation hub with a public mission and The Data City, its commercial services arm, both of which were among the organisations whose services and expertise were harvested.

Follow the opportunity

There was a commercial revenue expectation embedded in the project, but at the outset it was not clear who the target market or the services provided might be to generate this. The call for ‘Trailblazer’ proposals for LSIPs was well-timed and being able to connect the KTP to develop the innovative CISO concept, gave the project a clear direction and the scope to create and deliver an AI and CI-based innovative approach. Achieving this created the platform for EMC to be recognised as an effective innovator, to exceed the revenue targets, and to develop an economically sustainable business intelligence unit which can permanently employ the Associate as well as two new posts and future intern or project opportunities.