An example of using ChatGPT for football recruitment purposes

Abstract

This work has been inspired by a recruiting system that a Spanish football club created with artificial intelligence making use of both numerical and textual descriptions of players. However, the news piece presenting this approach did not give many details. Within this context, I was curious to know whether ChatGPT could provide me with some more information, which further led me to the idea of trying to develop my own player matching procedure. Following several constructive exchanges with ChatGPT and after performing some successful tests, I present the algorithm in this paper. The main questions that this research aims to discuss are as follows: how ChatGPT can be utilized in autonomous research projects, particularly in sports; how to overcome the lack of public football data, also considering textual information about players; and how to support the expert knowledge of recruiters and coaches with data-based tools. Indeed, one limitation when carrying out football analysis is the difficulty accessing public data, not to mention unstructured texts. I propose to use data from video games to mimic recruitment tasks. The methodology behind the algorithm is based on text embeddings and the Gaussian kernel to work with the text and numbers that represent each player, respectively. A new R package called fawir that includes the recruiting algorithm and a data set to reproduce the results is also presented.

Keywords

Artificial intelligence data science expertise playing styles soccer analytics video games

Introduction

Data analytics has come to play an important role in football. Proof of this is the constant flow of publications related to different topics, such as the analysis of passing behaviour,¹ the prediction of player performance through expected and actual goals² and the evaluation of the strategies that teams adopt to achieve their objectives.³ Indeed, more and more professional clubs are assigning increasing economic and human resources to gain a competitive edge on and off the pitch. For a football club as a whole, one of the most important decisions that they have to make is recruiting players who can improve the team.⁴ Recent news about a novel system presented by the Spanish club Sevilla FC attracted my attention.¹ The report explained that Sevilla FC have collaborated with IBM to develop a scouting tool based on Artificial Intelligence (AI) and Natural Language Processing (NLP) principles, in such a way that a list of potential signings is obtained after evaluating textual and quantitative data for each individual player.² IBM (International Business Machines Corporation) is an American multinational technology company.

Among the emerging AI models, one that has become very popular is Chat Generative Pre-trained Transformer, also known as ChatGPT.⁵ This technology is capable of providing answers to questions posed by users. Its impressive performance indeed opens a range of possibilities for the development of any discipline.⁶ In particular, the potential use of ChatGPT and its impacts and applications has already been discussed in areas such as medicine,⁷ global warming,⁸ tourism,⁹ supply chains¹⁰ and human resources.¹¹ From my personal perspective, two recurrent thoughts have had a bearing on this paper. On the one hand, as a football fan and as a statistician, I was interested to know how data analytics could help in the unseen world of scouting and talent identification.¹² On the other hand, I was also curious to see first-hand how ChatGPT could help me leverage an autonomous new line of research. At this point, I share the same concerns that have already been expressed in the scientific community in terms of honest practices.¹³ In this paper, I aim to advocate for responsible use of generative AI tools by critically verifying the contents provided by ChatGPT and including a transparency statement about how they have been generated. One of the branches of digital humanities is software studies, which refer to the question of digital technologies as a research subject in order to understand their structure and functioning.¹⁴ In this regard, my interaction with ChatGPT has focused on the creation of a player matching software algorithm with the constraint of using both structured and unstructured inputs. I got involved in a conversational exchange, where I conveyed to ChatGPT my goals and concerns with each piece of code that it was returning. My experience has been rewarding, since this technology is able to understand the momentary deficiencies of the algorithm and to propose different approaches when I did not agree with the mathematical procedures returned. I also carried out code cleaning and coherence tasks, because sometimes the programming was somewhat messy. After several iterations, we reached a final version of the algorithm that was ready to be tried with football players for recruitment purposes.

In professional football, the people in charge of recruitment receive the name “scouts”. Their main task consists of monitoring players and determining whether they fulfill the specific needs of the team. Trying to reveal the intricacies of the scouting procedure has been receiving increasing academic attention.¹⁵ Observing players in action, live or online, has been the traditional way that scouts have searched for suitable candidates, but this approach remains subjective because it is only based on the previous personal experience of the scout. With the advent of systems that generate data describing the players’ performance, a layer of objectivity can be added. The statistical analysis of these data helps to bring to light new patterns and insights. It is also well known that coaches can take advantage of this in their continuous learning process.¹⁶ However, to the best of my knowledge, there are few statistical contributions in the scientific literature. The related works include the following: A one-class support vector machine has been used to analyze which players show the most promising performance from a sample of junior players.¹⁷ Convolutional Neural Networks have been proposed to process video images and detect from them three tactical statistics for a player, namely, the duration of ball possession and the number of successful passes and steals.¹⁸ Artificial neural network modelling has also been applied to identify position-specific performance indicators that predict out-field players’ league status.¹⁹ All these approaches suffer from the same drawback, namely, the sole use of structured data in the form of numerical variables. However, talent identification approaches can also benefit from unstructured data sources, such as textual descriptions summarizing some key performance indicators of the athletes in words. Information provided in textual format has been wide spread on the internet in recent years. This has given rise to text data mining, which is the process of extracting knowledge from text documents.²⁰ Another limitation of the aforementioned approaches is that their software implementations are not available to check their internal structures. The method developed by Sevilla FC and IBM also remains private. Inspired by all these works, my research goal has been to develop a public player matching algorithm that transforms both numerical and textual data into actionable insights for football recruitment purposes.

Although there has been a rapid increase in data studies in the sports industry in recent years,²¹ working with real data in football analytics is a challenge because the data collected by official providers are usually inaccessible. Luckily enough, video games have turned out to be an important alternative source of data.²² Nowadays, the release of highly interactive football videogames is accompanied by an extensive scout for players worldwide that is a faithful representation of their actual performance.²³ The virtual data used in this work come from PES Master, which is the source of information for Pro Evolution Soccer, one of today’s most popular football video games. The most remarkable advantage of this source of data is the inclusion of both numerical and textual variables describing the players.

A further point of personal concern is adhering to the FAIR guiding principles by providing the scientific community with Findable, Accessible, Interoperable and Reusable research results.²⁴ In this regard, a new R package called fawir has been developed to include all the methods and a portion of the data in order to support a critical understanding and extensions of the current work. Its latest version is freely available on GitHub.³ As widely recognized, R software²⁵ is a leading tool for analyzing any kind of data, including football data. An easy way to check the number of football-related R packages is by consulting the CRAN task view about sports analytics.⁴ Some of the best-developed utilities focus on extracting world football data²⁶ and Fantasy Premier League data,²⁷ as well as fitting a collection of well-known football models.²⁸ There is also a recent survey worth consulting, which examines the catalogue of sports-related R packages.²⁹

In summary, this study aims to discuss the following questions: Q1: how ChatGPT can leverage autonomous research; Q2: how to collect textual descriptions of football players; and Q3: how football recruitment can be supported by means of automatized algorithms. The rest of the paper is organized as follows. Section “Methodology” outlines the details of the methodology developed. Section “Data” introduces the data set. Section “Results” presents the results and finally, the concluding remarks are summarized in Section “Conclusions”.

Methodology

The goal of the study is to select players using both a textual description and a set of numerical variables related to their skills. In practical terms, the desired algorithm should receive a query about finding a player with certain characteristics and return the most suitable player(s) from the input data set. The methodology presented in this paper is based on two ideas: text embeddings and the Gaussian kernel. Both are explained below in detail. For guidance, Algorithm 1 explains the workflow.

Algorithm 1 Player search workflow.

1. Text processing setup:

1.1. Create iterator over tokens.

1.2. Create vocabulary.

1.3. Construct the term-co-occurrence matrix (TCM).

1.4. Factorize the TCM with GloVe.

1.5. Compute text embeddings for reports,

r_{e m b}

1.6. Compute text embeddings for the query,

q_{e m b}

1.7. Compute textual similarity:

\, t e x t_{s i m} = c o s i n e_s i m (r_{e m b}, q_{e m b})

= s u m (r_{e m b} \times q_{e m b}) / (\sqrt{s u m (r_{e m b}^{2})} \times \sqrt{s u m (q_{e m b}^{2})})

2. Numbers processing setup:

2.1. Extract query numbers for abilities.

2.2. Normalize numbers for reports,

r_{n u m}

2.3. Normalize numbers for the query ,

q_{n u m}

2.4. Compute numerical similarity:

\, n u m_{s i m} = \sum_{i = 1}^{n} (e^{- | q_{n u m_{i}} - r_{n u m_{i}} |}) / n

, where

n

is the number of abilities.

3. Final similarity setup:

3.1. Combine textual similarity and numerical similarity:

α \times t e x t_{s i m} + (1 - α) \times n u m_{s i m}

Text embeddings

The word to vector concept, shortened to “word2vec” is an NLP technique for obtaining real-valued vector representations of words. Instead of computing scalar distances between words, the evaluation of analogies between them has been shown to be better at identifying contextual meanings.³⁰ Two traditional families for learning word vectors are global matrix factorizations and local context windows. Separately, both methods show structural weaknesses in terms of performing poorly on the word analogy task and the statistical analysis task, respectively. The Global Vectors approach, known as GloVe, was developed to make use of the advantages of these two families while removing their drawbacks.³¹ The GloVe model is summarized as follows:

Let $X$ be the word-word co-occurrence matrix that collects the word co-occurrence statistics. This means that each entry $X_{i j}$ indicates the number of times (how often) word $j$ occurs in the context window of word $i$ . Let also $X_{i} = \sum_{k} X_{i k}$ be the number of times any word occurs in the context of word $i$ .

Let $P_{i j} = P (j | i) = X_{i j} / X_{i}$ be the probability that word $j$ appears in the context of word $i$ . The experimentation carried out in Pennington et al.³¹ shows that ratios of these co-occurrence probabilities are more suitable for word vector learning than the probabilities themselves. Thus, since the ratio $P_{i k} / P_{j k}$ depends on three words $i$ , $j$ and $k$ , the most general model is expressed as:

F (w_{i}, w_{j}, {\tilde{w}}_{k}) = P_{i k} / P_{j k},

(1)

where

w_{i}

is the vector for the word

i

w_{j}

is the vector for the word

j

and

{\tilde{w}}_{k}

is the vector for the context word

k

The choice of the function $F$ is determined by the requirements of restricting the difference for the two target words $i$ and $j$ , preventing the combination of the vector dimensions in undesirable ways and being a homomorphism between groups $(R, +)$ and $(R_{> 0}, \times)$ . All of this means that equation (1) can be simplified as:

w_{i}^{T} {\tilde{w}}_{k} + b_{i} + {\tilde{b}}_{k} = l o g (X_{i k}),

(2)

where $b_{i}$ and ${\tilde{b}}_{k}$ are scalar biases for the main and context words.

The model of equation (2) weighs all co-occurrences equally, even those that happen rarely or never. In order to overcome this problem, a least squares regression model that includes a weighting function $f (X_{i k})$ is presented:

J = \sum_{i = 1}^{V} \sum_{k = 1}^{V} f (X_{i k}) (w_{i}^{T} {\tilde{w}}_{k} + b_{i} + {\tilde{b}}_{k} - l o g (X_{i k}))^{2},

(3)

where $V$ is the size of the vocabulary and $f (X_{i k})$ aims to prevent learning only from extremely common word pairs. The function selected in Pennington et al.³¹ is parameterized as follows:

f (X_{i k}) = {\begin{matrix} (\frac{X_{i k}}{X_{m a x}})^{α_{glove}} & i f X_{i k} < X_{m a x} \\ 1 & o t h e r w i s e \end{matrix}

(4)

As a result, GloVe is a log-bilinear regression model whose purpose is the unsupervised learning of word representations. It has been able to outperform related models on word analogy, word similarity and named entity recognition tasks.³¹ In this way, in order to represent textual descriptions in a vector space, first we have to create a set of meaningful words, the so-called vocabulary. We then construct the term-co-occurrence matrix (TCM), which is the mathematical representation that captures the frequency with which pairs of words appear together. Finally this TCM is factorized via the GloVe algorithm. Glove belongs to the family of models that are pre-trained on massive data sets and is available in the R package text2vec.³²⁵ Once we have vector representation of words, we use the cosine similarity to compare words. If two words have a cosine similarity close to 1, it means that they are used in similar contexts. On the contrary, if it is close to -1, they express opposite meanings.

Gaussian kernel

To incorporate numerical variables effectively into the player matching algorithm, the numbers related to the abilities mentioned in the search query must be identified and processed. In order to detect and extract these numbers, a natural language parser is used. This parser is able to match the numerical values that follow the string formed by an ability from the input data set and the word “of”. For example, if the query text contains the string kicking power of 65, the regular expression procedure extracts the number 65.

Once these numbers are identified, the procedure to compute how close the query value (target) is to the player’s value (actual) is as follows: First, both the target and actual numbers are standardized in each case by subtracting the mean and dividing by the standard deviation. This is intended to make the data comparable. Then, a score is computed using this form of Gaussian-like kernel $K (t a r g e t, a c t u a l) = \exp (- | t a r g e t - a c t u a l |)$ . It is bounded between 0 and 1. The similarity is close to 1 if the values are very close, and it decreases exponentially as the difference increases. Lastly, a final score is obtained by averaging all the similarities.

Hyperparameters

The player recruitment algorithm depends on several hyperparameters. The GloVe model has two main parameters that should be taken into account. The first is the rank value, which specifies the desired dimensionality of the embeddings, that is to say, how many dimensions or features are used to represent each word in the latent vector space. A large rank value is usually preferred, but there are some trade-offs to consider. The main benefit of a large value is higher precision, because more dimensions should be able to capture more nuanced and complex relationships between words. However, if the data set is relatively small (e.g., a few thousand sentences), there is a risk of overfitting, as well as a greater computational burden.

The second parameter to remember is the maximum number of co-occurrences to use in the weighting function during the training process. In GloVe, co-occurrence counts between words are weighted to give more importance to medium-frequency pairs while reducing the influence of very frequent or very rare pairs. In this way, this parameter helps limit the influence of extreme co-occurrences to prevent bias during co-occurrence weighting.

Finally, when constructing the TCM, it is also suggested that we should reflect on the size of the context window. This context window determines how many words before and after a target word are considered its context for co-occurrence. In order to create the TCM, the user must indicate the maximum distance (how many word positions away) within which two words are considered co-occurring.

R package fawir

The reproducibility of experiments and analysis is a fundamental premise in science. This has also been highlighted in the sports community.³³ Following the best practices of sharing all the methods developed, this paper is accompanied by the new R package fawir.⁶ It includes the necessary functions and data sets to reproduce the results presented here. It is worth clarifying that fawir does not provide the whole database used in this study. This is to preserve the data maintenance carried out by the PES Master website owners. Since the data collected belongs to the PES Master site, I do not see myself in the position of sharing all the data for free. However, I do wish to encourage an accurate review of my research, so fawir includes a toy data set so that potential readers can get acquainted with data analysis, learn how to use the algorithms and reduce the complexities associated with real-world data processing. A documenting guide (the so-called vignette) is also available, which includes the precise instructions for using each function.

The package also contains a user-friendly multilingual dashboard that will allow users to get acquainted with how the player matching algorithm works without the need to understand the methodology or have advanced knowledge of R. The app is launched within R with these two commands: library(fawir) ; run_app_recruit(). A toy data set to play with is available at https://www.uv.es/vivigui/docs/data_dashboard.xlsx. Overall, fawir aims to be the football contribution to a constantly growing infrastructure of sports analytics that I am developing, which already includes the BAwiR package for basketball.^34,35

Data

In sports, free data are rare. Leagues do not usually release their data in the form of public downloadable files. Nowadays, sports data management is in the hands of service providers and they set the rules for the use of data. Two well-known football analysis platforms are StatsBomb (https://statsbomb.com/) and Wyscout (https://footballdata.wyscout.com/). StatsBomb collects data about competitions, games in each competition and both lineups and events in each game, including 360 metrics with contextual information and the exact location of all players on the field. In addition, they offer R and Python packages that are adapted to these data (https://github.com/statsbomb). As for Wyscout, it collects data about competitions, games in each competition, player stats and events in each game.⁷

In the case of this research, finding data is even more difficult because the algorithm is fed with textual descriptions about players’ characteristics and style of play. On the aforementioned platforms, players are only described by means of numerical variables, to the best of my knowledge. A website that contains textual data is WhoScored.com.⁸ In addition to an in-depth analysis of football data, this site includes textual descriptions about the players’ performance, skills and characteristics. However, WhoScored.com suffers from the same structural disadvantage in terms of providing free data. As mentioned in their Terms of Use (https://www.whoscored.com/TermsOfUse), the use of WhoScored.com content is prohibited without an official license. I asked them for permission via their contact channel, but I got no answer.

It was at this point that I realized that obtaining a mixture of numerical and textual data about real players was going to be a hopeless task. However, there was still one path of research: video games. Connections between real players and their virtual counterparts have generated a unique universe of ratings that accurately describe players’ performances. Therefore, in the search for an alternative virtual database I landed upon FIFA Index,⁹ which is the largest FIFA player database on the internet. FIFA is a football-themed simulation video game, created by EA Sports. Players are described by a number of attributes as well as a short textual description of their specialties and traits (see for example https://www.fifaindex.com/player/243812/rodrygo/). The problem that arises here is that these data are not downloadable and copying everything into a spreadsheet for all the players is neither time efficient nor possible.

In order to overcome this lack of open data, we can make use of the data science technique called web scraping. Web scraping is the process of automatically extracting data from a website that cannot be obtained by manual procedures, either because the data are not available as downloadable files or because the manual copy-and-paste alternative is time and cost intensive.³⁶ The most important aspect of web scraping is to respect the website owners. This makes web scraping user friendly and efficient. In this regard, a file called robots.txt is used on every website to specify which information may or may not be accessed. The first step in web scraping is to check this file. In the case of FIFA Index, there is no robots.txt file (https://fifaindex.com/robots.txt was not found on this server). However, it must be highlighted that the lack of a robots.txt file or the fact that it permits scraping does not automatically mean that we are allowed to obtain the data. On the contrary, we must always read the website’s Terms of Use or similar pages. For example the NBA 2K25 Player Ratings, which are the official source for all news and updates regarding the basketball video game NBA 2K25 ratings, states in their Terms that scraping data from this website is explicitly prohibited and measures will be taken against individuals found in violation (https://www.2kratings.com/terms). In the case of FIFA Index, they state that users cannot make use of the Content without the express written permission of Company and/or Protected Content’s third party owner. With all these insights, I decided to stop trying to use data from FIFA Index.

In the football gaming world, another popular videogame is Pro Evolution Soccer (PES), created by Konami. In terms of player descriptions, PES Master (https://www.pesmaster.com/) includes the eFootball 2022 Database based on the 1.0 version released on April 14th, 2022. By December 2024, it contained 56,568 players across $847$ teams (https://www.pesmaster.com/efootball-2022/). As was the case with FIFA Index, players are also described by a number of attributes and textual information of their characteristics, skills and playing style (see for example https://www.pesmaster.com/william-saliba/efootball-2022/player/126689/). Therefore, this site is also an interesting option for the purpose of this research. There is a robots.txt file according to which all search engines are allowed to harvest data from the website (https://www.pesmaster.com/robots.txt). In addition, its privacy policy does not stipulate any prohibition for obtaining data (https://www.pesmaster.com/privacy-policy/). Therefore, since neither the robots.txt file nor the privacy terms explicitly prohibit the use of scraping scripts, we can collect the data. However, it must again be emphasized that this must be done in a friendly and responsible way to avoid problems with the flow of information on the website. In order to do so, I have followed the recommendations of the robotstxt³⁷ and polite³⁸ R packages, namely: (i) Check for permission; (ii) Introduce yourself by means of a user agent string; and (iii) Take it slowly by pausing for a number of seconds between requests. Finally, once data of interest are detected on any website, it is necessary to inspect how the data are internally structured. In this case, the PES data are part of an HTML (HyperText Markup Language) document, so by using straightforward R instructions, the data collection procedure can be easily automated and the data smoothly obtained.

So as not to overwhelm the target website with too many queries, but at the same time, with the aim of obtaining a representative and sufficiently large database, I have scraped the data from six top European domestic leagues, namely, Eredivisie from the Netherlands, LaLiga from Spain, Liga Portugal from Portugal, Ligue 1 from France, the Premier League from England and Serie A from Italy. Some may note that the Bundesliga from Germany is missing. The reason is that this league does not have a separate category in the eFootball 2022 database on PES Master (https://www.pesmaster.com/efootball-2022/).

The PES web page characterizes players in the following way: in terms of field players, their abilities are described with five major categories, namely, attacking, dribbling, defending, passing and physicality. These categories encompass a total of $21$ subcategories. However, one of them is again dribbling. As a result, the number of variables considered is $5 + 21 - 1 = 25$ . Regarding goalkeepers, there is one category (goalkeeping) and five subcategories, resulting in a total of six variables for them. It is worth emphasizing that the value assigned for each category on the PES web page is the average of the values of the subcategories. Therefore, in a real application it could make sense to analyse only the subcategories. It will be up to the recruiter to select the ultimate number of variables to feed the algorithm.

In addition to these numerical variables, there are four textual variables. The characteristics describe the frequency and accuracy of the player with his weaker foot, physical condition (form) and resistance to injuries. The skills describe the players’ performance by means of 44 different competencies. Furthermore, the player’s style is described in two different ways: each player can only have one playing style as such (some have none), but up to five AI (artificial intelligence) playing styles. In PES, the AI playing styles describe the AI behaviour and playing tendencies of computer-controlled players during matches. There are $22$ playing styles and $7$ AI playing styles. The textual description of the skills and all playing styles come from https://www.fifplay.com/ (https://www.fifplay.com/efootball-2022-player-skills/ and https://www.fifplay.com/efootball-2022-players-playstyles/, respectively). It is also worth mentioning that AI playing styles are PES specific and might not be considered as real variables that we could find in a real player’s report. However, since the goal here is to provide the algorithm with as much information as possible about players (even more so when it comes to textual information), I have maintained AI playing styles in the analysis. But again, in practice, it would be up to the recruiter to leave them out.

In order to illustrate the performance of the algorithm presented in this paper, the information collected from the PES Master database fits perfectly with my intentions since it includes numerical variables and textual descriptions.

Results

This section presents three analyses. First, the PES data collected are explored and described. Then, the performance of the player matching algorithm is illustrated with a small toy data set. Lastly, an experiment is proposed to find out whether the playing styles might be presumed from their textual definition and the averages of their related numerical variables. This experiment includes a sensitivity analysis aimed at examining the influence of the hyperparameters on the performance of the algorithm and obtaining some insights about their optimal values for recruitment tasks.

PES data exploration

By using the scraping procedure explained earlier, a total of $3491$ players were initially obtained. These are the players that the PES web page made available from the six leagues selected, $433$ of whom were goalkeepers. From these $3491$ players, $340$ were removed because of missing information regarding skills, playing style and AI playing style ( $102$ were goalkeepers). Figure 1 shows the number of teams and players per league. There are three leagues with 18 teams (the Dutch Eredivisie, the Portuguese Liga and the French Ligue 1) and another three with 20 teams (the English Premier League, the Italian Serie A and the Spanish LaLiga). The number of players ranges from the 414 of the Eredivisie to the 585 players of LaLiga.

Figure 1.

Number of teams and players per league obtained from the PES Master site. There are either 18 or 20 teams. Players range between approximately 400 and 600 players, the Eredivisie and LaLiga being the leagues with fewest and most players, respectively.

Figure 2 shows the number of playing styles provided by PES Master. The most frequent style, with 364 players, is the offensive full-back, which refers to a player who contributes to the attack whenever there is a chance. In the opposite corner is the classic number 10, a player who is expected to be creative and imaginative enough to connect with the front line in the best possible way. There are only 16 players with that label.

Figure 2.

Unique playing styles and number of players representing them. The offensive full-back is the most common player, whereas the classic number 10 is underrepresented.

PES Master established 13 different playing positions (one for the goalkeeper and 12 for the field players). It is interesting to know how the skills and AI playing styles of the players assigned to these positions are described. Figure 3 shows the most relevant words for each position by means of the statistic tf-idf (term frequency-inverse document frequency), which measures how important a word is to a text in a collection (or corpus) of texts.³⁹ Goalkeepers are best described by punting the ball, being reactive and penalties. Positions related to defense (center-back, right-back and left-back) share stopping traits such as chasing, tackling or intercepting (terms for right-back and left-back are the same). Positions related to midfield (attacking midfielder, defensive midfielder, central midfielder, left midfielder and right midfielder) are different depending on whether their purpose is more related to attacking or defending. The defensive midfielder is like an additional member of the defense. The central midfielder has both missions, intercepting in defense and playing the ball with accuracy in attack. The attacking midfielder must also approach the goal, while one of the most important actions for the left and right midfielders is cutting to the goal. Finally, forward positions (central, left winger, right winger and supporting striker) are very much related to shooting and goals. The central forward must also be good at heading.

Figure 3.

The x-axis refers to the highest tf-idf words in player description texts.

Numerical variables (the abilities) start at 40 and the highest value is 90. Table 1 shows the variables and the players who show this high value of 90. James Ward Prowse is the only player who has two values of 90, specifically in set piece taking and stamina abilities. It is interesting to highlight that together with some of the highest-profile players, such as Mbappé and Vinícius (who play for Real Madrid) and Doku (who play for Manchester City), this simple analysis allows us to discover other less known players who show the same great performance in these abilities, such as James Ward Prowse (Nottingham Forest), Kevin Schade (Brentford) and Tariq Lamptey (Brighton & Hove Albion).

Table 1.

Abilities and players where the highest value 90 is achieved. James Ward Prowse appears twice. This simple analysis allows us to discover less-known players, such as Kevin Schade, together with some of the highest-profile players, such as Kylian Mbappé.

feature	max. value	players
set piece taking	90	James Ward Prowse (Nottingham Forest)
speed	90	Kevin Schade (Brentford),
		Kylian Mbappé (Real Madrid)
acceleration	90	Tariq Lamptey (Brighton & Hove Albion),
		Adama Traoré (Fulham),
		Jérémy Doku (Manchester City),
		Vinícius Júnior (Real Madrid)
stamina	90	James Ward Prowse (Nottingham Forest)

Basic performance of the algorithm

Before applying the algorithm to the PES data, it is worth giving insights into the algorithm’s performance. To this end, I use the small toy data set displayed in Table 2. It is made up of three players, who are described by their stamina values as well as the textual scouting report. After receiving the user’s query, the algorithm will try to return the most suitable player from this input data set. The goal of this analysis is to check that the algorithm returns the expected results.

Table 2.

Toy data set to check how the player matching algorithm works.

name	stamina	scouting_report
Player A	85	fast and strong forward
		with excellent goal-scoring ability
Player B	90	moderate speed, good stamina
		and decent passing
Player C	80	quick and aggressive forward,
		good at scoring under pressure

Let us suppose that our aim is to search for players who are quick and aggressive forwards. The expected outcome of the algorithm must be Player C in terms of textual similarity. As can be seen in Table 3, player C is the one with the highest textual similarity, with a value of 0.618. In addition, in terms of numbers, we consider three cases. In the first one, the player we are searching for (the target) has a stamina value of 70. The expected outcome of the algorithm in terms of numerical similarity must be Player C. In the second case, the target stamina value is 85, so the expected outcome must now be Player A. In the third case, the target stamina is 89, so the expected outcome must now be Player B. Table 3 shows that the algorithm does indeed return these results. As an illustration, the computation of the numerical similarity for the query Quick and aggressive forward with stamina of 70 is detailed. This is analogous for the rest. The players’ actual stamina values are $85$ , $90$ and $80$ , the average and standard deviation being $85$ and $5$ , respectively. Therefore, after standardization (subtracting the average and dividing by the standard deviation), the values become $0$ , $1$ and $- 1$ . On the other hand, the target value is 70, which becomes $- 3$ with the same standardization procedure ( $(70 - 85) / 5$ ). Finally, the Gaussian kernel formula gives $\exp (- | - 3 - c (0, 1, - 1) |) \approx 0.05, 0.018,$ $0.135$ . In a real situation aimed at obtaining a final recommendation for a player to recruit, the procedure will end by combining the textual and numerical similarities into a final similarity score. This will be done with the hyperparameter $α$ , whose purpose is to create a weighting between the similarities. Indeed, $α$ is used to establish a compromise between the importance of the text and the importance of the numbers. However, it is worth clarifying that this decision may vary between data sets. If the working data set involves high-quality text, such as text that includes a number of relevant keywords or is rich in shades of meaning, it would make sense to give a higher weight to textual similarity. Therefore, $α$ will be close to $1$ . On the contrary, if the data set contains few lines of text or the numerical variables represent the players’ characteristics better, more weight will be given to the numerical similarity, in which case $α$ will be close to $0$ . The recruiter’s feelings may also influence the selection of $α$ . Depending on the importance that the recruiter gives to text or numbers, $α$ will be modulated accordingly.

Table 3.

Algorithm’s performance with the toy data set. The highest values are shown in bold. The expected results are obtained in all cases.

Quick and aggressive forward with stamina of 70
Player	Textual similarity	Numerical similarity
A	0.266	0.05
B	0.046	0.018
C	0.618	0.135

Quick and aggressive forward with stamina of 85
Player	Textual similarity	Numerical similarity
A	0.266	1
B	0.046	0.368
C	0.618	0.368

Quick and aggressive forward with stamina of 89
Player	Textual similarity	Numerical similarity
A	0.266	0.449
B	0.046	0.819
C	0.618	0.165

Utility for scouting purposes

In order to evaluate the overall activity of the algorithm, it is worth admitting that the best practice would be for professional scouts to try it with their numerical and textual reports from a collection of players of interest. Until we reach this ideal situation, we can play with the algorithm in other ways and can also confirm its usefulness for player identification purposes.

In the collected database from PES Master, the scouting report is made up of the textual information from the characteristics, skills and AI playing styles. In this section, the experiment investigates whether the unique playing style assigned to each player can be inferred from a user’s query that includes the average numerical values from the players that are labelled with that playing style and the textual description of the same playing style. Just as a reminder, there are $22$ different playing styles. The basic procedure of the experiment for every playing style is as follows: (i) we feed the algorithm with the corresponding user’s input query, as explained above; (ii) we take the first 10 players with the highest similarities that the algorithm recommends; and (iii) we count how many of these 10 players correspond to the target playing style.

To illustrate this procedure here, I use goalkeepers because they have the most manageable number of numerical variables. On the PES Master website, variables start off red (poor performance) and change to orange at a value of 70 (acceptable), changing to green at 80 (good) and blue at 90 (excellent). Figure 4 shows the average values for all the goalkeepers labelled with the two playing styles defined for them, either offensive or defensive. It can be seen that the two classes are similar, but offensive goalkeepers are slightly better in their reflexes and worse in how quickly they respond to the ball when playing in goal (awareness) and how they can cover the goal and area (reach). Both classes move within acceptable values slightly above 70 units. Having said that, as an example for the playing style defensive goalkeepers, the input query aimed at finding them would be: A solid goalkeeper who prefers to stay around the goal line, goalkeeping of 72, gk awareness of 72, gk catching of 70, gk parrying of 70, gk reflexes of 73, gk reach of 73. As can be observed, the query must contain the same names of the numeric variables available in the PES Master reference database.

Figure 4.

Average numerical profile for the two playing styles defined for goalkeepers: either offensive or defensive. Both types are very similar, but offensive goalies are slightly better in reflexes and somewhat worse in awareness and reach.

As mentioned above, the player matching algorithm depends on several parameters. This practical application also presents a good opportunity to check in real time the influence they may have on the results provided by the algorithm. In an ideal situation, an optimal combination of parameters should guarantee that the algorithm returns sufficiently reliable recommendations. In a classification setting like this, where class-specific performance is evaluated, a common approach is to compute the classifier’s sensitivity, that is, we measure the percentage of true playing styles that are identified for each of the $22$ playing styles, or in other words, the percentage of matches between the target playing style and the 10 recommendations:

(\frac{1}{220} \sum_{i = 1}^{22} \sum_{j = 1}^{10} I (y_{i} = {\hat{y}}_{i j})) \times 100.

(5)

Here $y_{i}$ is the $i$ th target playing style and ${\hat{y}}_{i j}$ is its corresponding $j$ predicted playing style. And $I (y_{i} = {\hat{y}}_{i j})$ is an indicator variable that equals 1 if $y_{i} = {\hat{y}}_{i j}$ and 0 if $y_{i} \neq {\hat{y}}_{i j}$ . If $I (y_{i} = {\hat{y}}_{i j})$ then the $i j$ th prediction was the same playing style as the target one; otherwise it was misclassified. Hence, equation (5) computes the fraction of correct classifications.

In order to carry out the experiment, I have created a grid of several combinations of parameters. For the rank value I have chosen $6$ values: [30,40,50,60,70,200]. The last value, 200, is added to examine overfitting and computational cost. Indeed, in the GloVe model a higher rank might imply that the embedding matrix memorizes strange co-occurrences that cannot be generalized to unseen text. In addition, more dimensions mean more operations and more parameters to store and update. For the maximum number of co-occurrences, I have chosen $3$ values: [5,10,15]; for the size of the context window, I have also chosen $3$ values: [3,5,7]; and for $α$ , I have chosen $9$ values: [0.1,0.2,…,0.9]. In total, there are $486$ different combinations of parameters. The grid search procedure is chosen since it is a traditional way of performing hyperparameter optimization and also allows us to cover a considerable portion of the hyperparameter space.

Figure 5 is a visualization of the performance of every single combination. The bottom x-axis shows the values for the rank and the top x-axis shows the values for the maximum number of co-occurrences. The right y-axis shows the values for the context window and the left y-axis shows the sensitivity values. Each combination is coloured according to the level of $α$ . A red point highlights the optimal combination, that is, the one that returns the highest sensitivity, namely, $46.88 %$ . This combination is the one that has a rank of 50, a maximum number of co-occurrences of 5, a context window also of 5 and an $α$ of 0.1. These values are quite reasonable. In terms of the statistics-related parameters, they are not hard defaults as such for the GloVe procedure, but they are common starting points in some implementations, particularly in R packages like text2vec.

Figure 5.

Sensitivity values obtained by the whole combination of hyperparameters using the grid search procedure. The red point indicates the optimal combination. It is the one that has a rank of 50, a maximum number of co-occurrences of 5, a context window also of 5 and an $α$ of 0.1.

If we check the top 5 sensitivities as shown in Table 4, the parameter that fluctuates most is the context window. The other ones are more stable. It is also worth highlighting that the value of $α$ is the smallest one, namely, $0.1$ . This is to be expected because the piece of text included in the user’s query is short and there are 25 numerical variables (6 for the case of goalkeepers), so we could anticipate that the average numbers have larger influence than text in the classification task. This means that the numerical similarity was more important than the textual similarity. In cases like this practical exercise, one may argue that the marginal contribution of textual information compared to numerical data questions the use of NLP here. However, in my opinion, this aspect must be looked at from another point of view, namely, the fact that the possibility of including two sources of information can help to compensate the situation when either text or numbers are not giving sufficient information about the players’ characteristics. This will allow recruiters to conduct more comprehensive player searches based on a wealth of information. Regarding the extreme rank value of 200, it has not had any particular effect on the performance of the algorithm. In fact, the highest sensitivity value from a combination of parameters containing a rank value of 200 was in 35th position. Overfitting was probably not an issue here. On the other hand, we can confirm that this rank value requires a slightly higher computational cost. As an illustration, with the optimal combination, the execution of the algorithm for the 22 playing styles took 64 seconds. When the rank was changed from 50 to 200, it took it 77 seconds.

Table 4.

Combinations of hyperparameters that returned the top 5 sensitivity values.

rank	max_co_occur	context_window	alpha	sensitivity
50	5	5	0.1	$46.88 %$
50	5	7	0.1	$46.49 %$
60	5	5	0.1	$46.19 %$
60	10	5	0.1	$46.19 %$
50	5	3	0.1	$45.98 %$

Having identified the optimal combination of hyperparameters, Figure 6 shows the overall results for all the unique playing styles. The x-axis shows the playing style that we are searching for (the target). The y-axis allows us to check for matches. Indeed, the numbers on the plot count how many of the top 10 predictions match the corresponding targeted playing style. The category that shows the best match is offensive full-back, where 9 out of the 10 players were labelled as such. The next categories that show a notable match are anchor man and goal poacher, with 8 of the players grouped under that label. On the other side of the coin, we find that none of the players fit the full-back finisher category. In fact, it is also interesting to note that 9 of the predictions were classed as offensive full-back. In fact, the largest class returned is offensive full-back, with 23 predictions. This is related to the fact that it is the most available category, as was displayed in Figure 2. In addition, for classic number 10 and dummy runner, only one out of the 10 players fitted that label.

Figure 6.

Matches between the targeted unique playing styles and the top 10 predictions. The offensive full-back category shows the most matches, while full-back finisher shows the least.

The conclusion that we can reach with this experiment is that slightly more than half of the unique playing styles assigned to the players do not correspond to the average numerical performance. The internal PES Master categorization must be based on other criteria. In addition, this analysis also allows us to confirm that the algorithm is useful for finding players with certain characteristics. In terms of the sensitivity analysis, it has enabled us to check a considerable number of different values of the hyperparameters until the optimal combination was identified. This should help us as a guide for future applications.

Conclusions

The advent of generative AI tools, such as ChatGPT, has enhanced studies that had not yet been sufficiently developed. One sector that has started to integrate this new technology is football, as shown by Sevilla FC. This Spanish football club has created an automatic scouting system in collaboration with IBM to streamline player recruitment. Talent identification is a key issue in team sports. As Sevilla FC is indeed a private organization, one logical limitation of using this approach is that the methodology cannot be used by external users. However, I was curious to find out the details. Equipped with my knowledge of statistics and natural language processing, I interacted with ChatGPT to understand and eventually develop a maybe simpler but still useful and mathematically rigorous player identification algorithm.

One limitation when carrying out research on football analytics is the difficulty accessing public data. This problem is exacerbated when the goal is to analyze players’ performance using unstructured data. Most available data sets focus primarily on numerical statistics and often lack textual descriptions of players. In order to overcome this obstacle, I propose using data from football video games. Indeed, these games have the advantage that they mimic the players’ performance. By carrying out a friendly web scraping procedure, I have been able to obtain a representative set of football data from the PES Master website, which is the source of data for the video game called Pro Evolution Soccer.

The algorithm presented in this study shows two main benefits. On the one hand, it unifies structured and unstructured information to work with a complete player profile. On the other hand, it helps the scout save time by avoiding reading through multiple reports to search for certain characteristics of interest. However, I have also noted two main limitations. First, the problem that arises when a query contains new words that do not appear in the training corpus (so-called out-of-vocabulary (OOV) words). In this case, one interesting feature of the methodology is that including numerical attributes of the players can compensate for the fact that texts do not match the corpus players well. And the reverse is also true: if numbers are scarce but there is plenty of text describing the players, the algorithm can also be considered as a powerful tool. Overall, having the possibility of including two sources of information can help to compensate the situation when either text or numbers are not giving sufficient information about the players’ characteristics. The second drawback is testing the algorithm with real data. In this context, future work involves incorporating feedback from scouts to improve the system iteratively. As the Referee rightly pointed out, another promising research direction could be working with the text from match reports.

There is a debate about how ChatGPT might replace humans with the argument that ChatGPT shows a vast knowledge about any field. From my user experience, this tool still makes some mistakes and questionable decisions, both mathematically and in programming. I argue that ChatGPT practitioners need to possess a certain amount of technical knowledge to stop the spread of wrong ideas and guarantee that the scientific output generated is reliable. Otherwise, the proliferation of unsupervised and uncontrolled content might have disquieting implications for the community. Without a critical self-reflexive procedure, the benefits of using generative AI tools are failing to contribute to the progress of science. Through conducting this research I have reached the conclusion that a proper use of ChatGPT can foster and leverage autonomous research, but we must advocate to safeguard critical and rational thought.

Last, but not least, I wish to promote best practices of scientific reproducibility, also in sports. In this context, the new R package fawir is publicly available and includes all the functions and a toy data set to reproduce the results reported in this article. In terms of my personal infrastructure of sports analytics, future work will involve adding more packages for other sports.

Footnotes

Acknowledgments

During the preparation of this work I used ChatGPT in order to create the player matching algorithm. After using this tool, I reviewed and edited the content as needed and take full responsibility for the content of the publication. The study design, data collection, data analysis, paper writing and creation of the fawir R package were entirely my own work. I would also like to thank the PES Master website for making the data accessible. I am also grateful to the Referee and Editor in Chief for the constructive criticism and commitment, which helped to enhance the manuscript.

Consent to participate

Not applicable.

Consent for publication

Not applicable.

Data availability

Data and code are available as part of the free R package fawir, which is available on GitHub. It includes documentation for all the files as well as a vignette and a dashboard.

Declaration of conflicting interest

The author declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Ethical considerations

Not applicable.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Guillermo Vinué

Notes

References

Palazzo

Ievoli

Ragozini

. Testing styles of play using triad census distribution: an application to men’s football. J Quant Anal Sports 2023; 19: 125–151.

Malikov

Kim

. Beyond xG: A dual prediction model for analyzing player performance through expected and actual goals in european soccer leagues. Appl Sci 2024; 14: 1–23.

Toma

Campobasso

. Using data analytics to capture the strategic and financial decision-making of Europe’s top football club. Technol Forecast Soc Change 2023; 186: 122116.

Butler

Farnell

Simmons

. Do sports analytics affect footballer pay? Front Behav Econ 2024; 3: 1–12.

OpenAI. GPT-4 Technical Report. Technical report, OpenAI, 2023. https://cdn.openai.com/papers/gpt-4.pdf.

Sohail

Farhat

Himeur

, et al. Decoding ChatGPT: A taxonomy of existing research, current challenges, and possible future directions. J King Saud Univ - Comput Inf Sci 2023; 35: 1–23.

Sedaghat

. Early applications of ChatGPT in medical practice, education and research. Clin Med 2023; 23: 278–279.

Biswas

. Potential use of Chat GPT in global warming. Ann Biomed Eng 2023; 51: 1126–1127.

Carvalho

Ivanov

. ChatGPT for tourism: applications, benefits and risks. Tour Rev 2024; 79: 290–303.

10.

Frederico

. ChatGPT in supply chains: Initial evidence of applications and potential research agenda. Logistics 2023; 7: 26.

11.

Zhang

. The impact of ChatGPT on HR Recruitment. J Educ Humanit Soc Sci 2023; 19: 40–44.

12.

Bergkamp

Frencken

Niessen

, et al. How soccer scouts identify talented players. Eur J Sport Sci 2022; 7: 994–1004.

13.

Castellanos-Gomez

. Good practices for scientific article writing with ChatGPT and other artificial intelligence language models. Nanomanufacturing 2023; 3: 135–138.

14.

Berry

Fagerjord

. Digital humanities. Cambridge, UK: Polity Press, 2017.

15.

Cotterill

. Anatomy of a football scout: An in-depth look at player recruitment. São Paulo, Brasil: Camara Brasileira do Livro, 2022.

16.

Coté

. The development of coaching knowledge. Int J Sports Sci Coach 2006; 1: 217–222.

17.

Jauhiainen

Ayramo

Forsman

, et al. Talent identification in soccer using a one-class support vector machine. Int J Comput Sci Sport 2019; 18: 125–136.

18.

Theagarajan

Bhanu

. An automated system for generating tactical performance statistics for individual soccer players from videos. IEEE Trans Circuits Syst Video Technol 2021; 31: 632–646.

19.

Barron

Ball

Robins

, et al. Science and football. Identifying and developing talent, chapter Identifying playing talent in professional football using artificial neural networks, 2023, pp.15–24. United Kingdom: Routledge. https://doi.org/10.4324/9781003375968.

20.

Feinerer

Hornik

Meyer

. Text mining infrastructure in R. J Stat Softw 2008; 25: 1–54.

21.

Morgulev

Azar

Lidor

. Sports analytics and the big-data era. Int J Data Sci Anal 2018; 5: 213–222.

22.

Al-Asadi

Tasdemir

. Predict the value of football players using FIFA video game data and machine learning techniques. IEEE Access 2022; 10: 22631–22645.

23.

Cotta

de Melo

Benevenuto

, et al. Using FIFA Soccer video game data for soccer analytics. In: Workshop on Large Scale Sports Analytics. pp.1–4.

24.

Barker

Chue Hong

Katz

, et al. Introducing the FAIR principles for research software. Sci Data 2022; 9: 1–6.

25.

R Core Team. R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria, 2024. https://www.R-project.org/.

26.

Zivkovic

. worldfootballR: Extract and Clean World Football (Soccer) Data, 2022. R package version 0.6.2, https://CRAN.R-project.org/package=worldfootballR.

27.

Little

. FPLdata: Read in Fantasy Premier League Data, 2021. R package version 0.1.0, https://CRAN.R-project.org/package=FPLdata.

28.

Egidi

Palaskas

. footBayes: Fitting bayesian and MLE football models, 2023. R package version 0.2.0, https://CRAN.R-project.org/package=footBayes.

29.

Casals

Fernández

Martínez

, et al. A systematic review of sport-related packages within the R CRAN repository. Int J Sports Sci Coach 2023; 18: 621–629.

30.

Mikolov

Yih

Zweig

. Linguistic regularities in continuous space word representations. In: Proceedings of the 2013 conference of the North American chapter of the association for computational linguistics: Human language technologies, 2013, p.746–751. https://aclanthology.org/N13-1090.

31.

Pennington

Socher

Manning

. Glove: Global vectors for word representation. In: Empirical methods in natural language processing (EMNLP), 2014, pp.1532–1543. http://www.aclweb.org/anthology/D14-1162.

32.

Selivanov

Bickel

Wang

. text2vec: Modern Text Mining Framework for R, 2023. R package version 0.6.4, https://CRAN.R-project.org/package=text2vec.

33.

Nguyen

Day

Warton

, et al. fitzRoy - an R package to encourage reproducible sports analysis. R J 2021; 12: 82–106.

34.

Vinue

. A basketball big data platform for box score and play-by-play data. Big Data 2024; 00: 1–19.

35.

Vinue

. A web application for interactive visualization of European basketball data. Big Data 2020; 8: 1–17.

36.

Munzert

Rubba

Meissner

, et al. Automated Data Collection with R: A Practical Guide to Web Scraping and Text Mining. Chichester, West Sussex, UK: Wiley, 2105.

37.

Baltazar

Meissner

Ren

. robotstxt: A ‘robots.txt’ Parser and ‘Webbot’/‘Spider’/‘Crawler’ Permissions Checker, 2024. R package version 0.7.15, https://CRAN.R-project.org/package=robotstxt.

38.

Perepolkin

. polite: Be Nice on the Web, 2023. R package version 0.1.3, https://CRAN.R-project.org/package=polite.

39.

Silge

Robinson

. Text Mining with R. Santa Rosa, CA: O’Reilly Media, Inc., 2017. https://www.tidytextmining.com/ .