Abstract
Scientometrics has become a popular discipline, both as a field of study and a set of methods. Scientometric studies are carried out at varying levels from micro to macro. Institutions of higher learning and governments adopt scientometric measures to inform their decisions and to develop policies pertaining to ranking, standing, funding, impact, visibility and future plans. Institutions, countries and scholars can be evaluated using reliable scientometric indicators which are meaningful instruments in the study of disciplines. Scientometrics serves the purpose of determining the future of academic disciplines as well. However, it has not been applied to the study of the social sciences, including sociology, as much as it has been used in science disciplines. In this essay the need for scientometric studies in the study of sociology is put forward. Reviewing some prominent studies, the essay shows applicable models for scientometric studies for the study of sociology. As the use of scientometrics has not been effectively applied to qualitative studies, the essay presents the potential of scientometric data for qualitative studies of sociology.
Introduction
Scientometrics, bibliometrics, informetrics, webometrics, cybermetrics and netometrics are often used synonymously. Publication documents such as journal articles, chapters, books, reports and reviews constitute the basic units of analysis for scientometric studies. Groups of scientists, institutions and countries (De Bellis, 2009) also constitute firm units of analysis. These basic units are capable of transforming into several variables from which measurable indicators for the study of disciplines can be developed. Documents of publications can be sourced from databases or from the individual web pages of journals and publishers. Data can be generated from documents for analysis at micro level and macro level (Wallin, 2005).
Scientometrics is a group of methods. Since its origin in the 1960s, the use of scientometrics is evident in a host of areas. The methods include the analysis of word frequency, citation, authorship, co-citation, co-author, co-word, and count of author, research groups, institutions or country (Benckendorff and Zehrer, 2013). Research assessment, research evaluation, research performance, collaboration, citation and impact are among the common uses of scientometrics.
The application of the method in the study of disciplines is one of the major uses of scientometrics. In the study of disciplines, scientometrics serves to reveal the complexities of disciplines that are valuable not only to sociologists but also to disciplines themselves. As far as disciplines are concerned, knowledge about their features, research areas and subfields, the characteristics of authors, the type of research produced, the methodological and contextual developments, and the use of referencing material and citations are all important.
Social scientists have employed scientometrics more diligently for the study of science disciplines than for their own disciplines in the social sciences. With the arrival of citation indexes, online data sources are now more common. This makes scientometrics studies in sociology feasible. Apart from key online sources such as the Web of Science (WoS), Scopus and Google Scholar (GS), data can be obtained directly from the websites of publishers of journals and books.
The aim of this essay is four-fold. First, to show the importance of scientometrics to the study of sociology. This is done by reviewing some key scientometrics studies of the discipline that have been used for different purposes. Second, to draw the attention of sociologists to the untapped potential for scientometric studies other than those based on citation indexes. Scientometric data can be obtained directly from sources such as journals. Using an example, the process and steps are explained in detail. Third, to present how qualitative data analysis, which is sparingly applied in scientometrics, can be performed. Using a sample of sociology books, the process of obtaining and analysing qualitative data is described. Finally, with the support of an example, the essay explains the benefits of studying cited references in publications to better understand the discipline.
Scientometrics studies
As a tool for the study of the features and dynamics of authorship, scholars have turned their attention to scientometrics. Godin (2002) mapped the publications of Canadian researchers for a period of 20 years. In this analysis, Godin (2002) chose the volume of papers, the geographical and sectoral origins of publications, the specialisation of research areas and collaboration of authors. Mosbah-Natanson and Gingras (2014) examined the evolution of world publications in the social sciences.
Several subfields and research areas exist within a discipline. Knowledge of these in any discipline is a means towards knowing the discipline itself. If the analysis of publications in a discipline pertains to a particular period, the trends (emergences, decline or growth) of subfields can be revealed. Scientometrics thus serves the purpose of tracking the subfields and research areas within a discipline. As a way of exploring new concepts and areas that are found in a discipline, Shen (2016) looked for a specific sociological concept, social capital, in Chinese sociology. This analysis brought to light the popularity of the concept in Chinese sociological research.
Clark (1999) found the method useful for comparing publications in sociology and other disciplines. His content analysis of publications drawn from a selected sociology journal was mainly to study the diversity of subject matter in sociology and the methodologies employed in the discipline. Hartley and Robinson (2001) found Sociological Abstracts useful in examining the research undertaken by sociologists in selected colleges in the US.
Journal publications are not the only material for scientometric studies. Monographs and edited books are important channels of communication. Ossenblok and Engels (2015) analysed 753 edited books and 12,913 chapters published in various disciplines in the humanities and social sciences. The focus was on the distribution of publishers and their location, the language in which the books were published, the structural format of the chapters and co-authorship patterns.
Social science disciplines differ from science disciplines in the use of resources, which is evident from the type of references used. The use of references in publications refers to typical citation patterns. Knievel and Kellsey (2005) found how citation patterns differ from discipline to discipline, which become obvious in the number and type of books, journals articles and the language of publications. This kind of analysis has importance for both the discipline and its users. Librarians make decisions on the purchase of books and subscriptions to journals that are the most used in a discipline.
One of the most extensively discussed topics in scientometric studies is collaboration. Collaboration among authors indicates typical features about the discipline. Henriksen (2015) explored authorship patterns that exist in the social sciences. He used the publications in the WoS for three decades to search for the increasing tendencies of collaboration amongst authors. Hunter and Leahey (2008) examined collaboration in sociology. By using publications from two major journals in sociology, they realised how collaboration is influenced by certain factors. These include the year of publication, gender, institutional affiliation, prestige of the department, geographical location of the authors, methodologies used in the publications and sources of data.
Having seen an overall use of the method for the study of disciplines, a few specific studies on sociology may be examined in detail for the type of data and methodology used.
Bjarnason and Sigfusdottir (2002) focused on the productivity and impact of the publications of the sociology faculty in 16 departments in five Nordic countries: namely Sweden, Finland, Norway, Denmark and Iceland. The study had three major objectives: firstly, to present the productivity of and citations to publications produced by the faculty; secondly, to find the connection between the citations to publications, academic rank and the productivity of authors; thirdly, to examine the effects of productivity and citations on the individuals and departments. The analysis was carried out for a period of 20 years between 1981 and 2000. The data for the study covered journal articles that are indexed in the WoS and Sociological Abstracts. The analysis is based on a dataset of 1205 articles in 329 journals.
The impact factors for journals, authors and departments were measured in the above study. The data were processed according to the rank of the faculty, country, departments and journals in which articles appeared, and the citations the articles earned. Multilevel modelling of citations by individual authors, departments, and for both the individual and departments was performed.
The method of analysis in the study is systematically organised. The outlets in which the publications of the sociology faculty appeared were organised according to general/speciality sociology journals, national language/English language journals and citations of these publications. The study used the sum of journal impact factors as a reliable measure of the impact of individual scholars, sociology departments and countries. As indicated in the findings, publications of the faculty in high impact journals can increase the profile of both scholars and sociology departments. The importance of publication in key sociology journals for the impact and advancement of sociology is emphasised.
The importance of this scientometric study to the discipline in general and Nordic sociology in particular lies in its ability to show the relationship between publication patterns, faculty, departments and country. The study also revealed how the publications and their impact contributed to the trajectories of both the faculty and the departments of sociology in Nordic countries.
Aaltojärvi et al. (2008) made a similar attempt, choosing a different set of data and research objectives. In contrast to the study by Bjarnason and Sigfusdottir (2002), Aaltojärvi et al. (2008) resorted to data from Google Scholar (GS). Although scholars undertake scientometric studies mainly using data from the WoS or Scopus, the data from GS is also worth examining. Nevertheless, the authors compared their findings drawn from GS with the data obtained from the WoS. The study focused on three aspects of publications by the members of the sociology faculty in 16 Nordic departments: publication productivity, impact and web visibility. The main focus was more on the web visibility of the researchers and the impact of their publications. They chose sociology departments from five countries: namely, Sweden, Finland, Norway, Iceland and Denmark.
The authors looked for factors regarding the visibility of sociology publications on the web and the impact of Nordic sociologists. Furthermore, the study examined whether policies affect the publication behaviour of scholars, specifically sociologists. Aaltojärvi et al. (2008) were looking for the patterns in the publication productivity of the sociology faculty in the selected departments and the impact of Nordic sociology as made evident through the publications. In processing the data for analysis, the authors collected the names and positions of the sociology staff from their respective departmental web pages. This was followed by a search on GS to gather publications and citations. A total of 250 publications by 353 faculty members were finally made available for analysis. Citations of most of the publications were successfully sourced. The data were then subjected to rigorous statistical analysis. Regression models were developed to find out the effects of factors such as gender and position of the authors, age of the publications, departments in countries and type of publication.
The findings revealed a correlation between visibility and the position of the faculty. Similarly, the mutual influence of visibility, citations and position as crucial variables for the study of disciplines was reinforced. Through the publications of sociologists some of the unknown dimensions of sociology are made known. The differential status of sociology departments across Nordic countries provides meaningful insights into the development of the discipline. In view of the visibility and impact of publications, the authors were able to emphasise the level of internationalisation that has been happening in the discipline. As is evident, the intention was to know more about a segment of global sociology, namely Nordic sociology.
A book-length study of sociology found in the literature offers a novel scientometric attempt. Sooryamoorthy (2016) used various sources of scientometric data to portray the historical and contemporary features of sociology in South Africa, over three distinct political phases. The study delineated sociology through colonialism, apartheid and democracy and mapped the history and development of sociology for over a century.
The study combined scientometric data with historical information from reports, gazettes, proceedings and other pertinent documents. Apart from the data obtained from the WoS, the study consulted relevant journals that carried sociology publications. The issues covered in the publications in the WoS were complemented by these additional data.
In order to enrich the data and the analysis, certain demographic variables of the authors were also gathered. Information about the gender and race of the authors was searched for from institutional and personal websites. By incorporating the details of gender and race which are not provided in citation indexes and journals, the analysis becomes meaningful in revealing the influence of these variables in sociology.
Employing a proper sampling procedure and applying statistical tools, the study integrated a great deal of empirical data with the historical analysis of sociology in South Africa. The growth or decline of sociology in South Africa is mapped over three distinct phases that are connected to the political transformation in the country. In sum, the book provides a detailed account of the shifting foci and methodological preferences in the discipline in South Africa through its developmental trajectories influenced by colonialism, apartheid and democracy.
Journal sources
Data obtained directly from journals, as against the metadata from citation indexes, are often found to be supplementary or independent in scientometrics (Sooryamoorthy, forthcoming in 2021). Like citation indexes, the data deduced from publication documents of core disciplinary journals are valuable resources in scientometrics. Note that since data have to be collected manually, the volume of data to be used for a study cannot be as large as the data collected from citation indexes. In this section of the essay, a journal is taken as an example and describes how information from it can be productively used for scientometric studies.
The general and broad objective of this example scientometric study is to map the development of the discipline of sociology by using the publications in the selected journal, International Sociology, which is a core journal in the discipline of sociology. A single decade is chosen as the period of the study, therefore the window of time is nine years, from the beginning of the decade 2010 to 2018.
Articles published in the journal can be viewed at the publisher’s site, https://journals-sagepub-com-s.web.bisu.edu.cn/home/iss. While on this page, move the cursor to ‘Browse Journal’ from the row of options under the journal’s title, i.e. ‘Journal Home’, ‘Browse Journal’, ‘Submit Paper’, ‘About’, ‘Subscribe’. The ‘Browse Journal’ has two links: ‘Current Issue’ and ‘All Issues’. Choose ‘All Issues’. A page will appear, as shown in Figure 1. Select the issues from the drop-down menu (see the menu for ‘Select Decade’, ‘Select Volume’ and ‘Select Issue’) on the page as indicated in the figure, and click the ‘View’ button. Issues can also be browsed by year, using the link below the three menu bars (Figure 2). The plus sign in the circle in front of the year expands the volume, and subsequently the issues under the selected volume. When the year 2010, which is the first year of the period examined, is expanded, volume 25 is displayed, and a click on the plus sign or on the volume opens all six issues that are linked to the publication.
Selecting the volumes and issues of the journal.
Volumes and issues.
The first issue of volume 25 in 2010 has six papers. The first two papers are displayed in Figure 3. On this page, the title of the paper, the name of the author/s, the published date of the paper, the page numbers of the paper, the link to the abstract and the preview of the paper are shown. The title of the article and abstract are linked to the metadata of the publication. Opening the link on the title of the first publication, ‘Non-Discrimination towards Homosexuality’, presents the full bibliographical information about the publication. The link ‘Article information’ under the DOI number should be opened at this stage to view the volume, issue and page numbers of the publication.
Table of contents of the selected volume and issue.
Figure 4 displays the full title of the article, the full name and surname of the author, publication data, DOI, volume, issue and page numbers, institutional address and email address of the author, the abstract and keywords. These are the raw data of the publication required for scientometric analysis of the discipline. The data from each of these publications for each issue and volume of all the selected years need to be captured into a software program to proceed with the analysis.
Metadata of the publication.
More information can be gathered from these publications, provided there is access to the full-text. ISA members have full access to this journal. Certain key variables can be extracted from the full-text and cited references. Compared to abstracts, titles and keywords, the full-text publications have more content for robust qualitative data analysis. With the advancements in online publications and open access, full-text publications can be accessed directly from several online databases such as EBSCO, JSTOR, Elsevier, Science Direct, Wiley Online Library and online journal platforms including Cambridge Online Journals, Oxford University Press, Taylor and Francis and several other national and regional databases. Full-text journal publications may be obtained from their journal web platforms. Subscriptions, except for open access sources, are a prerequisite to access. Institutional libraries subscribe to many of these databases for their students, teachers and researchers.
Book publications
In this example, a sample is taken of significant book publications. Searching the Scopus database, which is known for its books collection, 88 records were retrieved. These were books published in sociology under the broad area of the arts and humanities in 2018. The document was edited to include only the information on the author, affiliation, title, abstract, publisher and language. The general objective, for the purpose of example, is to look for prominent and emerging topics and relevant aspects of the authors of these books in the field of the arts and humanities. The specific focus is obviously on the discipline of sociology.
The first step in qualitative data analysis is categorising, indexing, labelling or coding. Coding is the beginning of the analysis. This entails attaching tags or labels to assign meaning to the descriptive material and may refer to words, phrases, sentences or even paragraphs (Miles and Huberman, 1994) (see the example in Figure 5). In this process, the textual material – the titles and abstracts of the books – is read several times to develop meaningful codes which emerge during the process of reading and carefully considering the document several times over. This is a reflexive process, rooted firmly in the study objectives. While engaging in this process, the objectives or the purpose of the study are to be kept in mind.
Coded qualitative data on book publications: a sample of a book from Scopus in the field of arts, humanities and social sciences in 2018 (the codes are explained in Table 1).
The preliminary analysis of the title and abstracts of all 88 books published in 2018 in the arts, humanities and social sciences is presented in Table 1. The codes are derived from the textual material of the titles and abstracts of the book publications. These codes are organised according to the major themes and topics that were dealt with in these books, the disciplinary background, the contexts and methodological approaches followed in the studies presented in the books, and the countries of the authors and editors. They could also be quantified by counting the frequencies of occurrence, but that is not the purpose of qualitative data analysis, but is a different method called content analysis.
Codes derived from book publications in Scopus, arts, humanities and social sciences, 2018.
What do these codes signify in regard to the discipline? A glance at the codes thus generated points to the disciplinary features derived from the books. The listed codes also represent contemporary disciplinary trends in books published in the arts, humanities and social sciences. The long list of themes and subject areas are again indicative of the current foci of the disciplines, which are valuable for scholars to be aware of the ‘hot areas’ in their disciplines and their ‘marketability’. From the publisher’s point of view, this is valuable information for the industry to invite timely proposals of book publications. The primary concern of publishers in general is whether a book is marketable or not. In the same way, the methodology/approach adopted by the authors of the books presents the preferred and popular methodology/approaches that are in demand.
The above codes are only preliminary. They should be constantly refined, regrouped and recoded, keeping the objectives of the study in perspective. The data should be further explored with refined codes. The interrelationship between the refined codes will produce more new findings. In that case, data can be examined again for powerful insights. For example, it is worth exploring the relation between themes/subject areas and methodology/approach. It would be a purposeful exercise to see how certain themes/subject areas are closely or loosely related to particular research methodologies/approaches in the discipline.
The given qualitative data can provide answers to several such questions. In what way are the themes/subject areas (gender and humanitarian feminism, to choose two codes within the themes/subject areas) interrelated? Does a discipline have more of an orientation to a particular theme/subject area than others? What connection can be discerned between author, affiliation, country and methodology or approach of the study? What association between the country of affiliation of authors and the study contexts can be established in the data? Answers to these questions might lead to a new theoretical understanding of the subject of enquiry using this inductive method. Nor does the analysis end here. These codes will lead to models and theories as well. Any trained qualitative researcher will know how to get the best out of qualitative data by subjecting it to the rigorous process of analysis. The purpose of the example here, as indicated earlier, is to show the richness and potential of qualitative scientometric data for varying purposes that are central to scientometric analyses in sociology.
Cited references
Cited references in publication records also constitute a good data resource. They have not been adequately used as qualitative data. They can be analysed quantitatively for the nature of cited references (monographs, books, chapters and journal articles and their proportion), the age of the cited references (calculated from the year of publication), the titles of publications (to identify core and non-core journals), and even the place of publication. Despite the questions raised by some scholars regarding the validity of citation counts to measure scientific growth and quality due to the perfunctory nature of the references (Chubin and Moitra, 1975; Moravcsik, 1973; Moravcsik and Murugesan, 1975, Schubert and Maczelka, 1993, for instance), the content of cited references remains a source of data which has been used as quantitative data for content analysis. The qualitative aspects of this information are discussed next.
The content of a publication is influenced by the content of the cited references. They mutually influence each other. References cited in a publication indicate the theme, focus, objectives, methodology and analysis of not only the publication but also the discipline. Some of the core purposes of scientometrics, namely mapping, assessment, evaluation and citation practices, can be applied to the source of cited references. The topic of the publication should rely on other publications that deal with the same or related topics. This is why cited references are connected to the publication that cites the references.
In the following example, cited references from journal publications are used for qualitative data analysis. Scopus, compared to the WoS, provides full details (authors, year of publication, title of the article or chapter, title of the journal or book, volume number, issue number, page numbers, publisher’s name and city of publication) of the cited references in its indexed publications. The data are sourced from the Scopus database, using the advanced search option and limiting the search to the subject area of social sciences, published year 2018, English language, access type of open and others, publication stage of final, and the source title of the journal, Sociology. The data therefore pertain to all papers published in a core sociology journal in 2018.
The search revealed 61 publication records, which were downloaded and saved for analysis. The objective of the analysis is modest for this example. It is to look for the key areas of research in sociology, the methodologies followed, and the context of the study as derived from the title of the publications listed in the cited references. The importance of monographs and edited books in these cited references can also be assessed but is not attempted in this study, which is based on qualitative methods and analysis. Neither is a quantitative content analysis attempted.
The cited references in the 61 publications provide a huge volume of data. It runs to 141 single-spaced pages, with more than 3,000 references. In view of the volume of data, coding was done only for the first 50 pages of references, representing only 21 publication records. Table 2 presents the codes generated from the data. They are grouped under three categories: key areas of research, methodologies/approaches/methods, and the study contexts of publications.
Codes collected from publications in sociology in a core journal, Sociology, published in 2018.
These preliminary codes, obtained from the cited references in the sampled publications, present a variety of key areas, methodologies, research approaches and methods followed in the selected publications, and the study contexts representing different countries. Some of the key research areas in sociology, as found in the journal publications in 2018, indicate the current foci of the discipline. Some are quite new and innovative. The codes enhance the knowledge about the discipline. Codes are like dots that are connected to make a map of disciplinary aspects. As methodology is a central concern in sociological research and in publications, the codes reveal a list of contemporary methods, approaches and methodologies pursued in the discipline. In the same way, the contexts of the study that formed the content of publications reveal the contextual relevance of the discipline. This source generates valuable data for both sociologists and the discipline. The codes are just the beginning of the analysis, which can be explored further for more detailed findings.
As opposed to quantitative data analysis, a huge sample of records is not necessary in qualitative data analysis. To process thousands of pages of articles, reviews or books for qualitative data analysis is not practical. For the study of recent developments in a specific subject area within a discipline, full-text publications, properly sampled from a core journal that publishes papers in that specific area would easily meet the objectives for a proper qualitative data analysis.
Conclusion
This essay has shown the importance of scientometrics as a tool for the study of disciplines, sociology in particular. A number of objectives in regard to the study of the discipline can be achieved using the method. A major advantage of scientometrics is its application to mapping the development of a discipline, which can be performed by analysing publications produced over a period of time showing its growth, stagnation or decline. The shifting foci of the discipline in research areas can be traced. Scientometric analysis therefore assists in knowing sociology better and guiding sociologists towards achieving more visibility and impact for their research outputs. The essay also shows the untapped potential of descriptive scientometric data that can be used for qualitative data analysis.
Footnotes
Funding
The author received no financial support for the research, authorship, and/or publication of this article.