Scholarly databases under scrutiny

Abstract

The study aims to investigate whether indices (e.g. SSCI/ SCI/AHCI/ESCI, SCOPUS, MEDLINE, HW Wilson, ERIC) include potentially fake journals (PFJs) identified earlier in Beall’s list. Fourteen indices were examined to find out if PFJs are indexed by indices. The study also aims to provide detailed data regarding which of these indices include such PFJs along with information about the number of issues and articles published per year and publication fees charged by PFJs indexed by ordinary indices. The universe of the PFJs is composed of those listed in the Beall list (n=2708), and the study aimed to reach all the universe instead of being content with a sample from it. The results of the study indicated that one PFJ was indexed by SCI and two were indexed by ESCI. In addition, it was found that 53 PFJs were indexed by SCOPUS, 12 by ERIC and six by MEDLİNE. It was revealed that PFJs indexed by indices have significantly higher numbers of articles published annually and charge significantly higher fees compared to PFJs that are not indexed in such indices.

Keywords

Beall’s list indices potentially fake journals predatory journals publication fee scholarly databases

Introduction

Bibliographic databases (BDs) play an important role in the dissemination of universal scientific knowledge as they systematically offer researchers scientific knowledge. In addition, BDs should play an active role in strengthening the quality control of indexed articles and journals (Frandsen, 2017). Also, researchers assume that the journals and the studies published in them are included in BDs after they are meticulously examined and filtered through a scientific filter. However, over the last few years, some researchers have argued that BDs neglect their vital roles.

This discussion was sparked off by the claim that potentially fake journals (PFJs), which are called “evil” by Das and Chatterjee (2017), “Trojan Horse” by Demir (2018a) and “pollution” by Beall (2016), are indexed by such BDs as SSCI/SC/AHCI, SCOPUS, MEDLINE, ERIC (Beall, 2016; Masten and Ashcraft, 2016; Roberts, 2016; Van Nuland and Rogers, 2016; Ward, 2016). Initially, no one in the scientific world other than some young researchers in developing countries respected them.

However, Shen and Bjork (2015) argued that the number of PFJs is increasing day by day and an estimated number of 53,000 so-called articles published in PFJs are indexed in DOAJ. In addition, Nelson and Huffman (2015) revealed that six PFJs are indexed in Academic OneFile, 55 in Academic Search Complete, 812 in DOAJ, and 299 in ProQuest Central. However, the studies by Shen and Bjork (2015) and Nelson and Huffman (2015) were ignored. In addition, until recently some researchers have argued that that PFJs are indexed by some indices is not correct (Eriksson and Helgesson, 2016; Masten and Ashcraft, 2016; Roberts, 2016; Van Nuland and Rogers, 2016; Ward, 2016). These researchers’ arguments are based on the assumption that PFJs are not generally indexed in indices like SSCI/SC/AHCI, SCOPUS, ERIC, Wilson, and PubMed index. However, Savina and Sterligov (2016) argued that 447 PFJs were indexed in SCOPUS as of May 2016. Similarly, Macháček and Srholec (2017) claim that 306,000 documents, published by 405 PFJs, were indexed in SCOPUS in 2004 and 2015. The world of science in particular was seriously shaken when Manca et al. (2017a) reported that some PFJs are indexed in PubMed.

Until now no researcher has been able to show that PFJs are indexed in indices (other than PubMed), though Kratochvíl and Plch (2017) argued that some PFJs can be indexed by some indices like Scopus and ERIC. However, this is still an assumption. Frandsen (2017) from a different perspective reported that so-called articles published in 124 PFJs indexed in SSCI/SCI/AHCI were referred to 1295 times. Based on qualitative data, Demir (2018a) reported that one basic factor that makes PFJs attractive for researchers in developing countries is that they are indexed by some indices. Demir (2018b) and Demir (2018c) found that researchers who tend to publish in PFJs for academic promotion state that they would not publish in these PFJs if they were not indexed by indices, which indicates that it is high time that the issue be considered. Furthermore, researchers call for a study on a current list of PFJs which are indexed by SSCI/SCI/AHCI and in Scopus, MEDLINE, ERIC, and Wilson, which are viewed as field indices in developing countries (Demir, 2018a; Demir, 2018c; Frandsen, 2017; Kratochvíl and Plch, 2017).

There are some studies indicating that PFJs are indexed in some indices (Demir, 2018a; Kratochvíl and Plch, 2017; Manca et al., 2017a, 2017b; Nelson and Huffman, 2015; Shen and Bjork, 2015). However, these studies have some limitations: (1) no study examined whether all journals in Beall’s list and predatory publishers were indexed in indices; (2) these studies examined whether PFJs were indexed in DOAJ, Academic OneFile, Academic Search Complete, ProQuest Central, and PubMed indices. However, according to Demir (2018c) the basic reason that encourages researchers to publish in PFJs is that they are indexed by SSCI/SCI /AHCI and in Scopus, PubMed, ERIC, and Wilson, which are regarded as field indices. Therefore, the aim of this study is to ascertain whether indices include the PFJs in Beall’s list.

With this aim in mind, this study seeks to answer the following research questions:

Are PFJs indexed by indices?

If they are indexed:

Which PFJs are indexed by which indices?

Do the annual numbers of articles PFJs indexed in indices publish differ from those of PFJs that are not indexed by indices?

Does the publication fee PFJs charge vary according to the indices in which they are indexed?

Related research

It is possible to categorize the studies on PFJs into five main categories: those focusing on the places where PFJs are founded/directed, those focusing on their basic characteristics, the countries which publish most in PFJs, the characteristics of researchers, and those studying the reasons why researchers publish in PFJs.

In the literature, it is frequently argued that PFJs often give developed countries as their addresses but they are mostly located in developing countries (Beall, 2015b; Gutierrezet al., 2015; Kurt, 2018). Demir (2018a), who examined 735 journals in Beall’s list whose websites are accessible, ascertained that 119 out of 735 PFJs use incorrect addresses according to WHOIS data and that 82.3% of the PFJS were located in developing countries.

Truth (2012) emphasizes that one of the basic philosophy of PFJs is “pay big, publish fast”. But some of the authors publishing in PFJs do not actually pay to publish. They are editors, invited writers or some writers who fall into the trap without knowing the features of PFJs. In the same way, Bohannon (2013) pointed out that PFJs publish for money and that the articles do not go through a robust editorial process and referee review processes. It has been claimed that a low quality referee examination process takes place, if any at all, and articles are published for money regardless of scientific concerns (Anderson, 2012; Poynder, 2013).

In a study by Demir (2018a), 24,840 so-called articles published in 2017 PFJs were examined and it was determined that researchers from 146 countries out of 195 countries in the world published in these journals and that mostly researchers from India, Nigeria, and Turkey published in PFJs. Simón (2016) argued that the highest number of researchers publishing in PFJs were from Nigeria which was followed by Turkey, Botswana, Jordan. The common point in the literature is that it is mostly researchers from developing countries who publish in PFJs (Demir, 2018a; Kurt, 2018; Lukić et al., 2014; Nwagwu, 2015, Nwagwu and Ojemeni, 2015; Omobowale et al., 2014; Raghavan et al., 2014; Simón, 2016; Truth, 2012).

Xia et al. (2015) and Lukić et al. (2014) asserted that mostly young and inexperienced researchers published in PFJs. In a study by Demir (2018b), it was determined that most of the researchers who published in PFJs in 2017 do not hold PhD degree.

Arunachalam (2006) argued that researchers who lacked research skills and/or had failed to publish in journals indexed in indices like SSCI/SCI tended to publish in PFJs. The pressure of “publish or perish” has been frequently cited as another reason that directs researchers to PFJs (Demir, 2018c; Kolata, 2013; Roberts, 2016; Shaw, 2013; Xia, 2015; Xia et al., 2015). While Kurt (2018) cites social identity threat, unawareness, high pressure, and lack of research proficiency as reasons that direct researchers to PFJs, Xia (2015) and Xia et al. (2015) argue that academic and socio-cultural environments in developing countries and the desire to make a reputation encouraged them to publish in PFJs. It is frequently claimed that the academic allowance paid to researchers for publications in journals, whether they are PFJs or not, is another factor that directs researchers to PFJs (Abraham and Minj 2007; Adomi and Mordi, 2003; Demir, 2018a; Pyne, 2017). Furthermore, there are also studies arguing that the desire to get promoted quickly causes researchers to publish in PFJs (Djuric, 2015; Lukić et al., 2014; Omobowale et al., 2014). Lack of awareness is also a factor (Darbyshire et al., 2017; Hadi, 2016; Kurt, 2018; Lukić et al., 2014).

Some researchers suggest fighting against PFJs (Anthony, 2015; Beall, 2015b; Betz, 2016; Clark and Smith, 2015; Demir, 2018a; Fitzpatrick, 2015; Flanagan, 2015; Kmietowicz, 2009; Mehrpour and Khajavi, 2014; Moher and Moher, 2016; Pickler et al., 2015). Beall (2015b) strongly emphasized that PFJs cause great damage to the scientific world and the academic culture. In line with this, the editors of some journals published warnings for researchers who want to avoid PFJs (Cartwright, 2016; Darbyshire et al., 2017; Fernandez-Llimos, 2014; Hadi, 2016; Lewis and Wulster-Radcliffe, 2014; Owens, 2015).

Method

Research design and procedure

As this study aims to determine whether PFJs are indexed by indices using quantitative methods, it employs descriptive survey design, which is a quantitative research method. The research process is shown in Figure 1.

Figure 1.

The diagram of the current study.

Data collection process

Identification of journals

First, the databases with PFJs were examined. There are several webpages where PFJs are listed, such as Cabell’s New Predatory Journal Blacklist, Beall’s list, etc. However, the list issued by Beall explains how these journals were chosen (Beall, 2015a). But there are some serious criticisms about Beall’s list (see Berger and Cirasella, 2015; Crawford, 2014b). Besides, the Beall list is actually a blog. Thus, it cannot be argued that it is a fully reliable list. In spite of these limitations, this study, like many others (e.g. Kozak et al., 2016; Shen and Bjork, 2015; Xia et al., 2015), examined the journals in the Beall list.

The study aimed to reach the entire universe instead of making do with a sample from the journals in the Beall list. Beall has curated two lists: (1) questionable, scholarly open-access standalone journals (see https://beallslist.weebly.com/standalone-journals.html) and (2) questionable open-access publishers (see https://beallslist.weebly.com/)

At this point, the researcher accessed the websites of the journals and publishers. It is actually possible to access the journals in the first list. The journals published by the publishers in the second list were also identified. There are a total of 3311 PFJs (Beall, 2018a, 2018b). However, the websites of 603 PFJs were closed or hacked. Therefore, 2708 PFJs were examined in the study, based on the February 2018 version of Beall’s blog.

Later, the ISSN of these journals were checked. The same procedure was also followed by another researcher. It was seen that there was 100% accordance between the researchers. Then the researcher checked if these journals were indexed in the related journals through the use of journal names and ISSNs. After that, the journals indexed in the related databases were listed. Web of Science and ERIC stated they included these journals as they meet their criteria for indexing. Those replies from the two indices which were allowed to be shared are as follows:

1. Web of Science

Thank you for your message you reference Beall’s list, which is no longer updated. Beall’s list identified potentially predatory publishers and should not be considered a definitive determination of whether or not a particular publisher did or did not engage in predatory behaviors. Clarivate evaluates journals using its own rigorous and intensive selection criteria, available for review at https://clarivate.com/essays/journal-selection-process Clarivate Analytics takes seriously the issues of scholarly and publishing ethics. We will investigate the issues you raise. Any resultant changes to a journal’s coverage will be communicated directly to the publisher.

2. ERIC

Thank you for your enquiry we conduct a formal review process to select journals for indexing in ERIC. In addition to reviewing the full-text content, we apply several best-practice activities to consider the affiliation of the editors and reviewers, and the journal’s selection and peer-review process. ERIC is currently indexing the 12 journals listed in your email. During the formal review process, we consider the merit of each journal. Each of the 12 journals met the standard and criteria outlined in the ERIC Selection Policy when they were reviewed. Our review process also includes consideration of the information available on Beall’s list of potentially predatory journals and publishers. The selected journals must continue to meet the standard and criteria outlined in the ERIC Selection Policy. We regularly monitor the publishers and journals indexed in ERIC, and will take another look at the journals you submitted in the next review cycle. If at any time a selected journal fails to meet our policy, ERIC will discontinue indexing it. Please see our Selection Policy for more information: https://eric.ed.gov/?selection Best regards

After the databases (Web of Science and ERIC) confirmed that the journals were indexed in them, the publication fee per article for each of the journals was identified. The researcher sent emails to the journals which do not provide publication fee information on their webpages and 79% of them responded. The data from journals which did not respond were regarded as missing data.

Then the researcher identified the number of articles published by each journal indexed in any of the indices presented in Table 1. The number of articles published was also verified using the records of the related indices.

Table 1.

Indices examined in the study.

Sciences	Index
Educational Sciences	1. Education Full Text (HW Wilson) 2. British Education Index 3. Australian Education Index 4. ERIC
Social Sciences	5. Art Index (Art Research Database)
Science and Engineering Sciences	6. The Avery Index to Architectural Periodicals, 7. ICONDA (The International Construction Database) 8. Ergonomics Abstracts (Ergo-Abs) 9. DAAI (Design and Applied Arts Index)
Health Sciences	10. Ergonomics Abstracts (Ergo-Abs) 11. MEDLİNE<
Common to all sciences	12. SSCI/ SCI/AHCI13. ESCI14. SCOPUS

Identification of indices

The countries that publish most in PFJs are India, followed by Nigeria, Turkey, the US, China, Saudi Arabia, Indonesia, and Iran, respectively and the indices to be mentioned below are considered as indices in all these countries. Furthermore, publications in these indices are required for academic promotion or for receiving a material incentive (Demir, 2018a). Therefore, this study examined whether these indices, which are also considered in the world, index PFJs. As can be seen in Table 1, this study examined whether 14 indices index PFJs.

Data analysis

After the data collection process, the data analysis process started. Quantitative data were analyzed using PASW 18 software. Initially, 2708 PFJs were categorized as those indexed and not indexed. In order to ascertain if the publication fees and total number of annual publication varied according to whether they were indexed in indices or not, the Mann Whitney-U test was used as data were not normally distributed. Furthermore, to determine if the number publications and publication fees varied according to the index, the Kruskal Wallis-H Test was used as data were not normally distributed. Moreover, the researcher collected rich descriptive data and presented them in tables throughout the paper.

Result

Web of Science (SSCI/SCI/AHCI, and ESCI)

The data indicated that only one PFJ out of 2708 PFJs was indexed in SCI Journal of Biomedical Nanotechnology (ISSN: 1550-7033), published by American Scientific Publishers. In addition, Genetics and Molecular Research (ISSN: 1676-56809) and Annals of Palliative Medicine (ISSN: 2224-5820) are indexed by ESCI.

SCOPUS

SCOPUS is an index in the scientific world. The data indicated that 53 out of 2708 PFJs were indexed in SCOPUS. It was also found that 9 PFJs previously indexed in SCOPUS are no longer indexed there. The mean publication fee charged by these PFJs currently indexed in SCOPUS is $667.5. However, the mean publication fee charged by the nine PFJs removed from SCOPUS is $121.4. The PFJs currently indexed by SCOPUS are presented in Table 2, while those indexed in it before 2018 (excluded in 2018) are presented in Table 3.

Table 2.

PFJs currently indexed in SCOPUS (March 2018).

Journal	ISSN	Journal	ISSN
Australasian Medical Journal (AMJ)	1836-1935	Journal of Artificial Intelligence	1994-5450
Biomedical and Pharmacology Journal	0974-6242	Journal of Biological Science	1727-3048
International Journal of Veterinary Science	2304-3075	Journal of Environmental Science and Technology	1994-7887
Journal of Clinical and Analytical Medicine (JCAM)	1309-0720	Journal of Agronomy	1812-5379
Pharmacology Online (PhOL)	1827-8620	Trends in Bioinformatics	1994-7941
Shiraz E-Medical Journal	1735-1391	The Open Biochemistry Journal	1874-091X
Journal of Pure and Applied Microbiology (JPAM)	0973-7510	The Open Cardiovascular Medicine Journal	1874-1924
Oriental Journal of Chemistry	0970-020X	Current Chemical Genomics and Translational Medicine	2213-9885
Journal of Thoracic Disease	2072-1439	The Open Dentistry Journal	1874-2106
Translational Cancer Research	2218-676X	The Open Medicinal Chemistry Journal	1874-1045
Translational Lung Cancer Research	2226-4477	The Open Microbiology Journal	1874-2858
Stem Cell Investigation	2306-9759	The Open Neurology Journal	1874-205X
Cardiovascular Diagnosis and Therapy	2223-3652	The Open Rheumatology Journal	1874-3129
Quantitative Imaging in Medicine and Surgery	2223-4292	AACL Bioflux	1844-8143
Translational Andrology and Urology	2223-4683	Problems and Perspectives in Management	1727-7051
Chinese Clinical Oncology	2304-3865	International Journal of Green Pharmacy	0973-8258
Annals of Palliative Medicine	2224-5820	Journal of Digital Information Management	0972-7272
Translational Pediatrics	2224-4336	International Journal of Clinical and Experimental Medicine	1940-5901
Chinese Journal of Cancer Research	1000-9604	International Journal of Physiology	0167-8760
Journal of Biomedical Nanotechnology	1550-7033	International Journal of Physiology, Pathophysiology and Pharmacology	1944-8171
Journal of Bionanoscience	1557-7910	The WSEAS Transactions on Business and Economics	1109-9526
Journal of Advanced Microscopy Research	2156-7573	WSEAS Transactions on Systems and Control	1991-8763
Science of Advanced Materials	1947-2935	Research Journal of Chemistry and Environment	0972-0626
Journal of Medical Imaging and Health Informatics	2156-7018	Disaster Advances	0974-262X
Asian Journal of Clinical Nutrition	1992-1470	International Journal of Safety and Security Engineering	2041-9031
Asian Journal of Crop Science	1994-7879	International Journal of Design and Nature and Ecodynamics	1755-7437
Asian Journal of Epidemiology	1992-1462

Table 3.

PFJs indexed in SCOPUS before 2018 (currently not indexed).

Journal	ISSN	Journal	ISSN
Der Pharma Chemica	0975-413X	Journal of Nanoneuroscience	1939-0637
European Journal of Scientific Research	1450-202X	The WSEAS Transactions on Mathematics	1109-2769
European Journal of Social Sciences	1450-2267	Carbon - Science and Technology	0974-0546
Dynamic Systems and Applications	1056-2176	Journal of Holography and Speckle	1546-900X
Neural, Parallel, and Scientific Computations	1061-5369

ERIC

ERIC is an index in education sciences. Of the 2708 PFJs 12 are indexed by ERIC. The mean publication fee PFJs indexed by ERIC charge authors is $266.6. The PFJs currently indexed by ERIC are presented in Table 4.

Table 4.

PFJs currently indexed in ERIC (March 2018).

Journal	ISSN	Journal	ISSN
College Student Journal	0146-3934	Higher Education Studies	1925-4741
Education	0013-1172	International Education Studies	1913-9020
Educational Research and Reviews	1990-3839.	International Journal of Higher Education	1927-6044
English Language Teaching	1916-4742	Journal of Curriculum and Teaching	1927-2677
Journal of Education and Learning	1927-5250	Universal Journal of Educational Research	2332-3213
Journal of Education and Training Studies	2324-805X	World Journal of Education	1925-0746

Education Full Text (HW Wilson), British Education Index and Australian Education Index

None of the PFJs examined in this study is indexed in Education Full Text (HW Wilson), British Education Index and Australian Education, which are regarded as field indices in education sciences in many countries in the world. The claim that one PFJ is indexed in HW Wilson is totally misleading.

ErgoAbs, ICONDA, DAAI, Art Index, and the Avery Index to Architectural Periodicals

None of the PFJs examined in this study is indexed in ErgoAbs, ICONDA, DAAI, Art Index, and the Avery Index to Architectural Periodicals, which are regarded as field indices in architecture and engineering in many countries in the world.

MEDLINE

Medline is an index in health sciences. Of 2708 PFJs six are indexed by Medline. The PFJs currently indexed by MEDLINE are presented in Table 5. The mean publication fee PFJs indexed by MEDLINE charge authors is $1386. Of the PFJs previously indexed by Medline, 48 were seen to have been excluded from the index. Those volumes or issues previously indexed by Medline are accessible for some journals (Table 6), while they are not for some others (Table 7).

Table 5.

PFJs currently indexed in MEDLINE (March 2018).

Journal	ISSN	Journal	ISSN
Chinese Clinical Oncology	2304-3865	Journal of Biomedical Nanotechnology	1550-7033
Journal of Nanoscience and Nanotechnology	1533-4880	Annals of Palliative Medicine	2224-5820
Journal of Biomedical Nanotechnology	1550-7033	Journal of Nanoscience and Nanotechnology	1533-4880

Table 6.

PFJs indexed in MEDLINE before 2018 (currently not indexed).

Journal	ISSN	Journal	ISSN
American Journal of Blood Research	2160-1992	International Journal of Molecular Epidemiology and Genetics (IJMEG)	1948-1756
American Journal of Cancer Research	2156-6976	Journal of Clinical Medicine Research	1918-3003
American Journal of Clinical and Experimental Immunology	21647712	American Journal of Stem Cells	2160-4150
American Journal of Neurodegenerative Disease	2165-591X	American Journal of Translational Research	1943-8141
American Journal of Nuclear Medicine and Molecular Imaging	2160-8407	Annals of Translational Medicine	2305-5839
International Journal of Physiology, Pathophysiology and Pharmacology	1944-8171	Bioinformation	0973-2063
Cardiology Research	1923-2829	Gastroenterology Research	1918-2805
Cardiovascular Diagnosis and Therapy	2223-3652	Gland Surgery	2227-684X
Chinese Journal of Cancer Research	1000-9604	Journal of Gastrointestinal Oncology	2078-6891
Current Chemical Genomics and Translational Medicine	2213-9885	Journal of Thoracic Disease	2072-1439
Hepatobiliary Surgery and Nutrition	2304-3881	mHealth	2306-9740
International Journal of Biochemistry and Molecular Biology	2152-4114	Quantitative Imaging in Medicine and Surgery	2223-4292
International Journal of Biomedical Science	1550-9702	Stem Cell Investigation	2306-9759
International Journal of Burns and Trauma	2160-2026	Journal of Visualized Surgery	2221-2965
International Journal of Clinical and Experimental Medicine	1940-5901	The Open AIDS Journal	1874-6136
International Journal of Engineering Research and Technology (IJERT)	2278-0181	The Open Biochemistry Journal	1874-091X
The Open Biomedical Engineering Journal	1874-1207	The Open Biomedical Engineering Journal	1874-1207
The Open Cardiovascular Medicine Journal	1874-1924	The Open Dentistry Journal	1874-2106
The Open Medical Informatics Journal	1874-4311	The Open Medicinal Chemistry Journal	1874-1045
The Open Microbiology Journal	1874-2858	The Open Orthopaedics Journal	1874-3250
The Open Neuroimaging Journal	1874-4400	The Open Rheumatology Journal	1874-3129
The Open Neurology Journal	1874-205X	The Open Virology Journal	1874-3579
The Open Ophthalmology Journal	1874-3641	Translational Andrology and Urology	2223-4683
Translational Lung Cancer Research	2218-6751	Translational Pediatrics	2224-4336

Table 7.

PFJs indexed in MEDLINE before 2018 (currently not indexed and not accessible).

Journal	ISSN	Journal	ISSN
Journal of Biomedical Graphics and Computing	1925-4008	Endocrinology and Diabetes (Journal of Endocrinology and Diabetes)	2374-6890
EC Dental Science	Not found	European Journal of Pharmaceutical and Medical Research (EJPMR)	2394-3211
EC Microbiology	Not found	Palliative Medicine and Care	2374-8362
EC Neurology	Not found	Psychology	2152-7180
EC Ophthalmology	Not found	Symbiosis Open Journal of Anesthesiology and Pain Management (SOJAPM)	2374-684X
EC Pharmacology and Toxicology	Not found	Clinical Medical Reviews and Case Reports	2378-3656

Does the money PFJs charge vary according to the indices where they are indexed?

When the publication fee PFJs indexed by indices charge and the fee charged by PFJs that are not indexed by these indices are compared using the Mann Whitney U test, it was found that there was a significant difference in favor of the PFJs indexed by indices (see Table 8). In other words, PFJs indexed by journals charge significantly higher publication fees (U=12822.00; p=0.000; p<0.05).

Table 8.

The result of the Mann Whitney-U Test conducted to determine whether the amounts of money PFJs charge differed in terms of their being indexed in indices or not.

Index	$\bar{x}$	Mean rank	$U$	$z$	$p$
Indexed PFJs	659.24	693.41	12822.00	−9.624	0.000
Not indexed PFJs	125.08	416.05	12822.00	−9.624	0.000

As a result of a Kruskal Wallis-H test conducted to determine if publication fees PFJs indexed by different indices charge differed, it was found that the fees significantly differed in favor of SCOPUS and MEDLINE (see Table 9). In other words, PFJs indexed by SCOPUS and MEDLINE charge higher publication fees (χ213.26; p=0.001; p<0.05).

Table 9.

The result of Kruskal Wallis-H Test Conducted to determine whether the publication fee charged varied according to the indices where PFJs were indexed.

Index	N	$\bar{x}$	Mean rank	χ²	df	$p$
ERIC	12	266.6	24.17	9.886	2	0.007
Scopus	47	667.5	45.62
MEDLINE	6	1386.7	58.17

Does the number of articles PFJs indexed in indices publish annually differ from that of PFJs that are not indexed by indices?

When the total number of publications in the PFJs examined in this study in 2017 were compared using the Mann Whitney U test, it was found that there was a significant difference in favor of PFJs indexed by indices (see Table 10). In other words, the PFJs indexed by indices publish more articles (U=14952 .00; p=0.000; p<0.05).

Table 10.

The result of Mann Whitney-U test conducted to determine whether total number of publications in PFJs in 2017 varied in terms their being indexed by indices or not.

Index	Mean rank	$U$	$z$	$p$
Indexed PFJs	1687.98	14952.00	−13.28	0.000
Not indexed PFJs	910.79	14952.00	−13.28	0.000

Discussion and conclusion

The researcher received some spam emails from PFJs which claimed that they were indexed by ERIC and SCOPUS, and he wanted to find out if they were telling the truth. He realized that these PFJs are listed in Beall’s list. As a result of investigations in the databases of indices, it was seen that these two PFJs are indexed in these indices. This caused the researchers to ask the question: “How come these two PFJs are indexed in these indices?”, which led to a paradox in the mind of the researcher. This was the starting point for this study. Following from this, the researcher sought the answer for the question of whether the PFJs listed by Beall (2018 a and b) were indexed by indices or not.

Among the 2708 PFJs in the Beall list examined in this study, it was determined that one journal was indexed by SCI and two PFJs were indexed by ESCI. In addition, it was found that 53 PFJs were indexed by SCOPUS and 12 by ERIC. It was determined that six PFJs were indexed by MEDLINE, which is considered to be an index in medical sciences. This result is compatible with the result of the study by Manca et al. (2017a) that some PFJs were indexed by PubMed. Another important finding of the current study is that Education Full Text (HW Wilson), British Education Index, Australian Education Index, ErgoAbs, ICONDA, DAAI, Art Index, and the Avery Index to Architectural Periodicals do not index any PFJs.

In the literature, it is stated that it is mostly researchers from developing countries who publish in PFJs (Demir, 2018a; Kolata, 2013; Lukić et al., 2014; Roberts, 2016; Shaw, 2013) and these PFJs are founded in developing countries (Demir, 2018a; Kurt 2018; Lukić et al., 2014; Nwagwu, 2015; Nwagwu and Ojemeni, 2015; Omobowale et al., 2014; Raghavan et al., 2014; Simón, 2016; Truth, 2012; Xia et al., 2015). In a study by Demir (2018a), it was determined that one of the basic factors that makes PFJs valuable for researchers is that they are indexed by some journals. Furthermore, it was determined that young researchers who tend to publish in PFJs would not publish in these PFJs if they were not indexed by indices (Demir, 2018a). Therefore, arguing that “PFJs are local problems and the problem of developing countries” would be hiding one’s head in the sand since all the indices that include the PFJs are not located in developing countries but in developed countries.

Another significant result from the study was that the PFJs indexed in indices published significantly more articles and charged higher publication fees than those PFJs indexed in ordinary indices. In other words, PFJs included in indexes charge their authors more, and they tend to publish more articles in each issue, possibly due to economic concerns. This implies that indexes somehow support PFJs by including them in their databases and provide them with the prestige that they need to make more money without doing much.

It is not known why these indices include these PFJs, but considering the fees charged for acceptance and indexing processes the simplest and clearest explanation for this would be found in capitalist philosophy. However, the current study calls for further studies that will examine why indices include PFJs based on the statements of the indices themselves.

In line with the developments in ICT, it is now very easy to have access to information. Because researchers take it for granted that scientific data accessed through such databases are presented to the science community after they are carefully examined and filtered, scientists hardly question the reliability and academic value of studies that they access through indexes. Yet, the finding that 74 PFJs are indexed in indices indicates that the scientific community should question the validity and reliability of articles published in such PFJs.

Determining predatory journals and their publishers has been a hotly debated issue in the world of science. In line with this, some researchers carried out studies to determine PFJs based on some criteria (Beall, 2015a, 2016; McCrostie, 2017; Nicholl and Chinn, 2015; Nolfi et al., 2015; Vinny et al., 2016; Xia et al., 2015). However, the list and criteria determined by Beall have drawn more interest and several studies were carried out based on Beall’s list (see Eriksson and Helgesson, 2017; Frandsen, 2017; Hadi, 2016; Moher and Moher, 2016; Nelson and Huffman, 2015; Pyne, 2017; Shen and Björk, 2015; Ward, 2016).

The blog on predatory publishers and journals issued by Jeffrey Beall is followed by blogs in the world (Nelson and Huffman, 2015). However, there have been serious objections that some journals and publishers in this list are not actually predatory. For example, it is claimed that high quality journals published by some publishers known worldwide are included in this list (Crawford, 2014a, 2014b; Coyle, 2013; Emery, 2013). In addition, Crawford (2014a) argued that Beall favored western publishers with a Eurocentric perspective and was also prejudiced in Elsevier’s favor. Berger and Cirasella (2015) pointed out the benefits and restrictions of Beall’s list of “potential, possible or probable” publishers. They claim that Beall ignored the fact that less reputable and even trustworthy academic journals publish flawed or debatable articles. Besides, it was also claimed that Jeffrey Beall had reflected his biases in his list and that he was personally prejudiced against open access journals based on his cultural and linguistic beliefs (Anderson, 2015; Berger and Cirasella, 2015; Crawford, 2014a).

In addition, there are some serious criticisms about Beall’s list (Berger and Cirasella, 2015). For example, researchers argued that some criteria were not built on strong foundations and they were not completely explained and were inherently confusing (Berger and Cirasella, 2015; Crawford, 2014b). In line with this, it is possible that some journals in Beall’s list are not actually PFJs; therefore, this possibility should be born in mind when reading the results of the current study.

Limitations

This study is limited to 2708 accessible PFJs included in Beall’s list and 14 indexes There is some possibility that other PFJs are included in indices. The data collection process was completed at the end of March 2018. Therefore, the readers should note that the PFJs listed in this study with their ISSN numbers might be excluded from the databases mentioned.

Footnotes

Author’s contributions

SBD carried out the complete process.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Selcuk Besir Demir

Author biography

Selcuk Besir Demir is a faculty member of the College of Education of Cumhuriyet University in Sivas, Turkey. He received his Doctor of Philosophy in Education and has done post-doctoral studies in education and research methods.

References

Abraham

Minj

(2007) Scientific journal publishing in India: Promoting electronic publishing of scholarly journals in India. First Monday 12(10). Available at: http://firstmonday.org/ojs/index.php/%20fm/article/view/1954 (accessed 2 April 2018).

Adomi

Mordi

(2003) Publication in foreign journals and promotion of academics in Nigeria. Learned Publishing 16(4): 259–263.

Anderson

(2012) Predatory open access publishers: The natural extreme of an author pays model. The Scholarly Kitchen, March 2. Available at: http://scholarlykitchen.sspnet.org/2012/03/06/predatoryopen-access-publishers-the-natural-extreme-of-an-author-pays-model/ (accessed 2 April 2018).

Anderson

(2015) Should we retire the term ‘predatory publishing’? The Scholarly Kitchen. Available at: http://scholarlykitchen.sspnet.org/2015/05/11/should-weretire-the-term-predatory-publishing (accessed 2 April 2018).

Anthony

(2015) Predatory journals. Home Healthcare Now 33(2): 65–66.

Arunachalam

(2006) Open access: Current developments in India. Available at: http://arizona.openrepository.com/arizona/handle/10150/105554 (accessed 2 April 2018).

Beall

(2015a) Criteria for determining predatory open-access publishers. Available at: http://ssau.ru:8080/files/science/crpd/CriteriaBillPublishers2015.pdf (accessed 2 April 2018).

Beall

(2015b) Predatory journals and the breakdown of research cultures. Information Development 31(5): 473–476.

Beall

(2016) Predatory journals: Ban predators from the scientific record. Nature 534(7607): 326.

10.

Beall

(2018a) Potential predatory scholarly open-access journals. Available at: https://beallslist.weebly.com/standalone-journals.html (accessed 24 February 2018).

11.

Beall

(2018b) Potential predatory scholarly open-access publishers. Available at: https://beallslist.weebly.com (accessed 24 February 2018).

12.

Berger

Cirasella

(2015) Beyond Beall’s list: Better understanding predatory publishers. College and Research Libraries News 76(3): 132–135.

13.

Betz

(2016) Authors beware: Open access predatory journals. Journal of Pediatric Nursing: Nursing Care of Children and Families 31(3): 233–234.

14.

Bohannon

(2013) Who’s afraid of peer review? Science 342: 60–65.

15.

Cartwright

(2016) Authors beware! The rise of the predatory publisher. Clinical and Experimental Ophthalmology 44(8): 666–668.

16.

Clark

Smith

(2015) Firm action needed on predatory journals. BMJ: British Medical Journal 350: h10.

17.

Coyle

(2013) Predatory publishers peer to peer review. Library Journal. Available at http://lj.libraryjournal.com/2013/04/opinion/peer-to-peer-review/predatory-publishers-peer-to-peer-review/ (accessed 4 April 2013).

18.

Crawford

(2014a) Ethics and access 1: The sad case of Jeffrey Beall. Cites and Insights 14(4): 1–14.

19.

Crawford

(2014b) Journals, journals and wannabes: Investigating the list. Cites and Insights 14(7): 1–24.

20.

Darbyshire

McKenna

Lee

, et al. (2017) Taking a stand against predatory publishers. Journal of Advanced Nursing 73(7): 1535–1537.

21.

Das

Chatterjee

(2017) Say no to evil: Predatory journals, what we should know. Asian Journal of Psychiatry 28: 161–162.

22.

Demir

(2018a) The new Trojan horses of the science world: Predatory fake journals (PFJs), who publish in PFJs and why do they publish in these journals? Journal of Informetrics 12(4): 1296–1311.

23.

Demir

(2018b) A mixed-methods study of ex-post funding incentive policy for scholarly publications: Academic or economic? Journal of Scholarly Publishing 49(4): 1–28.

24.

Demir

(2018c) Pros and Cons of the New Fınancıal Support Polıcy For Turkısh Researchers. Scientometrics 116(3): 2053–68.

25.

Djuric

(2015) Penetrating the omerta of predatory publishing: The Romanian connection. Science and Engineering Ethics 21(1): 183–202.

26.

Emery

(2013) Heard on the Net: It’s a small world after all: Traveling beyond the viewpoint of American exceptionalism to the rise of the author. Charleston Advisor 15(2): 67–68.

27.

Eriksson

Helgesson

(2017) The false academy: Predatory publishing in science and bioethics. Medicine, Health Care and Philosophy 20(2): 163–170.

28.

Fernandez-Llimos

(2014) Open access, predatory publishing and peer-review. Pharmacy Practice 12: 427–428.

29.

Fitzpatrick

(2015) Predatory journals: When outcome is valued over quality. Applied Nursing Research 1(28): 1.

30.

Flanagan

(2015) Predatory publishers: Authors beware. International Journal of Nursing Knowledge 26(1): 1–1.

31.

Frandsen

(2017) Are predatory journals undermining the credibility of science? A bibliometric analysis of citers. Scientometrics 113(3): 1513–1528.

32.

Gutierrez

Beall

Forero

(2015) Spurious alternative impact factors: The scale of the problem from an academic perspective. Bioessays 37(5): 474–476.

33.

Hadi

(2016) Fake peer-review in research publication: Revisiting research purpose and academic integrity. International Journal of Pharmacy Practice 24(5): 309–310.

34.

Kmietowicz

(2009) Students warned not to subscribe to journal ‘scam’. BMJ: British Medical Journal 338: 3.

35.

Kolata

(2013) Scientific articles accepted (personal checks, too). The New York Times, 7. Available at: http://art.yale.edu/file_columns/0000/2306/kolata.pdf accessed 7 April 2013).

36.

Kozak

Iefremova

Hartley

(2016) Spamming in scholarly publishing: A case study. Journal of the Association for Information Science and Technology 67(8): 2009–2015.

37.

Kratochvíl

Plch

(2017) Predatory journals: How their publishers operate and how to avoid them. Vnitrni Lekarstvi Journal 63(1): 5–13.

38.

Kurt

(2018) Why do authors publish in predatory journals? Learned Publishing 31(2): 141–147.

39.

Lewis

Wulster-Radcliffe

(2014) Predators and impersonators: A new breed of journals. Animal Frontiers 4(1): 46–47.

40.

Lukić

Blešić

Basarin

, et al. (2014) Predatory and fake scientific journals/publishers: A global outbreak with rising trend: A review. Geographica Pannonica 18(3): 69–81.

41.

McCrostie

(2017) Warning: Conmen and shameless scholars operate in this area. Times Higher Education, 12 January. Available at: https://www.timeshighereducation.com/comment/warning-conmen-and-shameless-scholars-operate-area (accessed 12 January 2018).

42.

Macháček

Srholec

(2017) Predatory journals in SCOPUS. Working Paper. Available at: https://idea-en.cerge-ei.cz/files/IDEA_Study_2_2017_Predatory_journals_in_Scopus/mobile/index.html (accessed 10 May 2018).

43.

Manca

Cugusi

Dvir

, et al. (2017a) PubMed should raise the bar for journal inclusion. The Lancet 390(10096): 734–735.

44.

Manca

Martinez

Cugusi

, et al. (2017b) The surge of predatory open-access in neurosciences and neurology. Neuroscience 353: 166–173.

45.

Masten

Ashcraft

(2016) The dark side of dissemination: Traditional and open access versus predatory journals. Nursing Education Perspectives 37(5): 275–277.

46.

Mehrpour

Khajavi

(2014) How to spot fake open access journals. Learned Publishing 27(4): 269–274.

47.

Moher

(2016) Stop predatory publishers now: Act collaboratively. Annals of Internal Medicine 164(9): 616–617.

48.

Nelson

Huffman

(2015) Predatory journals in library databases: How much should we worry? The Serials Librarian 69(2): 169–192.

49.

Nicholl

Chinn

(2015) Caught in the trap: The allure of deceptive publishers. Nurse Author and Educator 25(4): 4.

50.

Nolfi

Lockhart

Myers

(2015) Predatory publishing: What you don’t know can hurt you. Nurse Educator 40(5): 217–219.

51.

Nwagwu

(2015) Counterpoints about predatory open access and knowledge publishing in Africa. Learned Publishing 28(2): 114–122.

52.

Nwagwu

Ojemeni

(2015) Penetration of Nigerian predatory biomedical open access journals 2007–2012: A bibliometric study. Learned Publishing 28(1): 23–34.

53.

Omobowale

Akanle

Adeniran

, et al. (2014) Peripheral scholarship and the context of foreign paid publishing in Nigeria. Current Sociology 62(5): 666–684.

54.

Owens

(2015) More trends in predatory publishing practices. Nurse Author and Editor 25(1): 3.

55.

Pickler

Noyes

Perry

, et al. (2015) Authors and readers beware the dark side of Open Access. Journal of Advanced Nursing 71(10): 2221–2223.

56.

Poynder

(2013) The OA interviews: Ashry Aly of Ashdin Publishing. Available at: http://poynder.blogspot.com/2013/01/the-oa-interviews-ashry-aly-of-ashdin.html (accessed 18 January 2013).

57.

Pyne

(2017) The rewards of predatory publications at a small business school. Journal of Scholarly Publishing 48(3): 137–160.

58.

Raghavan

Dahanukar

Knight

, et al. (2014) Predatory journals and Indian ichthyology. Current Science 107(5): 740–742.

59.

Roberts

(2016) Predatory journals: Illegitimate publishing and its threat to all readers and authors. Journal of Sexual Medicine 13(12): 1830–1833.

60.

Savina

Sterligov

(2016) Potentially predatory journals in Scopus: Descriptive statistics and country-level dynamics. In: Nordic workshop on bibliometrics and research policy 2016. Available at: http://www.communication.aau.dk/digitalAssets/252/252872_nwb-2016-proceedings.pdf (accessed 10 May 2018).

61.

Shaw

(2013) Hundreds of open access journals accept fake science paper. The Guardian, 4 October. Available at: http://www.theguardian.com/higher-education-network/2013/oct/04/open-access-journals-fake-paper (accessed 4 October 2013).

62.

Shen

Björk

B-C

(2015) ‘Predatory’ open access: A longitudinal study of article volumes and market characteristics. BMC Medicine 13(1): 230.

63.

Simón

(2016) Pitfalls of predatory journals: A personal account. Comprehensive Psychology 5: 1–5.

64.

Truth

(2012) Pay big to publish fast: Academic journal rackets. Journal for Critical Education Policy Studies (JCEPS) 10(2): 54–105.

65.

Van Nuland

Rogers

(2017) Academic nightmares: Predatory publishing. Anatomical Sciences Education 10(4): 392–394.

66.

Vinny

Vishnu

Lal

(2016) Trends in scientific publishing: Dark clouds loom large. Journal of the Neurological Sciences 363(Supplement C): 119–120.

67.

Ward

(2016) The rise of predatory publishing: How to avoid being scammed. Weed Science 64(4): 772–778.

68.

Xia

(2015) Predatory journals and their article publishing charges. Learned Publishing 28(1): 69–74.

69.

Xia

Harmon

Connolly

, et al. (2015) Who publishes in ‘predatory’ journals? Journal of the Association for Information Science and Technology 66(7): 1406–1417.