Why do we encounter studies with older versions of the Oswestry Disability Index? Awareness of researchers about the version updates and the lifespan of older versions

Abstract

BACKGROUND:

The Oswestry Disability Index (ODI) is a common outcome-measure used for assessment of spinal disorders with three officially updated versions. However, there are recently published articles that still use ODI-1.0. This suggests the existence of a block on information flow between developers and end-users.

OBJECTIVE:

To investigate the articles with missing citation to the latest ODI version, to calculate the life span of previous ODI version and to investigate the existence of a block on information flow.

METHODS:

PMC-PubMed citation dataset is analyzed by CNA and text processing methods.

RESULTS:

The most important finding of this study is that there is no block on information flow between developers and users in terms of visibility in citations and full text access restrictions. Three different sub-networks are identified between ODI-1.0 and ODI-2.0 articles. 14% (165) of articles have citation to ODI-1.0 and ODI-2.0 version-articles. The time limit that is required for the new ODI version to become dominant in the literature is between 4 and 9 years.

CONCLUSION:

There is a missing citation problem in the ODI literature which is not a result of a block on information flow between developers and end-users.

Keywords

Oswestry Disability Index citation network analysis ODI-1.0 ODI-2.0 ODI-2.1 ODI-2.1a version updates dissemination PMC PubMed physiotherapy

1 Introduction

The Oswestry Disability Index (ODI) is one of the most common condition-specific outcome measure used in the management of spinal disorders [1 –3]. ODI measures the intensity of low-back pain and its effect on nine different daily living activities (lifting, personal care, walking, sitting, sexual function, standing, social life, sleep quality, and traveling). Each item consists of six statements which describes disability from least amount (0) to severe (5). Patient performs a self-assessment by selecting a statement for each question which most closely resembles his/her situation.

The original version of ODI (version 1.0, ODI-1.0) was published in 1980 by Fairbank et al. [1]. The pain section of ODI-1.0 was modified according to the suggestions of Medical Research Council (MRC) [4, 5]. This modified version (ODI 2.0) was published in 2000 [2, 6]. Afterwards, Fairbank identified an error in the travel section of ODI-2.0 and he released the revised version (ODI-2.1) [6, 7]. The final adjustment was made in the opening statement of ODI-2.1 and then ODI-2.1a was released [7]. All stated version updates and modifications were made by the developers of ODI. The developers ignored or rejected all the other suggestions for modification of ODI [2].

The developers update ODI for better assessment of patients. The latest and approved ODI version is recommended in clinical practice. The improvements made by the developers will not influence the assessment results unless the updated version become available in clinical practice.

The updated ODI version can be implemented to clinical practice in a very short time in English speaking countries. This update may only affect ongoing studies and rehabilitation programs. It is recommended to use a metadata field for version information of ODI in electronic patient record systems [8]. If ODI version update causes a score comparison issue between pre-assessment and post-assessment, the use of this metadata field will be helpful.

The implementation of updated ODI version takes more time in non-English speaking countries due to cross-cultural adaptation procedures. Häkkinen et al. reported lower correlation for corrected pain intensity item due to the use of an older ODI version (ODI-1.0) in 2005 [9, 10]. They clearly suggested the use of ODI-2.0 for this issue. Cross cultural adaptation to Thai language is another study with ODI-1.0 after version updates in 2000 [11].

The information about the latest ODI version and cross-cultural adaptation studies has been provided by the web site of MAPI Trust after the transfer of ODI license to MAPI Trust [12]. This web site requires registration and some features requires subscription.

PubMed is one of the most common and non-commercial medical database since 1997 [13]. PubMed Central (PMC) is the open-access full-text collection of the National Library of Medicine (NLM, since 2000) [14]. PMC articles may be the unique source of scientific information for medical workers who do not have access to commercial databases. For this reason, bibliometric relations in PMC articles may also represent an important non-commercial clinical reasoning foundation [15].

Awareness of researchers about the latest ODI version may be one of the important reason why do we encounter studies with older ODI versions. The researchers who are aware the latest ODI version have higher probability to include version update articles in their reference lists. The articles are linked to each other through their reference lists. These links can form citation networks. Citation networks can be analyzed and visualized by scientometric network analysis methods [15]. The awareness of researchers about the latest ODI version and the life span of older ODI versions can be presented in a quantitative way by using scientometric network analysis.

The Entrez Programming Utilities (E-utilities) is the API for databases of National Center for Biotechnology Information (NCBI). E-Link functions of this API can be used to retrieve citation relation of a PMC article in PubMed (PMC-PubMed Citation). This dataset can be used to form citation networks. The PubMed summary dataset can be used to improve qualifiers of every article in the citation network [15].

The purpose of this study is to investigate the visibility of ODI version articles in PMC subset of PubMed, to investigate the awareness of researchers about the latest ODI version and to calculate the life span of older ODI versions.

2 Material and methods

The design of this study was a citation network analysis.

2.1 Version articles

Fairbank’s first article in 1980 was used to identify articles that cited ODI-1.0 [1]. MRC articles in 1985 and 1996, reviews of Fairbank and Roland in 2000 were used to identify articles that cited ODI-2.0 [2 , 4–6]. The review of Fairbank in 2014 was used to identify articles that cited ODI-2.1a [7].

The researchers who developed a new outcome measure sometimes cited ODI articles. These articles were not closely related to ODI version updates. The older ODI version articles and the article of Beaton et al. in 2000 were used to identify these unrelated articles [17]. Article of Davidson M. in 2008 was used to identify other ODI versions which are not approved by the developers [18].

2.2 Citation dataset

The PMC-PubMed citation records of version articles were retrieved from NCBI database by an MS EXCEL VBA macro in PMID-PMID format. E-Link function was placed inside a loop in this macro (Syntax 1) [16]. Uncited articles were excluded and final data was saved in edge file format. Syntax 1 Citation query https://eutils.ncbi.nlm.nih.gov/entrez/eutils/elink.fcgi?dbfrom=pubmed&linkname=pubmedpubmedcitedin&id=

The corresponding MEDLINE data for each version article was directly downloaded from PubMed in CSV file format. The additional MEDLINE data fields were; PMID, first author, publication year, journal name, article title and article type. This dataset was converted to node file format. Node and edge files were merged in Gephi for citation network analysis.

The citations of each version article was validated with PubMed web interface results. The citation and MEDLINE data were retrieved on 12 December 2018.

2.3 Citation network analysis

Node and network metrics of citation network were calculated with GEPHI 0.9.0. Force atlas-2 and Yifan Hu layouts were used to visualize the citation network.

The article of Beaton et al. in 2000 was not an ODI version article but this article was cited by articles of cross cultural adaptation studies of ODI. Also this article was highly cited by most of the cross cultural adaptation studies of other outcome measures [17]. The number of citations of this article was higher than ODI version articles. For network simplification this article and the articles that solely cited it (other outcome measure articles) were removed from the network.

The version articles (nodes) of same ODI version was not merged together. The version articles with a few number of citations (very small ego network) were investigated manually.

2.4 Word profile list

Corresponding MEDLINE data elements for author keywords (OT), MeSH terms (MH), affiliation of first author (AD) and abstract (AB) were downloaded from PubMed and text processing was performed.

2.5 Researchers awareness about the latest ODI version

The criteria for researchers’ awareness about the latest ODI version were as follows; 1) ODI version update article should be available before the publication date of any citing article. 2) The researchers should include version update article in their reference list even if they used older version in data collection. Failure to meet this criteria was considered as lack of awareness and the articles were classified as articles with missing references.

2.6 Life span

The life span of the older version article was calculated with the help of a citation ratio. This ratio was calculated by dividing citations of version update article to citations of older version article. The number of citations after the publication date of version update article was considered in calculations. When the ratio was equal or greater than 1, it was assumed that the older version completed its life span.

The life span of ODI-1.0 was calculated by two different methods. In the first method, the number of articles that cited one of ODI-2.0 version articles was divided by the number of articles that solely cited ODI-1.0 version article. In this method the effect of co-citing articles was not considered.

In the second method, the number of articles that cited one of ODI-2.0 version articles and co-citing articles in subnetworks (SN-1-3-4) were calculated. The sum of these articles was divided by the number of articles that solely cited ODI-1.0 version article. In this method effect of co-citing articles was considered.

3 Results

The number of citations retrieved by VBA macro for each version article is valid when compared with PubMed web interface.

Citation network of ODI version articles consist of 1,143 nodes (articles) and 1,347 edges (citation, Fig. 1). The Fairbank JC-2000 article has the highest degree centrality and eigenvector centrality (Table 1). MRC articles and Fairbank JC-2014 article had too small node metrics as compared with Fairbank JC-2000 article. The size and quality of final data was not sufficient to perform analysis between ODI-2.0 version articles and upper version articles.

Fig. 1

Citation network of ODI version articles. a) whole network, b) subnetworks, c) articles with missing references. Black color represents review and cross cultural adaptation studies. Grey color represents original articles. Node radius represent in-degree centrality.

Table 1

Node (article) metrics

Article	ODI version	In-degree	Out-degree	Degree	Page ranks	Eigenvector centrality
Fairbank, 2000	2.0	564	0	564	0.193	1.000
Fairbank, 1980	1.0	485	0	485	0.173	0.870
Roland, 2000	2.0	251	0	251	0.083	0.445
MRC, 1995	2.0	27	1	28	0.010	0.048
Fairbank, 2014	2.1a	8	0	8	0.003	0.014
	1.0
Davidson, 2008	2.0	8	0	8	0.003	0.014
	Other version
MRC, 1986	2.0	4	1	5	0.001	0.007

In-degree is the measure of citations. Out-degree is the measure of references. Degree is the sum of citations and references. Eigenvector centrality is the improved form of degree centrality that also measures the importance (citations) of neighbors. Page rank is a special form of eigenvector centrality that Google uses for web page rankings. Higher page rank represents the relative order of version articles in Google search results.

The entire citation network consisted of a single connected component and network metrics were as follows; average degree is 2.357 (ratio between citations and articles), graph density is 0.001 (ratio of citations to maximum possible number of citations), modularity is 0.522 (a measure of communities), average clustering coefficient is 0.001 (a measure of how network close to a network with maximum possible citations), average path length is 1.02 (a global measure of path between two articles).

The articles that cited (co-citation) at least two version articles forms a subnetwork (SN). Four major subnetworks (SN) were identified. The total number of articles in subnetworks was 165 (14%). The researchers who were aware of version updates, have both cited ODI-1.0 and at least one of ODI-2.0 articles. The SN-1 (94, 8%), SN-3 (22, 2%) and SN-4 (13, 1%) were formed by articles of researchers who are aware of version updates. SN-II (36, 3%) is formed by articles which cited both of ODI-2.0 articles published in 2000.

Single out-degree is a term for a single citation. One degree ego network is a term for the set of citing articles of a single article. The nodes with single out-degree in one degree ego-network of ODI-1.0 version article and SN-V subnetwork were the articles with missing references (Fig. 1). 325 of 333 articles with missing reference were published after 2000.

The ratio of articles of researchers who were aware of version updates increased linearly (Fig. 2c). However, the ratio did not exceed 30% of single ODI-1.0 citations. The number of articles that both cited OD-1.0 and one of ODI-2.0 version articles increased more than the articles in other subnetworks (Fig. 2d).

Fig. 2

The life span and citation network parameters of ODI version articles.

All the articles in SN-V both cited Beaton et al. in 2000 and ODI-1.0 version article. All the articles in SN-V were development studies of new outcome measures.

The life span of ODI-1.0 version article were about 9 years when the effect of co-citing articles was not considered in calculations (Fig. 2a, the ratio is 50/38 in 2006 and 132/134 in 2010). The life span of ODI-1.0 article was about 4 years when the effect of co-citing articles was considered in calculations (Fig. 2b, the ratio is 36/31 in 2005 and 50/54 in 2006).

The articles in subnetworks and the articles that solely cited ODI 2.0 version article had at least 91% free full text access ratio (Fig. 2f). Difference in free access rate was only % 1 between ODI-1.0 and ODI-2.0 version articles. The out-degree distribution followed power law distribution in our data set (Fig. 2e). English speaking countries showed 17% lower number of missing reference articles (Table 2).

Table 2

Country of origin of missing reference articles. Percentage represents the ratios over total number of missing reference articles

Country of origin	Counts	Total
USA	76 (23%)
UK	30 (9%)
Canada	15 (5%)	131 (40%)
Australia	9 (3%)
New Zealand	1 (0%)
Germany	21 (6%)
Norway	20 (6%)
China	17 (5%)
Korea	15 (5%)
Sweden	15 (5%)	185 (57%)
Switzerland	12 (4%)
Finland	11 (3%)
Italy	10 (3%)
Other	64 (20%)
Missing	-	9 (3%)

The term “Oswestry Disability Index” was identified in only % 2 of author keywords of missing reference articles (Table 3). MESH terms returned better results than author keywords. “Treatment outcome” (30%), “disability evaluation” (20%) and “pain measurement” (17%) were the most frequent MESH term that were identified in missing reference articles.

Table 3

Terms of missing reference articles. Percentage represents the ratios over total number of missing reference articles

ID	MESH term	Counts	Author keywords	Counts
1	Treatment Outcome	99 (30%)	Low back pain	35 (11%)
2	Disability Evaluation	65 (20%)	Chronic low back pain	9 (3%)
3	Pain Measurement	55 (17%)	Spine	9 (3%)
4	Surveys and Questionnaires	45 (14%)	OswestryDisability Index	8 (2%)
5	Prospective Studies	42 (13%)	Rehabilitation	7 (2%)
6	Follow-Up Studies	38 (12%)	Disability	5 (2%)
7	Retrospective Studies	26 (8%)	Lumbar disc herniation	5 (2%)
8	Time Factors	24 (7%)	Quality of life	5 (2%)
9	Lumbar Vertebrae/
	^*surgery	23 (7%)	Sciatica	5 (2%)
10	Quality of Life	22 (7%)	Spinal stenosis	4 (1%)
11	Cohort Studies	18 (6%)	Surgery	4 (1%)
12	Radiography	18 (6%)	Assessment	4 (1%)
13	Reproducibility of Results	18 (6%)	Back pain	4 (1%)
14	^*Disability Evaluation	16 (5%)	Chiropractic	4 (1%)
15	Severity of Illness Index	15 (5%)	Depression	4 (1%)
16	Patient Satisfaction	14 (4%)	Pain	4 (1%)
17	Prognosis	14 (4%)	Randomized controlled trial	4 (1%)
18	Chronic Disease	13 (4%)	Spondylolisthesis	4 (1%)
19	Cross-Sectional Studies	12 (4%)	Chronic pain	3 (1%)
20	^*Surveys and Questionnaires	11 (3%)	Degenerative disc disease	3 (1%)

Text processing of abstracts returned detailed information that suggest clear use of ODI as an outcome measure in the studies. The term “Oswestry Disability Index” is identified in 52% of missing reference articles and 42% of articles that solely cited ODI-2.0 version articles (Table 4). The term “Oswestry Low Back Pain Disability” which was used in title of ODI-1.0 version article have lower frequency in both ODI-1.0 and ODI-2.0 version articles.

Table 4

Abstract terms

Abstract term	ODI-1.0 articles	ODI-2.0 articles
Oswestry Disability
/ Index / Questionnaire / Score / Scale / Classification	169 (52%)	244 (42%)
Oswestry Low Back Pain Disability / Index / Questionnaire /Score	14 (4%)	11 (2%)
Oswestry Disability Index v2.0	1 (0%)	7 (1%)
Oswestry Disability Index version 2.1a	–	1 (0%)
Other	1 (0%)	5 (1%)
Total	185 (57%)	268 (46%)

4 Discussion

In this study visibility of version update articles of ODI was compared with visibility of older version articles of ODI in PMC subset of PubMed. Methods of citation network analysis were used in this comparison.

The parameters of citation network, the patterns in citation network graph and the number of freely available full-text articles do not suggest any block of information flow between developers of ODI and researchers. If the version update articles are removed from the citation network, the information about the version updates can still be provided by the articles left in the citation network.

There are articles with older version of ODI in the literature. However, the findings do not support a lack of researchers’ awareness about the ODI version update articles. The most important finding of this study is that there is no block on information flow between developers of ODI and researchers.

The life span of older versions of ODI was calculated by two different methods. The life span of ODI-1.0 version article is about 9 years when the effect of co-citing articles were not considered in calculations. The life span of ODI-1.0 article is about 4 years when the effect of co-citing articles were considered in calculations. It may be useful to use these two lifespan as the lower and upper time limits required for the new version to become dominant in the literature. This is the second important finding of this study.

The articles that cited MRC articles and Fairbank JC-2014 article do not form a citation network that can be compared with ODI-2.0 version articles. We could not succeed to determine the life span of ODI-2.0 version articles.

Web of Science and Scopus also support citation network analysis. Nonetheless, these commercial databases do not provide free database or full text to make a citation network analysis [13]. PubMed has been improving its open access policy with PMC database since 2000. The publication date of ODI-2.0 version articles coincided with the inception date of PMC. This situation may contribute the researchers’ awareness about the ODI version updates and formation of multiple information channels seen in citation network.

Badhiwala et al. reported 1,456 citations for the article by Fairbank et al. [2] in commercial citation databases. However, this article was cited by 564 times by PMC articles (Fig. 1) [19]. PubMed keeps citation record of only PMC articles. Commercial databases keep citation records of every article in their databases. Therefore, there is a difference between numbers of citations. However, force law distribution is clearly identified in citation network.

The force-law distribution is a characteristics of real-life networks and the citation networks are classified under real-life networks. The properties of real-life networks are scale free. For this reason, it is not expected a meaningful difference in network characteristics by the increase in number of citing articles [20].

Fairbank also identified spelling and typological errors in reference lists of citing articles [2]. In this study these type of errors were also controlled in the text processing stage. A suspicious situation was found in an article by Brodke et al. [21]. In the abstract of this article a tag was identified related to ODI-2.0. The tag was; “Oswestry Disability Index v2.0 (ODI)”. However, in the citation network the article did not cite any of ODI-2.0 articles. The full text of this article was investigated. In the reference list only ODI-1.0 article was listed with two other studies as in citation network (Table 4) [21 –23]. There is no spelling or typological error in reference of this article. This article is a missing reference article with “v2.0” tag.

English speaking countries showed lower number of missing reference articles as expected.

The full text analysis was only performed for suspicious articles. This may be a limitation of this study. A text mining on full text of missed reference articles may improve the findings of this study.

5 Conclusion

The ODI is a gold standard outcome measure for self-assessment of patients with low back pain. Developers of ODI made minor modifications when necessary. They suggest the latest version. However, we encounter studies with older ODI versions. There are limited number of studies that report possible issues related to the use of older ODI versions.

In this study, the reason for the existence of the articles with older ODI versions in the literature was investigated. The visibility of version update articles of ODI was compared with visibility of older version articles of ODI in PMC subset of PubMed by methods of citation network analysis. The findings do not suggest any block on information flow between developers and researchers that can be a reason for use of older ODI versions. The conclusion is that we encounter studies with older versions of ODI in the literature and this is not a result of block on information flow between developers and researchers.

5.1 Disclaimer

The study does not include any human subject or medical data. Ethical approval is not required for this study according to national regulations of medical ethics. There is no conflict of interest. There is no financial support.

During the review of this article a newer version of ODI (2.1b) was announced on the MAPI website. However, no article was found in PubMed with query key [“Oswestry Disability Index” AND 2.1b] on 20 January 2021.

Footnotes

Acknowledgment

The author would like to thank Dr. Donald Lipardo for his suggestions for revision of the manuscript. The author also would like to thank the editor and reviewers for giving the opportunity to publish an article with an uncommon method for medical journals.

The university where the author worked was closed after the failed coup attempt in 2016. He is currently unemployed. He endeavours to continue his studies in the library specified in the affiliation. However, the use of the library is not always possible. ().

References

Fairbank

, Couper

, Davies

, et al. The Oswestry low back painquestionnaire. Physiotherapy 1980;66:271–3.

Fairbank

, Pynsent

. The Oswestry Disability Index.Spine (Phila Pa 1976). 2000;25:2940–2952.

Irmak

, Baltaci

, Ergun

. Long term test-retest reliability of Oswestry Disability Index in male office workers. Work 2016;53:639–642.

Meade

, Browne

, Mellows

, et al. Comparison of chi ropractic and outpatient management of low back pain: a feasibility study. J Epidemiol Commun Health 1986;40:12–7.

Meade

, Dyer

, Browne

, et al. Randomized comparison of chiropractic and hospital outpatient management for low-back-pa results from extended follow-up. BMJ. 1995;311:349–51.

Roland

, Fairbank

. The Roland-Morris Disability Questionnaire and the Oswestry Disability Questionnaire. Spine (Phila Pa 1976). 2000;25:3115–3124.

Fairbank

. Why are there different versions of the Oswestry Disability Index? J Neurosurg Spine (Phila Pa 1976). 2014;20:83–6.

Irmak

(2008). Design of a database for Oswestry Disability Index (ODI) in multi languages. (Master dissertation). Turkish Council of Higher Education Thesis Center https://tez.yok.gov.tr/ (Access Number: 224210).

Häkkinen

, Kautiainen

, Järvenpää

, et al. Changes in the total Oswestry Index and its ten items in females and males pre- and post-surgery for lumbar disc herniation: a 1-year follow-up. Eur Spine J. 2007;16:347–52.

10.

Puolakka

, Ylinen

, Neva

, et al. Risk factors for back pain-related loss of working time after surgery for lumbar disc herniation: a 5-year follow-up study. Eur Spine J. 2008;17:386–392.

11.

Sanjaroensuttikul

. The Oswestry low back pain disability questionnaire (version 1 0) Thai version. J Med Assoc Thai. 2007;90:1417–22.

12.

Oswetry Disability Index [eProvide Web Site]. Avaliable at: https://eprovide.mapi-trust.org/instruments/oswestry-disability-index. Accessed September 13, 2018.

13.

Falagas

, Pitsouni

, Malietzis

, et al. Comparison of PubMed, Scopus, Web of Science, and Google Scholar: strengths and weaknesses. FASEB J 2008;22:338–42.

14.

PMC FAQs [PMC-NCBI Home Page]. Available at https://www.ncbi.nlm.nih.gov/pmc/about/faq/. Accessed September 1, 2018.

15.

Irmak

. The most cited and co-cited COVID-19 articles: Knowledge base for rehabilitation team members. Work 2020;66:479–489.10.3233/WOR-203193.

16.

Entrez Programming Utilities [Entrez Pro-gramming Utilities Help Site]. Avaliable at: https://www.ncbi.nlm.nih.gov/pmc/tools/cites-citedby. Accessed September 1, 2018.

17.

Beaton

, Bombardier

, Guillemin

, et al. Guidelines for the process of cross-cultural adaptation of self-report measures. Spine (Phila Pa 1976). 2000;25:3186–91.

18.

Davidson

. Rasch analysis of three versions of the Oswestry Disability Questionnaire. Man Ther 2008;13:222–31.

19.

Badhiwala

, Nassiri

, Witiw

, et al. Highly Cited Works in Spinal Disorders: The Top 100 Most Cited Papers Published in Spine Journals. Spine (Phila Pa 1976). 2018;15:1746–1755.

20.

Bornholdt , Stefan , Heinz Georg . Handbook of Graphs and Networks: From the Genome to the Internet. Wiley, 2006.

21.

Brodke

, Goz

, Voss

, et al. PROMIS PF CAT Outperforms the ODI and SF-36 Physical Function Domain in Spine Patients. Spine (Phila Pa 1976). 2017;42:921–929.

22.

Dawson

, Steele

, Hodges

, et al. Utility of the Oswestry Disability Index for studies of back pain related disability in nurses: evaluation of psychometric and measurement properties. International Journal of Nursing Studies 2010;47:604–607.

23.

Lochhead

, MacMillan

. Psychometric properties of the Oswestry disability index: Rasch analysis of responses in a work-disabled population. Work 2013;46:67–76.