Validation of the Kidney Failure Risk Equation in Kidney Transplant Recipients

Abstract

French

Background:

Predicting allograft failure in kidney transplant recipients can help plan renal replacement therapy and guide patient-provider communication. The kidney failure risk equation (KFRE) accurately predicts the need for dialysis in patients with chronic kidney disease (CKD), but has not been validated in kidney transplant recipients.

Objective:

We sought to validate the 4-variable KFRE (age, sex, estimated glomerular filtration rate [eGFR], and urine albumin-to-creatinine ratio [ACR]) for prediction of 2- and 5-year death-censored allograft failure.

Design:

Retrospective cohort study.

Setting:

Four independent North American Cohorts from Ontario, Canada; Alberta, Canada; Manitoba, Canada; and Wisconsin, United States, between January 1999 and December 2017.

Patients:

Adult kidney transplant patients at 1-year posttransplantation.

Measurements:

Kidney failure risk as measured by the KFRE (eGFR, urine ACR, age, and sex).

Methods:

We included all adult patients who had at least 1 serum creatinine and at least 1 urine ACR measurement approximately 1 year following kidney transplantation. The performance of the KFRE was evaluated using the area under the receiver operating characteristic curve (C-statistic). C-statistics from the 4 cohorts were meta-analyzed using random-effects models.

Results:

A total of 3659 patients were included. Pooled C-statistics were good in the entire population, at 0.81 (95% confidence interval: 0.72-0.91) for the 2-year KFRE and 0.73 (0.67-0.80) for the 5-year KFRE. Discrimination improved among patients with poorer kidney function (eGFR < 45 mL/min/1.73 m²), with a C-statistic of 0.88 (0.78-0.98) for the 2-year KFRE and 0.83 (0.74-0.91) for the 5-year KFRE.

Limitations:

The KFRE does not predict episodes of acute rejection and there was heterogeneity between cohorts.

Conclusions:

The KFRE accurately predicts kidney failure in kidney transplant recipients at 1-year posttransplantation. Further validation in larger cohorts with longer follow-up times can strengthen the case for clinical implementation.

Keywords

transplant kidney failure risk equation KFRE allograft prediction

Introduction

Patients with chronic kidney disease (CKD) are at risk of progression to kidney failure requiring renal replacement therapy.¹ Kidney transplantation is the ideal form of renal replacement treatment for kidney failure as it offers a survival advantage, improvement in quality of life, and cost-utility when compared with treatment with dialysis.^2-4 However, kidney transplants can fail and patients with failing transplanted kidneys (allografts) need education and preparation for dialysis and/or retransplantation.

In 2011, the kidney failure risk equation (KFRE) was developed by Tangri et al and validated as a highly accurate model for predicting kidney failure in patients with CKD. It has demonstrated to be more accurate in predicting kidney failure than estimated glomerular filtration rate (eGFR) or albuminuria alone, and outperforms models that incorporate routinely collected clinical risk factors. Since 2011, the KFRE has been validated in cohorts from more than 30 countries across 4 continents and was demonstrated to be accurate in predicting kidney failure in these diverse CKD populations.^5,6 In some jurisdictions, the KFRE is used to guide dialysis and transplant planning in nontransplanted patients with CKD.⁷

However, no studies to date have evaluated the diagnostic accuracy of the KFRE in patients who have received a kidney transplant. CKD in transplant recipients may differ from CKD in nontransplant kidneys due to the presence of immune factors that can lead to rejection, and routine use of immunosuppressive medications such as calcineurin inhibitors which can lead to chronic allograft nephropathy.⁸ If the KFRE accurately predicts kidney failure in transplant recipients, it could be used to guide location and intensity of monitoring and follow-up as well as preparation for kidney replacement therapy for patients at high risk of allograft failure. The purpose of this study is to evaluate the accuracy of the 4-variable KFRE in kidney transplant recipients with a functioning allograft at 1-year posttransplant from 4 independent clinical cohorts.

Materials and Methods

Study Cohorts

We combined data collected from 4 large, separate cohorts. The first cohort was extracted from linked health care databases from the Alberta Kidney Disease Network (AKDN) for kidney transplant recipients from Alberta, Canada, between May 2002 and March 2015. The second cohort was extracted from the Transplant Manitoba Database, which includes all patients who have received a kidney transplant in Manitoba, Canada, between January 1999 and December 2017. The third cohort was extracted from electronic medical databases from St. Michael’s Hospital, an academic and tertiary care center located in inner-city downtown Toronto, Canada, between July 2004 and June 2014. The last cohort was the Wisconsin Allograft Recipient Database (WisARD), where kidney transplant recipients were extracted between January 2004 and June 2013.

We included all adult patients (age 18+) who had at least 1 serum creatinine measurement and at least 1 urine albumin-to-creatinine ratio (ACR) measurement approximately 1 year following kidney transplantation. Patients who died or had allograft failure in the first year posttransplant were excluded.

Variables and Outcomes

The risk of kidney failure was predicted for each patient at 2 years and 5 years following the serum creatinine test performed closest to 1-year posttransplant using the 4-variable KFRE. The 4-variable KFRE consists of eGFR, age, sex, and urine ACR (equation provided in supplemental material Item S1).^5,6 We estimated eGFR using the Chronic Kidney Disease Epidemiology Collaboration (CKD-EPI) equation.⁹ Urine protein-to-creatinine ratios (PCRs) were used to approximate some urine ACRs in the Manitoba cohort (supplemental material Item S2).⁶

Variables were taken closest to the 1-year posttransplant date. In addition to variables in the KFRE, we collected other patient clinical characteristics required for more complex KFREs when available from each of the 4 cohorts. Calcium, hemoglobin, bicarbonate, and phosphate were additionally collected in the Toronto and Alberta cohorts.

The primary outcome was death-censored allograft failure, defined as starting dialysis or undergoing retransplantation. The calculated kidney failure risks from the KFRE were compared to the actual patient kidney failure outcomes collected from each cohort.

Statistical Analysis

We evaluated the performance of the KFRE for predicting allograft failure using patient data 1 year after kidney transplantation with discrimination and calibration statistics.

Discrimination

We calculated discrimination by generating an area under the receiver operating characteristic curve (AUROC) for the risk scored evaluated by the KFRE modeled as a continuous variable using Harrell’s overall C-statistic. The C-statistic presents the proportion of times the KFRE correctly discriminates a pair of high risk and low risk individuals. A C-statistic of 0.70 or higher indicates good discrimination, while a C-statistic of 0.50 means the model predicts no better than chance.^10,11

Calibration

The difference between observed and predicted risk over 2 and 5 years was examined by plotting the probabilities of kidney failure at each risk decile in our validation cohorts using line graphs.¹²

Subgroup Analysis

As the KFRE was originally developed in patients with CKD Stages G3-G5,⁵ and GFR estimating equations may underestimate measured GFR in kidney transplant recipients,¹³ we performed a subgroup analysis examining discrimination and calibration in patients with poorer kidney function (Stages G3b-5, eGFR < 45 mL/min/1.73 m²).

Meta-analysis

C-statistics from each of the 4 cohorts were subsequently meta-analyzed using random-effects models using the DerSimonian and Laird method with generic inverse variances.¹⁴ Heterogeneity between studies was calculated using the I² statistic.¹⁵

All findings were presented at a level of significance of α = 0.05. Statistical analysis was performed using SAS version 9.4 (Cary, NC). Forest plots for the meta-analysis were generated using RevMan version 5.3 (Copenhagen, Denmark).¹⁶

Results

A total of 3659 patients were included across the 4 independent cohorts. The cohorts were very similar in the average age of the patients, percentages of males and females, and the level of kidney function. The Wisconsin cohort had a higher absolute rate of kidney failure with 2.3 per 100 person-years compared with Toronto at 1.5 per 100 person-years, Alberta at 1.2 per 100 person-years, and Manitoba at 1.1 per 100 person-years. Demographics and available clinical characteristics of the patients in the 4 cohorts are shown in Table 1.

Table 1.

Characteristics of Patients Included by Cohort at 1 Year Post Kidney Transplantation.

	TorontoN = 993	WisconsinN = 1263	AlbertaN = 940	ManitobaN = 463
Age	52.7 ± 13.0	52.8 ± 13.2	50.7 ± 14.3	47.8 ± 32.2
Female	36.5%	38.9%	34.0%	40.3%
Living donor	39.5%	38.7%	NR	49.7%
Systolic BP	129.4 ± 16.1	NR	NR	NR
Diastolic BP	79.3 ± 9.5	NR	NR	NR
eGFR	54.8 ± 17.9	56.4 ± 19.0	60.3 ± 19.4	63.1 ± 20.4
eGFR < 45 mL/min/1.73 m²	30.3%	29.1%	21.0%	18.4%
Urine ACR (mg/mmol)	2.2 (1.0 - 6.3)	9.8 (6.4-16.7)	6.3 (3.8-11.7)	5.9 (4.0-10.7)
Albumin (g/L)	41.4 ± 4.0	NR	39.4 ± 3.6	NR
Calcium (mmol/L)	2.4 ± 0.2	NR	2.4 ± 0.1	NR
Hemoglobin (g/L)	131.8 ± 18.4	NR	134.2 ± 16.8	NR
Bicarbonate (mEq/L)	25.5 ± 3.1	NR	24.8 ± 2.5	NR
Phosphate (mmol/L)	1.0 ± 0.2	NR	1.0 ± 0.2	NR
Death censored graft failure 5 years	5.2%	9.2%	3.8%	4.1%

Note. Continuous variables are presented as mean ± standard deviation for normally distributed variables and median (interquartile range) for urine ACR as it was not normally distributed. Categorical variables are presented as percentages. BP = blood pressure; NR = not reported; eGFR = estimated glomerular filtration rate; ACR: albumin-to-creatinine ratio.

Alberta Cohort

In the Alberta cohort, a total of 940 recipients were deemed eligible for the study. The mean eGFR was 60.3 mL/min/1.73 m². Of these patients, 36 developed kidney failure within 5 years following the 1-year posttransplant date, a total of 53 died before kidney failure and were censored for the study, and 851 patients did not develop kidney failure and did not die.

Manitoba Cohort

In the Manitoba cohort, a total of 463 recipients were deemed eligible for the study. The mean eGFR was 63.1 mL/min/1.73 m². Of these patients, 19 developed kidney failure within 5 years following the 1-year posttransplant date, a total of 30 died before kidney failure and were censored for the study, and 414 patients did not develop kidney failure and did not die.

Toronto Cohort

In the Toronto cohort, a total of 993 recipients were deemed eligible for the study. The mean eGFR was 54.8 mL/min/1.73 m². Of these patients, 52 developed kidney failure within 5 years following the 1-year post-transplant date, a total of 45 died before kidney failure and were censored for the study, and 896 patients did not develop kidney failure and did not die.

Wisconsin Cohort

In the Wisconsin cohort, a total of 1263 recipients were deemed eligible for the study. The mean eGFR was 56.4 mL/min/1.73 m². Of these patients, 116 developed kidney failure within 5 years following the 1-year posttransplant date, a total of 119 died before kidney failure and were censored for the study, and 1028 patients did not develop kidney failure and did not die.

Model Performance

The KFREs discriminated well in patients in the Alberta cohort with an overall C-statistic of 0.71 (0.55-0.87) at 2 years and 0.69 (0.58-0.80) at 5 years. In the a priori defined subgroup of patients with an eGFR < 45 mL/min/1.73 m² at 1 year (n = 197), the C-statistic was excellent at 0.97 (0.94-1.00) at 2 years and 0.93 (0.88-0.98) at 5 years. Calibration was good in the overall cohort and improved in those with more advanced CKD.

The KFREs in the Manitoba cohort had C-statistics of 0.93 (0.856-1.00) at 2 years and 0.61 (0.43-0.79) at 5 years. In patients with an eGFR < 45mL/min/1.73 m² (n = 85), discrimination was similar with C-statistics of 0.63 (0.28-0.98) at 2 years and 0.74 (0.60-0.88) at 5 years. Calibration in the Manitoba cohort was suboptimal, likely attributable to the small number of events.

The KFREs discriminated well in patients in the Toronto cohort with an overall C-statistic of 0.74 (0.64-0.84) at 2 years and 0.73 (0.65-0.81) at 5 years. In patients with an eGFR < 45 mL/min/1.73 m² at 1 year (n = 301), the C-statistic was also good, with a value of 0.79 (0.64-0.94) at 2 years and 0.77 (0.67-0.87) at 5 years. Calibration was good in the overall cohort and improved in those with more advanced CKD.

The KFREs also discriminated well in the Wisconsin cohort with C-statistics of 0.82 (0.75-0.89) and 0.79 (0.74-0.84) at 2 and 5 years, respectively. This improved in the subgroup of patients with an eGFR < 45 mL/min/1.73 m² (n = 368) to 0.87 (0.81-0.93) at 2 years and 0.82 (0.77-0.87) at 5 years. There was slight underprediction in the entire cohort, but this improved in the subgroup with eGFR < 45 mL/min/1.73 m² at 5 years.

Meta-analysis

The pooled C-statistic for the KFRE at 2 years was 0.81 (0.72-0.91) and at 5 years 0.73 (0.67-0.80) in the entire study population (n = 3659). These improved in the subgroup of patients with baseline eGFR < 45 mL/min/1.73 m² (n = 951), with a pooled C-statistic of 0.88 (0.78-0.98) for the 2-year KFRE and a pooled C-statistic of 0.83 (0.74-0.91) for the 5-year KFRE. Significant clinical heterogeneity was present for all the meta-analyses except for the 5-year KFRE in the entire study population.

Findings are summarized in Figures 1 to 3.

Figure 1.

Results of discrimination analyses and meta-analyses.

Figure 2.

Calibration plots.

Figure 3.

Calibration plots in patients with eGFR < 45 mL/min/1.73 m².

Discussion

In this validation study involving 3659 kidney transplant recipients across 4 independent cohorts in Canada and the United States, we demonstrated that the 4-variable KFRE at 1 year posttransplant provides good discrimination for the outcomes of 2-year and 5-year risk of progression to kidney failure. This discrimination improved among patients who had an eGFR of less than 45 mL/min/1.73 m² (CKD Stages 3b-5) at 1 year posttransplant, suggesting that it could be used in this population for determining prognosis, communicating risk, and planning for renal replacement therapy.

Several prediction models for long-term allograft and patient survival in kidney transplant recipients have been developed using both clinical and registry data. In a large study of patients from the United States Renal Data System (USRDS), several models were developed that predicted allograft failure within 5 years using data from the time of transplantation and 1-year posttransplant.¹⁷ Their simplest model had modest discrimination (C-statistic 0.65) and required the input of clinical variables including race, history of hospitalization, as well as primary health insurance. As such, these models may not have been generalizable in universal health care settings or in countries with varying racial distributions. Similarly, other investigators conducted single-center studies and found models with better accuracy, but found that these models were unable to outperform eGFR alone in the development population.¹⁸

In a series of recent studies involving 3 independent cohorts from the United Kingdom, France, and Canada, Shabir et al studied patients at 12-month posttransplantation and developed models to predict 5-year allograft and overall survival, and found good discrimination (C-statistics 0.78-0.90) and calibration with models that included measures of albuminuria as well as eGFR.¹⁹ Their final models included ethnicity and previous acute rejection, and validated well in all external validation cohorts. The same investigators also studied the additional predictive ability of kidney biopsy findings, and donor specific alloantibodies on risk prediction, and found improvement with the inclusion of the biopsy findings, but not with the addition of antibody levels. Furthermore, a study published in 2019 that included functional, histological, and immunological factors derived an 8-variable equation that demonstrated excellent discrimination (C-statistic 0.81, 95% confidence interval 0.79-0.83) which was subsequently validated in cohorts from both the United States and Europe, and for different time horizons posttransplant.²⁰ Our findings complement the data from these investigations, as they show the excellent predictive ability of the KFRE for allograft survival, but highlight the potential for improvement in risk prediction with the addition of histological variables or a history of rejection. These variables can improve predictive accuracy, but require a biopsy or at minimum manual data entry, and may not be suitable for integration into automated reporting systems from laboratory reports to electronic medical records. As such, we demonstrated that the KFRE, which is sometimes routinely reported, can still provide reasonable utility in the clinical decision-making process in the absence of these additional predictors.

Our findings have important clinical, research, and policy implications for patients with CKD with kidney transplants. First, they highlight the potential utility of the KFRE as a tool for prognostication in this population, particularly for patients with more advanced decline in kidney function. If the KFRE is reported by laboratory information systems and electronic medical records for all patients with CKD, clinicians can be assured that the prognosis is accurate for allograft recipients, similar to eGFR. From a policy perspective, our work provides additional evidence to complement the work by Shabir et al and the iBox prediction system and we would recommend that our equations can be used in routine reporting and clinical practice, and the equations by Shabir et al or the iBox system can and should be used for additional prognostic accuracy if histological or rejection history data are available.^19,20

There are limitations to this study. First, it is important to note that the KFRE was not developed specifically for use in transplant patients, and does not include some alloimmune factors such as donor type and characteristics, expanded donor criteria, delayed graft function, human leukocyte antigen antibodies, histopathology parameters, or immunological parameters that have been evaluated in transplant-specific algorithms.¹⁹ As such, the intent of this analysis was to demonstrate that the KFRE, which is routinely reported in some electronic medical record (EMR) systems, may still provide valuable prognostic information despite lacking these particular parameters. The KFRE was initially developed and validated in patients with an eGFR < 60 mL/min/1.73 m² (CKD stages 3-5), and indeed in this study, it was most accurate in patients with an eGFR < 45 mL/min/1.73 m². Patients with an eGFR ≥ 60 mL/min/1.73 m² at 1-year posttransplant are likely at low risk of allograft failure in the next 5 years and should not be risk stratified using the KFRE. In addition, the KFRE predicts death-censored kidney failure and does not predict all-cause mortality or episodes of acute rejection. Although we found statistically significant heterogeneity between the cohorts in our meta-analyses, we felt this approach was appropriate as the 4 cohorts were all in a North American setting with similar clinical management guidelines and immunosuppressive agent use. There may have been differences in induction and maintenance agents over time in the individual cohorts, but we were unable to evaluate these effects due to the aggregate nature of our analyses. Finally, our study population included patients with a functioning allograft at 1-year posttransplantation. As such, survivor bias would limit the applicability of our findings to predictions after the first year of transplantation.

Our study is the first to examine the accuracy of the KFRE, a prediction model that uses readily available clinical data that are routinely collected as part of posttransplant care, in kidney transplant recipients. Additional strengths include the diversity of our 4 cohorts, which differed significantly in their ethnicity and access to care, as well as their outcome rates. The fact that the KFRE was accurate in all cohorts further supports its generalizability and provides evidence for its clinical use.

In summary, the 4-variable KFRE developed by Tangri et al^5,6 accurately predicted kidney failure progression for kidney transplant recipients in both contemporary Canadian and American cohorts. The KFRE demonstrated adequate discrimination and calibration in these diverse populations. The KFRE can be a useful tool to help guide the clinical decision making process and to help appropriate risk stratify patients post kidney transplant.

Supplemental Material

supplemental_content – Supplemental material for Validation of the Kidney Failure Risk Equation in Kidney Transplant Recipients

Supplemental material, supplemental_content for Validation of the Kidney Failure Risk Equation in Kidney Transplant Recipients by Navdeep Tangri, Thomas W. Ferguson, Chris Wiebe, Frederick Eng, Michelle Nash, Brad C. Astor, Ngan N. Lam, Feng Ye, Jung-Im Shin, Reid Whitlock and Darren A. Yuen in Canadian Journal of Kidney Health and Disease

Footnotes

Acknowledgements

This study is based in part on data provided by Alberta Health and Alberta Health Services. The interpretation and conclusions contained herein are those of the researchers and do not necessarily represent the views of the Government of Alberta or Alberta Health Services. Neither the Government of Alberta nor, Alberta Health or Alberta Health Services express any opinion in relation to this study.

Ethics Approval and Consent to Participate

All studies were approved by the respective research ethics boards at the Universities of Alberta and Calgary, the University of Manitoba, the St. Michael’s Hospital institutional review board for de-identified data, and the University of Wisconsin Health Services Institutional Review Board. Patient consent was not required as the study used restrospective data collected in several health registries.

Consent for Publication

All co-authors reviewed this final manuscript and consent to its publication.

Availability of Data and Materials

Data sources used to inform these analyses are confidential and held in four registries: the Manitoba Transplant Database (University of Manitoba), the Wisconsin Allograft Recipient Database (WisARD - University of Wisconsin), electronic medical records at St. Michael’s Hospital (University of Toronto), and from the Alberta Kidney Disease Network (University of Calgary and University of Alberta) and are only available with their respective approvals. Any data used to derive figures or obtain values in this manuscript is available by contacting the corresponding author (Navdeep Tangri, ntangri@sogh.mb.ca).

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iDs

Thomas W. Ferguson

Reid Whitlock

Supplemental Material

Supplemental material for this article is available online.

References

Levey

De Jong

Coresh

, et al. The definition, classification, and prognosis of chronic kidney disease: a KDIGO Controversies Conference report. Kidney Int. 2011;80(9):1000.

Wolfe

Ashby

Milford

, et al. Comparison of mortality in all patients on dialysis, patients on dialysis awaiting transplantation, and recipients of a first cadaveric transplant. N. Engl. J. Med. 1999;341:1725–1730.

Laupacis

Keown

Pus

, et al. A study of the quality of life and cost-utility of renal transplantation. Kidney Int. 1996;50(1):235-242.

Wyld

Morton

Hayen

Howard

Webster

AC.

A systematic review and meta-analysis of utility-based quality of life in chronic kidney disease treatments. PLoS Med. 2012;9(9): e1001307.

Tangri

Stevens

Griffith

, et al. A predictive model for progression of chronic kidney disease to kidney failure. JAMA. 2011;305(15):1553-1559.

Tangri

Grams

Levey

, et al. Multinational assessment of accuracy of equations for predicting risk of kidney failure a meta-analysis. JAMA. 2016;315:164-174.

Wojciechowski

Tangri

Rigatto

Komenda

Risk prediction in CKD: the rational alignment of health care resources in CKD 4/5 care. Adv Chronic Kidney Dis. 2016;23(4):227-230.

Issa

Kukla

Ibrahim

HN.

Calcineurin inhibitor nephrotoxicity: a review and perspective of the evidence. Am J Nephrol. 2013;37(6):602-612.

Levey

Stevens

Schmid

, et al. A new equation to estimate glomerular filtration rate. Ann Intern Med. 2009;150:604-612.

10.

Pencina

D’Agostino

RB.

Overall C as a measure of discrimination in survival analysis: model specific population value and confidence interval estimation. Stat Med. 2004;23:2109–2123.

11.

Pencina

D’Agostino

Vasan

RS.

Evaluating the added predictive ability of a new marker: from area under the ROC curve to reclassification and beyond. Stat Med. 2008;27:157–172.

12.

D’Agostino

Nam

BH.

Evaluation of the performance of survival analysis models: discrimination and calibration measures. Handb Stat. 2003;23:1-25.

13.

White

Akbari

Doucette

Fergusson

Knoll

GA.

Estimating glomerular filtration rate in kidney transplantation: is the new chronic kidney disease epidemiology collaboration equation any better?

Clin Chem. 2010;56:474–477.

14.

DerSimonian

Laird

Meta-analysis in clinical trials. Control Clin Trials. 1986;7:177-188.

15.

Higgins Thompson

Deeks

Altman

DG.

Measuring inconsistency in meta-analyses. BMJ. 2003;327(7414):557-560.

16.

The Nordic Cochrane Centre: Review Manager (RevMan). London, England: Cochrane Collaboration; 2014.

17.

Kasiske

Israni

Snyder

Skeans

Peng

Weinhandl

. A simple tool to predict outcomes after kidney transplant. Am J Kidney Dis. 2010;56(5):947-960.

18.

Moore

Shabir

, et al. Development and evaluation of a composite risk score to predict kidney transplant failure. Am J Kidney Dis. 2011;57(5):744-751.

19.

Shabir

Halimi

Cherukuri

, et al. Predicting 5-year risk of kidney transplant failure: a prediction instrument using data available at 1 year posttransplantation. Am J Kidney Dis. 2014;63(4):643-651.

20.

Loupy

Aubert

Orandi

, et al. Prediction system for risk of allograft loss in patients receiving kidney transplants: international derivation and validation study. BMJ. 2019;366:l4923.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.11 MB