Optimizing laboratory quality with sigma metrics: Application of CLIA 2024 total allowable error guidelines

Introduction

A majority of tests critical in diagnosing and managing diseases are done in a biochemistry laboratory, making this an essential component of the healthcare system. Thus, the accuracy and reliability of tests performed in these laboratories are crucial for effective patient care.^1,2 Throughout the analytical phase of laboratory testing, result reliability is maintained by means of internal quality control (IQC) and external quality control (EQC). By keeping a real time check on the analytical system, IQC offers assurance that laboratory results are dependable. In EQC, control samples supplied by independent agencies at scheduled intervals, typically once per month, are analyzed and reported in clinical biochemistry. The results are interpreted using either a standard deviation index or a Z score which shows how many standard deviations (SD) a control result differs from the expected mean for that material. A drawback of using internal and external QCs is that these do not directly quantify the number of analytical defects. Sigma metrics, in contrast, provide an estimate of assay performance (estimated defect rate) by integrating imprecision, bias and total allowable error (TEa). ³

The sigma methodology is an essential tool for assessing clinical biochemistry laboratory performance. These metrics aid in identifying areas that need to be improved in order to raise the overall quality of testing procedures. By computing sigma metrics for individual biochemical parameters, laboratories can determine each parameter’s performance level. ⁴ This systematic evaluation helps in determining appropriate IQC levels to enhance the performance of target analytes. ³ By integrating internal and external QC performances and using the total allowable error from various sources, laboratories can redefine analytical quality goals and improve overall quality management practices. ⁵

To calculate the number of quality control measures required to achieve desired quality, we used sigma metrics and principles for the assessment of quality of laboratory testing procedures in the current study. This study was undertaken in light of the recent revision of TEa guidelines by Clinical Laboratory Improvement Amendments (CLIA), with the aim of assessing the performance of our laboratory’s biochemistry parameters against the updated goals. We have evaluated various biochemical parameters on a sigma scale and calculated their sigma metrics. The resulting data will be used to apply Westgard guidelines, determining the appropriate Westgard rules and IQC levels to enhance target analyte performance.

Material and methods

The Dimension® Xpand® Plus Integrated Chemistry System (Siemens Healthineers, Forchheim, Germany) was used to analyze the quality control data of various biochemical parameters. The parameters included glucose, creatinine, urea, uric acid, electrolytes (sodium, potassium, phosphate, calcium), bilirubin (total and direct), aspartate transaminase, alkaline phosphatase, alanine transaminase, total protein, albumin, triglycerides, total cholesterol, high-density lipoprotein, iron and total iron-binding capacity. Two levels of IQC material provided by Randox were run twice a day. Out of the tri-level IQC material, Level 2 and Level 3 controls were selected as they better represent clinically relevant decision ranges. Level 1 corresponds to lower concentration ranges that are rarely encountered while running routine patient samples. The use of two levels of IQC run twice daily is in accordance with National Accreditation Board for Testing and Calibration Laboratories (NABL) recommendations for internal quality control. The percentage of coefficient of variation (CV%) was derived from this data. Additionally, data from monthly participation in RIQAS (Randox International Quality Assessment Scheme) based on peer-group mean values were used to calculate percent bias. The data for this study was collected from December 2023 to May 2024. IQC and RIQAS data were systematically recorded and managed utilizing Microsoft Excel (Microsoft Corporation, Redmond, WA, USA). For the selected parameters, sigma metrics were calculated based on Total Allowable Error (TEa), percentage of coefficient of variation (CV%), and percent bias, according to the following formulas:

Sigma Metrics = TEa % − Bias % CV %

Total allowable error (TEa)

The extent to which an individual result may differ from the reference value while still being acceptable. Analytical quality requirements (TEa) for the evaluated parameters were primarily adopted from CLIA 1992 and CLIA 2024 guidelines. For certain analytes, biological variation-based TEa values (Ricos 2014 database, currently hosted by the European Federation of Clinical Chemistry and Laboratory Medicine) were used. These include sodium, potassium and calcium as CLIA TEa values are expressed in absolute units and cannot be directly used for percentage-based sigma calculations. For direct bilirubin and TIBC, biological variation-derived TEa values were preferred as these provide clinically meaningful performance goals.^6,7

Coefficient of variation %

The ratio of standard deviation (SD) to mean, expressed as a percentage. It measures the variability of an assay. The CV% was calculated as under:

CV % = SD * 100 Mean

Bias (%)

Systematic difference between result obtained by testing laboratory and expected result from peer-group mean. The bias percentage was calculated on the basis of RIQAS results, as under:

Bias % = ( Lab result – Peer − group mean ) * 100 Peer − group mean

Quality goal index (QGI)

The QGI ratio was used to explore the underlying cause of reduced sigma performance, and to evaluate how well precision and bias align with established quality standard requirements. This distinction aided in identifying whether the problem resulted from imprecision, or inaccuracy, or from both. Calculation of the QGI ratio was done as below:

QGI = Bias % 1.5 * CV %

When QGI <0.8, it typically indicates an issue primarily related to imprecision, a QGI >1.2 points towards an inaccuracy problem. On the other hand, a QGI ranging from 0.8 to 1.2 suggests a combination of imprecision with inaccuracy.

Ethical review exemption was obtained from the Institute Ethics Committee for Biomedical and Health Research as the study did not use data from patient samples.

Results

The study assessed 20 biochemical parameters over the period from December 2023 to May 2024, employing sigma metrics for evaluation. As per laboratory protocol, two levels of IQC material provided by Randox, run twice daily, were used for calculation of mean, standard deviation (SD) and coefficient of variation (CV%). Bias (%) was calculated using data from the RIQAS, based on peer-group mean values.

Table 1 summarizes the CV% and Bias% of 20 clinical chemistry parameters, based on IQC and EQAS performance data. Table 2 depicts the sigma metrics of the parameters for which CLIA TEa goals were available. It also includes a comparison of the sigma values calculated using CLIA ‘1992 and 2024 guidelines. Table 3 shows the parameters for which sigma was calculated based on the 2014 Ricos Biological variation guidelines. The QGI has been calculated for the parameters with sigma <3 to perform root cause analysis. It revealed that for most of the parameters lack of precision was the hindrance in achieving the desired quality goals (Table 4).

OPEN IN VIEWER

Table 1.

Calculated CV % and Bias % for various clinical chemistry parameters.

S. No.	Parameter	Dec-23			Jan-24			Feb-24			Mar-24			Apr-24			May-24
		CV % L2	CV % L3	Bias %	CV % L2	CV % L3	Bias %	CV % L2	CV % L3	Bias %	CV % L2	CV % L3	Bias %	CV % L2	CV % L3	Bias %	CV % L2	CV % L3	Bias %
1	Albumin	1.3	1.9	0.7	2.2	2	0.2	2.5	2.8	1.4	1.7	2.3	1.1	2.5	1.7	0.3	4.8	4.5	1.5
2	Alkaline phosphatase	2.2	2.5	0.7	7.3	5.5	3.8	5.6	5.2	2.7	4.1	3.7	0.2	4.5	2.9	4.5	3.9	3.2	2.2
3	ALT	9.4	3.9	4.5	7.3	6.1	5.5	5.9	5.7	0.2	5.1	4.8	9.3	8.4	6.1	3.6	6.1	4.2	2.6
4	AST	3	1.8	10.1	6.7	4.1	8.7	6.2	4.2	2.3	5.8	2.9	4.8	8.8	3.4	13.8	7.2	2.4	1.4
5	Bilirubin, direct	4.5	5.4	1.8	8.2	6.6	14.2	6.6	5.9	0.8	5	3.7	1.9	4.2	5.4	9	6.2	4.9	6.4
6	Bilirubin, total	5	4.3	10.9	5.9	3.3	16.9	3.5	3.4	16.1	3.9	3.5	7.1	3.6	4.8	8	4.9	4.6	7.9
7	Calcium	5	3.5	3.9	3.9	4	4.9	4	4.2	4.9	3.6	3.8	7.7	4.5	2.8	3.6	4.7	3.8	1.4
8	Cholesterol	2.6	2.3	6.8	2.7	2.4	2.2	3.1	2.7	0.4	1.8	1.7	0.8	2.8	1.6	2.3	2.5	2.6	2
9	Creatinine	3.4	4	3.9	3.8	2.8	6.6	4	2.4	0.3	3.5	2.1	0.5	5.8	3	5.8	4	2	1.6
10	Glucose	1.6	1	0.9	2.9	3	1.1	3.5	2.5	0.2	3.9	3.8	2.8	2.3	2.2	0.4	2.8	2	2.2
11	HDL-cholesterol	2.5	2.5	3.4	4.7	2.6	2.1	5	2.7	1.4	2.1	2.6	0.4	2.3	2.1	3.1	4.5	4.8	4.5
12	Iron	2.8	1.9	7.9	3	2.9	4.5	2.8	2.6	3.5	2.6	2.2	2.7	4.9	2.9	4.3	5	3.1	0.8
13	Phosphate	0.9	1.2	3.2	3.9	2	3.8	3.4	3	4.6	2	2.6	5	2	1.9	4.7	3.3	2.4	3
14	Potassium	0.6	0.7	4.6	1.3	1.8	2.6	1.6	2.9	1	1.8	1.9	1.2	2.2	1.9	3.8	3.7	5.7	4.1
15	Protein, total	1.4	2.9	3.5	2.2	3.2	0.2	3.5	4.4	5.1	1.4	2	3.9	2.4	2	0.1	2.9	3.7	0.5
16	Sodium	0.7	0.9	3.5	1.3	1.9	1.9	1.4	1.9	0.5	1.5	1.2	1	1.7	1.2	0.6	1.7	2	4.1
17	TIBC	4.8	3.3	0.6	3.6	4.4	10.3	4.1	4.3	6.1	7.3	3.6	5.3	6.9	3.8	6	11.4	5.4	0.5
18	Triglycerides	1.6	1.8	3	2.4	3.4	5.2	2.7	3.2	1.9	3.9	3.9	9.3	5.6	2.6	6.4	3.1	2.4	0.8
19	Urea	2.1	2.3	4.5	6.4	3.5	2.5	5.7	3.2	5.1	6.5	7.1	3.2	6.2	2.7	7.1	4.9	2.8	2.4
20	Uric acid	2.3	2.9	4.1	3	2.5	1.8	3.1	3.7	0.8	2.2	2.2	1.6	2.4	1.6	5.6	2.9	3	1.1

OPEN IN VIEWER

Table 2.

Comparison of sigma values for clinical chemistry parameters calculated using CLIA 1992 and CLIA 2024 guidelines.

S. No.	Parameter	TEa CLIA’92	TEa CLIA’24	Sigma L2		Sigma L3
				With TEa CLIA’92	With TEa CLIA’24	With TEa CLIA’92	With TEa CLIA’24
1	Albumin	10	8	4.0	3.1	3.8	3.0
2	Alkaline phosphatase	30	20	6.4	4.1	7.5	4.8
3	ALT	20	15	2.2	1.5	3.1	2.0
4	AST	20	15	2.4	1.5	4.8	3.1
5	Bilirubin total	20	20	1.8	1.8	2.0	2.0
6	Cholesterol	10	10	2.8	2.8	3.3	3.3
7	Creatinine	15	10	2.9	1.6	4.4	2.4
8	Glucose	10	8	3.2	2.4	3.9	3.0
9	HDL-cholesterol	30	20	8.3	5.3	9.9	6.3
10	Iron	20	15	4.7	3.2	6.2	4.2
11	Phosphate inorganic	10	10	2.5	2.5	2.8	2.8
12	Protein total	10	8	3.5	2.5	2.6	1.9
13	Triglycerides	15	15	3.4	3.4	3.6	3.6
14	Urea	9	9	0.9	0.9	1.3	1.3
15	Uric acid (urate)	17	10	5.5	2.8	5.6	2.8

CLIA: Clinical Laboratory Improvement Amendments, TEa: Total allowable error, ALT: Alanine transaminase, AST: Aspartate transaminase, HDL: High-density lipoprotein.

OPEN IN VIEWER

Table 3.

Sigma values calculated using Ricos 2014 biological variation-based guidelines.

S. No.	Parameter	TEa Ricos 2014	Sigma- L2	Sigma- L3
1.	Bilirubin, direct	44.5	6.8	7.4
2.	Calcium	10	1.2	1.4
3.	Potassium	8	3.0	2.4
4.	Sodium	5	2.0	1.8
5.	TIBC	20	2.5	3.6

TEa: Total allowable error, TIBC- Total iron-binding capacity.

OPEN IN VIEWER

Table 4.

Quality goal index (QGI) ratio of analytes with low-sigma performance due to accuracy and precision issues.

Analyte	QC level	Average CV %	Average bias %	Sigma	QGI	Problem
ALT	Level 2	7.0	4.3	1.5	0.4	Imprecision
	Level 3	5.1		2.0	0.6	Imprecision
Bilirubin total	Level 2	5.8	11.2	1.8	1.7	Inaccuracy
	Level 3	5.3		2.0	1.9	Inaccuracy
Calcium	Level 2	4.3	4.4	1.2	0.7	Imprecision
	Level 3	3.7		1.4	0.8	Imprecision
Creatinine	Level 2	4.1	3.1	1.6	0.5	Imprecision
	Level 3	2.4		2.4	0.9	Imprecision
Phosphate inorganic	Level 2	2.6	4.1	2.5	1.0	Imprecision & inaccuracy
	Level 3	2.2		2.8	1.2	Imprecision & inaccuracy
Protein total	Level 2	2.3	2.2	2.5	0.6	Imprecision
	Level 3	3.0		1.9	0.5	Imprecision
Sodium	Level 2	1.4	1.9	2.0	0.9	Imprecision
	Level 3	1.5		1.8	0.8	Imprecision
Urea	Level 2	5.3	4.1	0.9	0.5	Imprecision
	Level 3	3.6		1.3	0.8	Imprecision
Uric acid (urate)	Level 2	2.7	2.5	2.8	0.6	Imprecision
	Level 3	2.7		2.8	0.6	Imprecision

ALT: Alanine transaminase, CV: Coefficient of variation.

Table 5 summarizes QC strategy recommendations based on sigma performance of various parameters: 6-sigma processes require only 1_3s rules with 2 controls per run, 5-sigma processes need multi-rule QC (1_3s/2_2s/R_4s) with 2 controls, while lower-sigma systems need stricter rules (including 4_1s, 8_x) and more control measurements (2 controls in each of the 2 runs).

OPEN IN VIEWER

Table 5.

Recommendations for various parameters according to Westgard sigma rules.

Sigma metrics	Chemistry level 2	Chemistry level 3	IQC procedure recommended (Westgard sigma rules)
<4	Albumin, ALT, AST, bilirubin total, calcium, cholesterol, creatinine, glucose, iron, phosphate inorganic, potassium, protein total, sodium, TIBC, triglycerides, urea, uric acid (urate)	Albumin, ALT, AST, bilirubin total, calcium, cholesterol, creatinine, glucose, phosphate inorganic, potassium, protein total, sodium, TIBC, triglycerides, urea, uric acid (urate)	13s/22s/R4s/41s/8x rule; four control measurements in each run or 2 control measurements in each of the 2 runs
4	Alkaline phosphatase	Alkaline phosphatase, iron	13s/22s/R4s/41s rule; four control measurements in each run or 2 control measurements in each of the 2 runs
5	HDL-Cholesterol	--	13s/22s/R4s rule; two control measurements in each run
6	Bilirubin direct	Bilirubin direct, HDL-Cholesterol	13s rule; two control measurements in each run; one on each level of control

Discussion

The laboratory achieved excellent performance (≥6 sigma) for two test parameters—direct bilirubin and HDL-cholesterol. Albumin, alkaline phosphatase, aspartate transaminase, cholesterol, glucose, iron, potassium, total iron-binding capacity and triglycerides achieved minimum sigma performance standards (>3) at one level. However, alanine transaminase, total bilirubin, calcium, creatinine, inorganic phosphate, total protein, sodium, urea, uric acid (urate) did not meet minimum sigma performance standards for both level 2 and level 3 controls (Tables 2 and 3). As is clear from the comparison, CLIA 2024 guidelines have tightened the allowable error thresholds leading to a decline in sigma performance of the parameters.

The root cause analysis revealed that lack of precision was the primary issue for majority of the parameters. However, for total bilirubin across all assay range, QGI was >1.2 indicating that inaccuracy could be the underlying problem for this test. The assay of phosphate showed lack of accuracy as well as precision suggesting a more complex issue requiring attention to both measurement variability and calibration errors (Table 4). Sigma metrics play a crucial role in clinical biochemistry laboratories’ performance evaluation and quality control procedures. These metrics provide a quantitative assessment of analytical quality as well as a standardized scale for evaluating laboratory activities. ⁸ Clinical biochemistry labs can use a sigma scale of 1 to 6 to determine the acceptability of their procedures. A performance level of 3 sigma is usually considered satisfactory. ⁹ This methodology facilitates the identification of errors and the development of logical quality control designs based on sigma values. ¹⁰ According to Westgard sigma rules, ¹¹ for processes operating at 6-sigma quality, the use of Levey-Jennings QC chart with control limits of mean ±3 standard deviations (SD), and analyzing two control samples every run, should effectively detect medically important errors. At the 5-sigma level, a straightforward multi-rule QC approach (e.g., 1_3s/2_2s/R_4s rules) with two control levels measured once each—should suffice. However, for lower-quality assays, more rigorous QC, such as including additional rules (4_1s, 8_x) and increasing to total four control measurements is recommended (Table 5).

Researchers have carried out various studies on the evaluation of sigma metrics in clinical biochemistry laboratories. This emphasizes the role that these metrics play in enhancing analytical performance of various laboratory parameters. These studies have demonstrated the importance of continuous assessment of performance of all laboratory parameters, and creating specialized quality control plans for each analyte. ¹² Geto et al used the COBAS 6000 analyzer to evaluate the sigma metrics for 21 clinical chemistry parameters. The laboratory demonstrated world-class performance (≥6 sigma) for amylase, triglycerides, magnesium, ALT, HDL, total bilirubin, and AST at both QC levels. Creatinine, urea, and chloride, however, did not achieve the minimum acceptable sigma metrics, indicating suboptimal analytical performance. An intermediate level of assay performance (3–6 sigma), was obtained for bilirubin (direct), alkaline phosphatase (ALP), total protein, phosphate, potassium, albumin, and glucose across both quality control levels. Hence, it is concluded that stringent IQC is unnecessary for analytes with sigma ≥6, while continuous QC monitoring is critical for renal function tests and process improvements are required for parameters with poor sigma performance. ¹³

In a similar retrospective observational study, sigma metrics for 20 analytes were calculated using internal and external quality control data. World-class performance (>6 sigma) was observed for total protein, glucose, urea, triglycerides, high-density lipoprotein (HDL) and low-density lipoprotein (LDL) across both control levels. Conversely, chloride, sodium, inorganic phosphate, creatinine, calcium and potassium yielded sigma metrics below 4 in at least one QC level. ¹⁴ Discrepancies in sigma metrics across various studies may be attributed to several factors. These include differences in analytical methodologies, equipment used, IQC materials, environmental and personnel factors and variations in bias calculations resulting from diverse external quality control testing organizations.^3,15

The newly introduced CLIA 2024 TEa guidelines have significantly tightened quality control standards in clinical laboratories. Hence, the laboratories need to review and update their QC protocols to meet the desired goals. As observed in this study, many parameters failed to meet the bare minimum quality goal, that is, sigma level of 3. Thus, the testing process should be consistently monitored to reduce the margin for error in diagnostic testing. To decrease imprecision of biochemical parameters showing low-sigma values, a comprehensive training program needs to be implemented for laboratory personnel. The training sessions should focus on reagent preparation, control material reconstitution, equipment maintenance, and proper handling and storage of reagents. The staff must be made aware of the impact of these daily practices on the quality of lab. results. ¹⁴ Sigma metrics give an objective evaluation of lab. performance, thus the lab. staff can be motivated to focus on improvement in sigma values of various parameters. This emphasis on stricter QC practices will increase the reliability of test outcomes, ultimately improving patient care and outcomes.

This study has certain limitations. Bias was derived from peer-group mean values obtained from the external quality assurance program rather than reference measurement procedures which may affect the estimation of accuracy. Additionally, the study was conducted on a single analyzer platform [Dimension® Xpand® Plus Integrated Chemistry System (Siemens Healthineers, Forchheim, Germany)] at a single-center and this may limit generalizability. Further studies addressing these limitations may provide more robust and generalizable assessment of analytical performance.

Conclusion

Six Sigma is a proven tool for evaluating laboratory test performance. Implementing optimal QC procedures reduces redundant QC runs and associated costs for high-sigma analytes. Focussed QC strategies for low-sigma parameters will improve their performance. The sigma calculations vary significantly depending on the source of TEa. The CLIA 1992 TEa guidelines were broad and were based on the technology available at that time. These focused on minimum performance standards for clinical laboratories, with the objective being the prevention of significant errors in patient care. In contrast, the 2024 TEa guidelines are more stringent, reflecting advancements in laboratory technology and a greater emphasis on precision. Consequently, laboratories should reassess the performance of various chemistry parameters and redefine the QC protocols.