Rapid clinical assessment of the sublingual microcirculation - Visual scoring using microVAS in comparison to standard semi-automated analysis

Abstract

RATIONALE:

Alterations in human microcirculation occur in many disease states leading to morbidity and mortality, however assessing the microcirculation is not standard clinical practice. Standard microcirculation analysis using semi-automated analysis is expensive, time consuming, and expertise dependent making it unfeasible. We proposed a novel visual scoring system (microVAS) for the analysis of microcirculation videos that can be performed at the patient bedside in real time.

OBJECTIVE:

Validate our microVAS score by training health professionals unfamiliar with the microcirculation field to use our microVAS score and compare their scores to the standard method of semi-automated analysis using AVA3 software.

METHODS:

Using a prospective double-blind study design, we recruited and trained 20 participants to use our microVAS score. Participants scored 40 videos (from 22 healthy and 18 septic patients) for MFI and PPV. The same 40 videos were analysed by an expert using the gold standard semi-automated method of analysis. The results of the participants and the expert were analysed by Pearson’s linear regression. Krippendorff’s alpha was used to assess inter-rater reliability of the participants.

RESULTS:

Overall correlation of MFI was r = 0.33 (95% CI 0.27–0.39), p < 0.05; overall correlation of PPV was r = –0.11 (95% CI –0.18 to –0.04), p < 0.05. The Krippendorff’s alpha for MFI was 0.56 (healthy videos: α= 0.34, sepsis videos: α= 0.31). For PPV Krippendorff’s alpha was 0.43 (healthy videos: α= 0.56, sepsis videos: α= 0.17).

CONCLUSIONS:

Overall for both MFI and PPV, there was a small correlation between our microVAS score and AVA 3 scores. Regarding inter-rater reliability both MFI and PPV showed fair agreement between raters. Going forward multiple improvements to the microVAS scoring system as well as the training program are suggested to improve reliability and consistency.

1 Introduction

Alterations to tissue blood supply, causing organ and tissue damage, are a common etiologic factor in many acute and chronic disorders with significant morbidity and mortality [1 –4]. Currently in clinical practice, hemodynamic assessment of the cardiovascular system mainly focuses on the macrocirculation (i.e. larger vessels). If macrocirculatory alternations were solely responsible for disturbances in tissue perfusion, modern medical imaging methods such as angiography or ultrasonography are readily available for diagnostics and therapeutic monitoring. However, most nutrient and oxygen transfer to tissues occurs in the smallest vessels, such as capillaries and arterioles. These small vessels (<100μm diameter) make up a network in the body known as the microcirculation [5]. Due of the small dimensions of the microcirculation, it can only be studied by microscopy or similarly sophisticated techniques. Currently, monitoring the microcirculation in patients is not standard practice, even though technology exists that allows clinicians the ability to observe the microvasculature at the bedside in real time [6].

In order to implement microcirculatory monitoring into clinical practice, validation of the modality, as well as standardized guidelines for all aspects of using that modality need to be established. Since the development of microscopic hand held cameras, multiple studies have extensively employed this technology in experimental and clinical studies assessing systemic microcirculatory function [2 , 7–10]. Guidelines for evaluating the microcirculation were initially established in 2007 outlining various aspects such as image acquisition, technical analysis requirements, modalities of analysis, as well as comments on interpretation of physiological and clinical aspects of microcirculatory monitoring [11]. Since then, microcirculatory assessment has also gained traction in the clinical field with clinical guidelines including recommendations regarding microcirculatory monitoring in critically ill patients [12]. Recently an update to the 2007 guideline was published due to advances in, technology, software, as well as modalities in analysis [13]. These guidelines also provide recommendations to future studies and clinical trails in order to improve validity and reliability, as well as quantitative cut-offs for microcirculatory variables that can guide intervention [13].

In this study we proposed a visual scoring system (microcirculatory Visual Analog Scale - microVAS) for the analysis of microcirculation videos. The microVAS score is a simple scoring system from 0 to 10 that quantifies two important microcirculatory parameters. We chose a scoring system from 0 to 10 because it offers the observer a wider discriminatory range when evaluating microcirculatory videos. The objective of this study was to validate our microVAS score by comparing it to the standard method of semi-automated analysis. We hypothesized that health care professionals with no microcirculation experience could be trained to use our microVAS score to rapidly and reliably analyse the microcirculation.

2 Methods

2.1 Volunteer raters and analysis setting

This prospective observational study was conducted at Dalhousie University in Halifax, Nova Scotia. The study was approved and conducted in accordance with the guidelines and standards set forth by the Nova Scotia Health Authority research ethics board (REB no. 100316). Twenty volunteers were recruited through flyers and word of mouth. Inclusion criteria included age >18 and currently in a medical profession at any stage of training. Exclusion criteria included any prior experience with analysing microcirculation videos, and any significant visual impairments affecting their ability to analyse videos. Informed consent had been obtained from all participants prior to participation in the study. The participants consisted of 14 medical students, 5 resident doctors, and 1 nurse, with ages ranging from 22 to 35.

2.2 Sample size calculation

Assuming a 95% confidence interval, a medium effect size (r = 0.30), and a margin of error of±0.10, we would need 320 observations [14]. Since each participant in our study would be scoring 40 videos each, recruiting 20 participants would give us 800 observations which is above the recommended minimum sample size four our desired level of precision.

2.3 Video samples and analysis setting

40 microcirculation videos were randomly chosen from a database of sublingual microcirculation videos collected from a previous study [15]. Of the 40 videos chosen, 22 videos were from healthy patients and 18 videos were of septic patients. All patient identifiers were stripped from the videos, then all 40 videos were randomly ordered and numbered in ascending order from 1 to 40. All observers scored the 40 videos in the same order to minimize confounders in the scoring based on video type. The analysis of videos was conducted in a conference room with the videos projected on a screen to maintain consistent screen size and resolution for all participants.

2.4 Introductory program

Prior to scoring microcirculation videos, all participants were given a 10 minute introductory presentation on microcirculation as part of the 40 minute training session. The introductory presentation highlighted the clinical importance of microcirculation, evolution of technology in acquiring microcirculation videos, the benefits and drawbacks of current analysis techniques, and the proposed benefit of a bedside visual score. Participants then got a short presentation describing the quality parameters required for video acquisition to obtain adequate videos for microcirculation analysis based on standard recommendations [11]. Recently an updated report was published, and the video acquisition techniques used in this study also adhere to the updated recommendations [13]. This presentation was given to get participants familiar with microcirculation videos. Following the presentation participants had the opportunity to visualize their own microcirculation using a sidestream dark field (SDF) videomicroscope (Microscan, Microvision Medical, Amsterdam).

2.5 microVAS score

Participants received a 30 minute training session that described all the microcirculatory parameters acquired in microcirculation analysis. There are 5 parameters currently used in microcirculation analysis that conveys information about different microcirculatory characteristics. The 5 parameters are: Proportion of perfused vessels (PPV), Microvascular flow index (MFI), Total vessel density (TVD), Perfused vessel density (PVD), and Heterogeneity index (HI). PPV quantifies the percentage of perfused vessels in the field of view, MFI quantifies different blood flow characteristics in the vessels, TVD quantifies the vascular density, PVD quantifies the density of perfused vessels, and HI quantifies the variability of blood flow. Based on the inherent nature of these analysis parameters, only MFI and PPV can be visually quantified, and were therefore incorporated into our microVAS score. Prior to developing our score, only one study had been conducted assessing a rapid visual score for microcirculation videos [16]. They used a 3 point scoring system for quality parameters, and a 5 point scoring system for perfusion parameters. Using our expertise in microcirculation research along with recommendations of their study, the microVAS score was developed. Based on their recommendations for an improved scoring system combined with the established visual analogue scale commonly used to describe pain levels in anesthesiology, we developed a scoring system from 0 to 10 to score microcirculatory videos for MFI and PPV. Traditional analysis techniques score MFI from 0–3 with 0 = no flow, 1 = intermittent flow, 2 = sluggish flow, and 3 = normal flow. Our hypothesis was that a 0 to 10 scoring system would offer better discrimination of various flow characteristics seen in human microcirculation that is not discriminated in a traditional 4 category system. In order to incorporate these same 4 flow characteristics into our scoring we created categories with 0 = no flow, 1–5 = intermittent flow, 5–9 = sluggish flow, and 10 = normal flow (Fig. 1).

Fig.1

Visual analogue scale of MFI scoring categorised into 4 standard flow characteristics.

Participants were trained to assign MFI scores using the recommended guidelines of splitting the video into 4 equal quadrants, assigning a flow value to each quadrant, and then taking the mean of all 4 quadrants to determine the microVAS MFI score for the video [17]. Participants were shown a series of sample videos highlighting capillaries with different flow characteristics and invited to ask any questions during the training period. The microVAS PPV score was also based on a scoring system from 0–10 (Fig. 2). Since the standardised PPV score is a percentage, the conversion of a microVAS PPV score to standard PPV score can be obtained with a simple conversion, (microVAS score×10 = PPV). Participants were shown various video clips of microcirculation demonstrating a range of different perfusion proportions to get them familiarised with employing the microVAS PPV score.

Fig.2

Visual analogue scale of PPV scoring with each score representing the corresponding percentage of perfused vessels.

After getting familiar with the microVAS scoring technique, participants were given a series of trial videos to score and their scores were compared to the scores of an expert observer. Participants were given the chance to clarify any discrepancies between scores, with the expert observer explaining his reasoning for assigning a microVAS score. Participants were informed that due to the subjective nature of the microVAS score, discrepancies between independent observers is inevitable. However, the aim of the training program was to ensure that scores between independent observers for the same video are comparable, and individual observers maintain a consistent subjective scoring technique for all videos.

Participants were shown the 40 videos in a sequential order and given as much time as needed to score the videos. However, all participants scored the videos in under 2 minutes each. All videos were looped so they played continuously without any interruption.

2.6 Statistical analysis

Pearson’s correlation was used to assess the similarity between individual microVAS and AVA3 scores both MFI and PPV. We also used the standard guidelines in interpreting the correlations with r = 0.1–0.3 indicating a small association, r = 0.3–0.5 indicating a moderate association, and r = 0.5–1.0 indicating a strong association [18]. The correlation coefficient was considered significant if p < 0.05. These calculations were carried out on Prism 5 (GraphPad Software, La Jolla, CA, USA). Krippendorff’s alpha was calculated to assess the inter-rater reliability when scoring videos for both MFI and PPV using the microVAS score. Standard guidelines in interpreting the agreement were used, with α>0.8 indicating good agreement, α>0.7 acceptable agreement, α>0.6 marginal agreement, and α>0.5 fair agreement. These calculations were carried out in SPSS 24 Software (IBM, Chicago, IL USA). Figures were created using the ggplot2 package in R (R Foundation for Statistical Computing, Vienna, Austria).

3 Results

In this study 20 volunteers scored 40 microcirculatory videos in real time for both MFI and PPV using the microVAS 10 point scale. The overall correlation for MFI combining healthy and septic videos was r = 0.33, p < 0.05, 95% CI [0.27, 0.39]. MFI scores for videos from healthy patients showed a correlation of r = 0.24, p < 0.05, 95% CI [0.15, 0.33]. MFI scores for videos from septic patients showed a correlation of r = 0.03, p = 0.26, 95% CI [–0.07, 0.14].

The overall correlation for PPV using both healthy and septic videos was r = –0.11, p < 0.05, 95% CI [–0.18, –0.04]. PPV scores for videos from healthy patients showed a correlation of r = –0.11, p < 0.05, 95% CI [–0.20, –0.02]. PPV scores for videos from septic patients showed a correlation of r = –0.02, p = 0.39, 95% CI [–0.12, 0.09].

The overall Krippendorff’s alpha for MFI using both healthy and septic patient videos was α= 0.56 (Fig. 3). When video type was separately analysed for MFI, the Krippendorff’s alpha for healthy patients’ videos was α= 0.34, and for septic patients’ videos was α= 0.31 (Fig. 4).

Fig.3

Superimposed histograms of the frequency of MFI microVAS scores based on video type. Blue bars indicate scores from healthy patient videos, red bars indicate scores from septic patient videos, and purple bars indicate their overlapping scores.

Fig.4

Histogram of the frequency of MFI microVAS scores based on video type. Blue bars indicate scores from healthy patient videos, red bars indicate scores from septic patient videos.

The overall Krippendorff’s alpha for PPV using both healthy and septic patient videos was α= 0.43 (Fig. 5). When video type was separately analysed for PPV, the Krippendorff’s alpha for healthy patients’ videos was α= 0.56, and for septic patients’ videos was α= 0.17 (Fig. 6).

Fig.5

Superimposed histogram of the frequency of PPV microVAS scores based on video type. Blue bars indicate scores from healthy patient videos, red bars indicate scores from septic patient videos, and purple bars indicate their overlapping scores.

Fig.6

Histogram of the frequency of PPV microVAS scores based on video type. Blue bars indicate scores from healthy patient videos, red bars indicate scores from septic patient videos.

4 Discussion

Currently, various modalities and parameters exist to analyse the microcirculation. To quantify microvascular changes, the microcirculatory video recordings can be analysed manually, automatically with the help of imaging software, or using recently developed visual scores. Initially, there was only complex, time-consuming, and semi-automated software available to analyse the microcirculation (∼1 hour per measurement). More recently, computer software has become available for automated analysis of the microcirculation. However, few studies comparing the accuracy of the automated software with the standard, semi-automated software have been conducted [15]. As a result, the automated method of analysis has not gained traction as the new standard of microcirculation video analysis. Furthermore, the cost associated with purchasing the automated software prevent its widespread use. Therefore, semi-automated analysis using software remains the gold standard of microcirculation analysis. Multiple barriers to clinical use exist with using software analysis (i.e., cost, delayed results, and expertise with using the software). Alternative, more practical approaches to microcirculation video analysis are needed to overcome these barriers. Visual analysis offers a solution to these barriers by providing a fast and inexpensive method to quantify certain microcirculatory parameters. Few published studies employ various visual scoring algorithms to assess the microcirculation either descriptively or quantitatively [19 –23].

The primary objective of this study was to validate our microVAS score by comparing it to the current standard method of analysis which is semi-automated using AVA3 software. Based on the correlation data obtained, the microVAS score showed poor correlation to standard analysis. However, in terms of clinical relevance, novice participants show some ability to distinguish between normal and pathological microcirculatory videos after only 40 minutes of training (Fig. 4, 6). This is evident in the histograms depicting a clear left skew distribution for MFI scores of healthy patient videos and a normal distribution for MFI scores of septic patient videos (Fig. 4). Analysing the frequency histograms for PPV indicate that novice raters tended to score both healthy and septic patient videos high (Fig. 6). Although both histograms had a left skew, healthy patient videos received higher PPV scores compared to septic patient videos evidenced by the higher mode (Fig. 6). This effect also explains the negative correlation for PPV between the two scores because in general, for the same video AVA3 scores tended to be lower while microVAS score tended to be higher. Furthermore, the PPV variance of AVA3 scores for septic videos was lower compared to microVAS scores, and the raw scores for PPV were all >60%. It could indicate that during the time of video recording, septic patients had not progressed to a stage where it was possible to observe a PPV under 60%. Alternatively, it may also indicate that the PPV of septic patients does not drop below 50% until they progress to septic shock. Similar values are evident in other published studies [22 , 25], with their PPV values all >60%. Overall this makes it challenging for novice observers to rate PPV and therefore contributes to a statistically nonsignificant correlation between our microVAS score and AVA3 scores for septic videos.

As a secondary objective we assessed the inter-rater reliability of our microVAS score using Krippendorff’s alpha. The microVAS results indicate that overall novice raters had scores that showed fair agreement for MFI, but when assessed based on video type the agreement was lower. Due to our microVAS score having 11 points for scoring MFI, variance between individual raters was inevitably higher compared to a score with fewer categories. The agreement was higher for both MFI and PPV when scoring healthy patient videos compared to septic patient videos. This observation can once again be attributed to the lack of variability in healthy patient’s microcirculation making it easier for novice raters to score, and vice versa.

Since the onset of our study, two papers have been published with their own individual visual scores [19, 22]. One study used an ordinal 5 point scale which is a composite score of both flow and heterogeneity [22]. They showed acceptable agreement between their POEM score and AVA3 score for PPV (R² = 0.71), and MFI (R² = 0.75). The other study only used descriptive categories of good, bad and very bad to rate MFI and PPV [19]. Both studies showed adequate agreement between their scores and AVA 3 scores, however they both employ descriptive measures. Assigning descriptive values is beneficial for individual clinicians to distinguish between different videos, however in the most recent consensus guidelines on microcirculatory analysis, certain quantitative cut-offs would be needed in order to guide intervention for treatments [13]. Once these cut-offs are established, the qualitative scores would be unable to incorporate these guidelines to help in microcirculatory monitoring and intervention. Therefore, the microVAS score, which is a quantitative score, still provides tremendous information for future diagnostic and therapeutic cut-offs, quantifiable comparisons between independent studies and standardization of visual microcirculation analysis. However, before these benefits can be materialised, the microVAS score needs to be improved and further testing needs to occur to improve its validity. Primarily it appears that the 0 to 10 point system for MFI is too broad for novice raters and therefore reducing the number of categories back to a 4 point system similar to standard MFI scoring from 0–3 would improve reliability and consistency. Furthermore, improvements in our training program would help novice participants use our microVAS system more consistently. This could include a longer practice scoring period, or repeated sessions to gain familiarity and experience with the scoring system. Similar to any training program, timely repeated exposure and practice will help improve reliability and consistency.

There are also a few limitations within our study that need to be addressed. Our microVAS score does not assess the other microvascular parameters of TVD, PVD, and HI due to its inherent visual characteristics. However, microcirculatory assessment guidelines indicate that only two components are needed that describe physiological function, namely the flow of blood through capillaries, and the density of perfused capillaries [13]. MFI and PPV encompass both these two requirements, therefore our microVAS score should contain enough information to be clinically beneficial at the bedside. In order to distinguish between various microcirculatory alterations for distributive shock, the minimum recommended variables are MFI, TVD+PVD, and HI [13]. In this regard our microVAS falls short because it does not address the heterogeneity of flow, but it does address diffusive capacity since PPV = PVD/TVD. Another limitation of our study was that video resolution could have been improved. In order to standardize the scoring setting for this study, the videos were projected onto a screen and multiple participants could independently score the videos at the same time. Having higher resolution monitors for scoring, and/or videos recorded in higher resolution with newer technology could help reduce the discrepancy in some of the microVAS scores.

Going forward, there are multiple potential areas where rapid clinical assessment of the sublingual microcirculation can be implemented. Primarily it can be used to improve diagnostics and monitoring in critically ill patients, or patients at risk for developing various types of shock. Equally as important it can be used to assess treatment efficacy in patients who are already in shock, by assessing changes in microcirculatory function after medical interventions. In addition, in current clinical practice, a patients’ fluid requirements during surgery are roughly estimated by clinical signs like blood pressure measurements, and urine output. More accurate assessments of a patient’s fluid status could be monitored in surgical procedures and trauma situations of haemorrhagic shock for guidance of fluid and blood products using microcirculatory monitoring. Furthermore, our microVAS score would also be a valuable screening tool for patients with acute and chronic cardiovascular health conditions in developing countries where more sophisticated laboratory and imaging methods are less accessible.

5 Conclusion

Our microVAS score provides a quantitative assessment of a patient’s microcirculation at the bedside in real time. Currently it has a small correlation with the standard method of analysis, however with adjustments to the microVAS scoring parameters, as well as training regimen, the future of the microVAS score looks promising. Using our current microVAS scoring technique, novice observers with minimal training could distinguish between a healthy and a septic patient. Therefore, as with other subjective modalities, experience and training will greatly enhance the robustness of our microVAS scoring system.

Footnotes

Acknowledgments

Funding for this project was made possible by: DMRF Cresco Summer Research Studentship 2016; & Tom Marrie Summer Studentship 2015, as well as support from the Department of Anesthesia, Pain Management and Perioperative Medicine, Dalhousie University, Halifax, Nova Scotia, Canada.

References

Ince

. The microcirculation is the motor of sepsis. Crit Care. 2005;9(Suppl 4):13–9.

De Backer

, Creteur

, Preiser

, Dubois

, Vincent

. Microvascular blood flow is altered in patients with sepsis. Am J Respir Crit Care Med. 2002;166:98–104.

Tikhomirova

, Oslyakova

, Mikhailova

. Microcirculation and blood rheology in patients with cerebrovascular disorders. Clin Hemorheol Microcirc. 2011;49:295–305.

Abularrage

, Sidawy

, Aidinian

, Singh

, Weiswasser

, Arora

. Evaluation of the microcirculation in vascular disease. J Vasc Surg. 2005;42:574–81.

Jung

, Leithäuser

, Landgraf

, et al. Laser Doppler flux measurement for the assessment of cutaneous microcirculation – critical remarks. Clin Hemorheol Microcirc. 2013;55:411–6.

Bosetti

, et al. Small Blood Vessels: Big Health Problems? Scientific Recommendations of the National Institutes of Health Worksho. J Am Heart Assoc (2016;5:1–12.

Bezemer

, et al. Rapid automatic assessment of microvascular density in sidestream dark field images. Med Biol Eng Comput. 2011;49:1269–78.

Boerma

, Mathura

, van der Voort

, Spronk

, Ince

. Quantifying bedside-derived imaging of microcirculatory abnormalities in septic patients: A prospective validation study. Crit Care. 2005;9:601–6.

De Backer

, Donadello

, Taccone

, Ospina-Tascon

, Salgado

, Vincent

J-L

. Microcirculatory alterations: Potential mechanisms and implications for therapy. Ann Intensive Care. 2011;1:27.

10.

Demir

, Hakimzadeh

, Hargraves

, Ward

, Myer

, Najarian

. An automated method for analysis of microcirculation videos for accurate assessment of tissue perfusion. BMC Med Imaging. 2012;12:37.

11.

De Backer

, et al. How to evaluate the microcirculation: Report of a round table conference. Crit Care. 2007;11:101.

12.

Cecconi

, et al. Consensus on circulatory shock and hemodynamic monitoring. Task force of the European Society of Intensive Care Medicine. Intensive Care Med. 2014;40(12):1795–815.

13.

Ince

, et al. Second consensus on the assessment of sublingual microcirculation in critically ill patients: Results from a task force of the European Society of Intensive Care Medicine, Intensive Care Med. 2018.

14.

Moinester

, Gottfried

. Sample size estimation for correlations with pre-specified confidence interval. Quant Methods Psychol. 2014;10:124–30.

15.

Sharawy

, et al. Preliminary clinical evaluation of automated analysis of the sublingual microcirculation in the assessment of patients with septic shock: Comparison of automated versus semi-automated software. Clin Hemorheol Microcirc. 2017;67:489–98.

16.

Gommeren

, Allerton

, Morin

, Reynaud

, Peeters

, Silverstein

. Evaluation of a rapid bedside scoring system for microcirculation videos acquired from dogs. J Vet Emerg Crit. Care (San Antonio). Aug. 2014.

17.

De Backer

, et al. How to evaluate the microcirculation: Report of a round table conference. Crit Care. 2007;11:R101.

18.

Cohen

. A power primer. Psychol Bull. 1992;112:155.

19.

Lima

, et al. Interrater Reliability and Diagnostic Performance of Subjective Evaluation of Sublingual Microcirculation Images by Physicians and Nurses: A Multicenter Observational Study. Shock. 2015;44:239–44.

20.

Tanaka

, et al. Qualitative real-time analysis by nurses of sublingual microcirculation in intensive care unit: The MICRONURSE study. Crit Care. 2015;19:388.

21.

Arnold

, et al. Point-of-care assessment of microvascular blood flow in critically ill patients. Intensive Care Med. 2009;35:1761–6.

22.

Naumann

, et al. Real-time point of care microcirculatory assessment of shock: Design, rationale and application of the point of care microcirculation (POEM) tool. Crit Care. 2016;20:1–9.

23.

Gommeren

, Allerton

, Morin

, Reynaud

, Peeters

, Silverstein

. Evaluation of a rapid bedside scoring system for microcirculation videos acquired from dogs. Journal of Veterinary Emergency and Critical Care. 2014;24:554–61.

24.

De Backer

, et al. Microcirculatory Alterations in Patients With Severe Sepsis: Impact of Time of Assessment and Relationship With Outcome. Crit Care Med. 2013;43:791–9.

25.

Filbin

, et al. The Microcirculation Is Preserved in Emergency Department Low-acuity Sepsis Patients Without Hypotension. Acad Emerg Med. 2014;21:154–62.