Abstract
Background
Patients’ adherence to elastic compression stockings is difficult to evaluate, and therefore we create a short self-questionnaire and validate its psychometric properties.
Methods
We reduce the questions with Varimed rotation analysis, evaluate its internal consistency using Cronbach’s alpha test and its external validity by comparison to electronic thermic captors. Receiver operating characteristic (ROC) analysis was used to determine a threshold and determine its sensitivity and specificity.
Results
The initial questionnaire was reduced from 22 to 5 questions rated from 0 to 4. The internal consistency and its external validity are good, and the ROC analysis shows that values > 3 correspond to poor compliance with a sensitivity of 88.1% and a specificity of 63.1%.
Conclusion
The adherence score is valid for the detection of patients poorly compliant to the wearing of elastic compression. Its small number of questions makes it a suitable for a screening in everyday practice.
Introduction
One of the methodological weaknesses commonly pointed up in many clinical studies is that adherence is not evaluated. 1 This is especially true of studies evaluating elastic compression stockings for want of suitable validated methods for quantifying their wear.
Admittedly, clinical studies do include evaluation factors. These are essentially self-questionnaires given to patients in which they are asked to record whether they have worn the compression stocking every day throughout the study. These questionnaires are essentially Likert scale or visual analogue scale evaluations, and the review of recent literature in standard databases and particularly Pubmed does not turn up any specific and duly validated adherence questionnaires. Moreover, as with all self-reporting questionnaires used for this purpose, whether for medicinal drugs or medical devices, the drawback is that they may influence how often elastic stockings are worn through the mere fact that they must be completed daily; or worse still, they may be completed out of complacency with no relation to actual wearing of the elastic stockings for fear of being reprimanded by the investigator.
Recently, sensors have appeared that can record temperatures tens of times per day over several months. These temperature recorders can be fitted into the top edge of the stocking and subsequently read by computers to reveal the precise times they were in contact with a surface at around 35℃ and so a priori when the stockings were worn. Several recent publications have shown them to be valid for monitoring the wearing of compression stockings2,3 except that they cannot be used in summertime because they are heat sensors.
However, although they can be used as measuring devices in clinical trials, they obviously cannot be used in day-to-day vascular medicine or general practice.
Besides, whether in the form of questionnaires or heat sensors, these evaluation tools relate only intrinsically to the wearing of compression stockings, which potentially biases results by focusing patients’ attention on this. It would be preferable to appraise the relation between patients and the wearing of their compression stockings more comprehensively through patient attitudes towards them, the way they perceive the medical service they provide and the constraints they are under in return. This would entail a shift from observance to adherence to treatment.
In the area of medicinal drugs, steps were taken many years ago with the introduction of questionnaires such as the Morisky-Green questionnaire. 4 Their transposition to elastic compression might be a solution, but several obstacles stand in the way. First, the questionnaires were initially devised for evaluating observance of taking medication, making the transposition to so remote an area as wearing compression stockings difficult. Second, this type of ‘uncontrolled’ transposition would be counter to good practice in terms of development of evaluation questionnaires which must now be devised according to well-defined rules. It would be particularly risky for the quality of evaluations if the tool used to evaluate the products had not itself been evaluated and if its psychometric properties were not known and validated.
In producing this questionnaire, a stringent, special-purpose method has been established 2 and a basic observance questionnaire was devised beforehand by Delphi-type qualitative methods involving patient interviews. This paper shows how the number of questions in the initial version was mathematically reduced to yield the final questionnaire and reports the results of validation of its psychometric properties.
Material and methods
Purpose of the trial
This clinical trial was intended to draw up a short questionnaire of patient adherence to medical compression stockings that could readily be used in day-to-day practice. This was to be done by reducing the number of questions by objective mathematical methods based on an initial questionnaire drawn up by means of Delphi-type qualitative interviews. The aim was also to validate the final questionnaire’s psychometric properties against the current gold standard of measuring observance via temperature sensors.
Design of the initial adherence questionnaire
Initial version observance questionnaire (n = 79).
Obtaining and devising the gold standard
Wymesure® temperature sensors were used to measure whether compression stockings were worn by recording the ambient temperature every 30 min. The sensors were sown into the top edge of the stocking and temperatures higher than 35 C were taken to indicate that the compression stockings in which they were fitted were actually being worn.
The information in the chip was read using special-purpose software. The plots produced by the software were used to identify the periods during which each of the two stockings given to each patient were actually worn, to sum the total time the compression stockings were worn and to calculate an observance index (OI) by dividing the total time of actual wear by the total time the patient should have worn the stockings (OI = actual time/prescribed time). This OI was the gold standard against which the external validity of the adherence questionnaire was evaluated. The sensors are waterproof and can be machine washed. The washing times were identified by spikes in the temperature curve and could not be confused with periods of actual wear. 2
Nature of the compression stockings
Over the four weeks of the study, all the patients had to wear the compression stockings given to them and for which the protocol measured observance. Each patient was given two pairs of class II Sigvaris Diaphane® stockings, exerting a pressure of 15–20 mmHg on the ankle (AFNOR G 30 102 B standard) and prescribed specifically for the measurements made for the patient. Choosing the same make for all the patients meant that adherence to wearing the stockings was not affected by variations in observance arising from the stockings varying in comfort.
Subject selection
Patients had to be female, at least 18 years old, have given their free and informed written consent and present chronic oedema related to bilateral venous hyperpressure that had set in more than 12 weeks before and required class II (15–20 mmHg) compression. Patients also had to be able to put on the class II compression stockings unassisted.
The study excluded patients:
who were pregnant or breast-feeding, as well as women of child-bearing age not on effective contraception; with known hypersensitivity to any of the constituents of the compression stockings or any contraindication to wearing medical compression stockings, or conversely patients treated currently or in the previous year by elastic or inelastic compression, whatever the type of strap, sock, stocking, or tights, except for occasional wear related notably to air travel; with unilateral oedema and especially post-thrombotic oedema, poorly controlled diabetes, deep vein thrombosis in the previous 12 months, obesity with a BMI of more than 35, distal systolic pressure of less than 0.6 or more than 1.3, or oedema whose aetiology was likely to be non-venous in origin; who were linguistically or mentally unable to understand the information provided, or with altered cognitive functions that might affect the validity of their consent, or without legal contractual capacity.
Conduct of the trial
The trial lasted four weeks for each patient with an inclusion visit on D0 and a final visit on D30. At enrolment, the doctor completed an observation form describing the patient’s demographic characteristics and history of venous complaints. He explained how to put the compression stockings on and how to take care of them, the need to wear them from getting up in the morning to going to bed at night and to bring them to the next visit so the heat sensor recordings could be read. After four weeks (D28 ± 3 d), the doctor saw the patients again at the same time as for the initial visit. He asked them to complete the adherence questionnaire that was the subject of the study and recovered the compression stockings, the sensors from which were sent to Cen Biotech for reading.
Statistical analysis
Population analysed
The analyses were performed on the population of patients who met the inclusion criteria and completed the adherence questionnaire. All of the variables were described by means and standard deviations for quantitative variables and by frequencies and percentages for the qualitative variables.
Reduction of the number of questions
The number of questions was reduced iteratively by combining analyses of internal consistency via Cronbach’s alpha 5 and analyses of external consistency by Pearson’s correlation coefficient between the adherence questionnaire and the gold standard of the OI. The number of items was reduced by the following process. First, missing data and the distribution of responses to each question were described so as to identify those for which missing answers exceeded 2.5% or for which the response distribution was highly asymmetric. Questions which went unanswered too often could mean patients failed to understand them or found them too difficult to answer. Excessive asymmetry was also a ground for excluding a question since it indicated that all patients provided similar answers and so it was not discriminating.
A second phase in shortening the questionnaire involved analysing its external correlation with the OI measured by sensors and by internal correlation studies among the questions; a correlation of more than 0.75 between two questions led to one of them being removed because they covered the same information. Next, the internal consistency of the questions of the entire questionnaire was evaluated using Cronbach’s alpha, and each question’s contribution to the total questionnaire was also measured by Pearson’s correlation coefficient. Values of at least 0.7 had to be obtained for Cronbach’s alpha and values of at least 0.4 for correlations between each question and the entire questionnaire.
A third phase of reduction used a principal components analysis (PCA) with VARIMAX rotation. Only the axes of rotation meeting the Kayser criterion, that is with an eigenvalue of 1 or more, were selected. Questions whose contribution was < 0.4 for all axes and those whose contribution was ≥ 0.4 on at least two axes were removed as not discriminating.
A fourth and final reduction phase used the Pearson coefficient again to check whether the inclusion of each question in the questionnaire raised or lowered its overall correlation with the OI measured by the sensors.
Psychometric characteristics of the final questionnaire and determination of a discriminant level
On completing these iterative procedures to reduce and optimize the questionnaire compared with the OI, the questionnaire’s final internal consistency was measured by Cronbach’s alpha and its external validity measured by determining its correlation with the OI using Pearson’s coefficient.
Its discriminant value was then determined by an ROC analysis of the relation between the values of the final adherence questionnaire and those of the OI so as to identify the limit separating patients who adhered well or poorly to the use of compression stockings and to determine the sensitivity and specificity of this threshold.
Justification of frequencies
The objective was to shorten the adherence questionnaire to fewer than 15 questions. The accepted rule in this area6,7 for calculating frequencies needed to validate the questionnaire by multifactorial analysis being at least five analysis units per question, the number of patients to be included was 75 (15 × 5).
Analysis software and statistical significance level
The data were captured using the Capture System software. The statistical analyses were performed with SAS software for Windows (version 9.3, SAS Institute, Inc., Cary, NC, USA) for all the analyses.
Regulatory and administrative analyses
The protocol, patient information form and patient consent form were presented to the ethics committee (CPP EST I) and approved on 11 June 2014. In agreement with European directive 2001/20/EC transposed by decree no. 2005-477 of 26 April 2006, patients had to give their free and informed consent in writing after being duly informed in writing and verbally of the purpose of the research, its methodology, duration, constraints and foreseeable risks. Data collection was reported to the data protection agency (CNIL) and pursuant to decree no. 2007-454 of 25 March 2007 amending the Public Health Code, it was submitted to the general medical council (Conseil National de l’Ordre des Médecins) before the study was implemented. Pursuant to article R4113-110 of the Public Health Code and the EFPIA Code, it was the responsibility of the investigating doctors to declare any connections they had with the study promoters.
Results
The study involved 79 female patients aged 54.5 ± 14.7 years on average. Three-quarters of them (73.4%) had a family history of venous disease and they also presented many personal venous risk factors including antecedents relating to pregnancy (75.9%), standing for more than 6 h per day (36.7%), sedentarism (36.7%), sitting for more than 6 h per day (22.8%) and obesity (17.7%). Their venous disease had begun 17.9 ± 13.4 years before on average and had gradually worsened for 53.2% of them. All had oedema, associated with varicose veins in 35.4% of patients and trophic disorders in 9.2%. The initial answers to all of the questions of the initial questionnaire of 22 questions are given in Table 1.
The electronic sensors provided objective data about wearing the compression stockings and, over the 28 days, they were actually worn for 22 days, making a mean OI of 80% and good level of observance defined by an OI ≥ 70% of 74.7%. When worn, the mean length of time the compression stockings was worn was 11 h.
Item reduction and score construction
The items were reduced and the score was constructed by the following process in accordance with the statistical analysis plan. First, missing data and the distribution of responses of each item were described to identify the questions for which missing answers exceeded 2.5% and the items with wholly asymmetrical distributions. On the basis of these computations, no questions were excluded outright, but the analyses revealed that four questions (QOBS 10, QOBS 13, QOBS 17 and QOBS 20) were highly asymmetrical in the responses, showing that all the patients overall gave the same answers. In the cases in point, the ‘never’ answer was ticked more than 80% of the time, which affected their discriminating character.
A second reduction phase used an external validity study by correlation with the OI measured by sensors. On completing the analysis, nine questions were deleted (QOBS 2, QOBS 10, QOBS 11, QOBS 13, QOBS 14, QOBS 15, QOBS 20, QOBS 21 and QOBS 22) because the answers reduced the correlation with the OI.
Next, a study of correlations among the 13 remaining questions was conducted; a correlation of ≥0.75 between two questions meant one was deleted as they covered the same information. Question QOBS 8 was deleted because of a 0.79 correlation with question QOBS 9, and question QOBS 3 because of a 0.79 correlation with QOBS 4. For the whole of the questionnaire, and for each dimension, internal consistency was evaluated by its Cronbach’s alpha which was 0.86 for a recommended value of at least 0.70. The correlation of each question with the entire questionnaire was studied using Pearson’s correlation coefficient to check that it was at least 0.40. Questions QOBS 6 and QOBS 7 were deleted because they presented correlations of just 0.32 and 0.33, respectively.
The third reduction phase used a PCA with VARIMAX rotation. The questions whose contribution was <0.4 for all the axes and questions whose contribution was ≥0.4 on at least two axes had to be deleted because they were not discriminating. Question QOBS 1 was deleted on this ground.
A final reduction phase was conducted on the remaining eight questions by measuring their contribution to the overall score and the quality of its correlation with the OI. On completing this analysis, three questions were deleted (QOBS 4, QOBS 5 and QOBS 18) because the answers to them reduced the correlation with the OI.
Final structure of the observance questionnaire.
Psychometric characteristics of the final questionnaire and discriminating limit
The adherence score was calculated by adding the scores to the five questions of the initial questionnaire ranging from 0 to 4. It varied therefore from 0 to 20 and its properties were analysed. Its internal consistency is good with a Cronbach’s alpha of 0.7. Its external validity is satisfactory with a correlation of −0.4 (p: 0.0003) with the OI. Analysis of the ROC curve shows that scores ≤ 3 attest to good adherence to compression stockings, while values > 3 suggest poor adherence with a sensitivity of 88.1% and specificity of 63.1% (Figure 1).
ROC curve analysis.
Discussion
The development of this score by validated psychometric methods is a major guarantee for its validity, and there are few scores in phlebology that have benefitted from such methodological stringency. Its internal consistency measured by a Cronbach’s alpha of 0.7 is good and calls for no discussion. Its external validity measured by its correlation with the OI established by temperature sensors is also highly satisfactory considering that the coefficient of −0.4 has been obtained from just 79 observations yet with a significance level of p: 0.0003. As for the threshold at ≤ 3 or >3 separating patients who adhere well or poorly, it attests to very high sensitivity of close to 90%, which makes it a suitable test for spotting poor adherence while retaining a specificity of 63%, which fully justifies its use.
Of course, it is likely that the measurement of observance via this score is less precise than that obtained by temperature sensors, but whether in clinical trials and above all in day-to-day use, the sensor solution is not well adapted. In clinical trials, the sensors are thought of as ‘spies’ and strongly influence patients’ adherence, which is not without its problems, even for our study, in which it would probably have been better still to have a patient distribution that was more consistent with actually wearing compression stockings with in particular more poorly adhering patients. In daily practice, having sensors in compressions stockings cannot be contemplated and would go against the principle of a ‘real’ life study.
It is also to promote the use of this adherence score in real life that we have endeavoured to drastically reduce the number of questions to be asked, even at the price of a little precision. Given the current conditions in which phlebological or general consultations are conducted, only short questionnaires can be contemplated. With just five questions requiring about 1-min at most to answer, this questionnaire can be considered in day-to-day practice, so that it could become not just an instrument of clinical research but a tool of day-to-day clinical evaluation.
The literature review of recent years in the reference databases and especially Pubmed has not revealed other adherence questionnaires apart from Likert scale or visual analogue scale evaluations, which cannot in any way be compared with the score established in this work. It would be desirable that future work on this adherence score should be done to judge how useful it is in day-to-day practice and how doctors take to it. It might also be used and evaluated by pharmacists who are involved in supplying compression stockings and who could also contribute to promoting patient adherence. Identifying patients who adhere poorly to their treatment opens the way to dialogue to convince them of the benefit of wearing compression stockings and what it means for their health.
Conclusion
What is named the Adhesig adherence combines the psychometric characteristics required to ensure its validity for identifying patients who adhere poorly to wearing compression stockings. The reduced number of questions and its detection level aimed at sensitivity while providing good specificity make it a suitable tool for such testing in day-to-day practice.
Footnotes
Declaration of Conflicting Interests
The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: FA Allaert is head of the medical evaluation Chair and Cenbiotech Dijon. D Rastel is consultant for Sigvaris. A Graissaguel and D Sion are employed by Sigvaris. C Hamel-Desnos has no conflicting interest for that study.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The study has been funded by Sigvaris.
Ethical approval
CPP Centre Est 1 2014/29 – ID RCB : 2014-A00767-40 agreement on June 14 2014.
Informed consent
All patients gave their written informed consent.
Guarantor
French society of Phlebology.
Contributorship
FA Allaert conducted the methodological aspects of the study and wrote the paper, D Rastel contributed to the methodology, was the main investigator and reviewed the paper, A Graissaguel and D Sion reviewed the paper, C Hamel-Desnos contributed to the methodology and reviewed the paper.
Acknowledgements
We thank A Massinissa, R Schueller and F Rigaudier of CenBiotech for their contribution to the statistical analysis.