Hering’s Law Assessment Tool Revisited

Abstract

Hering’s Law Assessment Tool emerged as a systematic outcome assessment tool following homeopathic intervention. The authors intend to modify it and develop a new tool—Patient Response Assessment Tool after Homeopathic Treatment (PRATHoT)—in chronic cases through Delphi technique for systematic categorization of probable outcomes following individualized homeopathic treatment in chronic cases. The PRATHoT was drafted after literature review and iterative Delphi rounds with multidisciplinary expert panel, setting Fleiss κ of 0.41 to 1.00 a priori as the desired level of multirater agreement. Following pilot testing, the tool was implemented on 37 patients suffering from knee osteoarthritis over 6 months. Logistic regression analysis confirmed that higher PRATHoT score was significantly associated with achieving pain visual analogue scale responses from the second follow-up visit onwards (B = 0.037-0.066; SE = 0.021-0.036; P = .003-.048). The tool appeared to have acceptable psychometric properties; hence, it may be considered as a promising tool, amendable for further development.

Keywords

Delphi Hering’s law homeopathy

Introduction

Assessing outcomes systematically following administration of “individualized” or “classic”^1

–6 homeopathic drugs in chronic cases has remained pivotal and debatable since the inception of homeopathy. In individualized homeopathy, following a conventional diagnosis, the selection of remedy and dosage is based on matching the patient’s symptom picture with the “remedy picture” along with consideration of different complex issues, such as miasmatic analysis, constitution, susceptibility, and so on. The varied bulk of empirical literature concerning the subject matter is ambiguous, inconclusive, and has rarely been subjected and verified through high-quality trials. Hering’s law of cure in homeopathy⁷ helps assess clinical outcomes by stipulating that during a curative response to a constitutional remedy,^5,8 “vital force”⁵ responds by eliciting a distinct and consistent pattern^9,10 in which the symptoms improve from above downward, from within outward, from most important to least important organs, and in the reverse order of their appearance.^6,7,11 In recent times, Hering’s Law Assessment Tool has emerged as an outcome assessment tool that holds promise to serve the purpose^12,13; however, its validity and reliability remained unaddressed formally.¹² The researchers found some insufficiencies in the content of the available Hering’s Law Assessment Tool and intend to modify it by assessing, summarizing, and simplifying all the probable outcomes through extensive literature review and incorporating expert comments and to validate formally the newly developed instrument. We present here the development of the Patient Response Assessment Tool after Homeopathic Treatment (PRATHoT) and the formal validation of this tool.

Materials and Methods

The goal of this study was to develop a validated knowledge-based tool for categorizing outcomes after a homeopathic prescription in chronic cases. In order to meet this goal, we intended to adopt the Delphi technique. The Delphi technique is an iterative and sequential, multistage, flexible, group-communication process for forecasting and decision-making purposes to derive informed quantitative anonymous agreement and consensus among a panel of experts in the field on a particular issue or problem through qualitative assessment of evidence, thereby minimizing the liabilities of individual expert decision.^14

–17 Representatives from all relevant domains were invited to participate—practice, teaching, and research. Experts were chosen to ensure diverse viewpoints with national, scholarly, and clinical perspectives. They were required to have a university master’s degree in homeopathy and at least 5 years of experience in practice, teaching, and/or research and familiarity in using Hering’s Law Assessment Tool in clinical practice. Invitation and participation in the Delphi process was completed via email outlining the aim, likely time commitment, and processes. Interested respondents were invited to provide their consent to be considered as a member of Delphi panel in presentations and publications arising from this research.

Delphi First Round

The first round was qualitative in nature. A brief preamble was provided concerning the aim of survey, definition of key terms, likely time commitment, plan for 4 to 5 rounds, as well as the necessity of completing all rounds. The first round commenced with inviting free opinions and suggestions involving experiences and judgments from the experts regarding the content of Hering’s Law Assessment Tool and a list of predictions and recommendations. Fifteen experts, 5 each from 3 relevant fields—research, academics, and practice—were invited to participate. Reminder to reply within 7 days accompanied the first round. All replies were analyzed by the research team after 15 days. Extensive efforts were made to include all the probable elements identified in the literature review conducted between November 2012 and January 2013 in 6 electronic databases (MEDLINE, EMBASE, PSYCHINFO, CINAHL, AMED, and Cochrane Central Register) and formal hand-searching of departmental files and bibliographies. A new tool, namely, Patient Response Assessment Tool after Homeopathic Treatment (PRATHoT), was drafted incorporating and compiling all possible outcomes in the original Hering’s Law Assessment Tool.

Delphi Second Round

A close-ended questionnaire with 3-point agreement Likert-type scale (0 = Disagree; 1 = Uncertain; 2 = Agree) was developed from the draft tool. Consensus to achieve was defined a priori as Fleiss’ κ observed agreement between moderate to almost perfect (0.41-1.00). Available published literature influenced the conceptualization of the questions. All the panel experts were provided with instructions to rate their level of agreement on the Likert-type scale. After completing the rating exercise, participants had the opportunity to provide the “rationale” in justification of their reply and giving further comments/suggestion(s) to revise/modify the item(s), if any. The total number of completed response sheets was compiled after an interval of 1 week. Thus, the areas of agreement and disagreement were identified.

Delphi Third Round

Items not achieving consensus and suggestions from participants of the Delphi second round were cross-checked against the items derived from the literature review. These were added as “additional items” for participants to consider in Round 3. This entire list of questions was randomly ordered and comprised the third round survey. Items that already had consensus were eliminated from the list for further evaluation and comments. In this round, participants rated their responses using the same Likert-type scale of 3 points and were asked to specify the reasons for remaining outside the consensus. After completing the exercise, participants again had the opportunity to provide further suggestions by adding comments. All the 7 expert panelists replied in this round. All responses were compiled again after an interval of 1 week. Analysis was replicated as in the second round by computing consensus for each item.

Delphi Fourth Round

The Delphi process was prospectively planned to have iterations until and unless consensus had been achieved in all the items and no further change is taking place from further rounds. Analyzing the inputs, participants were invited for any further and final suggestions to modify the tool. We planned to ask the experts to either revise their opinions or discuss the reasons for not coming to consensus with the group. The analysis of the previous rounds was replicated in the final round analysis. Thus, the list of information items formed the basis for the content of the first draft of the PRATHoT, as compiled by the research team.

Pilot Testing

The tool was subjected to pilot testing for length, content, clarity, language, relevance, and overall adequacy in 5 patients suffering from knee osteoarthritis in follow-up visits every month over 6 months. With little modifications and approval of the entire team members, the tool appeared clear, language acceptable, and relevant to the topic under investigation.

Field Testing

The instrument was applied over 1 year on 37 patients suffering from knee osteoarthritis, diagnosed clinically as per American College of Rheumatology criteria¹⁸ with radiographic evidences. Patients recruited were involved in an observational study in the outpatients wing of Mahesh Bhattacharyya Homeopathic Medical College and Hospital, Howrah, West Bengal, and treated by individualized (“classic”) approaches. There was a 6-month study duration per patient, that is, 1 follow-up visit every month over 6 months for each patient. Outcome measures were 100 mm pain visual analogue scales. A minimum of 5.1 to 13.3 mm reduction was taken as the responder criteria.¹⁹ Every month responses were recorded in a specially designed excel spreadsheet using the PRATHoT codes.

Similar to Hering’s Law Assessment Tool, the PRATHoT was scored by the prescribing homeopath using a value of 0 or 1 for each of the options when the patient was reviewed at each consultation every month over 6 months, to assess the clinical effects of the previous prescription. Ascribing a score of 1 confirms that the patient had experienced at least one symptom from the Hering subscale (H), whereas a score of 0 indicates that symptom classification did not belong to that category. If patients report 2 symptoms considered by the homeopath for the Hering subscale, a score of 2 would be given for that option. Thus, to identify a healing response according to Hering’s law at each consultation, the total number of scores related to cure according to Hering’s law was summed (ie, for Ha, Hb, Hc, and Hd) to give an overall score at the end of treatment, the total PRATHoT score.

The PRATHoT was planned to undergo logistic regression analysis to test whether higher PRATHoT score was significantly associated with achieving pain visual analogue scale responses; analysis for internal consistency by Cronbach’s α, concurrent and discriminant validity by Pearson’s r, and repeated measure analysis of variance, respectively, interrater reliability by kappa statistics, sensitivity, and specificity by receiver operating characteristics analysis using different computational Web sites.

Results

Experts’ Responses

In the beginning, invitations asking suggestions and opinions were sent to a total of 15 experts of whom 10 (3 researchers, 4 academicians, 3 practitioners) responded and took part in Delphi Round 1. Others remained silent in spite of repeated reminders by telephone and email. Out of 10 respondents in the first round, only 7 (2 researchers, 3 academicians, 2 practitioners) replied in the second and subsequent rounds. The response rate could not be enhanced further in spite of repeated attempts. Profiles of the 7 experts are shown in Table 1.

Table 1.

Profiles of Experts.

Expert	Age	Gender	Qualification	Background
1	39	Male	Postgraduate	Researcher
2	40	Male	Postgraduate	Practitioner
3	59	Male	Postgraduate	Academician
4	48	Male	Postgraduate	Academician
5	28	Male	Postgraduate	Researcher
6	35	Male	Postgraduate	Practitioner
7	47	Male	Postgraduate	Academician

Tool Content

The PRATHoT was drafted on the basis of existing Hering’s Law Assessment Tool, literature review, and expert opinions. It intends to incorporate all the probable responses to an individualized constitutional homeopathic drug (not necessarily “the similimum”). The modified categorical and subcategorical coding of the PRATHoT outcome responses were as follows (Figure 1):

Figure 1.

Patient Response Assessment Tool after Homeopathic Treatment (PRATHoT).

Initial responses—new symptoms (P), symptoms relieved (Q), symptoms worsened (R), and no changes felt by the patient or the physician either (S)

Symptoms due to adverse events unrelated to remedy (eg, extraneous events)—categories A, G2, K

Symptoms due to adverse drug reaction—categories B, I2

Symptoms due to idiosyncratic or hypersensitivity reaction—category C

Accessory symptoms due to presence of one-sided disease—category D

New symptoms arising from progression of disease pathology—category E

Initial change (improvement/worsening) that resolves, no change thereafter (dose/repetition insufficient/overt, incorrect remedy, extraneous events, maintaining cause, fundamental/“miasmatic cause,” etc)—categories F(1-4), J(1-4)

Symptoms due to presence of subcurable state—categories G1, I1

Initial improvement of existing symptoms followed by worsening due to maintaining cause—category G3—or incorrect remedy and/or dose—category G4

Symptoms due to healing effect—category H

Symptoms under category H may arise from 2 probable conditions: (a) initial improvement of existing symptoms that persists and progresses toward recovery—H₁; and (b) initial worsening of existing symptoms (“homeopathic aggravation”) followed by improvement at different pace toward recovery—H₂. Further subdivisions of category H symptoms were kept intact as in Hering’s Law Assessment Tool, that is, Ha (from top down), Hb (from more to less vital organs), Hc (from center to periphery), and Hd (in reverse order of appearance). Progression of healing response may be interrupted by many factors as listed under categories F, G, and J and may be coded accordingly.

Adverse drug reactions are not usually expected from homeopathic drugs in high dilutions when prescribed by trained professionals; however, it is difficult drawing definite conclusions on account of low methodological quality of reports.²⁰ So, though not rigorously investigated, category B and I₂ of adverse drug reactions has been incorporated and retained in the tool. Any augmented (dose-related), chronic (dose and time related), delayed (time-related), and end of use (withdrawal) effects have not been recorded; yet they have been included under the name of “homeopathic drug disease” (category I₂).^21,22

Face and Content Validity

It was enhanced by first referring to the available literature related to the topic of research. Second, the tool was pilot-tested on 30 follow-up visits to ensure that the concepts included were actually related to assessing patients’ response following homeopathic intervention. Third, viewpoints of a panel of experts have been used in the development of the tool.

Kappa Statistics

The draft PRATHoT was converted into 14 and 4 close-ended questions in rounds 2 and 3, respectively. The interrater reliability of agreement was computed between the responses of the experts for each question in rounds 2 and 3 using Fleiss’ κ. In the second round, 3 questions achieved moderate level of agreement (between 0.41 and 0.60), 2 achieved substantial agreement (between 0.61 and 0.80), and 5 achieved perfect agreement (ie, 1.00). Four questions achieved fair agreement (between 0.21 and 0.40) and were subjected to Delphi round 3 with necessary modifications. In round 3, out of 4 questions, 3 achieved substantial agreement and a single question achieved perfect agreement (Tables 2 and 3).

Table 2.

Interrater Reliability Among the Experts During Delphi Round 2.

Question	Rating Score, Mean (SD)	Fleiss κ Observed Agreement (Consensus %)	Agreement	Acceptance
1	1.71 (0.76)	0.656 (85.71)	Substantial	A
2	1.43 (0.98)	0.524 (71.43)	Moderate	A
3	1.29 (0.95)	0.333 (57.14)	Fair	NA
4	1 (1)	0.286 (42.86)	Fair	NA
5	1.71 (0.76)	0.714 (85.71)	Substantial	A
6	1.14 (0.89)	0.238 (42.86)	Fair	NA
7	2 (0)	1 (100)	Perfect	A
8	1.57 (0.79)	0.476 (71.43)	Moderate	A
9	2 (0)	1 (100)	Perfect	A
10	2 (0)	1 (100)	Perfect	A
11	1.57 (0.79)	0.476 (71.43)	Moderate	A
12	2 (0)	1 (100)	Perfect	A
13	1.29 (0.95)	0.333 (57.14)	Fair	NA
14	2 (0)	1 (100)	Perfect	A

Abbreviations: SD, standard deviation; A, accepted; NA, Not accepted.

Table 3.

Interrater Reliability Among the Experts During Delphi Round 3.

Question	Rating Score, Mean (SD)	Fleiss κ Observed Agreement (Consensus %)	Agreement	Acceptance
3	1.86 (0.38)	0.714 (85.71)	Substantial	A
4	1.86 (0.38)	0.714 (85.71)	Substantial	A
6	2 (0)	1 (100)	Perfect	A
13	1.86 (0.38)	0.714 (85.71)	Substantial	A

Abbreviations: SD, standard deviation; A, accepted; NA, Not accepted.

Regression Analysis

Logistic regression confirmed that higher PRATHoT score (achieved on Hering subscale) was significantly associated with achieving pain visual analogue scale responses and therefore improved outcome from the second to the sixth follow-up visits (B = 0.037-0.066; SE = 0.021-0.036; P = .003-.048, 2-tailed; Table 4).

Table 4.

Logistic Regression Analysis of PRATHoT Scores and Pain VAS.

Follow-Up Visit	Coefficient	SE	OR (95% CI)	χ₁ ²	P Value
1	−.003	0.056	0.997 (0.894-1.113)	0.002	.961
2	.037	0.021	1.038 (0.996-1.082)	3.883	.048*
3	.055	0.025	1.057 (1.006-1.110)	7.426	.006*
4	.062	0.027	1.064 (1.009-1.121)	8.598	.003*
5	.050	0.029	1.052 (0.993-1.113)	4.506	.034*
6	.066	0.036	1.068 (0.996-1.146)	6.817	.009*

Abbreviations: PRATHoT, Patients’ Response Assessment Tool after Homeopathic Treatment; VAS, visual analogue scale; OR, odds ratio; CI, confidence interval.

*P < .05 (2-tailed) considered as statistically significant.

Reliability

Cronbach’s α measure of internal consistency or reliability of the PRATHoT was assessed on each visit yielding “good” acceptability of .729 to .814. These values suggested that the tool was internally consistent, satisfying the minimum criteria of .7 (Table 5).

Table 5.

Cronbach’s α Measures of the PRATHoT Over 6 Follow-Up Visits.

Visit	Cronbach’s α^a	Acceptability
All	.794	Good
1	.814	Good
2	.766	Good
3	.774	Good
4	.741	Good
5	.736	Good
6	.729	Good

Abbreviation: PRATHoT, Patients’ Response Assessment Tool after Homeopathic Treatment.

^a A value of .7 ≤ Cronbach’s α < .9 considered as good.

Discriminant Validity

The tool reflected acceptable discriminant validity by comparing repeatedly obtained PRATHoT scores in different visits (F = 10.100; P < .0001, 2-tailed; Table 6).

Table 6.

Discriminant Validity of PRATHoT Over 6 Follow-Up Visits.

Visit	n	Scores, Mean (SD)
1	37	0.03 (0.16)
2	37	0.68 (0.87)
3	37	0.81 (0.77)
4	37	1 (0.96)
5	37	1.19 (0.98)
6	37	1.35 (1.24)
F score		10.100
P value		<.0001*

Abbreviations: PRATHoT, Patients’ Response Assessment Tool after Homeopathic Treatment; SD, standard deviation.

*P < .05 (2-tailed) considered as statistically significant.

Concurrent Validity

Comparison between PRATHoT scores with pain visual analogue scale responses showed acceptable concurrent validity (Pearson’s r = .388-.441; P = .017-.006, 2-tailed; except visit 1; Table 7).

Table 7.

Concurrent Validity Testing: PRATHoT Versus Pain VAS.

Visit	Pearson’s r	P Value
1	−.008	.962
2	.424	.009*
3	.388	.017*
4	.420	.009*
5	.429	.008*
6	.441	.006*

Abbreviations: PRATHoT, Patients’ Response Assessment Tool after Homeopathic Treatment; VAS, visual analogue scale.

*P < .05 (2-tailed) considered as statistically significant.

Sensitivity and Specificity

With the PRATHoT score cutoff ≥ 1, sensitivity % and specificity% of the tool were measured to be 79.7% (95% confidence interval = 71.3-86.5) and 69.2% (95% confidence interval = 59.4-77.9), respectively. The accuracy, area under curve, positive and negative predictive values, and positive and negative likelihood ratios were 74.8%, 0.748 (area under curve = 0.70-0.79 indicates fair test), 74.6% (95% confidence interval = 66.1-81.9), 75% (95% confidence interval = 65.1-83.3), 2.59 (95% confidence interval = 1.91-3.50), and 0.29 (95% confidence interval = 0.20-0.43), respectively.

Discussion and Conclusion

The developed PRATHoT incorporates the heterogeneous subjective judgments of experts and rigorous analytical techniques as well. Experts from diverse backgrounds with respect to experience and practice contributed significantly to the examination of many broad and complex issues; thus, convergence was gained in place of clustering or polarization of results to a single point. The drawbacks of Delphi, such as “bandwagon effect,” “noise,” and “peer pressure for conformity,” could be avoided successfully by protecting the identification of each expert with strict confidentiality and by avoiding personal contacts among experts. Agreement was paid greater importance over consensus to emphasize on differing opinions and divergent responses. We tried to minimize “respondent’s fatigue” by limiting the Delphi rounds to only 4. However, there are some obvious limitations. The number of experts could not be enhanced in spite of repeated attempts, probably due to nonfamiliarity with the Hering’s Law Assessment Tool in clinical practice. It also inevitably raises concerns regarding the degree of expertise incorporated in the forecast. Lack of heterogeneity in the expert group raises inevitable concern regarding the incorporation of some bias into the tool; still the authors think that it would have been more problematic for experts from other background to assess outcomes using PRATHoT in homeopathic practice.

We found the wording and content of the Hering’s Law Assessment Tool problematic and incomplete, and there were invariably insufficiencies regarding the validity and reliability of the tool, as also identified by the authors themselves.¹² The simplified flow diagram¹³ was less complicated and hence better comprehensible than the original one. We have modified the existing Hering’s Law Assessment Tool, and though we believe that the instrument developed in this article is a promising one for assessing responses after administration of a homeopathic drug, this tool should not be used as the complete solution. The “new symptoms appear” section is more meticulous in PRATHoT than in Hering’s Law Assessment Tool. Time restrictions executed in Hering’s Law Assessment Tool were deemed to be arbitrary and not includible in PRATHoT. We also refrained from using the newly coined term “healing crisis” to aid easy comprehension and avoid confusion. Negative correlation was obtained in the first visit after 1 month between the PRATHoT scores and pain visual analogue scale responses because of inability to categorize reactions as Hering responses in the very first follow-up. Reduction or increase in the pain visual analogue scale does not necessarily mean advancement toward Hering subscale and may be dubious when scores cannot be ascribed (Figure 1). Another tool, namely Outcome in Relation to Impact on Daily Living, formerly referred to as the Glasgow Homeopathic Hospital Outcomes Scale, was developed and validated to measure patients’ views of the outcome of their care by asking about change in main complaint and relating this to impact on daily life by rating the outcomes between −4 and +4.²³ PRATHoT, though, does not quantify outcomes on such ratings but gauges outcomes qualitatively and more rigorously.

Though PRATHoT seems promising and competent, it should be interpreted with caution, and further valid, reliable, and convincing assessment tools are welcome to be implemented in larger settings and different clinical conditions. Osteoarthritis itself is a chronic progressive degenerative disease that needs to be followed-up for a longer period and in larger number of patients using PRATHoT to arrive at a definite conclusion; however, the tool seemed to serve the purpose of assessing outcomes after any individualized homeopathic prescription. It should be improved and validated further with due consultation with a number of homeopathic experts. There will always be scope for improvement when a study reported consensus of only 33% of practitioners for a case analysis, interpretation, and selection of the correct medicine.²⁴ It was also not possible to test the convergent validity of PRATHoT as currently there are no other measures available to serve the purpose. The tool was dependent on the interpretative skills of the therapist, ability to distinguish and differentiate symptoms, thus open to interobserver or intraobserver bias, as well as to within-subject variation due to response instability. These biases are being addressed in ongoing clinical trials by the authors by concealing assessments made in previous visits and analyzing the correlation coefficients among multirater observations. The recommendations of this study may be extrapolated in case reports, well-planned case series, observational studies, clinical trials, as well as in everyday practice settings. Further research is already ongoing with an aim to combine PRATHoT and the Outcome in Relation to Impact on Daily Living tool and validating it in clinical settings.

Footnotes

Acknowledgments

The authors are grateful to Dr Shailaja Upadhyaya for her valuable comments and suggestions to improve the tool. The authors are also thankful to the patients for their participation in this study.

Author Contributions

SS, MK: concept, design, literature search, data interpretation, statistical analysis, preparation of the article. JSA: concept, design. GC: concept, design, expert panelist. SMG: expert panelist, clinical study, and data acquisition. SG, TG, AG, SAA, NG: Expert panelist. All the authors edited, reviewed, and approved the final article.

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors received no financial support for the research, authorship, and/or publication of this article.

Ethical Approval

Approval was obtained from the Institutional Ethics Committee of Mahesh Bhattacharyya Homeopathic Medical College & Hospital prior to the initiation of the study. The study is registered in the Clinical Trials Registry, India (Registration No. CTRI/2013/06/0037280).

References

Linde

Melchart

. Randomized controlled trials of individualized homeopathy: a state-of-the-art review. J Altern Complement Med. 1998;4:371–388.

Saha

Koley

Ganguly

Rath

Roy Chowdhury

Hossain

. Developing the criteria for evaluating quality of individualization in homeopathic clinical trial reporting: a preliminary study. J Integr Med. 2014;12:13–19.

Kleijnen

Knipschild

ter Riet

. Clinical trials of homeopathy. BMJ. 1991;302:316–323.

Linde

Clausius

Ramirez

. Are the clinical effects of homeopathy placebo effects? A meta-analysis of placebo-controlled trials. Lancet. 1997;350:834–843.

Hahnemann

CFS

. Organon of Medicine . 5th and 6th editions combined. New Delhi, India: B. Jain Publishers; 2013.

Kent

. Lectures on Homoeopathic Philosophy . Reprinted 4th ed. New Delhi, India: B. Jain Publishers; 1997.

Hering

. Hahnemann’s three rules concerning the rank of symptoms. Hahnemann Monthly. 1865;1:5–12.

Hahnemann

CFS.

The medicine of experience. In: The Lesser Writings of Samuel Hahnemann. New Delhi, India: B. Jain Publishers; 1991.

Vithoulkas

. A New Model for Health and Disease. Blue Jay, CA: Health & Habitat; 1997:150–151.

10.

Vithoulkas

. The Science of Homeopathy. New Delhi, India: B. Jain Publishers; 2000:231–240.

11.

Kent

. Correspondence of organs, and directions of cure. Trans Soc Homoeopathicians. 1911;1:5–12.

12.

Brien

Harrison

Daniels

Lewith

Monitoring improvement in health during homeopathic intervention. Development of an assessment tool based on Hering’s Law of Cure: the Hering’s Law Assessment Tool (HELAT). Homeopathy. 2012;101:28–37.

13.

Harrison

The development and use of Hering’s Law Assessment Tool (HELAT) in clinical trials. Homeopathy Research Institute (HRI) Research Article, Issue 18. http://www.homeoinst.org/sites/default/files/newsletters/HRI_ResearchArticle_18_Winter2012_Harrison_HELAT.pdf. Published 2012. Accessed June 27, 2014.

14.

Yousuf

. Using experts’ opinion through Delphi technique. Practical Assessment Res Eval. 2007;12(4). http://pareonline.net/pdf/v12n4.pdf. Accessed June 27, 2014.

15.

Kalaian

Kasim

. Terminating sequential Delphi survey data collection. Practical Assessment Res Eval. 2012;17(5). http://pareonline.net/pdf/v17n5.pdf. Accessed June 27, 2014.

16.

Hsu

Sandford

. The Delphi technique: making sense of consensus. Practical Assessment Res Eval. 2007;12(10). http://pareonline.net/pdf/v12n10.pdf. Accessed June 27, 2014.

17.

Ferri

Prince

Brayne

. Alzheimer’s Disease International. Global prevalence of dementia: a Delphi consensus study. Lancet. 2005;366:2112–2117.

18.

Peat

Thomas

Duncan

Wood

Hay

Croft

. Clinical classification criteria for knee osteoarthritis: performance in the general population and primary care. Ann Rheum Dis. 2006;65:1363–1367.

19.

Angst

Aeschlimann

Stucki

. Smallest detectable and minimal clinically important differences of rehabilitation intervention with their implications for required sample sizes using WOMAC and SF-36 quality of life measurement instruments in patients with osteoarthritis of the lower extremities. Arthritis Rheum. 2001;45:384–391.

20.

Dantas

Rampes

. Do homeopathic medicines provoke adverse effects? A systematic review. Br Homeopath J. 2000;89 (suppl 1):S35–S38.

21.

Edward

Aronson

. Adverse drug reactions: definitions, diagnosis, and management. Lancet. 2000;356:1255–1259.

22.

Riedl

Casillas

. Adverse drug reactions: types and treatment options. Am Fam Physician. 2003;68:1781–1790.

23.

Reilly

Mercer

Bikker

Harrison

. Outcome related to impact on daily living: preliminary validation of the ORIDL instrument. BMC Health Serv Res. 2007;7:139.

24.

Aqhadiuno

. A study of the inter-observer reliability of paper case analysis. Homeopathy. 2002;91:10–17.