Complex Adaptive Systems Theory and Inter-Rater Reliability: Proposed Answers to Challenging Questions

Overview of CAS Theory Pertaining to TCM

A complex systems approach to research is one that considers how the relationships between parts of system lead to collective behaviors, and further how the system interacts with the environment. ³ These components are in mutual interaction ⁴ ; the relationship between them is one of interdependence; each element of a system can be constrained by, conditioned by, or be dependent on the state of the other units ⁵ ; and together they create an indivisible whole, sometimes called a “whole system.” ^5,* Complex systems are self-organizing and there is no hierarchy of command and control; they are adaptive, having the capacity to change and learn from experience, and they are constantly evolving toward greater complexity. Behaviors of complex systems result in emergent properties not predicted by the properties of the parts, the whole is always more than the sum of its parts; they are nonlinear, a small perturbation anywhere in the system can have disproportionately significant effects; they are multicausal and dynamic. These properties or behaviors emerge only when the parts interact in a wider whole. Systems are adaptive and self-organize around new information introduced to the system.

Many authors have noted the similarities between TCM and CAS-related theories. ^{6 –8} We envision key components in a “patient–system” CAS to be the patient–practitioner relationship including their communication and history, TCM theoretical frameworks, the clinical setting, TCM practitioner training, clinical reasoning, DDx, and treatment. TCM practice adjusts the DDx to reflect clinical change in the patient–system over time. Thus in this model, every patient–practitioner dyad is inherently unique and variation in information collected longitudinally across clinical encounters is an expected part of the patient–system.

A TCM treatment is conceptualized as a perturbation ^† introduced to the patient–system. The patient–system reorganizes around the perturbation to achieve a new dynamic equilibrium or a new dynamically stable state through a nonlinear process over time. In practice, as well as in clinical trials (e.g. Fig. 2 in Ref. ⁹ ), some patients show no improvement for weeks, then experience a sudden improvement (Fig. 1). This nonlinear behavior of the system, seen by the TCM practitioner as additional clinical information, will lead to an update of the DDx.

OPEN IN VIEWER

FIG. 1.

X-axis represents time and Y-axis represents system properties.

Modeling Problems

TCM theory allows for multiple simultaneous and correct DDxs; one symptom can be part of many DDxs, and change (here improved health of the patient) can be elicited by many distinct perturbations/treatments. This is the viewpoint of Jacobson et al. in this issue, ¹⁰ and is in distinct contrast to the assumptions of a singular and solely correct TCM diagnosis presented by Poppelwell et al. and Schnyer et al. (also this issue). In the CAS framework, because all parts of the system are interconnected, there are multiple right ways to start the patient toward healing. Therefore, while different practitioners may start with varying DDxs and treatment plans, all may be able to deliver a healthful outcome.

Also, the present discussion on low IRR does not allow for information that emerges as a result of system parts interacting within the whole in a clinical interaction. For example, in a CAS model of the patient–practitioner interaction, differing information could emerge based on degree of patient trust or, the practitioner's history or training. This raises another important point: in most discussions of low IRR in TCM practice, clinicians are considered interchangeable. MBM training strives to create interchangeability between clinicians and thereby achieve high IRR. This approach dismisses the valuable knowledge that clinicians accumulate over time and how this can effect what emerges in the clinical encounter. Poppelwell et al. mention in this issue CAS terms such as “nonlinearity” and “wholistic,” they are mentioned in isolation, without a theoretical framework such as CASs theory.

Making sense of patients' complicated presentations, organizing that information into a DDx, is a strength of TCM and varies by TCM practitioner.

Measurement Problems

When using IRR as an assessment method (as proposed by Poppelwell and Schnyer in this issue), we are making the following assumptions: first, that a DDx can stand alone, devoid of its context; and second, that there needs to be uniformity of DDx between practitioners. This is a reductionist non-CAS approach.

A CAS approach would attempt to understand every function (such as diagnosis) within its context of the patient–practitioner system. Therefore, any assessment or representational method would need to include both the function and its context (its connections to the whole). This means that the method would need to approach diagnosis from a higher conceptual level.

Let us look at some of the assessment methods used in TCM. The body is considered to be made of five elements: wood, fire, earth, metal, and water. Here “elements” represent patterns of bodily behavior, not the literal substances. For example, “wood” can only be excess in relation to the other elements and is devoid of context if examined alone. This is reflective of the generative and controlling relationships between the elements.

One way to make the IRR a more theoretically valid approach for TCM would be to re-establish the connections between the DDx and the CASs from which it was derived, creating a measurement model that allows for the actions of a CAS model already mentioned.

Evaluation Problems

Poppelwell et al. compare two probabilistic statistical approaches and propose one over the other. ¹¹ Relying on probability distributions is not congruent with TCM. In TCM, diagnosis and treatment are learning processes. Every piece of information adds to the practitioner's understanding and no information is dismissed (i.e., called an outlier). Compare this approach with that of probabilistic statistics and the assumption of central tendency, that the most likely predicted values are the most informative for any particular group of patients. “Statistically relevant” data imply that there are also nonstatistically relevant data. The latter type of data is typically ignored. This is in direct conflict with the TCM approach that includes all information to formulate the DDx.

Once the measurement tool is changed to a context-aware tool (as discussed above), evaluation will also need to change accordingly. We suggest that evaluation could better focus on the relationships between signs, symptoms, and context as they make a DDx rather than the recorded presence or omission of particular signs and symptoms.

Conclusions

TCM is not looking for the “right” DDx, but rather positive clinical outcomes. We have strong meta-analysis evidence that TCM works for pain. ¹² We appreciate the contributions of Poppelwell et al. that move the field forward, and more specifically closer to clinical reality by studying open populations in this issue. ¹³ However, the authors still make the assumption that IRR is related to clinical outcomes. This is not theoretically or ecologically valid for TCM. Some researchers suggest that TCM theory would be without significance in light of insufficient IRR, as noted by Poppelwell et al. We agree with the conclusion of Poppelwell et al. that further evaluation of TCM is necessary but we propose a CAS-informed methodology and strategy for modeling, measurement, and evaluation.

A strength of the TCM framework is its wholistic framing of the patient as an intelligent self-adapting organism influenced by biological, social, and spiritual information. We support conversations to develop specific rigorous applications of CASs to research design and analysis. These endeavors may generate relevant scientific evidence of what TCM practitioners and their patients have experienced for millennia: exceptionally profound improvement in health and well-being and the transformation of lives.