Doubts About the Clinical Effectiveness of Community Treatment Orders

CTOs’ Effectiveness is a Clinical and a Legal Concern

This subject—the evidence of clinical effectiveness of CTOs—is the focus of this In Review section. It is an important subject for clinicians who possess the power under mental health legislation to put patients on CTOs. It is also vital from a legal point of view, given the impact CTOs can have on a person’s right to autonomy, protected by the Canadian Charter of Rights and Freedoms. ³ Being placed on a CTO—and required to accept continuing medication—can obviously affect a person’s right to make fundamental choices about the conduct of their life. The question then arises, under the Charter, whether that limit on rights is demonstrably justified. ⁴

Under the famous Oakes test, set by the Supreme Court of Canada, ⁵ to justify placing limits on Charter rights, the state must show that a rational connection exists between the means it proposes to use to limit people’s rights and the ends it seeks to achieve. In this situation, establishing that connection would seem to require the state to show that some clear link exists between the use of CTOs (the means) and improvements in people’s mental health (surely the main end sought). CTOs might have other benefits, such as reducing the threat of harm posed to others, or preventing unnecessary arrests, homelessness, victimization, and so on. But surely those are secondary to the primary aim—securing clinical benefits for patients.

If it could be shown, then, by convincing evidence that CTOs were effective in improving people’s mental health that could justify both the clinical decision to put a person on a CTO and the impact on their rights, satisfying both therapeutic and Charter concerns.

The 2 Reviews of the Outcome Research

The articles by Steve Kisely ⁶ and Jorun Rugkåsa, ⁷ focus on this question of CTOs’ effectiveness. First, Kisely reviews the small group of studies conducted in Canada—mainly in Ontario and Quebec—that try to evaluate CTO regimes operating under the authority of provincial mental health legislation. He notes that only a few such studies have been conducted. They have small numbers of participants. No randomized controlled trial (RCT) has been conducted in Canada, and only one study uses a control group. A narrow range of methods has also been employed, usually mirror image designs that scrutinize the position of patients before and after they go on a CTO, using patients as their own control subjects. These studies suggest that CTOs in Canada decrease admissions to hospital, increase people’s use of crisis intervention and outpatient services, and promote access to supportive housing and general medical care.

As Kisely observes, however, the before and after method used in most of these Canadian studies is particularly prone to bias. First, it does not control for the increase in services that may be offered to people when they are placed on CTOs. The report of one study from Ottawa says, for instance: “Patients being issued CTOs were prioritized” for case management. ^{8, p 418} Delivering more services may certainly be part of the reason for putting people on CTOs, but is it then the increase in services offered, rather than the element of compulsion, that produces any improvement in their condition? This problem can be avoided only by following a research method that ensures the same level of services is offered to patients before and after they are placed on CTOs.

The second problem with before and after studies is regression to the mean. This arises from the prospect that people nominated for CTOs will be recovering from an acute episode of illness at the time. That episode may have produced their recent admission to hospital, and may be the reason why they are nominated for compulsory outpatient care. Then, when they are placed on a CTO, their condition may stabilize. But perhaps it would have done so anyway, with or without the CTO, due to the natural fluctuations in the course of their condition (that is, due to regression to the mean). The positive outcomes observed afterwards may not therefore be due to the CTO.

Unfortunately, these 2 methodological problems throw a cloud over much of the Canadian outcome research. The same phenomena may explain why clinicians who use CTOs are often convinced of their effectiveness. They observe patients improving when they are placed on a CTO. That improvement may not, however, have been caused by the CTO.

Rugkåsa reviews research from other countries, particularly the United States, Australia, and the United Kingdom. A wider range of studies has been conducted there, some using control groups, including RCTs. This research has generally produced less positive results than the Canadian studies. The substantial group of nonrandomized studies conducted internationally has produced mixed results, for instance, some showing patient benefits, some not. Rugkåsa emphasizes particularly the results of the 3 RCTs of CTO regimes, conducted in North Carolina, ⁹ New York, ¹⁰ and England. ¹¹ She emphasizes their results, firstly, because the methods of the RCT have the capacity to reduce the sources of bias that may influence the results, and, secondly, because all 3 of these RCTs reached the same finding on their primary outcome measure—that placing patients on CTOs did not reduce their rate of admission to hospital in the follow-up period.

In addition, Rugkåsa points to the conclusions of no effect reached in systematic reviews of CTO outcome research, particularly meta-analyses that pool the data from the RCTs. ¹² Ultimately she reaches the sobering conclusion that: “There is no evidence of patient benefit from current CTO outcome studies.” ^{7, p x} A similar conclusion was reached by Churchill et al in their review of the research at the Institute of Psychiatry in London ¹³ (though that review, commissioned by the Department of Health, did not deter the UK Parliament, which enacted a new CTO regime for England and Wales). ¹⁴

Time for the CPA to Review its Position

The stance taken by the CPA in its Position Paper on CTOs, issued in 2003, and revised in 2009, therefore looks increasingly out of step with the results of the international outcome research. The CPA’s statement that, “Most studies found a statistically significant reduction in the frequency of hospitalization” when patients were placed on CTOs is not consistent with the international evidence, particularly from rigorous controlled studies. The same is true of the CPA’s statement that, “There is a consistent finding that patients on [CTOs] are more likely to follow up with mental health services.” ^{2, p 3} In fact, the international studies show mixed results on this measure. Maughan et al’s recent review concluded: “there is now a strong level of evidence that CTOs have no significant effect on hospitalisation outcomes or community service use.” ^{15, p 12}

In the Charter challenge to the Ontario CTO regime, the court said: “where the evidence is inconclusive and the efficaciousness of a legislative remedy is difficult to measure, it is for the legislature and not the courts to decide upon the appropriate course of action….” (paragraph 91). ¹⁶ However, in recent years the evidence has become more conclusive, against the clinical efficacy of CTOs. One wonders how long CTO regimes can therefore survive Charter scrutiny in Canada.

There is no particular reason to believe that specific features of the Canadian CTO schemes are more likely to make them effective than those in other countries. The Canadian schemes contain no extra powers, for example, that make them more enforceable, and the descriptive studies indicate that patients placed on CTOs in Canada generally have the same characteristics as patients placed on CTOs elsewhere. ^17,18 Nor is there any particular reason to believe that the Canadian outcome studies are more likely to give an accurate indication of CTOs’ effectiveness than the much larger body of research conducted internationally, some of which uses methods less prone to bias and methods that are considered more convincing under the usual canons of evidence-based medicine. Perhaps the CPA should therefore think of reconvening its expert committee, reviewing the evidence afresh, and revising its position.