Re: Outcome measurement tools for communication,voice and speech intelligibility in the ICU and their clinimetric properties: A systematic review

Dear Prof Daniel

Zaga CJ, Cigognini B, Vogel AP, Berney S. Journal of the Intensive Care Society. November 2020. doi:10.1177/1751143720963757

Considering the psychometric properties and being aware of the strengths and weaknesses of any outcome measurement tool used in clinical practice and research is certainly important. However, when inspecting the information on one of the reviewed measures referred to in this paper i.e. The Therapy Outcome Measure (TOM) and AAC-TOM we were disappointed that the method underpinning the ‘systematic review’ reported was not appropriate for the task and resulted in factual inaccuracies leading us to doubt the information on the other measures referred to.

The paper aims to summarise issues of reliability and validity along with other psychometric properties of several tools. However, the method of identifying the literature pertaining to these issues was unlikely to elicit the main sources of this information. Outcome measures and formal clinical assessments are usually supported by published manuals detailing the psychometric properties and what may influence these. This source material is often referred to as a key when reporting research using the tools. The search method used by Zaga et al would not have identified this information. It is clear from the reference listed tosupport this paper that the study did not take account all of the information relating to TOM and AAC-TOM which is publicly available.^1,2 The authors did not inspect the books detailing the development and properties of the tool. No reference was made to ‘Therapy Outcome Measures for Rehabilitation Professionals (third edition) Enderby P., John A., 2015 published by J&R or the previous editions.’ The 2015 edition updated the information provided in the previous editions related to its theoretical underpinning, use, reliability and validity.

The authors imply that if a measure does not have criterion validity it is not valid. However, when new measurement tools for making clinical judgements are developed they frequently go through a number of stages from face validity, to content validity, to construct validity. The TOM has been through these stages to establish its validity. There is no comparable gold standard to compare it with which is not unusual with many clinical assessments and outcome measures, but this does not mean that as a tool it is clinically inadequate or cannot be used reliably or with stability as many reliability checks have shown. The Therapy Outcome Measure has been used effectively in a number of clinical and research studies including those with post operative subjects such as those undergoing a laryngectomy. ³

Furthermore, the authors provide an inaccurate summary in the table ‘’inadequate/doubtful’’ of the information provided in the paper you cite: John A, Enderby P. Reliability of speech and language therapists using therapy outcome measures. Int J Lang Commun Disord 2000; 35: 287–302. This is factually incorrect given that even the abstract provides clear information of three studies related to these issues.