Abstract
BACKGROUND AND OBJECTIVE:
Using articles from our systematic review of research on computer text entry by people with physical disabilities [1,2], we assessed the design, conduct, and reporting of text entry studies, in order to strengthen the utility and replicability of future studies.
METHODS:
We analyzed study designs, participant characteristics, study procedures, intervention specification, dependent variables, and data analysis, rating 40 indicators for each of 42 studies.
RESULTS:
The average study fully met 59% of the indicators, ranging from 48% for participant characteristics to 69% for intervention specification. Based on that assessment, we present some recommendations for improving future studies. Key recommendations to consider include: conducting more studies in a service delivery context; reporting information for every participant, including functional scores, experience with the text entry interfaces, and body sites used for typing; providing details regarding the typing task and text entry metrics; and using inferential statistics to inform conclusions.
CONCLUSIONS:
Following these recommendations can help ensure that a study can address its own specific goals as well as support a powerful synthesis of information across studies.
Background
Providing effective computer access interventions to people with physical disabilities often includes the important task of facilitating an easy and productive means of text entry. Computer-based text entry is vital for participation in education, employment, social activities, leisure and instrumental activities of daily living. Assistive technology (AT) control interfaces compensate for an individual’s motor deficit and enable the physical access required for text entry. Assistive technology practitioners may recommend a wide array of alternative interfaces including adapted keyboards, speech recognition systems, on-screen keyboards, one- or two-switch scanning input, brain interfaces, and text enhancement programs [3]. The selection of the most appropriate control interface is dictated by several factors including the client’s text entry needs, neuromuscular or musculoskeletal impairment, the resulting motor deficits, identification of an ideal motor (body) site, and his or her prior experience with interfaces [3, 4].
Research on text entry with alternative access interfaces spans over three decades. The research conducted has collectively included a wide range of subjects, interfaces, training and testing protocols, and interface set ups [1]. Utilization of this wide-ranging and heterogeneous body of research requires access to literature, translation of evidence, and replication of the training and setup in clinical settings.
We have been working to organize the available published evidence in this area, to help answer questions such as: What is the typical typing speed when an individual with a C6 spinal cord injury uses typing splints on a standard keyboard? Our goal is to build foundational knowledge that can inform decision-making for device selection and configuration and provide rough expectations for learning and long-term performance. The purpose of this paper is to assess the methods and reporting of studies that have been conducted in the past, and to provide methodological recommendations to strengthen utility and replicability of studies for the future.
Significant work has been conducted related to best practices for researching the effectiveness of rehabilitation and assistive technology in general [5, 6, 7, 8] and alternative access in particular [9]. Reporting guidelines developed for medical research, such as the CONSORT statement [10], can apply to this domain as well, at least to some extent. The field of human-computer interaction also contains some useful literature regarding methods for text entry research [11, 12]. While this work provides a valuable foundation for conducting research on alternative access for people with disabilities, there are specific issues for the text entry and access domain that are not covered by more general guidelines.
We recently completed a systematic review on computer text entry by people with physical disabilities. As summarized in methods below, we found 39 studies that met all inclusion criteria, dating back to 1986, and used the data to report on the text entry rates associated with different interfaces, diagnoses, and body sites [1, 2]. We also incorporated the data into a database and free search tool, called AT-node [13].
While the systematic review yielded useful results, it also revealed that many additional studies are needed, and that those studies that have been conducted do not always report important details or use consistent methods. Study authors have an opportunity, and in some ways an obligation, to design methods that not only address the specific questions of that individual study, but also allow use of the data for meta-analysis and cumulation across multiple studies. By reporting key details for the methods and results, a study’s contribution can be leveraged for greater impact on the field, enabling more effective replication, evidence synthesis, and knowledge translation.
Objectives
The immediate goal of this paper is to contribute to the conduct of strong and useful studies in the alternative access domain. First, we assess the conduct and reporting of the studies in our systematic review. Then, based on that assessment, as well as our own experience conducting similar studies [A9–A15, A23, A34], we offer recommendations for the design, conduct, and reporting of research on text entry with alternative access interfaces.
In the longer term, we hope to refine these recommendations, with input from other stakeholders, to define a common structure for performing text entry studies. Such a common structure will provide a stronger platform for cumulating results across studies over time and building a strong knowledge base to inform decision-making.
Methods
Articles included
Articles were retrieved using a systematic search of the literature, as detailed in our first systematic review article [1]. Studies were included if individuals with physical impairments were in the study population, typing speed was reported in words per minute or equivalent, and the access interface was available for public use. Studies were excluded if the method of measuring typing speed clearly did not follow appropriate techniques, or if the results reported were anecdotal or unclear. Additionally, because we quantitatively combined results across different studies, studies needed to provide sample size, average, and standard deviation for text entry rate or to give enough information for us to calculate those summary statistics.
Since that initial review, we have added three more recent studies that meet the review criteria, for a total of 42 studies that comprise the article set for this paper. Citations for all studies in the article set are provided at the end of this paper, and the full extracted datasets used in our previous systematic reviews [1, 2] are publicly available at kpronline.com/ter-review [14].
Study areas and indicators
In order to assess the conduct and reporting of the studies in our article set, we defined the following six main areas that apply to a text entry study:
Study Design. Includes the research questions and factors examined in the study, as well as the protocol design used to establish control and gather evidence about the research questions. Participant Characteristics. Defines the people who entered text in the study. Intervention Specification. Specifies the key ingredients that could affect the outcomes [7], i.e., the conditions under study. For text entry studies, relevant ingredients could include the interface(s) used for text entry, the context of use, training provided, etc. Procedure and Typing Task. Defines exactly what subjects were asked to do in the study, including the typing (text entry) task. Measuring Dependent Variables. Specifies the exact metrics used to assess text entry performance. Data Analysis and Presentation. Defines how statistical tests and other results were used to address the research questions.
We defined 40 indicators across these areas, then assessed each of the 42 studies in our article set with respect to these indicators. Most of the indicators rated how much information about that indicator was provided in the study manuscript, as defined in Table 1. The Typing Task, Dependent Variables, and Data Analysis areas used a 0–2 rating scale for each indicator, to indicate no, partial, or full information, respectively. The Participant Characteristics and Intervention Specification indicators also included a top rating of 3 to indicate providing all necessary information for each individual subject in the study; for indicators in those areas, a rating of 2 denoted providing full information, but only as a summary for the group as a whole (e.g., average age for the subject group).
Definitions for indicators that rate the level of information provided
In addition to the informational indicators described above, other indicators noted specific protocol decisions, such as whether non-disabled subjects were used, what type of text entry task was used, etc. These indicators had values that were either predefined categories (such as study designs) or Yes/No/Unspecified.
To refine the rating definitions and develop consistent guidelines for the indicators, both authors independently rated all indicators for three studies. We compared all ratings, came to consensus on any initial disagreements, and clarified the rating guidelines. We then rated five more studies independently, and measured the inter-rater agreement at 91%. This was judged to be sufficient for the purposes of using these indicators to get a basic sense of how these 42 studies conducted and reported their work. The first author then completed the ratings for the remaining articles, consulting with the second author for the few ratings that were not immediately clear.
Summary scores
To summarize how fully each study met the indicators in each area, we computed the percent of indicators that received full marks for that area, then took the average of those percentages across studies as a summary score. (We did not compute a summary score for the Study Design area, because its indicators are descriptive and generally do not have a “better” or “worse” connotation.) For informational indicators, “full marks” was defined as the maximum scale value (either 2 or 3, depending on the indicator). For protocol decision indicators, the preferable rating was used (e.g., a rating of Yes for whether the study used unique text for each measurement); the few indicators that had no obvious preferable rating were not included in the summary score.
Analysis of trends over time
Given the 30-year span of these studies, we examined any differences or trends in the design and conduct of these studies over time. We split the article set into groups for each 5-year period since 1986, then averaged the summary scores across studies within each time period.
Design types used for the primary research question
Design types used for the primary research question
This section presents the indicator ratings for each of the six study areas we defined. It also reports the overall summary scores and examines whether there are any trends in those scores across time.
Study design
The overwhelming majority of studies (92%) examined some aspect of interface as their primary factor, whether it was comparing two or more interface types (
The most common study design for examining the primary factor was some form of single subject design, used by 50% of studies (see Table 2). Eleven used a single subject series, involving a single subject design with multiple subjects and multiple data points per condition. Seven studies used a single subject design with one subject and multiple data points per condition. Those 18 single subject studies were generally well-controlled. An additional three studies, comprising 7% of the article set, used a weak form of single subject design, with only one data point per condition.
Thirty-eight percent of studies used a group controlled design, with 9 using random assignment and 7 using non-random assignment. (Although “RCTs” often use a between-subjects design, all of these randomized controlled studies used within-subjects design, using random assignment to the sequence of conditions.) Another 10% of studies combined both single subject series with a non-randomized controlled group design [A10–A13]. The advantage of this combined approach is that effects for each subject can be examined individually, and any common effects across all subjects can be examined on a group basis.
Ratings for indicators related to participant characteristics, showing percent of studies receiving each rating
Ratings for indicators related to participant characteristics, showing percent of studies receiving each rating
Ratings for indicators related to intervention specification, showing percent of studies receiving each rating
Ratings for indicators related to procedure & typing task, showing percent of studies receiving each rating
In the 42 studies we reviewed, 100% included subjects with disabilities, by definition of our review inclusion criteria. Non-disabled subjects were also included in 21% of studies (see Table 3). The majority of studies (64%) reported the age and gender of each subject. However, 43% provided no information on additional demographics including education, employment, or socioeconomic status with 36% providing partial information on these characteristics. The vast majority of studies (90%) only provided partial information on each subject’s experience and preferences with computers and alternative control interfaces. Seventy-four percent of studies reported diagnostic information on individual participants, and 64% provided descriptions of their functional ability relevant to computer access. However, 86% of the studies used no established metric or tool to report a functional score for the participants (with only 10% doing so).
Intervention specification
Seventy-one percent of studies provided full information on specific interfaces used by each participant with 60% providing details on specific body sites (see Table 4). For participant experience with the tested interface, most studies (67%) provided specific details on the amount of in-study experience. However, only 38% reported pre-study participant experience on the relevant interface. About three-quarters of the studies (72%) provided replicable information on the training protocol. Almost 90% of the studies reflected a real-world scenario, where at least one interface was identical or very similar to something the subject already used or plans to use in real life. Only 7% were conducted in a service delivery context, with the vast majority (93%) representing a research context.
Procedure and typing task
Almost every study (98%) described what participants were required to do in the protocol (see Table 5). However, only 19% of studies provided information on treatment fidelity. Transcription was used as the typing task in 93% of the studies. Almost half (48%) did not explicitly state how subjects were instructed to handle typing errors during the typing task. Only 45% of studies with multiple typing measurements explicitly reported using unique text for each measurement, and a similar portion of studies did not specify the tool used for presentation of the typing task.
Measuring dependent variables
Operational definition of text entry rate (TER) was provided in 69% of the studies, and 55% noted the tool(s) used to measure TER, ranging from manual timing to specific text entry software (see Table 6). Only 29% explicitly stated whether TER included errors in typing. The large majority of studies (74%) did not report any measure of validity or reliability for the dependent variables.
Ratings for indicators related to measuring dependent variables, showing percent of studies receiving each rating
Ratings for indicators related to measuring dependent variables, showing percent of studies receiving each rating
Ratings for indicators related to data analysis and presentation, showing percent of studies receiving each rating
‘Mult vals’
Summary scores, showing percent of indicators that were fully met in each area, averaged across studies.
Many studies (83%) reported TER for each participant, with 41% depicting it in numerical and graphical format (see Table 7). Descriptive statistics for subject groups (N, mean, and standard deviation) were available in every study (by definition of our inclusion criteria), although calculations were required to compute these values in 71% of studies. To interpret the main effect of interest, only about half the studies (52%) used a statistical test; the rest used primarily visual analysis, either of multiple values across time (29%) or single values for each condition (19%). For studies that had possible interactions between factors, about 1/3 used a statistical test to check for interactions.
Summary scores
Figure 1 illustrates the summary scores for each area. Across the five areas, the average study fully met 59% of the indicators, ranging from 48% for Participant Characteristics to 69% for Intervention Specification. The complete indicators and ratings for each study are available at a publicly available Google Sheet [15].
Trends over time
There appears to be a trend toward more group designs over time, which is generally a positive trend (see Fig. 2). However, we observed very little change in the overall indicator ratings across time. The overall summary score averaged 53% for studies published between 1986 to 1995, and 56% for studies published since 2011, with very little variation during the intervening years. The summary scores for each area also remained quite steady across this time period, with the possible exception of Intervention Specification, which actually showed a steady decrease across time (from about 80% to 60%). While the uneven sampling across time periods limits the strength of this evidence, it does suggest that there remains a need for recommendations that can improve study protocols and reporting.
Number of studies with each type of design, since 1986.
The results presented above provide some insights into how previous studies have addressed the challenges of conducting and reporting research on text entry by people with physical impairments using alternative access interfaces. Study designs were generally appropriate and established a reasonable level of control, most often by using a within-subjects design in which subjects are their own controls. This was true for both group designs as well as single-subject designs.
As noted previously, while we did analyze and report the study design, we did not rate the type of design as an indicator within the summary score for each study. In contrast to approaches like the OCEBM Levels of Evidence [16], our goal was not to critique and rank studies by their design. Rather, our indicators and scoring apply to all designs, which means that, for example, a single subject design could score equally as well as a randomized controlled group study.
The greatest opportunities for improvement lie mostly in the specific details of implementing and reporting a study. These opportunities include the need for more:
Studies conducted within a service delivery context. Only 7% of these studies were so conducted. Reporting participant characteristics on an individual basis. Depending on the characteristic, 5–74% of studies did this. Reporting of functional scores for participants. Ten percent of studies reported these. Details on exact body sites used for typing, provided by 60% of studies. Details on pre-study and in-study experience, provided by 38 and 67% of studies, respectively. Clarity on error handling and text characteristics in the typing task, provided by 38 and 74% of studies, respectively. Details on text entry metrics, fully provided by about half of studies (49% summary score for Measuring Dependent Variables). Greater user of inferential statistics. About half (52%) of studies used statistical tests when analyzing the main effect in the study.
These and other recommendations for future research are presented and discussed more fully in the Recommendations section below.
The inclusion and exclusion criteria for this article set influenced the results to some extent. For example, all studies included subjects with disabilities, by definition, so this analysis cannot offer information about how many studies only included non-disabled participants. However, our first systematic review [1] noted that 33 studies were excluded from the article set because they included only non-disabled subjects. Additionally, studies covering aspects of alternative access other than text entry rate are not addressed in this analysis.
Checklist of recommendations for studies on text entry by people with disabilities.
Our scoring rubric was ad hoc and not intended to be a standardized, highly precise metric of study quality. We defined our own rating method because we needed to look at the text entry studies in a more detailed way than is supported by more general methods of evidence appraisal [16] or study reporting [10]. This method, while homemade, proved to have reasonable inter-rater reliability and seems to have reasonably served its purpose of allowing us to track key aspects of the studies in this article set. The opportunities for improvement identified in our analysis of 42 studies support the need for recommendations for strengthening text entry studies.
The rating scale itself did reward studies for providing information for each individual participant, particularly for the Intervention Specification, Participant Characteristics, and the Data Analysis areas. It is reasonable to question whether this biased the scoring against studies that had relatively large participant groups, since it could be more difficult for them to report on a subject-by-subject basis. There were nine studies that had more than 10 subjects in a group [A1, A7, A10, A11, A15, A29–31, A38]. Only three of these (all by the same author) did not report characteristics and results for each individual participant, and the summary scores for these nine studies were the same as the overall average across all studies. This suggests that the indicators for participant-level information did not strongly drive the summary scores. The possibility that participant-level data reporting may not fit in space-limited journal articles is a real one, however, and suggestions for handling this are presented below.
Introduction to recommendations
The following recommendations include specific suggestions for each of the key areas, to enhance the scientific rigor of text entry studies and support future meta-analysis by other researchers. These are listed in Fig. 3 and described in more detail below. These recommendations are suggestive and meant to be implemented to the best extent possible, realizing that particular study contexts may require various compromises.
Although many of these recommendations may seem methodologically obvious, many studies in the past (including our own) have not followed all of them. This does not mean that these studies did not provide useful evidence to the field. Indeed, as already noted, almost all of the studies used appropriate designs that allowed them to meaningfully address their research questions. It does mean, however, that there was scope for improvement to contribute even more. We acknowledge the challenges involved, given constraints on time and other resources, as well as page limits in many journals. However, with only a handful of studies per year in this area, it is critical to leverage data as fully as possible, and a key way of doing that is to support the use of study data by other researchers. To work around page limits, authors can provide full protocols and data sets using links to publicly shared documents (e.g., via Google Drive) or a data enclave [17], with the caveat that provision of such data must be in accordance with human subjects confidentiality guidelines.
These recommendations are not an exhaustive list touching every aspect of a research study, but are meant to complement research fundamentals and other listings such as the CONSORT statement [10]. While some of the recommendations readily apply to a range of assistive technology areas, many are specific to the domain of text entry by people with physical impairments using alternative access interfaces. This is particularly true for the areas of Intervention Specification, Procedure and Typing Task, and Measuring Dependent Variables. Those recommendations that are not uniquely applicable to the text entry domain still have direct and specific consequences for text entry studies, and are included here to provide a relatively complete reference list covering the key issues.
Study design
Use a randomized controlled group design where possible, keeping in mind that a within-subjects design can be randomized by randomly assigning the order of conditions. Single subject designs can be appropriate, particularly when multiple data points are collected per phase; consider the possibility of combining single subjects over time for future group analysis. Whatever design is chosen, establish controls for sources of bias including variation between subjects, practice effects, prior experience, and other confounding variables.
Participant characteristics
We strongly recommend including target users as participants, i.e., people with physical impairments who already use the interface or who fit the characteristics of the interface’s target user group. Users without impairments may have a role in some studies, such as providing a convenient sampling pool for early stage, proof of concept work, or serving as a complementary group in a study that also includes people with impairments. But unless people who are potential users of the interfaces in terms of their diagnosis and motor deficits are also included, the external validity of the study is likely to be limited.
Reporting complete participant characteristics allows for stronger interpretation of the findings and supports meta-analysis based on participant characteristics. If available, provide information on key participant demographics, such as age and gender, as well as education, socioeconomic status, and employment. Also include information about each participant’s disabling condition. This is commonly presented as the individual’s medical diagnosis, which is useful but can be insufficient to understand the degree of impairment in diagnoses such as cerebral palsy (CP). Reporting functional measures such as the Gross Motor Function Classification System (for people with CP) can be challenging but can provide a much stronger basis for identifying and grouping similar participants, both within and between studies.
Intervention specification
Providing a full description of the interface used for text entry, the user’s interaction with the interface, and the context of its use enhances our ability as a field to understand the influence of various factors on user performance. The intervention may be specified for the participant group as a whole when homogeneous across the entire group. Otherwise, report for each participant. Some specific aspects to consider include:
Interface used: Report the general type of interface, specific setup (e.g., use of word prediction, letter layout, all configuration settings), and availability of the interface (prototype, commercially available). Body site: An exact description of how the participant interacted with the interface helps promote knowledge translation and meta-analysis. Images or visual representation of the setup (without revealing participant identity) are also helpful. Participant experience: Consider reporting the amount of experience with the interface prior to the study (in months or years of regular use), as well as during the study period itself (hours of use during study sessions and estimated hours of use outside of study sessions, where applicable). Training: Details about the training protocol, including length of time, frequency, duration, goals set, etc., support replicability and application of the findings. Other service delivery features: For studies conducted in a service delivery context, researchers could provide information about the practice setting, practitioner knowledge and skills, assessment and recommendation process, participant context, etc.
Ensuring that each participant received the same instructions and reporting the procedure in sufficient detail to be replicable helps demonstrate treatment fidelity [7]. Consider using a transcription task unless the goals of the study make that inappropriate [9, 11]. A composition task can also be included if that is of interest. Instructions and subsequent report on the study should be explicit about how errors are handled; it is usually appropriate to allow participants to correct errors that they notice during text entry. The length of the task can be defined by the length of text (e.g., 3 sentences) or amount of time (e.g., 5 minutes). Either way, it should represent at least a few minutes’ worth of text entry. It is advisable that the text be representative of basic English (or the language used in the study), and if participants enter text in multiple sessions over time, the text in each session should be similar in characteristics (word length, reading level) but not identical, in order to mitigate practice effects.
Measuring dependent variables
In a text entry study, the most commonly reported dependent variables are text entry rate (TER) and error rate. Clear operational definitions of each of these variables (and any others that are measured) are necessary.
For TER, the measure should be correct characters per minute. It can be useful to report TER in correct words per minute in English by dividing by 5 [11]. The key point is that only characters that are correct at the end of the task count toward TER, and this needs to be explicitly reported. In our systematic review of TER [1, 2], we did include results based on occasionally vague descriptions of how TER was measured. It would have provided more confidence if all studies had explicitly specified how errors were considered in the TER measurement.
Error rate reflects the number of incorrect characters entered during the task and can seem intuitive to measure. But the specifics of measuring error rate can be tricky [11, 12]. If circumstances require a manual measure of error rate, it can be approximated by counting the incorrect characters remaining in the transcription text divided by the total number of characters entered.
An easier and more accurate approach to measuring text entry performance is to use a software tool designed for that purpose. These tools help ensure the validity and replicability of the text entry task by providing consistent task presentation, data collection, and data analysis. MacKenzie and Wobbrock each have made available the software they use in their text entry research [18, 19]. Compass software for access assessment can also be used, and this tool provides additional flexibility in test presentation and setup [20]. The usability, measurement accuracy, and psychometric validity and reliability of Compass has also been demonstrated [21, 22, 23].
Data analysis and presentation
We urge authors to present enough information to allow future researchers to include the data in future meta-analyses or systematic reviews. The most flexible approach is to report results for each individual participant, including key characteristics of that participant (e.g., diagnosis, body site, demographics, functional measures, interface setup). If only group-level data can be reported, the following descriptive statistics must be included to allow for pooling text entry rate across studies: number of participants, average TER, and standard deviation of TER. The range for the group, as well as outliers or quartile ranges, can also be useful.
To the extent possible, use inferential statistics to support conclusions about the research questions. Visual analysis is a powerful and legitimate approach. However, combining it with inferential statistics provides more rigorous evidence by assessing the likelihood that visually convincing data might be due to chance.
Exemplar studies
It can be useful to examine articles that represent some best practices relative to these recommendations. Tam’s 2002 study on word prediction fully meets 78% of our indicators [A37]. Strengths of this study include: comprehensive reporting of functional scores, complete details of body sites, assessment of metric reliability, full details on the methods for computing text entry rate and error rate, and the combined use of group inferential statistics with single-subject visual analysis. Sears’ 2001 study on a prototype speech recognition system provides full details on the typing task, including the appropriate use of unique text [A33]. Finally, for good examples of studies within a service delivery context, see the studies by Blain et al. [A2] and Chan et al. [A3].
Conclusion
These recommendations can help address the clear need to develop and implement standard and replicable methods with reliable outcome tools for examining text entry performance. Incorporating these recommendations helps ensure a well-designed study, one that can address its own goals in a valid and robust way as well as support a powerful synthesis of information across studies. These recommendations can also be used by practitioners wishing to collect text entry data with their clients for clinical purposes. Text entry studies should strive to use appropriate and consistent methods in order to facilitate effective replication, evidence synthesis, and knowledge translation. This helps ensure that the considerable time and effort put in to conducting the study will provide lasting impact to the field.
Footnotes
Conflict of interest
None to report.
Appendix
Citations for the 42 studies in the article set
These are the studies analyzed here, with inclusion criteria that they reported typing speed, employed subjects with physical impairments, and used at least one access interface that is readily available for consumer use.
A1. Alcantud F, Dolz I, Gaya C, Martín M. The voice recognition system as a way of accessing the computer for people with physical standards as usual. Technology & Disability. 2006 Aug; 18(3): 89–97.
A2. Blain S, McKeever P, Chau T. Bedside computer access for an individual with severe and multiple disabilities: a case study. Disabil Rehabil Assist Technol. 2010; 5(5): 359–69.
A3. Chan J, Falk TH, Teachman G, Morin-McKee J, Chau T. Evaluation of a non-invasive vocal cord vibration switch as an alternative access pathway for an individual with hypotonic cerebral palsy – a case study. Disabil Rehabil Assist Technol. 2010 Jan; 5(1): 69–78.
A4. Chiapparino C, Stasolla F, de Pace C, Lancioni GE. A touch pad and a scanning keyboard emulator to facilitate writing by a woman with extensive motor disability. Life Span and Disability. 2011 Jan; 14(1): 45–54.
A5. DeVries RC, Deitz J, Anson D. A Comparison of Two Computer Access Systems for Functional Text Entry. American Journal of Occupational Therapy. 1998 Sep; 52(8): 656–65.
A6. Garrett JT. Using speech recognition software to increase writing fluency for individuals with physical disabilities. [US]: ProQuest Information & Learning; 2008.
A7. Guy V, Soriani M-H, Bruno M, Papadopoulo T, Desnuelle C, Clerc M. Brain computer interface with the P300 speller: Usability for disabled people with amyotrophic lateral sclerosis. Annals of Physical and Rehabilitation Medicine [Internet]. 2017 Oct 10 [cited 2017 Dec 12]; Available from:
A8. Hsieh M, Luo C. Morse code typing training of an adolescent with cerebral palsy using microcomputer technology: case study. AAC: Augmentative & Alternative Communication. 1999 Dec; 15(4): 216–21.
A9. Koester HH, Levine SP. Effect of a word prediction feature on user performance. AAC: Augmentative & Alternative Communication. 1996 Sep; 12(3): 155–68.
A10. Koester HH, Lopresti E, Simpson RC. Toward automatic adjustment of keyboard settings for people with physical impairments. Disability and Rehabilitation: Assistive Technology. 2007 Jan 1; 2(5): 261–74.
A11. Koester HH, Mankowski J. Automatic adjustment of keyboard settings can enhance typing. Assistive Technology. 2015 Jul; 27(3): 136–46.
A12. Koester HH, Simpson RC. Method for enhancing text entry rate with single-switch scanning. J Rehabil Res Dev. 2014; 51(6): 995–1012.
A13. Koester HH, Simpson R. Effectiveness and usability of Scanning Wizard software: a tool for enhancing switch scanning. Disability and Rehabilitation: Assistive Technology. 2017; ePub ahead of print.
A14. Koester HH. Quantitative Modeling in Augmentative Communication – A Case Study. In: RESNA 1990 Annual Conference [Internet]. 1990 [cited 2016 Feb 10]. Available from:
A15. Koester HH. Usage, performance, and satisfaction outcomes for experienced users of automatic speech recognition. Journal of Rehabilitation Research & Development. 2004 Oct 9; 41(5): 739–54.
A16. Lancioni G, O’Reilly M, Singh N, Green V, Chiapparino C, De Pace C, et al. Use of microswitch technology and a keyboard emulator to support literacy performance of persons with extensive neuro-motor disabilities. Developmental Neurorehabilitation. 2010 Aug; 13(4): 248-257 10p.
A17. Lancioni GE, Singh NN, O’Reilly MF, Green VA, Ferlisi G, Ferrarese G, et al. A man with amyotrophic lateral sclerosis uses a mouth pressure microswitch to operate a text messaging system with a word prediction function. Developmental Neurorehabilitation. 2013 Oct; 16(5): 315-320 6p.
A18. Lancioni GE, Singh NN, O’Reilly MF, Sigafoos J, Green V, Chiapparino C, et al. A voice-detecting sensor and a scanning keyboard emulator to support word writing by two boys with extensive motor disabilities. Res Dev Disabil. 2009 Apr; 30(2): 203–9.
A19. Lancioni GE, Singh NN, O’Reilly MF, Sigafoos J, Green V, Oliva D, et al. Microswitch and keyboard-emulator technology to facilitate the writing performance of persons with extensive motor disabilities. Research in Developmental Disabilities. 2011 Mar; 32(2): 576–82.
A20. Lau C, O’Leary S. Comparison of Computer Interface Devices for Persons With Severe Physical Disabilities. American Journal of Occupational Therapy. 1993 Nov; 47(11): 1022–30.
A21. Levine S, Gauger J, Bowers L, Khan K. A comparison of Mouthstick and Morse code text inputs. Augmentative and Alternative Communication. 1986; 2(2): 51–5.
A22. Manaris B, Harkreader A. SUITEKeys: A Speech Understanding Interface for the Motor-control Challenged. In: Proceedings of the Third International ACM Conference on Assistive Technologies [Internet]. New York, NY, USA: ACM; 1998 [cited 2015 Dec 8]. p. 108–115. (Assets ’98). Available from:
A23. Mankowski R, Simpson RC, Koester HH. Validating a model of row-column scanning. Disability and Rehabilitation: Assistive Technology. 2013 Jul 1; 8(4): 321–9.
A24. Mezei P. Effects of word prediction on writing fluency for students with physical disabilities. Doctoral dissertation, Georgia State University. [Internet]. 2009 [cited 2015 Dec 15]. Available from:
A25. Mezei P. Mezei Evaluating Word Prediction Software for Students with Physical Disabilities.pdf. Physical Disabilities: Education and Related Services, v23 n2 p93-113 Spr 2005; 2005.
A26. Nguyen T, Le D-A, Dang T-V, Nguyen T-A. The UITK: Towards the Designing a Ubiquitous Soft Keyboard for Disabled People. In: Proceedings of the 14th International Conference on Information Integration and Web-based Applications & Services [Internet]. New York, NY, USA: ACM; 2012 [cited 2015 Dec 8]. p. 399–402. (IIWAS ’12). Available from:
A27. Pires G, Nunes U, Castelo-Branco M. Comparison of a row-column speller vs. a novel lateral single-character speller: Assessment of BCI for severe motor disabled patients. Clinical Neurophysiology. 2012 Jun; 123(6): 1168–81.
A28. Pouplin S, Robertson J, Antoine J-Y, Blanchet A, Kahloun JL, Volle P, et al. Effect of dynamic keyboard and word-prediction systems on text input speed in persons with functional tetraplegia. Journal of Rehabilitation Research and Development. 2014; 51(3): 467–79.
A29. Pouplin S, Roche N, Antoine J-Y, Vaugier I, Pottier S, Figere M, et al. The effect of word prediction settings (frequency of use) on text input speed in persons with cervical spinal cord injury: a prospective study. Disabil Rehabil. 2016 Jun 13; 1–6.
A30. Pouplin S, Roche N, Vaugier I, Cabanilles S, Hugeron C, Bensmail D. Text input speed in persons with cervical spinal cord injury. Spinal Cord. 2015 Sep 15.
A31. Pouplin S, Roche N, Vaugier I, Jacob A, Figere M, Pottier S, et al. Influence of the Number of Predicted Words on Text Input Speed in Participants With Cervical Spinal Cord Injury. Archives of Physical Medicine and Rehabilitation [Internet]. 2016 [cited 2015 Dec 8]; Available from:
A32. Roark B, Fried-Oken M, Gibbons C. Huffman and Linear Scanning Methods with Statistical Language Models. Augmentative and Alternative Communication. 2015; 31(1): 37–50.
A33. Sears A, Karat C-M, Oseitutu K, Karimullah A, Feng J. Productivity, satisfaction, and interaction strategies of individuals with spinal cord injuries and traditional users interacting with speech recognition software. Universal Access in the information Society. 2001; 1(1): 4–15.
A34. Simpson R, Koester H, LoPresti E. Selecting an appropriate scan rate: the “.65 rule.” Assistive Technology. 2007 Summer; 19(2): 51–60.
A35. Smith JM, Simpson RC. Analyzing performance with computer access technology using unconstrained text entry protocol. Journal of Rehabilitation Research & Development. 2009 Aug; 46(7): 1059-1068 10p.
A36. Smith RO, Christiaansen R, Borden B, Lindberg D, Gunderson J, Vanderheiden G. Effectiveness of a writing system using a computerized long-range optical pointer and 10-branch abbreviation expansion. J Rehabil Res Dev. 1989; 26(1): 51–62.
A37. Tam C, Reid D, Naumann S, O’Keefe B. Effects of word prediction and location of word prediction list on text entry with children with spina bifida and hydrocephalus. AAC: Augmentative & Alternative Communication. 2002 Sep; 18(3): 147–62.
A38. Tam C, Wells D. Evaluating the benefits of displaying word prediction lists on a personal digital assistant at the keyboard level. Assistive Technology. 2009 Fall; 21(3): 105–14.
A39. Tumlin J, Heller KW. Using Word Prediction Software to Increase Typing Fluency with Students with Physical Disabilities. Journal of Special Education Technology. 2004; 19(3): 5–14.
A40. Wobbrock JO, Myers BA, Aung HH, LoPresti EF. Text Entry from Power Wheelchairs: Edgewrite for Joysticks and Touchpads. In: Proceedings of the 6th International ACM SIGACCESS Conference on Computers and Accessibility [Internet]. New York, NY, USA: ACM; 2004 [cited 2015 Dec 8]. p. 110–117. (Assets ’04). Available from:
A41. Wobbrock JO, Myers BA. Enabling devices, empowering people: The design and evaluation of Trackball EdgeWrite. Disability and Rehabilitation: Assistive Technology. 2008 Jan 1; 3(1–2): 35–56.
A42. Wobbrock JO, Myers BA. From Letters to Words: Efficient Stroke-based Word Completion for Trackball Text Entry. In: Proceedings of the 8th International ACM SIGACCESS Conference on Computers and Accessibility [Internet]. New York, NY, USA: ACM; 2006 [cited 2015 Dec 8]. p. 2–9. (Assets ’06). Available from: