Very little research exists on the dependability of handheld computers used in public school classrooms. This study addresses four dependability criteria—reliability, maintainability, availability, and safety—to evaluate a data collection tool on a handheld computer. Data were collected from five sources: (1) time-use estimations by 19 special education teachers using handheld computers, (2) observations of time use from eight external data collectors, (3) teacher interviews, (4) technical reports, and (5) teacher satisfaction surveys. Results indicate that this data collection tool satisfied all four dependability criteria. The study also confirms that dependable handheld computers can replace paper-based data collection in educational settings.
Get full access to this article
View all access options for this article.
References
1.
AnfaraV.BrownK.MangioneT. (2002). Qualitative analysis on stage: Making the research process more public. Educational Researcher, 31(7), 28–38.
BattenL.ChowdhuryM.DrewJ. (2003). The major drawbacks to the use of wireless communication products in education. In Proceedings of the 1st Australian Information Security Management Conference. Joondalup WA, Australia: We-B Centre, Edith Cowan University.
4.
BerryL. H. (2000). Cognitive effects of Web page design. In AbbeyB. (Ed.), Instructional and cognitive impacts of Web-based education (pp. 41–55). Hershey, PA: Idea Group.
5.
BerthelsenC. L.StilleyK. R. (2000). Automated personal health inventory for dentistry: A pilot study. Journal of the American Dental Association, 131, 59–66.
6.
BlakeE. H. (2002). A field computer for animal trackers. In Proceedings of CHI ′02 Conference (pp. 532–533). Minneapolis, MN: ACM Press.
7.
BlantonL. P.SindelarP. T.CorreaV. I. (2006). Models and measures of beginning teacher quality. The Journal of Special Education, 40, 115–127.
8.
BrownellM. T.RossD. D.ColonE. P.McCallumC. L. (2005). Critical features of special education teacher preparation: A comparison with general teacher education. The Journal of Special Education, 38, 242–252.
9.
CohenJ. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20, 37–46.
10.
CrawfordV.VaheyP. (2002). Palm education pioneers program, March 2002 evaluation report. Menlo Park, CA: SRI International.
11.
CroweA.van 't HooftM. A. H. (2006). Technology and the prospective teacher: Exploring the use of the TI (Texas Instrument)-83 handheld devices in social studies education. Current Issues in Technology and Teacher Education, 6(1), 99–119.
12.
DavisS. M. (2002). Research to industry: Four years of observations in classrooms using a network of handheld devices. In MilradM.HoppeU.Kinshuk (Eds.), IEEE International Workshop on Wireless and Mobile Technologies in Education (WMTE 2002) (pp. 31–38). Los Alamitos, CA: IEEE Computer Society.
13.
de LeeuwE.NichollsW. L. (1996). Technological innovations in data collection: Acceptance, data quality and costs. Sociological Research Online, 1(4). Retrieved January 2, 2007, from http://www.socresonline.org.uk/socresonline/1/4/leeuw.html.
14.
de MulM.BergM. (2007). Completeness of medical records in emergency trauma care and an IT-based strategy for improvement. Informatics for Health and Social Care, 32, 157–167.
15.
DillmanD. (2000). Mail and Internet surveys: The tailored design method (2nd ed.). New York: John Wiley.
16.
DixonM. R. (2003). Creating a portable data-collection system with Microsoft embedded visual tools for the Pocket PC. Journal of Applied Behavior Analysis, 36, 271–284.
17.
DumontR.ChafouleasS. M. (1999, May). Conducting behavior observations: Some technical support?Communiqué, 32–33. Washington, DC: National Association of School Psychologists.
18.
EdyburnD. L. (2001). Critical issues in special education technology research: What do we know? What do we need to know? In MastropieriM.ScruggsT. (Eds.), Advances in learning and behavioral disabilities (pp. 95–118). New York: JAI Press.
19.
EmersonE.ReevesD. J.FelceD. (2000). Palmtop computer technologies for behavioral observation research. In ThompsonT.FelceD.SymonsF. (Eds.), Behavioral observation: Technology and applications in developmental disabilities (pp. 47–60). Baltimore: Paul H. Brookes.
20.
FischerS.StewartT. E.MehtaS.WaxR.LapinskyS. E. (2003). Handheld computing in medicine. Journal of the American Medical Informatics Association, 10, 139–149.
21.
FisherD. (2009). The use of instructional time in the typical high school classroom. The Educational Forum, 73, 168–176.
22.
FitzgeraldM. (2002). Educational media and technology yearbook 2002. Westport, CT: Libraries Unlimited.
23.
FletcherL. A.EricksonD. J.ToomeyT. L.WagenaarA. C. (2003). Handheld computers: A feasible alternative to paper forms for field data collection. Evaluation Review, 27, 165–178.
24.
GongJ.TarasewichP. (2004). Guidelines for handheld mobile interface design. Proceedings of the Decision Sciences Institute 2004 Annual Meeting (pp. 82–88). Boston: Decision Sciences Institute.
25.
GravleeC. C. (2002). Mobile computer assisted interviewing with handheld computers: The entryware system 3.0. Field Methods, 14, 322–336.
26.
GravleeC. C.ZenkS. N.WoodsS.RoweZ.SchulzA. J. (2006). Handheld computers for systematic observation of the social and physical environment. Field Methods, 18, 382–397.
27.
GreenwoodC. R.CartaJ. J.KampsD.TerryB.DelquadriJ. (1994). Development and validation of standard classroom observation systems for school practitioners: Ecobehavioral assessment systems software (EBASS). Exceptional Children, 61, 197–210.
28.
HayhoeG. F. (2001). From desktop to palmtop: Creating usable online documents for wireless and handheld devices. Paper presented to the Communication Dimensions, International Professional Communication Conference, Santa Fe, NM.
29.
HoppeH. U.JoinerR.MilradM.SharplesM. (2003). Wireless and mobile technologies in education. Journal of Computer Assisted Learning, 19, 255–259.
30.
HutterD.MullerG.StephanW.UllmannM. (2003). Security in pervasive computing. New York: Springer.
31.
IceG. H. (2004). Technological advances in observational data collection: The advantages and limitations of computer-assisted data collection. Field Methods, 16, 352–375.
32.
Individuals with Disabilities Education Act of 2004, Pub. L. No. 108–446, 118 Stat. 2647 (2004). Retrieved January 2, 2007, from http://idea.ed.gov/download/statute.html.
33.
JohannesC. B.CrawfordS. L.WoodsJ.GoldsteinR. B.TranD.MehrotraS., (2000). An electronic menstrual cycle calendar: Comparison of data quality with a paper version. Menopause: The Journal of the North American Menopause Society, 7, 200–208.
34.
JohnsonR. B.OnwuegbuzieA. J. (2004). Mixed methods research: A research paradigm whose time has come. Educational Researcher, 33(7), 14–26.
35.
JohnstonJ. M.PennypackerH. S. (1993). Strategies and tactics of behavior research (2nd ed.). Hillsdale, NJ: Erlbaum.
36.
KahngS. W.IwataB. A. (1998). Computerized systems for collecting real-time observational data. Journal of Applied Behavior Analysis, 31, 253–261.
37.
KarampelasP.AkoumianakisD.StephanidisC. (2003). User interface design for PDAs: Lessons and experience with the WARD-IN-HAND prototype. Lecture Notes in Computer Science, 2615, 474–485.
38.
LalD. O.SmithF. W.DavisJ. P.CastroH. Y.SmithD. W.ChinkesD. L. (2000). Palm computer demonstrates a fast and accurate means of burn data collection. Journal of Burn Care & Rehabilitation, 21, 559–561.
39.
LuchiniK.QuintanaC.SolowayE. (2003). Pocket PiCoMap: A case study in designing and assessing a handheld concept mapping tool for learners. In Proceedings of the SIGCHI conference on Human factors in computing systems (pp. 321–328). Ft. Lauderdale, FL: ACM Press.
40.
MalanT. K.HaffnerW. H. J.ArmstrongA. Y.SatinA. J. (2000). Handheld computer operating system program for collection of resident experience data. Obstetrics & Gynecology, 96, 792–794.
41.
McAlpineL.WestonC.BeauchampC. (2002). Debriefing interview and colloquium: How effective are these as research strategies?Instructional Science, 30, 403–432.
42.
MikkelsenG.AaslyJ. (2005). Consequences of impaired data quality on information retrieval in electronic patient records. International Journal of Medical Informatics, 74, 387–394.
43.
MiltenbergerR. G.RappJ. T.LongE. S. (1999). A low tech method for conducting real time recording. Journal of Applied Behavior Analysis, 32, 119–120.
44.
MorganD. L.MorganR. K. (2008). Single-case research methods for the behavioral and health sciences. Los Angeles, CA: Sage Publications.
45.
MotroA. (1990). Imprecision and incompleteness in relational databases: Survey. Information and Software Technology, 32, 579–588.
OnwuegbuzieA. J.LeechN. L. (2006). Linking research questions to mixed methods data analysis procedures. The Qualitative Report, 11, 474–498.
48.
PaceW. D.StatonE. W. (2005). Electronic data collection options for practice-based research networks. Annals of Family Medicine, 3, 21–29.
49.
PascoeJ.RyanN.MorseD. (2000). Using while moving: HCI issues in fieldwork environments. ACM Transactions on Human-Computer Interaction, 7, 417–437.
50.
ReschlyD. J. (2000). The present and future status of school psychology in the United States. School Psychology Review, 29, 507–523.
51.
SarkarA.DuttaA.DhingraU.DhingraP.VermaP.JuyalR. (2006). Development and use of behavior and social interaction software installed on Palm handheld for observation of a child's social interactions with the environment. Behavior Research Methods, 38, 407–415.
52.
SchaffJ. I.JeromeM. K.BehrmannM. M.SpragueD. (2005). Science in special education: Emerging technologies. In EdyburnD. L.HigginsK.BooneR. (Eds.), The handbook of special education technology research and practice (pp. 642–662). Whitefish Bay, WI: Knowledge by Design.
53.
SmithB.RoderickM.DegenerS. C. (2005). Extended learning time and student accountability: Assessing outcomes and options for elementary and middle grades. Educational Administration Quarterly, 41, 195–236.
54.
SpinuzziC. (2003). Using a handheld PC to collect and analyze observational data. In Proceedings of SIGDOC Conference (pp. 73–79). San Francisco, CA: ACM Press.
55.
StaceyR. (2000). The emergence of knowledge in organizations. Emergence, 2(4), 23–39.
56.
SterrittR.BustardD. (2003). Autonomic computing: A means of achieving dependability? InProceedings of 10th IEEE International Conference on the Engineering of Computer Based Systems, Huntsville, AL, 247–251.
57.
StichterJ. P.LewisT. J.WhittakerT.RichterM.JohnsonN.TrussellR. (2009). Assessing teacher use of opportunities to respond and effective classroom management strategies within inclusive classrooms: Comparisons among high and low risk elementary schools. Journal of Positive Behavior Interventions, 11, 68–81.
SuliburkJ. (2003). Which handheld should I buy?Current Surgery, 60, 75–76.
60.
SutherlandK.Lewis-PalmerT.StichterJ. P.MorganP. (2008). Examining the influence of teacher behavior and classroom context on the behavioral and academic outcomes for students with learning and behavior problems. Journal of Special Education, 41, 209–222.
61.
TappJ. (2008). MINI-MOOSES: A multiple option observation system for experimental studies. [Handheld computer application]. Nashville, TN: Vanderbilt Kennedy Center. Retrieved October 25, 2007, from http://mooses.vuinnovations.com/.
62.
TraplE. S.BorawskiE. A.StorkP. P.LovegreenL. D.ColabianchiN.ColeM. L.CharvatJ. M. (2005). Use of audio-enhanced personal digital assistants for school-based data collection. Journal of Adolescent Health, 37, 296–305.
63.
VannestK. J.Hagan-BurkeS.ParkerR. (2006). Teacher allocation and use of teacher time. Annual report to Texas Education Agency. College Station, TX: Texas A&M University.
64.
WalkerI.SiqouinC.SekJ.AlmonteT.CarruthersJ.ChanA. (2004). Comparing hand-held computers and paper diaries for haemophilia home therapy: A randomized trial. Haemophilia, 10, 698–704.
