IMPAIRMENT OF MEMORY functioning is among the major causes of disability following traumatic brain injury (TBI).1–4 Persons with TBI and/or their family members find memory impairment to be among the most challenging residual problems resulting from TBI.5,6 Memory impairment is a primary contributor to failure to return to work.7 It is a predictor of unsuccessful rehabilitation and vocational training and has a detrimental impact on quality of life.6,8 Even mild memory impairment can interfere with basic daily activities.9
Prospective memory has been defined as “the ability to remember to perform an intended action at a specific time in the future.”10,11 Recent research suggests that prospective memory impairments may be particularly detrimental to individuals following TBI because they affect the ability to perform daily activities reliably and thereby gain increased self-sufficiency.9,12,13
Although a number of intervention approaches have shown promise for restoring impaired memory functions or learning internal strategies such as association and visualization to compensate for ongoing impairment,14 such approaches may fail to address prospective memory impairment adequately; approaches that emphasize compensation by external cuing may prove more efficacious.10 It is becoming generally accepted that external memory aids are among the most effective rehabilitation tools available to compensate for prospective memory impairment following TBI.15
Historically, the most common external memory aids have involved written reminders in paper media ranging from unformatted notebooks to individually customized memory books to published organizers and planners.16–18 A key shortcoming of paper-based reminder systems is that users must remember to check their reminder systems for information about upcoming appointments. That is, paper-based memory aid systems are crucially dependent for their effectiveness on the functioning of those very impaired memory capacities for which they are intended to compensate.
A variety of electronic devices have been explored as potential compensatory tools for individuals with memory impairment.19 One series of studies has investigated the use of commercial pagers as the reminding component of electronic reminder systems for assisting with daily scheduling needs.20 In pager-based systems, reminder content and timing information are typically entered by staff personnel into software running on remote servers. At the scheduled time, event-specific reminders are generated automatically on the remote servers, and then communicated automatically through commercial paging networks to pager devices carried by users with TBI-related memory impairments. Pager-based systems have been shown to improve timely task completion rates both in case studies21 and in group studies.22
The introduction of handheld computers in the 1990s provided previously unavailable levels of portability that allowed users for the first time to carry sophisticated computer technology on their person. This significantly expanded the ways in which computer technology could be used and the venues in which it could be helpful to users. The introduction of ever more powerful portable computer technologies has provided a continually expanding collection of tools for potential use in a variety of clinical populations to compensate for impaired cognitive functioning. Studies have described how electronic reminder-generating software running on handheld computers can assist in maintaining daily schedules during inpatient rehabilitation with adults who have sustained brain injury.23,24 Gentry et al25 examined the efficacy of PDAs as cognitive aids to the performance of everyday tasks in a sample of community-dwelling individuals with severe TBI and memory-based difficulties. Eight weeks following brief training in PDA usage, participants rated occupational performance, and performance on everyday life tasks more favorably than they had prior to PDA usage.
With the incorporation of alarms and scheduling software into portable telephones, studies have demonstrated the use of such functions in cellphones—sometimes referred to as “smart phones” when so equipped—to assist patients in maintaining daily schedules in community settings.26,27 Other studies have described the effective use of alarm-enabled voice recorders for providing reminders for everyday tasks.12,28
Another series of studies utilized early full keyboard-equipped “clamshell” type handheld computers (Psion Series 3 and Series 5 devices developed in Great Britain) initially in open clinical usage trials with patients who had sustained brain injury and were undergoing outpatient rehabilitation. An immediate and instructive finding was that these patients, in spite of persisting impairment related to brain injury, were able to learn the use of reminder-generating software on these devices far more readily than had been anticipated—within the course of a 1-hour training session rather than needing the 8 weekly 1-hour sessions that had been envisioned. Device usage proved generally positive and helpful, and nearly two-thirds of the sample subsequently chose to obtain devices, often at their own expense, and continue using them following completion of the research trials.29 A subsample contacted for follow-up months to years later provided strong endorsement for the use of such systems.30
A subsequent study using the same kind of device and a comparable participant population collected concrete behavioral data in a controlled crossover study31 of prospective memory task performance rates during 6-week usage trials. Twice-daily tasks were assigned at randomly chosen times 5 days a week and consisted of making phone calls at each assigned time to the research office, where the calls were automatically recorded with date- and time-stamping. Participants demonstrated highly significant differences in timely task completion rates between the palmtop computer usage condition and both a paper planner usage condition and a baseline condition in which no memory aid was provided. Performance levels in the final (sixth) week of usage trials, however, showed a decline to baseline levels, regardless of memory aid condition, suggesting the possibility of a generalized extinction effect. Despite the growing evidence suggesting the effectiveness of PDAs as memory aids, other studies have reported that clinicians with less training in therapeutic PDA use, less personal PDA use, and less confidence regarding therapeutic PDA use are less likely to introduce device usage clinically, and may contribute to underutilization of these tools.32,33
In 2001, the National Institute on Disability and Rehabilitation Research (NIDRR) awarded a one-time, multi-year, multi-site grant to investigate the potential of electronic devices to enhance the independence of adults and children suffering from memory and organizational challenges resulting from TBI or intellectual disability (ID). A partnership was established among the School of Speech-Language Pathology and Audiology at The University of Akron, The Institute on Disabilities at Temple University, Spaulding Rehabilitation Hospital/Harvard Medical School, Moss Rehabilitation Research Institute, and the Brain Injury Association of America to develop an Assistive Technology Collaborative that would implement this research objective.
The initial stage of the project consisted of surveys administered to the designated target populations. Few adult participants with TBI or with ID reported use of electronic devices to assist with deficits in memory and organization, although most reported that they had been exposed to such devices.34,35 Children with either TBI or ID reported exposure to a greater variety of devices and greater familiarity with computer operations in general than did adults. The children's primary use of electronic devices, however, was for entertainment, and few reported use for assistance in performing everyday functional tasks.36
Following these surveys, additional data collected from focus groups indicated clear interest in the use of such electronic devices among all sampled groups. On the basis of the survey and focus group findings, a series of second stage investigations was designed and implemented. The goal of these studies was to determine—through extended device usage trials in participants' real-world environments—whether PDA technology could be used by members of our target populations, and whether usage would improve completion rates for time-related tasks in spite of ongoing prospective memory impairments.
A common research design and protocol for usage trials was developed for use at all collaborative sites. The protocol utilized an adaptation of the call-in procedure described earlier.31 It was decided to make use of commercially available, unmodified consumer PDA devices and applications in these second stage studies rather than devices with specialized assistive hardware or software features.36,37 The choice of basic commercial PDA devices was based on several factors. First was the greater availability and lower-cost of these products by comparison with specialized adaptive devices. If less costly devices could be shown to be effective, then use of such devices could be available to more widespread populations. Second was the expectation gained from stage one survey data that the more “normal” appearance of commercial devices would give them significantly greater appeal for potential users.
Although studies on reminding technologies have employed a variety of devices ranging from pagers to telephones to PDAs, systematic investigation of the impact on performance of device and feature differences has been limited to date. The present research sought to begin examining this issue by utilizing 2 different brands of device hardware running 2 different operating systems, the Palm OS (POS) and Microsoft's Pocket PC OS (MOS), and allowing direct device comparisons to be made.
Outcome measures useful for investigating device effectiveness tend to be either highly resource intensive—for example, direct in vivo observation-based behavior counts—or global and generalized—for example, self-report ratings of “usefulness.” The present research adopted the call-in approach described earlier because of its efficiency, convenience, and ability to produce highly accurate and concretely behavioral outcome data.
In the first of the second stage usage studies to be completed by another of the partners in the Collaborative, children with TBI and ID in a school setting were more likely to complete tasks in a timely manner when using a reminder-generating PDA than when using a paper planner system. Unexpectedly, they did not do as well using the paper planner as they did in a baseline condition without use of a memory aid, perhaps because the reminders in the baseline condition came from classroom teachers who were directly observing the results. Intensive follow-up study of a subset of these students demonstrated that they could learn to use a PDA and a smart phone more effectively through in-depth training.36
The questions to be addressed by the current study are (1) With adults with TBI, will improvements in timely task completion rates result from the use of automated reminders generated by contemporary PDA devices as was seen with the use of a prior generation of handheld computers?31 (2) Will the magnitude of observed improvements seen with PDAs be greater than those seen with paper-based devices? (3) Will improvements resulting from device usage vary in magnitude depending on the nature of the task involved? (4) Will improvements resulting from electronic device usage vary as a function of the device employed? (5) Will individual demographic or test-derived cognitive characteristics be predictive of differential device usage improvements?
This was an 8-week memory aid usage study with a multiple cross-over design in which completion rates for assigned memory tasks were recorded under 4 conditions: a baseline condition relying on participants' usual approach to remembering without introduction of a memory aid, a condition using a supplied paper memory aid and 2 conditions using different palmtop computers as PDA memory aids. Memory aid condition was changed each week for each participant in a randomly determined order. Rate of timely task completion was used as the outcome measure of performance effectiveness. Selective psychometric testing was performed to examine individual cognitive differences as potential predictors of performance effectiveness.
Study participants were recruited from among persons with TBI participating in outpatient programs, group homes, and other long-term care settings in the Greater Boston area, including Spaulding Rehabilitation Hospital Network sites and other rehabilitation settings. Participants were referred by caregivers, other rehabilitation professionals and state rehabilitation agencies in response to one-page recruitment notices distributed at local conferences, sent to potential referral sources, and posted at Spaulding sites. Inclusion criteria included: (1) age between 18 and 64 years; (2) documented history of TBI; (3) self-perceived memory impairment; (4) not currently participating in training in the use of external memory aids in a rehabilitation setting or elsewhere; (5) corrected visual acuity adequate to read from paper-based and computer-based displays; and (6) meeting screening criteria for hearing, motor functioning, and cognitive functioning adequate for device usage. To address items (5) and (6), participants had to be able to find a date on a calendar; tell the time on a clock; read a message on a PDA device; respond to an alarm on a PDA device; use a stylus to press 3 locations on a PDA screen; and copy their name, 2 words, and a 4-word sentence.
Thirty-nine consecutively referred participants meeting inclusion criteria were initially enrolled in the study. Three participants were subsequently dropped from participation, one who withdrew for unanticipated medical reasons, one because of unresolvable scheduling conflicts and one who failed to participate after initial enrollment in spite of multiple attempts at contact by study staff.
Descriptive characteristics of study participants are summarized in Table 1. The average age of the 36 participants retained for analysis was 42.1 years (SD = 14.5); the youngest was 18.5 years of age and the oldest was 66.9 years. The one participant exceeding the maximum age was enrolled as a result of administrative error. Because this individual demonstrated performance capabilities on criterion testing generally comparable to those of other study participants, it was decided to waive the age maximum and maintain study enrollment in this case. Participants were divided nearly equally by sex and were predominantly white.
A selection of well-established cognitive measures in widespread clinical and research use and with strong reliability and validity properties was administered to participants. They included the Vocabulary and Matrix Reasoning subtests of the Wechsler Abbreviated Scale of Intelligence (WASI);38 the first measures word knowledge by asking for word definitions and is highly correlated with both verbal and general intelligence, and the second measures nonverbal reasoning ability utilizing matrix illustrations for which a missing component must be chosen from a multiple-choice array. They also included measures of immediate and delayed verbal memory assessed by the retelling of paragraph length stories in the Logical Memory (I and II) subtests of the Wechsler Memory Scale-III;39 measures of immediate and delayed nonverbal memory assessed by providing descriptions of previously viewed pictures of family interaction in the Family Pictures (I and II) subtests; and measures of concentration on the Mental Control subtest. Finally, written computation and single word reading were assessed with the Math and Reading subtests of the Wide Range Achievement Test 3 (WRAT3).40
Sample-wide performance levels on cognitive testing are reported in Table 2. Although a considerable range of between-participant variability was seen on these measures, most mean performance values for the participant sample as a whole fell in the average to low average range.
Two models of electronic device were used in this study: a Palm Operating System (POS)-based PDA (the Palm Zire 71 or 72) and a Microsoft Operating System (MOS)-based PDA (the Dell Axim X30).* The Palm and Dell PDAs were both palmtop-sized devices with full-sized touch-screen color displays and data input via stylus movement on the touch screen using on-screen keyboard displays and other control displays (eg, icons, menus). Both PDAs were equipped with calendar applications into which appointments and other reminders (such as those for prospective memory tasks) could be entered, and which produced audible alarm signals at the time a given calendar entry was due for completion (or at an appropriate pretask interval set to permit needed preparation).
Sequence of procedures
Participants selected for inclusion in this 8-week study met with one of the study staff members at their outpatient or community setting, accompanied by a designated support person where appropriate, to give informed consent, have an initial enrollment interview, and undergo administration of the pretrial assessment battery. The study was approved by the Spaulding Rehabilitation Hospital institutional review board.
At the outset of week 1, each participant was assigned one time per workday (5 per week) at which they were to call the answering machine at the hospital research office. Call-in times for each subject were randomly chosen each week from all possible times distributed at 15-minute intervals across each workday. Where necessary, assigned times were then modified for individual participants to accommodate conflicts with preexisting commitments. Three additional time-related personalized tasks (eg, taking medication doses, walking the dog, exercising, taking out the trash) were then chosen in collaboration with each participant from their own everyday activities. The 3 personalized tasks were to be completed at the same personally selected time each week throughout the study. Completion of the personalized tasks was reported by means of a call to the research office when each task was done. Instructions were provided on how to perform the daily call-in tasks and to report the 3 weekly personalized tasks. Week one was the baseline condition, in which participants were instructed to remember the 5 call-in tasks and the 3 personalized tasks using only whatever approach they usually employed for remembering future tasks. They were allowed to continue using any paper-based devices that were a part of their usual preexisting approach.
In each of weeks 2 through 4, the participants were trained in the use of one randomly assigned memory aid, either a paper-based schedule book with weekly format, a POS-based PDA or a MOS-based PDA.
Participants were trained in the use of each type of memory aid just prior to use and instructed to enter assigned reminders into the palmtop computer or paper organizer at the time they received their weekly assignments. As in week 1, they were instructed to call-in to the office answering machine at the time specifically assigned for each day (Monday through Friday), and told to perform each personalized task at the assigned time and then call in to report task completion. Weeks 5 to 8 involved a repetition of weeks 1 to 4, with week 5 again conducted under the baseline condition, and weeks 6 to 8 conducted with memory aids assigned in rerandomized order.
Main measures and statistical analyses
Rate of timely completion of assigned call-in tasks and personalized tasks served as the principal outcome measure. Timely completion was defined as task completion within 15 minutes of the assigned time for that task. Data analyses were performed using the Stata (Version 10) data analysis and statistical software package (StataCorp LP). Poisson regression was not used because underlying assumptions (variance = mean) were not met, and task completion rates were analyzed using negative binomial regression.41,42 Because conditions at the time of participant performance could render a response opportunity invalid, an exposure variable equal to the actual denominator (number of valid attempts) was utilized to adjust for inequality of opportunity. A very low rate of response opportunity invalidity was actually observed (0.3%, or 7 invalid opportunities out of a possible 2304 opportunities sample-wide—8 per week for 36 participants for 8 weeks), and the needed adjustment proved minimal. Sex, age, cognitive test scores, and final choice of device were used as potential independent predictors.
Timely task completion
It had been anticipated that task completion rates would vary in magnitude depending on the nature of the task to be completed. Specifically, it was expected that completion rates would be lower for the assigned tasks—that is, simply making a phone call at a particular time as instructed, and higher for the personalized tasks—such as walking the dog or emptying the trash—which participants assisted in choosing and in which they might be expected to have a greater investment. Contrary to that prediction, completion rates were comparable in magnitude and not significantly different (call-in task, rate = 0.403, personalized task, rate = 0.392, t = 0.588, P = .5) regardless of the nature of the task assigned. Data for call-in tasks and personalized tasks were therefore collapsed into a single task completion rate variable.
Task completion rates for each of 4 study conditions are presented in Figure 1. In the baseline condition, timely task completion occurred for 27% of the assigned tasks. In the paper planner condition, it was seen for 26% of the tasks. The MOS PDA condition produced a higher rate (38%), and the POS PDA condition yielded a yet higher rate of completion (56%).
As seen in Figure 1, task completion rates for the paper organizer condition were comparable to those seen in the baseline condition (Incidence Rate Ratio [IRR] = 0.95, P > .65, CI = 0.76–1.18). Data for both conditions were therefore combined into a single variable (Baseline+) for use in subsequent analyses.
Task completion rates were substantially higher under the MOS condition than under the baseline+ condition, a difference that proved to be highly significant (Table 3). Task completion rates were also substantially higher under the POS condition than under the baseline+ condition, a difference that also proved to be highly significant. Differences in task completion rates between the 2 electronic device conditions—MOS and POS—were also highly significant.
Sex did not predict task completion rate (P > .10), nor did age (P > .07). None of the assessed cognitive variables were predictive of response rate (P > .4 through .99).
In previous research, marked declines in performance levels had been observed on trials occurring at the end of the research sequence for both electronic device conditions and paper-based organizer conditions.31 We therefore examined average task performance rates for each condition as a function of week of participation in the study. Values averaged for each treatment condition were comparable throughout the 8-week sequence, failed to show significant performance differences as a function of week of participation, and failed to show a drop-off at the end of the sequence (IRR = 0.895, P < .0005, CI = 0.84 3–0.950).
At the conclusion of study participation, 27 participants (75%) chose to retain a Palm PDA, 8 participants (22%) chose to retain a Dell PDA, and 1 participant (3%) chose to take cash payment instead of a device. Participants were not found to differ on task completion rate as a function of their choice at the end of the study (P > .1).
Findings from the current study make clear that, for survivors of TBI with self-reported memory difficulties, receiving reminder cues from contemporary palmtop PDA devices results in higher rates of timely task completion than does use of a baseline condition relying on participants' usual approach to remembering tasks or use of a paper-based memory aid. These findings are consistent with those from studies using earlier portable computer technologies.
Our data suggest that timely task completion rates do not vary with the nature of the assigned task. Rates are comparable whether the task is a less personalized one such as making an assigned phone call or a more personally relevant task such as walking the dog or emptying the trash. An important implication of this finding is methodological: use of an impersonal assigned task such as the call-in task, with its ease of implementation and data collection through time-stamped voicemail recording, appears to yield data comparable to those produced by use of such calls for reporting completion of a personally chosen task. Although our data suggest that these approaches are interchangeable, future work would clearly be needed to corroborate whether tasks are actually completed when using the call-in paradigm.
Use of a PDA for reminding can be viewed as a 2-step process. First, data has to be entered into the device to set an alarm and provide a written cue at a given future time (input). Second, the person carrying the device has to hear the alarm, read the written cue and follow through with the task (output). This study evaluated the output phase of this equation, as the participants were assisted as much as needed to enter reminders successfully. Our focus on this phase of the process will be useful for future studies, enabling them to focus on requirements of the input phase while relying on the knowledge that the output phase is successful a significant amount of the time. Studies are needed that compare different amounts of training and subsequent cuing and assistance in entering data for the reminders.
The findings from the current study establish that the improvements in timely task performance seen with automatically generated reminders are not device specific, being seen with both of the technological systems for delivering the reminders in this study. However, the significant differences in strength of effect seen between the 2 electronic devices we used indicate that device differences can indeed influence the magnitude of device effect. The higher rate of timely task completion while using a Palm device with POS than a Dell palmtop with MOS could be related to the louder alarm incorporated into the former, or some other aspect of the hardware or software. Further studies are needed to evaluate aspects of the output alarm, the cues provided, and the hardware and software utilized that might facilitate follow-through with the task in question.
The current study has several limitations. It has focused on a narrowly defined aspect of behavior change. It provides no data on the effects of long-term continuous use or on more broadly defined outcomes such as community integration, personal satisfaction with the device, or measurable indices of quality of life. Future research will need to examine these dimensions to determine the nature and extent of more global impacts on adaptation to impairment and adequacy of overall postinjury functioning. The choice of many participants to keep the PDAs indicates that they feel that device usage is helpful and/or satisfying. Collection of systematic data on attitudes toward usage and on the long-term experience of users would be a crucial next goal in this field.
The strength of the current findings also suggests methodologies for testing device differences—either in hardware or software—that can be utilized in future research on reminding technologies and in evaluating emerging features. Specifically, tracking the timely completion of a simple call-in task provided results comparable to recording successful and timely completion of personalized tasks normally performed in the daily life of the participant.
Cognitive performance levels did not predict task completion rates. Although the primary focus of this study was not the examination of individual differences variables as predictors of outcome, the failure to find cognitive predictors was unexpected. Future research on individual differences in this area should consider the use of subject selection criteria beyond the self-perceived memory impairment utilized here. Furthermore, studies should consider inclusion of specific tests of both prospective memory and of executive function, in particular, the ability to follow-through. Since neuropsychological testing is typically done in a structured setting, it can fail to detect problems with initiation and follow-through, and evaluation methods for improving the ecological validity of the findings should be explored.43
As noted at the outset, impairment of memory functioning occurs frequently following TBI and has disabling consequences. Although our findings reflect the performance of adults with a history of TBI, comparable findings are emerging in children,36,44 and we have no reason to suspect that the positive effects of electronic memory aid use would be limited only to adults with TBI, but could prove useful in addressing prospective memory impairment as the result of other conditions as well. Usage studies of electronic memory aids would certainly be warranted in other populations with memory dysfunction.
1. Baddeley A, Harris S. Closed head injury and memory. In: Levin HS, Grafman J, Eisenberg HM, eds. Neurobehavioral Recovery from Head Injury. New York: Oxford University Press; 1987:295–315.
2. Brooks N, Campsie L, Symington C, Beattie A, McKinlay W. The five year outcome of severe blunt head injury: a relative's view. J Neurol Neurosurg Psychiatry 1986;49(7):764–770.
3. Consensus conference. Rehabilitation of persons with traumatic brain injury. NIH consensus development panel on rehabilitation of persons with traumatic brain injury. JAMA. 1999;282(10):974–983.
4. Sohlberg M. External aids for management of memory impairment. In: High W, Sander AM, Struchen MA, Hart KA, eds. Rehabilitation for Traumatic Brain Injury. New York: Oxford University Press; 2005:47–70.
5. Oddy M, Coughlan T, Tyerman A, Jenkins D. Social adjustment after closed head injury: a further follow-up seven years after injury. J Neurol Neurosurg Psychiatry 1985;48(6):564–568.
6. Ryan TV, Sautter SW, Capps CF, Meneese W, Barth JT. Utilizing neuropsychological measures to predict vocational outcome in a head trauma population. Brain Inj. 1992;6(2):175–182.
7. McKinlay W, Watkiss AJ. Cognitive and behavioral effects of brain injury. In: Rosenthal M, Griffith ER, Kreutzer J, Pentland B, eds. Rehabilitation of the Adult and Child with Traumatic Brain Injury. Philadelphia: F.A. Davies; 1999:74–116.
8. Kennedy M. Managing memory and metamemory impairment in individuals with traumatic brain injury. The ASHA Leader. 2006;11(14):8–9, 34–36.
9. Sohlberg M, Mateer C. Cognitive Rehabilitation: An Integrative Neuropsychological Approach. New York: Guiliford Press; 2001.
10. Fleming JM, Shum D, Strong J, Lightbody S. Prospective memory rehabilitation for adults with traumatic brain injury: a compensatory training programme. Brain Inj. 2005;19(1):1–10.
11. Ellis J, Kvavilashvili L. Prospective memory in 2000: past, present and future directions. Appl Cogn Psychol. 2000;14:S1–S9.
12. van den Broek MD, Downes J, Johnson Z, Dayus B, Hilton N. Evaluation of an electronic memory aid in the neuropsychological rehabilitation of prospective memory deficits. Brain Inj. 2000;14(5):455–462.
13. Mathias JL, Mansfield KM. Prospective and declarative memory problems following moderate and severe traumatic brain injury. Brain Inj. 2005;19(4):271–282.
14. O'Neil-Pirozzi TM, Strangman GE, Goldstein R, et al. A controlled treatment study of I-MEMS: a group intervention emphasizing internal memory strategies following traumatic brain injury. J Head Trauma Rehabil. In press.
15. Sohlberg M, Kennedy M, Avery J, Coelho C, Turkstra L, Ylsvisaker M. Evidence-based practice for the use of external aids as a memory compensation technique. J Med Speech-Lang Pathol. 2007;15(1):xv–li.
16. DePompei R, Tyler J. Learning and Cognitive Communication Challenges: Developing Educational Programs for Students with Brain Injuries. Wake Forrest: LA Publishing; 2004.
17. Constantinidou F, Thomas R, Best P. Principles of cognitive rehabilitation: an integrative approach. In: Ashley M, ed. Traumatic Brain Injury: Rehabilitative Treatment and Case Management. Boca Raton: CRC Press; 2004:337–366.
18. Parente R, Di Cesare A. Retraining memory: theory, evaluation, and application. In: Kreutzer J, Wehman P, eds. Cognitive Rehabilitation for Persons with Traumatic Brain Injury: A Functional Approach. Baltimore: Paul Brooks; 1991:147–163.
19. Inglis EA, Szymkowiak A, Gregor P, Newell AF, Hine N, Wilson B. Usable technology? Challenges in designing a memory aid with current electronic devices. Neuropsychol Rehabil. 2004;14(1/2):77–87.
20. Hersh NA, Treadgold LG. The rehabilitation of memory dysfunction by prosthetic memory and cueing. Neuro Rehabil. 1994;4:187–197.
21. Wilson B, Emslie H, Quirk K, Evans J. George: learning to live independently with neuro page. Rehabil Psychol. 1999;44(3):284–296.
22. Wilson BA, Emslie HC, Quirk K, Evans JJ. Reducing everyday memory and planning problems by means of a paging system: a randomised control crossover study. J Neurol Neurosurg Psychiatry. 2001;70(4):477–482.
23. Kim HJ, Burke DT, Dowds MM, George J. Utility of a microcomputer as an external memory aid for a memory-impaired head injury patient during in-patient rehabilitation. Brain Inj. 1999;13(2):147–150.
24. Wright P, Rogers N, Hall C, Wilson B, Evans J, Emslie H. Comparison of pocket-computer memory aids for people with brain injury. Brain Inj. 2001;15(9):787–800.
25. Gentry T, Wallace J, Kvarfordt C, Lynch KB. Personal digital assistants as cognitive aids for individuals with severe traumatic brain injury: a community-based trial. Brain Inj. 2008;22(1):19–24.
26. Thone-Otto AIT, Walther K. How to design an electronic memory aid for brain-injured patients: Considerations on the basis of a model of prospective memory. Int J Psychol. 2003;38(4):1–9.
27. Wade TK, Troy JC. Mobile phones as a new memory aid: a preliminary investigation using case studies. Brain Inj. 2001;15(4):305–320.
28. Hart T, Hawkey K, Whyte J. Use of a portable voice organizer to remember therapy goals in traumatic brain injury rehabilitation: a within-subjects trial. J Head Trauma Rehabil. 2002;17(6):556–570.
29. Dowds MM, Robinson K, Kim HJ, Burke DT, Park G. Electronic Memory Aids for Outpatient Head Injury. Poster presented at the 3rd World Congress on Brain Injury; 1999.
30. Kim HJ, Burke DT, Dowds MM Jr., Boone KA, Park GJ. Electronic memory aids for outpatient brain injury: follow-up findings. Brain Inj. 2000;14(2):187–196.
31. Dowds MM, Sheer JB, Zainea K. Current Research on PDA-Based Electronic Memory Aids. Poster presented at Rehabilitation Psychology 2005, the Seventh Annual Conference of the American Board of Rehabilitation Psychology and the Division of Rehabilitation Psychology of the American Psychological Association; 2005.
32. Hart T, O'Neil-Pirozzi T, Morita C. Clinician expectations for portable electronic devices as cognitive-behavioural orthoses in traumatic brain injury rehabilitation. Brain Inj. 2003;17(5):401–411.
33. O'Neil-Pirozzi TM, Kendrick H, Goldstein R, Glenn M. Clinician influences on use of portable electronic memory devices in traumatic brain injury rehabilitation. Brain Inj. 2004;18(2):179–189.
34. Hart T, Buchhofer R, Vaccaro M. Portable electronic devices as memory and organizational aids after traumatic brain injury: a consumer survey study. J Head Trauma Rehabil. 2004;19(5):351–365.
35. Carey A, Friedman M, Bryen DN. Use of electronic technologies by people with intellectual disabilities. Ment Retard. 2005;34(5):322–333.
36. Gillette Y, DePompei R. Do PDAs enhance the organizational and memory skills of students with cognitive disabilities. Psychol Schools. 2008;45(7):665–677.
37. Bryen DN, Carey A, Friedman M. Cell phone use by adults with intellectual disabilities. Intellect Dev Disabil. 2007;45(1):1–9.
38. Wechsler D. Wechsler Abbreviated Scale of Intelligence (WASI). San Antonio: Harcourt, Brace & Company; 1999.
39. Wechsler D. Wechsler Memory Scale. 3rd ed. San Antonio: The Psychological Corporation; 1997.
40. Wilkinson GS. Wide Range Achievement Test 3 (WRAT3). Wilmington: WIDE RANGE, Inc.; 1993.
41. Rothman KJ, Greenland S. Modern Epidemiology. Philadelphia: Lippincott-Raven Publishers; 1998.
42. Hilbe JM. Negative Binomial Regression. Cambridge: Cambridge University Press; 2007.
43. Manchester D, Priestley N, Jackson H. The assessment of executive functions: coming out of the office. Brain Inj. 2004;18(11):1067–1081.
44. DePompei R, Gillette Y, Goetz E, Xenopoulos-Oddsson A, Bryen D, Dowds M. Practical applications for use of PDAs and smartphones with children and adolescents who have traumatic brain injury. Neurorehabilitation. 2008;23(6):487–499.
* The two devices using the “Palm Operating System (POS),” the Zire 71 and the Zire 72, were manufactured by Palm, Inc., 950 W. Maude Ave. Sunnyvale, CA 94085, now a subsidiary of Hewlett-Packard Company, 3000 Hanover Street, Palo Alto, CA 94304, and ran on Palm OS 5.2.1 and 5.2.8 respectively. The device using the “Microsoft Operating System (MOS),” the Axim X30, was manufactured by Dell Products, L.P., One Dell Way, Round Rock, Texas 78682, and ran on Microsoft Windows Mobile 2003 Second Edition. Innovation in mobile technologies has resulted in rapid turnover of device models in the PDA and smartphone markets. Established models are quickly superseded by newer models, and that has happened with the PDA models used during this study. Although these specific devices are no longer commercially available, the calendar and reminding functionalities upon which our findings are based remain securely incorporated in newer models, and are as available to today's users as they were to our subjects.