WHITE OAK, MD—The US FDA is actively seeking ideas from experts on what criteria it should use in evaluating new devices for detecting seizures, assessing traumatic brain injury (TBI), concussion, and cognitive function. As part of that effort, the FDA co‐sponsored a workshop at its headquarters here with speakers and attendees from academia, government, private practice, professional organizations, and industry.
The workshop, co‐sponsored by the FDA, the AAN, the American Epilepsy Society, and the National Academy of Neuropsychology, featured general‐session speakers and breakout sessions for group discussion. Information from the sessions will be used to help the agency craft a guidance document for industry in developing new devices for patients. The FDA evaluates and regulates devices through its Office of Device Evaluation, Center for Devices and Radiologic Health (CDRH).
Brian Litt, MD, associate professor of neurology and bioengineering at the University of Pennsylvania, said that seizures affects 60 million people worldwide, one‐third of whom are not controlled and one‐half unaware of when they are having them. In the hospital setting, “we can't keep physicians up all night” monitoring patients with seizure disorders, he said, adding, “we need technology to help us.” And, he said, in the case of a child with epilepsy, “how many times are parents willing to get up at night for false alarms?” But, he noted, the challenge for industry is that currently neurologists do not know the exact mechanism of a seizure. In the face of that gap, Dr. Litt said that one especially fruitful area for new diagnostic devices is typing the kind of seizure the patient is having. “Not all epilepsy is the same; it's a heterogeneous disease. We need to assess the burden, map, monitor, predict, warn and treat,” he said.
“There is no generally agreed upon definition of seizure,” said Natalie Getzoff, MD, medical officer/neurologist in the FDA Neurodiagnostic and Neurotherapeutic Devices Branch of the CDRH Office of Device Evaluation. In fact, the lack of agreement on the exact definition of a seizure causes “inherent issues with the validation of seizure detection algorithms,” she said.
Dr. Getzoff said that in addition to meeting the FDA requirements for safety and efficacy, a seizure detection device must have full and accurate labeling. “Labeling that describes performance may help mitigate risks and allow for proper usage,” she said. Dr. Getzoff also cautioned that a seizure detection device that monitors EEG data will, by its very design, not be able to capture clinical data. For example, she said, the device will not capture a patient saying before a seizure, “I smell a bad smell.”
Just as detection of seizures is hampered by lack of a professional consensus on a definition, so assessment of TBI and concussion is hampered by the lack of a true consensus on concussion, said Ruben Echemendia, PhD, a sports neuropsychologist who consults for major‐league teams such as those in the National Hockey League (NHL). “We test cognitive domains that are believed to be affected by concussion…we are not measuring concussion per se,” said Dr. Echemendia.
Dr. Echemendia warned that new devices for evaluating concussion in professional sports must be quick (time is of the essence) and must take into account the fact that professional athletes may be suspicious of the professional giving the test and may try to “sandbag” or manipulate the test results because they want to return to the field of play. “In the professional leagues where I work, this [sidelining a player] can mean millions of dollars,” noted Dr. Echemendia.
He also warned that test results may be influenced by the setting in which a device is used: whether it's the locker room, office, or sidelines. A computerized assessment device must take the player's language into account, he said. “In the NHL we have eight different languages,” he said. Dr. Echemendia said that in general computerized test batteries for concussion give a less complete evaluation of memory than paper and pencil tests, lack direct observation of behavior and do not measure the effort and motivation of the test taker. Dr. Echemendia also noted that test results can be muddied by a player's confounding variables, such as ADHD, depression, anxiety and fatigue.
Currently, “there is no single test to diagnose sports concussion,” said Barry D. Jordan, MD, MPH, associate professor of neurology at Weill Medical College of Cornell University and director of the Brain Injury Program, Memory Evaluation Treatment Service at Burke Rehabilitation Hospital. While various devices and tests can assist in the diagnosis of concussion, “it still remains a clinical diagnosis,” said Dr. Jordan. He noted that neuroimaging testing techniques such as DTI, PET, fMRI, SPECT and NIRS (near infrared spectroscopy) can assist in making a diagnosis. “When someone is concussed, the [brain] activation pattern is different from what you see in controls,” he said. “Although athletes are able to do the same tasks, they have to use more of their brain to do them” following a concussion.
Dr. Jordan said that in the future biomarkers may be part of the assessment of concussion. He noted that S100beta — a marker of brain damage in serum and cerebrospinal fluid — has been the most extensively studied, but it needs to be collected within four hours of injury. He cautioned that “genes cannot be used to diagnose concussion,” although the APOE4 allele — which has been implicated in Alzheimer disease — may be associated with a poorer outcome from TBI at six months.
Capt. Paul S. Hammer, MD, director of the Defense Centers of Excellence for Psychological Health & Traumatic Brain Injury (DCoE), put in a strong plea for new devices which can accurately, quickly and reliably diagnose TBI in men and women in uniform — both in non‐combat and combat settings. In June 2010 DCoE initiated a directive requiring evaluation of all deployed military personnel who may have developed a concussion or mild TBI, he said. “Line leaders are screaming at us” to give them a quick, accurate test for TBI, noted Dr. Hammer. “…We need to understand blast dynamics better.”
Capt. Hammer noted that any new device for evaluating TBI in servicemen and women would need to take into account co‐morbidities such as post‐traumatic stress disorder (PTSD). A new diagnostic device for TBI would be especially helpful if it could help clear an individual to return to active duty or determine that he or she could not return to active duty based on the person's pre‐injury functioning compared to post‐injury functioning, noted Capt. Hammer.
As for new devices for assessment of cognitive function, Gordon J. Chelune, PhD — professor of neurology and senior neuropsychologist at the Center for Alzheimer's Care, Imaging and Research at the University of Utah School of Medicine — noted that the evolution of the computer has had a major impact on measurement of cognitive abilities. He cited software for complex and labor‐intensive analyses of cognition along with software for scoring assessment programs that can be easily used by clinicians. “Documentation is key,” said Dr. Chelune. As in any test, validity, reliability and fairness to the test subject are keys. “If you start using labels such as ‘impaired,’ you'd better tell me what is ‘impaired,’” he cautioned.
In cognitive assessment today, “we're really moving from the adaptation of existing [test] measures to stand‐alone batteries to Internet‐based assessment to virtual reality assessment,” said Alison Cernich, PhD, Veterans Affairs (VA) senior liaison for TBI at the DCoE. But Dr. Cernich cautioned that there are technical limitations of computerized cognitive assessment tools, citing “a potential for error introduced by the system.” For example, computer keyboards slow processing speeds, people vary in their ability to use a mouse, computer monitors differ in resolution according to pixels and there are user variations with touch screens.
Other test‐subject considerations must be taken into account, such as familiarity with a computer, disabilities (if any), influence of the testing environment, test‐taker anxiety, language issues and cultural background. Dr. Cernich also stressed the challenges of user verification, privacy and security issues, informed consent and difficulties of use of computerized cognitive assessment tools in unmonitored and uncontrolled situations. Speakers and workshop participants agreed that a “plug and play” mentality with computerized cognitive assessment devices — without a thorough reading and understanding of the accompanying manual — is not in anyone's best interests. Responsible use based on user qualification and user accountability must be a given, they said.