The NIH Toolbox for Assessment of Neurological and Behavioral Function: The Cognition Team Experienc

1
The NIH Toolbox for Assessment of Neurological
and Behavioral Function The Cognition Team
Experience
S. Weintraub, D. Tulksy, S. Dikmen, R. Heaton, R.
Havlik, N. Fox, D. Blitz, N. Chiaravalotti, P.
Zelazo, J. Machamer, N. Carlozzi, D. Mungas, C.
Nowinski, J. Slotkin, R. Gershon, B. Borosh
Center on Education Research and Outcomes,
Evanston Northwestern Healthcare, Evanston,
Illinois

• AIMS
• The NIH Toolbox, part of the NIH Neuroscience
  Blueprint initiative, seeks to develop brief, yet
  comprehensive, assessment tools measuring motor,
  cognitive, sensory and emotional health and
  function. Upon completion, the Toolbox will be
  available for use in longitudinal epidemiologic
  studies and prevention or intervention trials for
  people ages 3-85. This is a preliminary report of
  progress from the Cognition Team, including
  identification of core constructs, selection of
  candidate instruments and a plan for validation.
• The mandate for all toolbox domain teams was to
• Cover 4-6 subdomains
• Take no more than 30 minutes to administer
• Cover the full range of normal function (not
  disease states)
• Cover ages 3-85 years
• Require minimal cost to the investigator
• Use state-of-the-art instruments and technology

• RESULTS
• 147 RESPONDENTS (63 OF INITIAL MAILINGS)
• 58 male
• Areas of expertise
• 57 adult only
• 18 pediatric only
• 25 adult and pediatric.
• Average years of research experience 25 (SD8)
• Average years of clinical experience 17
  (SD13)
• Experience conducting clinical research/trials
  86
• Experience conducting longitudinal/epidemiologic
• research 87

• Results In-depth Phone Interviews
• 9 cognition experts (5 males, 4 females)
  participated.
• 6 reported expertise in neuropsychology.
  Overlapping specialties cognitive neuroscience
  (3), gerontology/geriatrics (2), psychometrics
  (3), epidemiology (1), clinical trials (1),
  neurology (2).
• Average of 27.8 years (SD7.8) research and 24.2
  ( SD4.3) clinical experience, respectively.
• Rankings of the sub domains previously identified
  by the RFI and fine tuning of categories (I.e.,
  breaking out working memory from executive
  function and attention) were 1) Executive
  Function, 2) Processing Speed, 3) Learning and
  Memory, 4) Language, 5) Attention 6) Working
  Memory, and 7) Visuospatial Functioning.
  Additional domains mentioned as potentially
  important included hot and cold cognition and
  social cognition.

SAMPLE TASKS
Flanker Task Adaptation For Use With Children
And Adults
Flanker conditions can be used to measure
different aspects of attention and executive
function. The standard congruent condition
provides a measure of a simple continuous
performance type of attention, while the
standard incongruent condition minus the
congruent condition serves as a measure of
inhibition.
RESULTS INSTRUMENT SELECTION COMPUTER DRIVEN
CAT METHODOLOGY DIF ANALYSES

• METHODS
• Procedures
• Request For Information (RFI) And Expert
  Interviews Online-surveys and subsequent
  in-depth interviews of experts in cognition
  and/or large cohort, pre-clinical and clinical
  studies were obtained to identify criteria for
  selection of Subdomains and measures. The
  Scientific Advisory Board met to select final
  criteria.
• Instrument Selection Library search tabulation
  of existing psychometric instruments and
  measurement tools meeting with cognitive
  consultants (July 27, 2007) to consider measures
  and narrow options.
• Instrument Development Subdomain teams disperse
  to work on measures development team provides
  technical support for computer administration
  development plan.
• Validation/Norming Plans Validation measures
  discussed by the Cognition Team and in the July
  27 meeting. Epi-Stats team designs sampling
  plan computerized adaptive testing (CAT) and
  Differential Item Functioning (DIF) Analyses.

Conceptual reasoning and set shifting are
measured with this task, designed for children,
but easily adaptable to adults.

• CONCLUSIONS
• A survey (Request for Information), in-depth
  interviews, and a consensus meeting were used to
  identify key constructs of Cognition and measures
  for inclusion in Toolbox. Similar procedures
  were used for the other three domains. Tests are
  in development using computer technology.
  Spanish version is planned. We anticipate the
  final Toolbox to be available in 2011.
  Ultimately, it can serve as a national resource
  for the scientific community.

Acknowledgement Funded in whole or in part with
Federal funds from the National Institute on
Aging, National Institutes of Health, Contract
HHS-N-260-2006 00007-C. (PI Gershon) additional
contributors to this work include Jennifer Manly
PhD, Patricia Bauer PhD, Jennifer Richler PhD,
and Joanne Deocampo PhD.