U'S' Army Competency Assessment for the 21st Century Perform21

1

• U.S. Army Competency Assessment for the 21st
  Century (Perform21)

2
Problem Overview

• Need Army Training and Leader Development Panel
  NCO survey (ATLDP June 2002) calls for objective
  performance assessment and self-assessment of
  soldier technical and leadership skills
• Limitation Systemic performance assessment
  requires considerable manpower and financial
  resources
• Response ARI is conducting a three-year
  research program to identify viable approaches
  for the development of a useful yet affordable
  performance assessment system

3
Approach to the Problem
Assessment Development

• Phase I
• Feasibility and Alternative Designs

• Phase II
• Design Selection
• Prototype Measure Development Testing

• Phase III
• Prototype Evaluation
• System Recommendation

Jan 04
Jan 05
Jan 03

• Common Core Performance Prototype
• Support Army Competency Assessment
• Identify Practical Issues

Army Development and Implementation of
Operational Assessment (Beyond Project Scope)

• Automated data collection
• Item analyses
• Operational assessment recommendations

Demonstration Competency Assessment Program (DCAP)
4
Phase I Feasibility and Alternative Designs
Approach
Needs Analysis
Automation Technology
Historical Perspective
Feasibility
Alternative Performance Assessment Systems
Military and Civilian Assessment and Certification
Demonstration Competency Assessment Program
5
Sergeant Major of the Army Guidance for DCAP

• Guidance for Competency Assessment
• Promotion-Oriented
• Web-Administered Test
• Whole Army (AC, USAR, ARNG)
• SPC/CPL (E4) to SFC (E7)
• Assessment Topics
• Leadership
• Training
• Army/NCO History
• Basic Soldier Skills
• MOS-specific assessment added later
• Multiple Choice including Situation-Based Items

6
DCAP Development
DCAP to investigate competency test system design
and to identify issues, shortcomings, and design
modifications
Identify test topics based on job analysis and
objectives identified by stakeholders
Develop test blueprint
Identify test content and format specific to the
selected prototype
Prepare a test prototype
Prepare for self-assessment
7
Phase II

• Phase I end 5 Jan 04
• Phase II approximately Nov 03 Jan 05

• MOS-Specific Assessment Development
• Develop/identify job analysis and assessment
  strategies
• Finalize representative MOS and identify SMEs and
  written sources
• Develop test blueprint and MOS prototypes
• Conduct pilot tests of MOS-specific prototypes

• DCAP
• Technology-enhanced data collection method
• Pilot test prototype
• Item analyses and assessment results
• Operational assessment recommendations

8
Phase III

• Evaluate 5 MOS-specific prototype designs
• Recommend system(s) for performance assessment
• Estimate alternative resource configurations to
  develop and maintain such system(s)

9
Status

• Visited Navy Advancement Center and Air Force
  Occupational Analysis Squadron
• Integrating research and generating a range of
  testing alternatives
• Army Test Program Advisory Team (ATPAT) workshops
  Jun 03, Aug 03, Oct 03
• Coordinating with Army Training System Center,
  and Distributed Learning System personnel for for
  secure internet assessment administration
• Identified existing sources and previously
  developed research test items for possible test
  content
• Developed test blueprint and began preliminary
  item writing
• SME reviews in Nov 03
• Pilot testing begins in Jan 04