Institute for Organizational Assessment Navy Personnel, Research, Studies,

1
Institute for Organizational Assessment Navy
Personnel, Research, Studies, Technology
(BUPERS-1/NPRST)January 2008
2
Institute for Organizational Assessment What We
Do

• Use scientifically defensible approaches to study
  human-related systems used by Navy to manage its
  workforce
• Conduct large-scale surveys, evaluate programs,
  provide technical review of Navy personnel
  surveys, and develop strategies to enhance data
  coordination and comparability
• Document findings to provide practical decision
  support to Navy policymakers

3
Surveys
4
Why the Navy Needs Surveys

• Cost-effective, scientifically sound, timely
  information for Navy decision makers
• Finger on the Pulse of the Navy
• Assess Navy programs policies
• Track major trends
• Surveys are structured conversations with
  Sailors
• Efficient method of gathering data on Sailors
• expectations, needs, wants, satisfaction,
  intentions, opinions about various aspects of
  Navy life

5
How the Navy Uses Surveys

• Provide accurate information to Navy Leaders
  about important areas for SECNAV, DOD and
  Congress
• SAVI quick poll provides Navy-wide data as
  follow-up to SASC Hearings on sexual assault
• NEOSH results cited by prior SECNAVs, ASNs, CNPs
  to demonstrate Navys progress in reducing sexual
  harassment in years following Tailhook
• Improve training and information for Sailors
• Results of Pregnancy Survey integrated into Navy
  GMT
• Answer data-based queries from Navy
  leaders/policymakers
• NPS results about religious affiliation provided
  to Navy JAG in 2004 to assistant Navy in defense
  against lawsuit brought against Chief of Navy
  Chaplains
• Provide information to key Navy stakeholders
  about topics of interest
• Results of Navy QOL Survey on Shipboard
  Habitability provided the basis for NAVSEA to
  develop the American Bureau of Shipping's (ABS)
  Naval Vessel Rules for ABS certification of Navy
  ships.

6
Strategic Survey Approach

• Regularly scheduled surveys
• Examples
• Navy-wide Personnel Survey
• Navy Climate Survey
• Argus Career Transitions Survey

• Navy Quick Polls

Did It Work?
Find Solutions
Regular Check-ups

• Marketing Research Surveys
• Polls
• Focus Groups

• In-depth, specialized surveys
• and program evaluations
• Focus Groups

In-depth Diagnostics
Fewer SurveysBetter InformationLower CostLess
Impact on Sailors
7
Navy Survey Strategy
8
Survey Topics Assessed by NPRST

• Job and Career Satisfaction
• Organizational Commitment
• Pay and Compensation
• Career Intentions
• Communications
• Detailing
• Leadership
• Assignments
• Performance Evaluations
• Command Climate
• CACO
• Media Use
• Retention
• Career Development
• Financial Status

• Training and Education
• Pregnancy and Parenthood
• Diversity
• Equal Opportunity
• Discrimination
• Sexual Harassment
• Sexual Assault
• Mentoring
• Spouse and Family Issues
• Shipboard Life
• Quality of Life
• MWR Programs
• FFSC Programs
• Community Specific Surveys
• Individual Augmentation

9
N1 Sponsored Navy-wide Personnel Surveys

• Navy-wide Personnel Survey (AD)
• Job Satisfaction, career development, leadership,
  training, education, financial status, morale,
  gender integration, PERSTEMPO, retention
  predictors, career plans. Began in 1990.
  Administered annually or biannually
• Quality of Life (AD and Navy spouses)
• Shipboard life, work satisfaction, health,
  preparedness, residence, leisure/recreation,
  relationships, personal development, standard of
  living, spiritual well-being, overall quality of
  life. Began 1999. Administered biannually or
  triennially
• MWR/FFSC/Personal and Family Readiness (AD, Navy
  leadership, and Navy spouses)
• Use, satisfaction, and importance of Navy MWR
  programs and services. Began 1997. Annually (to
  one patron groupe.g., active, spouses, etc.
• Argus (AD and Reserves)
• Reasons for leaving/staying in the Navy,
  retention indicators, organizational commitment,
  reenlistment/ continuation intentions, perceived
  civilian employment opportunities. Began around
  2002. Reserves added in 2006. Continuous
  administration.
• Pregnancy and Parenthood (AD)
• Rate of pregnancy, rate of parenthood, attitudes
  toward pregnancy, attitudes toward parenthood,
  and attitudes toward contraception. Began 1988.
  Administered biannually.
• Diversity (AD)
• Perceptions of equal opportunity, command climate
  for EO/SH, frequency and nature of
  racial/ethnic/gender related insensitive remarks
  and behavior, participation in EO/SH training.
  Began 1989 as NEOSH Survey. Changed to Navy
  Climate Survey in 2006. Administered biannually
  or triennially.

10
NPRST Survey Sponsors and Customers

• CNP Navy Personnel Survey Navy QOL Survey
• N12F FC (Aegis) Quick Poll
• N13, ASN Pregnancy and Parenthood Survey
• N134 Navy Climate Survey
• N135 Navy Spouse Survey, Navy IA Spouse Survey
• PERS-4 Captains Retention Quick Poll
• CCD Special Forces Retention Quick Poll,
  Communication Quick Poll
• CPPD Enlisted Education, Character Quick Polls
• CNRC DEP Survey
• LANTFLEET Sea Swap Surveys
• Naval Aviation Enterprise Aviation Diversity
  Quick Poll
• Supply Corps USS Lincoln Storeroom Consolidation
  Prototype Poll
• Reserves Reserve Career Intentions Survey
• Marine Corps Marine Corps Climate Assessment
  Survey

11
Quick Polls

12
Problem

• Navy typically obtains Sailor attitude and
  opinion data on personnel-related issues through
  large scale surveys
• Mailed to 10,000
• Time consuming
• Costly
• Navy leaders need a mechanism to quickly yet
  accurately pulse the Fleet and obtain valid,
  reliable information to aid decision-making

13
Solution

• Conduct quick, scientifically sound, web-based
  polls of Navy personnel
• Examples of Quick Polls conducted to date

• Retention
• Kitty Hawk
• Vinson SG
• SRB
• ET/FC
• Communities
• LDO/CWO
• Medical
• Supply
• USS Lincoln
• Officer
• FTS
• SWO
• IT AIP
• Naval Aviation Enterprise

• Media/Communications
• CHINFO Summer Pulse
• Navy Communications/Sea Warrior
• CHINFO Media Usage
• Program Evaluations
• Navy Sailor and Family Services
• Personal Financial Management
• Diversity
• SAVI
• Sea Swap (I - V)
• Personal Family Readiness
• Operational Risk Management
• Enlisted Education Requirement
• Character Development

14
Navy Quick Poll Technical Approach
1. Select Target Population
2. Contact Target Population
3. Deliver Polling Instrument
4. Analyze Data

• Navy Message System -- Until NMCI/IT-21 fully
  implemented

• Internet-based poll
• Poll open 5 days

• Random samples extracted from electronic
  personnel files

• Minimal breakouts
• Standardized reporting for all polls
• Graphical representation of data limited text

15
Technical Approach (Cont.)

• Provide reliable, credible, and representative
  data with
• Margin of error of 5
• Standard data tabulations and limited breakouts
• Take a quick pulse of Sailors attitudes and
  opinions through brief, focused polls
• Results provided within 13 working days from
  topic selection to reporting of results
• Entirely web-based

http//quickpolling.nprst.navy.mil/
16
Navy Quick Poll Standard Timeline
Days
1
2
3
4
5
6
7
8
9
10
11
12
13
Receive Polling Topics
Question Development
Draft Notification Letter
Pull Sample
Sponsor Approval
IRB/ Survey Approval
Verify Sample
Web Site Development
Test Web Survey
Prepare Send Navy Messages
Survey in Field
Survey Closed
Develop brief
Weighting Analysis
Report Reviewed/Approved
Send brief to sponsor
17
Return on Investment

• Provide Navy leadership with rapid yet accurate
  mechanism to determine Fleet attitudes and
  opinions on personnel-related issues

• Measures of Success
• Number of days to complete each poll
• Response rate
• Completion of projected polls
• Use of data by sponsors
• Number of flag-level briefings
• Number of days to post results on Quick Poll
  website

18
Survey Operations Center (SOC)
19
SOC Background

• Navys in-house Survey Operations Center (SOC)
  was created in early 1990s at NPRDC in San Diego,
  CA

20
Moving the SOC

• Relocated to Millington, TN in 1999
• New name
• Navy Personnel, Research, Studies Technology
  Department (NPRST)
• SOC renamed Sue Ryan Survey Operations Center

21
Current SOC Capabilities

• Administrative
• Accurate cost estimates
• Separate funding lines
• Electronic status reports
• Operational
• Paper, internet, or email surveys
• Single or multi-wave survey administration
• Survey design and tracking
• Survey processing and scanning
• Internet data downloadssurvey counts in real
  time

22
Current SOC Capabilities (contd.)

• Paper surveys designed in DesignExpert (currently
  on legacy)
• Paper surveys scanned using OpScan 10 scanners
  and ScanTools 2 software (on legacy,
  non-networked)
• Data saved in multiple file types using
  StatTransfer

23
Internet Surveys

• Sample selected from EMF/OMF databases using SPSS
• Survey is programmed in SSIWeb which creates Perl
  files to load on web server
• Data is downloaded through Internet Explorer and
  then loaded into SSIWeb for export to SPSS or
  other

24
Internet Surveys (contd.)

• Data is analyzed in SPSS or SAS
• Follow-on analyses may be conducted in SUDAAN,
  AMOS, Polyanalyst, or Mplus
• Survey pages printed using Acrobat
• Results pages programmed using FrontPage and
  Photoshop

25
Emerging Capabilities

• Capability to mass email notifications/reminders/
  thank yous
• Capability to email completable PDF survey forms
  for those having problems access the Internet
• Capability to instantaneously conduct surveys
  through text messages sent via cell phones