From Start to Pilot: Comprehensive Redesign of a Questionnaire Measuring Foreign Direct Investment - PowerPoint PPT Presentation

Loading...

PPT – From Start to Pilot: Comprehensive Redesign of a Questionnaire Measuring Foreign Direct Investment PowerPoint presentation | free to view - id: 710aea-ODNjO



Loading


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation
Title:

From Start to Pilot: Comprehensive Redesign of a Questionnaire Measuring Foreign Direct Investment

Description:

From Start to Pilot: Comprehensive Redesign of a Questionnaire Measuring Foreign Direct Investment Alfred D. Tuttle Rebecca L. Morrison United States Census Bureau – PowerPoint PPT presentation

Number of Views:32
Avg rating:3.0/5.0
Slides: 32
Provided by: tut59
Category:

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: From Start to Pilot: Comprehensive Redesign of a Questionnaire Measuring Foreign Direct Investment


1
From Start to Pilot Comprehensive Redesign of a
Questionnaire Measuring Foreign Direct
Investment
  • Alfred D. Tuttle
  • Rebecca L. Morrison
  • United States Census Bureau
  • David H. Galler
  • United States Bureau of
  • Economic Analysis

2
Overview Redesign of BEA Form BE-605
  • Reimbursable project with BEA
  • Multiple, intensive stages of research, design,
    testing, revision
  • Qualitative and quantitative evaluations
  • New BE-605 form mailed out in March 2007

3
Outline
  • Background BEA Form BE-605
  • Summary of stages of research
  • Findings
  • Next steps

4
Background
  • BE-605 (quarterly) and BE-15 (annual)
  • Foreign Direct Investment (FDI) gt10 of voting
    rights in an incorporated entity (or equivalent
    interest in unincorporated entity)
  • Sample foreign-owned businesses in US

5
Research stages and methods
  • Background investigations
  • Expert review of form
  • Focus groups with survey analysts
  • Observations of callbacks
  • Respondent debriefings

6
Research stages and methods
  • Cognitive testing of mockups

7
Research stages and methods
  • Pilot test
  • Respondent debriefings
  • Evaluation questions

8
Findings from background research
  • Data collection and design issues
  • Dense formatting
  • Economic accounting concepts differ from US GAAP
  • Separate instructions

9
Recommendations
  • Formatting recommendations
  • Letter size pages (not legal)
  • More open visual design
  • Incorporate instructions into form
  • within questions
  • adjacent to questions on facing page
  • key instructions become questions
  • Diagrams to augment questions

10
Original form
11
Recommendation More open design
12
Recommendation Embedded instructions
13
Recommendation Adjacent instructions
14
Recommendations Questions from instructions, and
diagrams
15
Cognitive testing
  • Problematic sections mocked-up and tested (not
    entire forms)
  • 5 rounds of cognitive interviews
  • Iterative process
  • mock-ups revised between rounds
  • design features modified re-tested

16
Findings from cognitive testing
  • New design features were effective
  • Majority of respondents liked the more open
    format
  • Embedded instructions would likely improve data
    quality
  • Some Instructions?Questions and diagrams
    required extensive modification, though most did
    not

17
Developing 605 pilot
  • Redesigned entire BE-605 form
  • Close collaboration with BEA survey staff

18
Pilot test
  • Mailed 2nd quarter 2006
  • Sent to sub-sample (n 653 survey sample
    4,000)
  • Completing pilot form optional
  • Response rate 53 (350 returned forms) during
    initial 8-week processing period

19
Pilot form evaluation
  • Respondent debriefings
  • Evaluation questions in form

20
Findings R debriefings
  • Favorable impressions of new form
  • More pleasing to the eye
  • White reporting spaces easier to see against
    colored background
  • Larger font, bold text, more space
  • Letter size pages easier to handle
  • Diagrams were helpful
  • Embedded instructions helpful

21
Findings R debriefings
  • Drawbacks
  • More pages
  • Booklet format makes it harder to fax
  • Took longer to complete than original form
    (disruption of reporting routine)

22
Limitations
  • Stymied by experienced Rs, reporting routines
  • though even experienced Rs were helped by new
    formatting and instructions.

23
Findings Evaluation questions
  • Four questions appended to survey
  • Each item received between 279 and 290 responses

24
Findings Evaluation questions
  • Instruction placement?
  • 79.3 - Near questions
  • 18.3 - In separate booklet
  • 2.4 - Not applicable I havent completed
    the usual BE-605

25
Findings Evaluation questions
  • Open space and number of pages?
  • 48.7 - less open space, fewer pages
  • 47 - more open space, more pages
  • 4.3 - Not applicable I havent completed
    the usual BE-605

26
Findings Evaluation questions
  • Use of diagrams?
  • 65.4 - very helpful or somewhat helpful
  • 11.7 - only a little helpful
  • 18 - did not need them
  • 3.9 - confusing

27
Findings Evaluation questions
  • Easy/difficult compared to usual form?
  • 50.5 - harder
  • 42.5 - easier
  • 7 - Not applicable I havent completed the
    usual BE-605

28
Findings Evaluation questions
  • Instructions preferred near questions
  • Organization charts helpful
  • Easier or more difficult?
  • Preference for more/fewer pages?

29
Summary
  • Redesigned BE-605 went into production in March
    2007
  • Recommended analysis of edit checks during form
    processing

30
Summary
  • Multi-method approach to comprehensive survey
    redevelopment

31
Thanks!
  • Alfred D. Tuttle
  • alfred.d.tuttle_at_census.gov
  • Rebecca L. Morrison
  • rebecca.l.morrison_at_census.gov
  • David Galler
  • david.galler_at_bea.gov
About PowerShow.com