Programmers and Analysts: The Yin and Yang of Policy Research

1
Programmers and Analysts The Yin and Yang of
Policy Research

• Nancy R. Burstein

October 4, 2006
2
The Yin and Yang of Policy Research
3
What Do Women Want?

• (Sigmund Freud)

4
What Do Analysts Want From Programmers?

• Three things!
• Your intelligent participation in the specs
• Your readable log file
• Your self-documenting output or list file

5
Your intelligent participation in the specs

• You know things that I dont for example,
• that a variable is mostly missing
• that a variable is unreliable
• that there are other values to a variable than
  appear in a codebook or instrument
• that the skip pattern wasnt followed
• I know things that you dontbut I can make
  mistakes
• Negotiate re. treatment of missing values,
  special values, data anomalies
• The forbidden phrase It did seem odd to me at
  the time
• Get it in writing!
• You can be a hero! Make a contribution to the
  analysis specs!

6
Example Survey Nonresponse Rate

• Me How many selected sample members were not
  interviewed because you couldnt get in touch
  with them?
• Programmer
• Do you mean How many were attempted 10 times and
  then we gave up?
• How about the ones we reached on the 10th attempt
  that declined to be interviewed?
• And what about the ones that scheduled a
  call-back but didnt do the interview?

7
Example (continued)

• Me AAAGH!!! I really need to know
• Total number of telephone numbers fielded (A)
• Number of interviews completed (B)
• Number of women determined to be eligible but not
  interviewed (C)
• Number determined to be ineligible, and why (D)
• Number for which eligibility not determined (E)
• Eligible NR C (BC)/(BCD) x E
• NRR Eligible NR / Eligible NR B
• THANK YOU, programmer!

8
Treat missing/special/anomalous values explicitly

• The task Create an indicator for being
  employed during the year, based on a measure of
  number of weeks employed
• Not good anywork(wkswork gt 0)
• Better if wkswork ne . then anywork (wkswork
  gt 0)
• Still better if 0 le wkswork le 52 then
  anywork(wkswork gt 0)
• Your code, like Caesars wife, should be above
  suspicion!

9
For extra credit, understand the research
questions

• RQ What is the occupational distribution of
  recently discharged veterans (RDVs) relative to
  that of similar non-veterans?
• We use the Survey of Income and Program
  Participation which has lots of useful
  information on employment and veteran status
• We create matched pairs of RDVs and non-veterans
  based on age, race/ethnicity, sex, education,
  region of the country, etc.
• We compare occupational distribution
• And what do we do if one member of the pair has
  an occupation and the other member does not?
• Do we drop the pair?
• AROOO yip yip yip!
• The more you know, the more you can contribute!

10
Your readable log file

• Why you should give the analyst your log file
• Because its part of the job you did
• Because the analyst should confirm that you
  understood the specs
• Because even you sometimes make mistakes
• Because it may prompt the analyst to ask for
  something else
• Can analysts read log files?
• Maybe not but you can help
• Analyst may bring the log file to another
  programmer
• Supplementary tools for the analyst
• PROC CONTENTS or codebook for the incoming data
  set
• Markup of questionnaire
• Before you give the analyst the log file of
  course you would read it yourself!

11
Some of my favorite NOTES and WARNINGS

• NOTE The data set WORK.ONE has 201 observations
  and 29 variables.
• NOTE Variable X is uninitialized.
• NOTE MERGE statement has more than one data set
  with repeats of BY values.
• NOTE Character values have been converted to
  numeric values at the places given by
  (Line)Column. 123
• WARNING Apparent symbolic reference X not
  resolved.
• WARNING The data set ONE may be incomplete.
  When this step was stopped there were 100
  observations and 10 variables.
• NOTE Only one observation in a stratum in
  domain SB2 for variable(s) SB7A. The variance of
  SB7A in that stratum is estimated by zero.
• You should never be surprised by a NOTE!

12
Making the log file more readable

• (Tastes may vary!)
• Appearance on the page
• Indentations
• Font size
• One line per line of code
• Comments
• Short variable names
• Logical clarity, even at the expense of brevity
• IF/THEN/ELSE
• Arrays, macros, link/return

13
Your self-documenting list file

• Titles for (almost) everything include date and
  whatever is special about this run for example
• Restricted sample
• Different version of key variable
• Different statistical procedure, different
  included variables, etc.
• FOOTNOTE (program name) - sysdate - systime
• For every data set you create, PROC CONTENTS and
  PROC PRINT
• Diagnostics means, freqs, crosstabs
• LIST MISSING options
• Formats (but not in permanent data sets)
• Label the variables

14
An Example
15
Conclusion Were in this together!