Surveys, Field Research

1
Research Methods

• Surveys, Field Research Secondary Data

2
Week 6 Lecture outline

• Survey data collection methods
• Types of Surveys
• Field Research
• Secondary Data
• Class Round on Methods

3
Topics appropriate to survey research

• Counting crime asking people about
  victimization counters problems of data collected
  by police
• Self-reports dominant method for studying the
  etiology of crime
• Frequency/type of crimes committed
• Prevalence (how many people commit crimes)
  committed by a broader population

4
Topics appropriate to survey research

• Perceptions and attitudes to learn how people
  feel about crime and CJ policy
• Policy proposals search for ways to respond to
  crime that are supported by the general public
• Targeted victim surveys used to evaluate policy
  innovations program success
• Other evaluation uses e.g., measuring community
  attitudes, citizen responses, etc.

5
Open- and close-ended questions

• Open-ended respondent is asked to provide his
  or her own answer
• What were your initial expectations of the
  program?
• ________________________________________________
• Closed-ended respondent selects an answer from
  a list or scale
• On a scale of 1 to 5, where 1 is low and 5 is
  high, rate your initial expectations of the
  program.

1 2 3 4 5
Low High
6
Open versus closed?

• Choice between these question types will depend
  on the aim of your research, the study design,
  the population being study, etc.
• Advantages and disadvantages to both.
• Possible gains in reliability (closed-ended)
  balanced with possible losses in breadth of
  information and validity.
• One strategy is to combine open- and close-ended
  questions.

7
Designing questions

• Make items clear avoid ambiguous questions do
  not ask double-barreled questions
• Short items are best respondents like to read
  and answer a question quickly
• Avoid negative items leads to misinterpretation
• Avoid biased items and terms do not ask
  questions that encourage a certain answer
• Closed question response categories must be
  exhaustive and mutually exclusive.

8
Survey data collection methods

• 1. Self-administered
• 2. In-person interview
• 3. Telephone interviews
• Focus Australias National Crime Victimization
  Survey Methodology

9
Australias Crime Victimization Survey

• The MPHS was conducted as a supplement to the
  monthly LFS.
• Each month one eighth of the dwellings in the
  LFS sample were rotated out of the survey.
• In 2012-13, all of these dwellings were selected
  to respond to the MPHS each month
• . In these dwellings, after the LFS had been
  fully completed for each person in scope and
  coverage, a person aged 15 years and over was
  selected at random (based on a computer
  algorithm) and asked the various MPHS topic
  questions in a personal interview.
• If the randomly selected person was aged 1517
  years, permission was sought from a parent or
  guardian before conducting the interview. If
  permission was not given, the parent or guardian
  was asked the crime questions on behalf of the
  1517 year old.
• Questions relating to sexual assault, alcohol or
  substances contributing to the most recent
  physical or face-to-face threatened assault were
  not asked of proxy respondents. Only those
  persons aged 18 years and over were asked
  questions on sexual assault.
• Data was collected using Computer Assisted
  Interviewing, whereby responses were recorded
  directly onto an electronic questionnaire in a
  notebook computer, usually during a telephone
  interview.

10
A Link to the USA National Crime Victimization
Survey

• http//www.bjs.gov/index.cfm?tydcdetailiid245

11
(No Transcript)
12
UCR and NCVS

• UCR data are based on reported criminal acts
  (offender characteristics)
• NCVS data based on individuals actually
  victimized (characteristics of victims)

13
Assessment of NCVS

• Document a massive amount of crime that goes
  unreported
• Underestimate crime rate
• Insignificant crimes tend to be forgotten
• Victims of several crimes may also forget about
  all the crimes
• Females do not report victimization if her abuser
  live in the same household
• Whites and college graduates are more likely to
  report being victimized

14
Assessment of NCVS

• NCVS respondents are interviewed every six months
  (7 interviews)
• Reported victimization rates usually decease with
  each interview (awareness of victimization)
• Overestimation of some crimes
• Respondents might mistakenly interpret some
  noncriminal events as crimes
• Telescoping effect

15
Self-reports data

• Created to complement UCR and NCVS
• Interviews or questionnaires
• Demonstrate the prevalence of offending (the
  proportion of respondents who have committed a
  particular offence)
• Incidence of offending (the average number of
  offences per person in the study)

16
Samples for self-reports

• Target Adult inmates of jails and prisons PREA
  findings on sexual assault
• Target Adolescents, usually high school students
• The most important finding delinquency is very
  common
• Middle-class youth commit as much crime as
  working-class youth
• Testing criminological theories Travis Hirschis
  Research Study testing Control Theory( compared
  to subculural Strain)

17
Assessment of self-report studies

• Focus on minor and trivial offenses (truancy,
  running away from home, minor drug and alcohol
  use)
• Although recent studies (NYS) asked subjects
  about rape and robbery
• Respondents might not to tell the truth
  (reliability issues)

18
If respondents lie.

• Self-report data can be checked against police
  records, school records, interviews with teachers
  and parents
• The use of, or threat of , polygraph validation
  (20 change their initial responses when
  threatened with a lie detector)
• Subsequent interviewing of subjects permits
  probing regarding the details and context of acts
• Use of lie scales

19
Focus Self Reported Substance Use Among
Detainees in Australia

• http//www.aic.gov.au/media_library/conferences/ev
  aluation/mcgregor.pdf
• http//192.190.66.70/documents/7/E/8/7B7E8D4A8E-A
  5AF-4D3B-8821-ED8A1BA489B67Drpp93.pdf

20
UCR, NCVS, and self-reports

• None of the three is perfect
• For the best estimates of the actual number of
  crimes, NCVS data are preferable
• For the best estimates of offender
  characteristics, self-reports and NCVS are
  preferable
• UCR are superior for understanding the
  geographical distribution of crime

21
Self-administered questionnaires

• Can be home-delivered
• Researcher delivers questionnaire to home of
  sample respondent, explains the study, and then
  comes back later
• Mailed (sent and returned) survey is most common
• Researchers must reduce the trouble it takes to
  return a questionnaire
• Goal is High Response Rate

22
Computer-based self-administration
22

• Via email, website
• Issues
• representativeness
• mixed in with, or mistaken for, spam
• requires access to Web
• sampling frame?

23
Week 7 More Survey Research

• What we will Cover
• 1.Interviewing techniques and Focus Groups
• 2. A Comparison of Survey Methods phone/
  computer( survey monkey), mail, and in-person
• 3. Field Research The good, the bad, and the
  ugly
• 4. Agency Records, Content Analysis and Secondary
  Data

24
In-person interview survey
24

• Typically achieve higher response rates than mail
  surveys (80-85 is considered good)
• Demeanor and appearance of interviewer should be
  appropriate interviewer should be familiar with
  questionnaire and ask questions precisely
• When more than one interviewer administers,
  efforts must be coordinated and controlled
• Practice interviewing

25
Specialized interviewing
25

• Two variations
• General interview guide less structured, lists
  issues, topics, questions you wish to cover no
  standardized order
• Standardized open-ended interview more
  structured, specific questions in specific order
  useful in case studies, retrieves rich detail in
  responses

26
Telephone surveys
26

• 94 of all households now have telephones
• Random-Digit Dialing
• Obviates unlisted number problem
• Often results in business, pay phones, fax lines
• Saves money and time, provides safety to
  interviewers, more convenient
• may be interpreted as bogus sales calls ease of
  hang-up

27
Comparison of the three methods
27

• Self-administered questionnaires are generally
  cheaper, better for sensitive issues than
  interview surveys
• Using mail local and national surveys are same
  cost
• Interviews more appropriate when respondent
  literacy may be a problem, produce fewer
  incompletes, achieve higher completion rates
• Validity low in survey research reliability high
• Surveys are also inflexible, superficial in
  coverage

28
Tips on self-report items
28

• Convince subjects you will guarantee
  confidentiality and anonymity
• Minimize possible social undesirability you are
  asking respondents to admit
• Phrase questions in non-judgmental manner
• Bear in mind fading memory when setting time
  frame

29
Focus groups
29

• 12-15 people brought together to engage in guided
  group discussion of some topic( e.g. addicts
  recovery)
• Members are selected to represent a target
  population, but cannot make statistical estimates
  about population
• Most useful when precise generalization to larger
  group is not necessary
• May be used to guide interpretation of
  questionnaires following survey administration
• Examples Drug addicts and reentry, homeless sex
  offenders, teenagers and sexting

30
Getting Out

• Field Research

31
Field Research

• Field research encompasses two different methods
  of obtaining data
• Direct observation
• Asking questions
• May yield qualitative and quantitative data
• Often no precisely defined hypotheses to be
  tested
• Can be used to make sense out of an ongoing
  process

32
Topics Appropriate to Field Research

• Gives comprehensive perspective enhances
  validity
• Go directly to phenomenon, observe it as
  completely as possible
• Especially appropriate for topics best understood
  in their natural setting
• How street-level drug dealers distinguish
  customers
• Studies of vice, e.g., prostitution and
  drug-use.
• Aspects of physical settings, Disney World,
  social control

33
Various Roles of the Observer (Gold, 1969)

• Complete participant participates fully true
  identity and purpose are not known to subjects
• E.g., posing as a bar patron becoming a police
  officer or corrections worker
• Participant-as-observer make known your
  position as researcher and participate with the
  group
• E.g., study of active drug users Julie Mueler
  and the Guardian Angels
• Observer-as-participant make known your
  position as a researcher do not actually
  participate
• E.g., Observational study of police
  patrolride-along research on gangs
• Complete observer observes without becoming a
  participant
• E.g., court observation, Chicago neighbourhood
  study( windshield studies)

34
Observer status

• Be aware of, and document role of researcher
  (extent of participation)
• Be aware that all observation is subjective.
• Be aware of the possible effect of
  participation.
• Be aware of often competing ethical and
  scientific values related to all observational
  studies.

35
Asking Questions

• Field research is often a matter of going where
  the action is and simply watching and listening
• Also a matter of asking questions recording
  answers
• Field research interviews are much less
  structured than survey interviews
• Ideally set up and conducted just like a normal,
  casual conversation

36
Preparing for the Field

• Access to formal organizations
• Find a sponsor, write a letter to executive
  director, arrange a phone call, arrange a meeting
• Access to subcultures
• Find an informant (e.g., person who works with
  offenders), use that person as your in
• Snowball sampling is useful as informant
  identifies others, who identify others, etc.

37
Sampling in Field Research

• Controlled probability sampling used rarely
  purposive sampling is common
• Bear in mind two stages of sampling
• To what extent are the situations available for
  observation representative of the general
  phenomena you wish to describe and explain?
• Are your actual observations within those total
  situations representative of all observations?

38
Recording Observations

• Note taking, tape recording when interviewing and
  when making observations
• Videotaping or photographs can make records of
  before and after some physical design change
• Field notes observations are recorded as
  written notes, often in a field journal first
  take sketchy notes and then rewrite your notes in
  detail
• Structured observations observers mark
  closed-ended forms, which produce numeric measures

39
Linking Field Observations and Other Data

• Useful to combine field research with surveys or
  data from official records
• Baltimore study of the effects of neighborhood
  physical characteristics on residents
  perceptions of crime problems (Taylor, Shumaker,
  Gottfredson, 1985)
• Perceptions surveys
• Physical problems (1) observations, (2) actual
  population and crime information - census data
  crime reports from police records

40
Strengths and Weaknesses of Field Research

• Provides great depth of understanding
• Flexibility (no need to prepare much in advance)
• More appropriate to measure behavior than surveys
• High validity quant. measures incomplete
  picture
• Low reliability often very personal
• Generalizability personal nature may produce
  findings that may not be replicated by another
• Precise probability samples cant normally be
  drawn

41
Other Sources to Consider

• Agency Records, Content Analysis and Secondary
  Data

42
Secondary Data

• Data from agency records agencies collect a
  vast amount of crime and CJ data
• Secondary analysis analyzing data previously
  collected
• Content analysis researchers examine a class of
  social artifacts (typically written documents)

43
Topics Appropriate for Agency Records

• Most commonly used in descriptive or exploratory
  studies
• Content analysis often center on links between
  communication, perceptions of crime problems,
  individual behavior, CJ policy

44
Types of Agency Records

• Published Statistics govt organizations
  routinely collect and publish compilations of
  data (e.g., ABS, BOCSAR, AIC) often available in
  libraries and online
• Nonpublic Agency Records agencies produce data
  not routinely released (e.g., police departments,
  courthouses, correctional facilities)
• New Data Collected by Agency Staff collected
  for specific research purposes less costly
  more control

45
Units of Analysis in Criminal Justice Data

• Criminal Activity
• Incidents
• Crimes violated
• Victims
• Offenders
• Court Activity
• Defendants
• Filings
• Charges and Counts
• Cases
• Appearances
• Dispositions
• Sentences

• Apprehension
• Arrests
• Offenders
• Charges
• Counts
• Corrections
• Offenders
• Admissions
• Returns
• Discharges

46
Sources of Reliability and Validity Problems

• Virtually all CJ record keeping is a social
  process social production of data
• Records reflect decisions made by CJ personnel as
  well as actual behavior by juveniles and adults
• Discretion factors in to recordkeeping
• CJ organizations are more interested in keeping
  track of individual cases than in examining
  patterns
• Potential for clerical errors due to volume of
  data

47
Content Analysis

• Systematic study of messages can be applied to
  virtually any form of communication
• Decide on operational definitions of key
  variables
• Decide what to watch, read, listen to time
  frame
• Analyze collected data
• Well suited to answer who says what, to whom,
  why, how, and with what effect?

48
Aspects of Sampling and Coding in Content
Analysis 2

• Reminders
• Remember operational definition of variables, and
  their mutually exclusive exhaustive attributes
• Pretest coding scheme
• Assess coding reliability via intercoder
  reliability method and test-retest method

49
Secondary Analysis

• Sources websites, libraries
• AIC, http//www.aic.gov.au/
• BOCSAR, http//www.bocsar.nsw.gov.au/
• BJS, NCVS, ICPSR, NACJD
• Advantages cheaper, faster, benefit from work
  of skilled researchers
• Disadvantages data may not be appropriate to
  your research question least useful for
  evaluation studies (which are designed to answer
  specific questions about specific programs)

50
Class Round
Whats the most appropriate method(s) for
investigating your research evaluation
question(s)?