Preliminary Data: Expert Views on Best Practices Related to Enforcing Underage Drinking Laws

1
National Evaluation Enforcing Underage Drinking
Laws Program
EUDL National Leadership Conference Dallas,
Texas September 19, 2002
Funding provided by the Office of Juvenile
Justice and Delinquency Prevention
2
Acknowledgements

• All 50 state EUDL coordinators
• All Key Actor Survey participants
• All Law Enforcement Agency Survey respondents
• OJJDP, PIRE, and USC

3
Evaluation Design
1999 2000 2001
Process Evaluation Key Actor Survey Case
Studies Local Project Director Survey
50 States
50 States D.C.
50 States D.C.
6 States
6 States
3 States (BP)
216 Projects
Impact Evaluation Law Enforcement
Agency and Youth Surveys
First Round of Discretionary Grants
52 Intervention 52 Comparison
52 Intervention 52 Comparison
52 Intervention 52 Comparison
Second Round of Discretionary Grants
17 Intervention 17 Comparison
17 Intervention 17 Comparison
4
Evaluation Design (cont.)
2002 2003 2004
Process Evaluation Key Actor Survey
Local Project Director Survey
50 States D.C.
50 States D.C.
50 States D.C.
36 Projects
Impact Evaluation Law Enforcement
Agency and Youth Surveys
Second Round of Discretionary Grants
17 Intervention 17 Comparison
Fourth Round of Discretionary Grants
36 Intervention 36 Comparison
36 Intervention 36 Comparison
36 Intervention 36 Comparison
5
Themes

• Galvanized Activity and New Relationships
• Increased Enforcement
• Youth Behavior Any Change?
• Best and Most Promising Practices

6
State Government Agencies Very Involved in
EUDL
N 49
7
State Government Agencies Very Involved in
EUDL (cont.)
N 49
8
Local Organizations Very Involved in EUDL
N 49
9
Other Groups Very Involved in EUDL
N 49
10
Types of Interventions (No. of States Very
Involved)
N 40-50
11
Additional Interventions (No. of States Very
Involved)
(N 50)
12
Interventions Funded with Local EUDL Grants
2001 Local Project Director Survey
(N 211)
13
Local Agency Focus on Underage DrinkingBefore
and After Receiving EUDL Funding
(N 72 125)
14
Perceived Obstacles
N 50
15
Themes

• Galvanized Activity and New Relationships
• Increased Enforcement
• Youth Behavior Any Change?
• Best and Most Promising Practices

16
Enforcement Activities (No. of States Very
Involved)

• Compliance Checks - 35 States
• Cops in Shops 22 States
• Shoulder Tap program 16 States

N 49-50
17
Target of Enforcement Intervention (No. of
States Very Involved)
N 49-50
18
Impact Evaluation Analysis Design
Year 1 Year 2 Year 3
Round 1 Communities (104)
1999 Data Collection
2000 Data Collection
2001 Data Collection
Round 2 Communities (38)
2000 Data Collection
2001 Data Collection
2002 Data Collection
19
Median Number of Compliance Checks
P-value 0.030 N (Comparison) 76 N
(Intervention) 105
Year 1
Year 2
20
Median Number of Cops in Shops Operations
P-value 0.005 N (Comparison) 39 N
(Intervention) 68
Year 1
Year 2
21
Median Number of Arrests for Purchase,
Possession, or Use of Alcohol
P-value 0.049 N (Comparison) 88 N
(Intervention) 81
Year 1
Year 2
22
Other Enforcement Findings No Statistically
Significant Changes

• Shoulder Tap Program
• Monitoring/patrolling parking lots
• Tracing alcohol source
• Citations/arrests for false ID
• Citations/fines/license suspension of businesses

23
Percentage of Agencies Reporting Strong Local
Support for Underage Drinking Efforts
P-value 0.610 Odds Ratio 0.82 N (Comparison)
200 N (Intervention) 207
Year 1
Year 2
24
Themes

• Galvanized Activity and New Relationships
• Increased Enforcement
• Youth Behavior Any Change?
• Best and Most Promising Practices

25
Alcohol Consumption in the Past 30 Days
P-value 0.662 Odds Ratio 0.94 N (Comparison)
2410 N (Intervention) 2370
Year 1
Year 2
26
Other Youth Behavior Findings No Statistically
Significant Changes

• Past 7 day alcohol consumption
• Binge drinking
• Drinking and driving
• Attempts to purchase alcohol
• Perception of alcohol use among peers
• Consequences of alcohol use

27
Why do Program Evaluations Seldom Detect an
Impact on Behaviors?

• Delayed, Incomplete, or Failed Program
  Implementation
• Flawed Program Theory
• Short Intervention Period
• Secular Trends
• Competing Programs
• Non-comparable Intervention and Comparison Groups
• Measurement Error
• Use of Wrong Measures
• Inadequate Statistical Power
• Adapted from Lipsey and Cordray, 2000

28
Themes

• Galvanized Activity and New Relationships
• Increased Enforcement
• Youth Behavior - No Solid Evidence of Change
• Best and Most Promising Practices

29
Expert Survey Methods

• Literature review identified experts in UAD
  research
• Practitioner experts were identified using
  snowball sampling techniques
• Web based survey
• N 40
• Response rate 75

30
Analysis
Empirical Evidence
Larger Quantity
Smaller Quantity
High
Perceived Effectiveness
Not High
31
Findings Best Practices
32
Findings Promising Practices
33
Findings Promising Practices
34
Findings Promising Practices

• DWI Enforcement
• Driving under the influence emphasis patrols
• Other Enforcement Approaches (including training)
• Training of law enforcement officers to promote
  better enforcement efforts
• Enforcement and education efforts focused on
  parents and landlords who allow underage drinking
  parties to take place on their property

35
Evaluation Products

• Bulletin
• Year 1, Year 2, and Year 3 Reports
• Design Paper
• Outcome Paper
• Best Practices Paper
• Best Practices Case Studies
• Papers on Assorted Topics LE, Sexual Assault,
  Parenting

36
End of Leadership Conference Presentation