Reporting and Dissemination of Evaluation Findings from Programs Serving Pregnant and Parenting Teen

1
Reporting and Dissemination of Evaluation
Findings from Programs Serving Pregnant and
Parenting TeensPresented by Olivia Silber
Ashley, Dr.P.H.Presented toOffice of
Adolescent Pregnancy Programs Care Grantee
Conference, February 1-2, 2007, New Orleans,
Louisiana
3040 Cornwallis Road P.O. Box 12194
Research Triangle Park, NC 27709
Phone [phone]
e-mail osilber_at_rti.org
Fax [phone]
RTI International is a trade name of Research
Triangle Institute
2
Overview

• Elements of a strong report
• Publishing
• Avenues for dissemination

3
Reports about Evidence-Based Programs for
Pregnant and Parenting Teens

• Summary of findings of the research community on
  pregnancy and parenting programs for adolescents
  (Hoyer, 1998)
• Preponderance of quasi-experimental designs with
  small samples
• Lack of theoretically driven, longitudinal
  research specific to the developmental level of
  the population
• Public health nursing intervention for adolescent
  mothers (Koniak, Koniak-Griffin, Anderson,
  Verzemnieks, Brecht, 2000)
• Nurse-Family Partnership home visitation program
  (Nguyen, Carson, Parris, Place, 2003)
• Project CHARM (Koniak-Griffin, Lesser, Nyamathi,
  Uman, Stein, Cumberland, 2003)
• Home visiting intervention to improve parenting
  skills among American Indian adolescent mothers
  (Barlow et al., 2006)
• Home-based mentoring program to delay second
  births (Black et al., 2006)

4
How AFL Fits In

• Support for publications
• Evaluation technical assistance and training
• Cross-site evaluation
• Capacity assessment
• Parent involvement assessment
• Partnering with scientific experts

5
Elements of a Strong Report

• End of year evaluation report templatea detailed
  guide
• Clear, detailed understanding of the program
• Theoretical basis for behavior change, captured
  in logic model and instrumentation
• Program objectives
• Specific
• Measurable
• Achievable (based on literature or existing data)
• Realistic (based on literature or existing data)
• Time-framed (appropriately to literature,
  existing data, and data collection schedule)
• Clearly articulated evaluation research questions

6
Process Evaluation Reporting

• Process evaluation measures assess qualitative
  and quantitative parameters of program
  implementation
• Attendance data
• Participant feedback
• Program-delivery adherence to implementation
  guidelines
• Facilitate replication, understanding of outcome
  evaluation findings, and program improvement
• Source USDHHS. (2002). Science-based prevention
  programs and principles, 2002. Rockville, MD
  Author.

7
Sampling Strategy and Evaluation Design

• Large, representative sample
• Control or comparison group
• Appropriate to answer evaluation research
  questions
• Random assignment is the gold standard to answer
  research questions about program effectiveness
• Units for study (such as individuals, schools,
  clinics, or geographical areas) are randomly
  allocated to groups exposed to different
  treatment conditions
• Begin with most rigorous design possible
• Assess baseline differences between treatment
  groups
• Source USDHHS. (2002). Science-based prevention
  programs and principles, 2002. Rockville, MD
  Author.

8
Attrition

• Number of participants lost over the course of a
  program evaluation
• Some participant loss is inevitable due to
  transitions among program recipients
• Extraordinary attrition rates generally lower the
  degree of confidence reviewers are able to place
  on outcome findings
• Source USDHHS. (2002). Science-based prevention
  programs and principles, 2002. Rockville, MD
  Author.

9
Outcome Measures

• Assess actual behavior change
• Assess various risk and protective factors
  associated with behavior
• Quantify what they proport to address (i.e., they
  should be valid)
• Must show consistent results (i.e., they must be
  reliable)
• Source USDHHS. (2002). Science-based prevention
  programs and principles, 2002. Rockville, MD
  Author.

10
Missing Data

• Not the same as attrition (rate at which
  participants prematurely leave an evaluation)
• Absence of or gaps in information from
  participants who remain involved
• A large amount of missing data can threaten the
  integrity of an evaluation
• Source USDHHS. (2002). Science-based prevention
  programs and principles, 2002. Rockville, MD
  Author.

11
Data Collection

• Quality of measurement procedures
• Strong evaluations collect data using unbiased
  procedures
• Participant subject data are anonymous or
  confidential
• Ensure that data are coded and stored to protect
  individual identities
• Source USDHHS. (2002). Science-based prevention
  programs and principles, 2002. Rockville, MD
  Author.

12
Analysis

• Appropriateness of data analytic techniques for
  determining the success of a program
• Employ state-of-the-art data analysis techniques
  to assess program effectiveness by participant
  subgroup
• Use the most suitable current methods to measure
  outcome change
• Subgroup analyses allow evaluation of outcomes by
  participant age and ethnicity, for example
• Multivariate analysis controlling for variables
  associated with baseline differences and
  attrition
• Source USDHHS. (2002). Science-based prevention
  programs and principles, 2002. Rockville, MD
  Author.

13
Threats to Validity

• Evaluation design must establish a causal link
  between the program and its presumed outcomes
• Must be able to rule out other factors that could
  explain outcomes, such as
• Competing programs
• Concurrent media campaigns
• Effects of maturation among evaluation
  participants
• Source USDHHS. (2002). Science-based prevention
  programs and principles, 2002. Rockville, MD
  Author.

14
Integrity of the Evaluation

• Overall confidence reviewers can place in the
  findings of a program evaluation
• Confidence is derived from the sum of the
  positive assessment of the
• Evaluation study design
• Actual conduct of the evaluation
• Reviewers rate the merits of the science that
  guided the evaluation
• Source USDHHS. (2002). Science-based prevention
  programs and principles, 2002. Rockville, MD
  Author.

15
Publishing

• Publications about all aspects of evaluation
• Recruiting
• Participation
• Formative research
• Program description
• Implementation evaluation
• Outcome evaluation
• Epidemiological and etiological research
• Methodological research

16
An Example Formative Research

• Study had adolescents evaluate a parent manual
  developed for parents of children from birth to
  age 3
• Written and verbal responses were gathered about
  format, readability, and content
• Adolescents were aged 14 to 18 and enrolled in a
  high school program for adolescent mothers
• Results showed that overall comments were
  favorable
• Adolescents suggested including more information
  about basic child care for first-time mothers
  instead of primarily focusing on typical
  development and parent-child interactions
• Source Lambert, C. (1998). Removing the mystery
  Evaluation of a parent manual by adolescent
  parents. Adolescence, 33(129), 61-72.

17
Example Program Description

• Description of a school-based program combining
  traditional parent education with creative
  physical activity classes to assist newly
  delivered adolescent mothers
• The program contains sessions on
• Parenting information
• Creative coping strategies
• Fitness
• Role modeling of positive parenting behaviors
• Source Sadler, L.S., Cowlin, A. (2003). Moving
  into Parenthood A program for new adolescent
  mothers combining parent education with creative
  physical activity. Journal of Specialists in
  Pediatric Nursing, 8, 62-70.

18
An Example Outcome Evaluation

• Study to increase parenting skills and improve
  attitudes about parenting among parenting teens
  through a structured psychoeducational group
  model
• All parenting teens (n91) were offered a 12-week
  group parenting curriculum
• Comparison group (n54) declined the curriculum
  but agreed to participate in evaluation
• Pre-test, post-test measures included
  Adult-Adolescent Parenting Inventory (AAPI), the
  Maternal Self-Report Inventory (MSRI), and the
  Parenting Daily Hassles Scale
• Analyses controlled for mothers age, babys age,
  and race
• Results showed that program participants or those
  who attended more sessions improved their
  mothering role, perception of childbearing,
  developmental expectations of child, empathy for
  baby, and reduced frequency of hassles in child
  and family events
• Source Woods et al. (2003). The parenting
  project for teen mothers The impact of a
  nurturing curriculum on adolescent parenting
  skills and life hassles. Ambulatory Pediatrics,
  3, 240-245.

19
An Example Epidemiological Research

• Study based on Theory of Reasoned Action tracked
  substance use behavior, intention, attitudes,
  perceived social norms, and beliefs from
  pregnancy to 12 months postpartum
• Cohort of 255 adolescents
• Results documented a pattern in which adolescent
  mothers engaged in low levels of cigarette,
  marijuana, and alcohol use during pregnancy
• Resumed substance use in the first 6 postpartum
  months
• Doctors perceived as most opposed to substance
  use
• Friends, boyfriends, and siblings were considered
  to be least opposed
• Negative outcomes from substance use were viewed
  less negatively and as less likely after the baby
  was born
• Source Morrison, D.M., Spencer, M.S., Gilmore,
  M.R. (1998). Beliefs about substance use among
  pregnant and parenting adolescents. Journal of
  Research on Adolescence, 8(1), 69-95.

20
An Example Etiological Research

• Exploratory investigation conducted within a risk
  and resilience framework
• Diverse sample of adolescent mothers (n149) aged
  15 to 19 completed measures of
• Parental stress (social isolation and role
  restriction)
• Maternal competence
• Weight/shape concerns
• Depression
• Regression analyses showed that social
  isolation, maternal competence, and weight/shape
  concerns predicted unique variance associated
  with depression level
• Source Birkeland, R., Thompson, J.K., Phares,
  V. (2005). Adolescent motherhood and postpartum
  depression. Journal of Clinical Child and
  Adolescent Psychology, 34, 292-300.

21
Additional Topics

• Literature review
• Qualitative research about adolescent mothers
  experiences
• Adolescent fathers experiences
• Lessons learned
• Barriers to program delivery or outcomes
• Tools for providers and data collectors

22
An Example Qualitative Research

• Study investigated pregnant adolescents
  experiences of receiving support
• Guided by feminist theory
• Participants (n30) ranged from aged 13 to 18
• 19 African American, 10 European American, 1
  Asian American
• Qualitative data collection through focus groups
• Results showed how adolescents piece together the
  support they need from adults, peers, and
  partners who are often unreliable
• Adolescents relationships with their families
  and their babies fathers, threats to their
  safety, and socioeconomic status affected receipt
  of social support
• Source Logsdon, M.C., Gagne, P., Hughes, T.,
  Patterson, J., Rakestraw, V. (2005). Social
  support during adolescent pregnancy Piecing
  together a quilt. Journal of Obstetric,
  Gynecologic, and Neonatal Nursing, 34, 606-614.

23
Publishing in Peer-Reviewed Journals

• Optimism
• A good idea or innovation
• A gap in the field
• Clear presentation, explanation, and
  justification
• Prioritizing
• Perseverance
• Knowledge of journal and audience
• Present at conferences
• Submit to a mix of top tier and mid-level
  journals
• Revise and resubmit

24
Utility

• Overall assessment of the pattern and value of
  program findings to guide subsequent programs
• Whether, and to what degree, a findings are
  usable and appropriate for widespread application
  and dissemination
• Source USDHHS. (2002). Science-based prevention
  programs and principles, 2002. Rockville, MD
  Author.

25
Strategies

• Find mentor/scientific reviewer
• Review other articles
• Go for the best
• Partner with co-authors
• Identify and tap into their strengths
• Think creatively

26
Target Audiences

• Nursing journals
• ANS Advances in Nursing Science
• Annual Review of Nursing Research
• Journal of Community Health Nursing
• Journal of Obstetric, Gynecologic, and Neonatal
  Nursing
• Journal of Pediatric Nursing
• Journal of School Nursing
• Journal of Specialists in Pediatric Nursing
• MCN American Journal of Maternal and Child
  Nursing
• Public Health Nursing
• Other health care journals
• Ambulatory Pediatrics
• Archives of Pediatrics and Adolescent Medicine
• Journal of Health Care for the Poor and
  Underserved
• Pediatrics
• Qualitative Health Research

27
Target Audiences (continued)

• Adolescent research journals
• Adolescence
• Journal of Research on Adolescence
• Journal of Adolescent Health
• Journal of Adolescent Research
• Journal of Early Adolescence
• Journal of Youth and Adolescence
• Reproductive health journals
• Perspectives on Sexual and Reproductive Health
• Journal of Midwifery Womens Health
• Journal of Pediatric and Adolescent Gynecology
• Journal of Sex Research

28
Target Audiences (continued)

• Psychology journals
• The American Psychologist
• Journal of Clinical Child and Adolescent
  Psychology
• Journal of Pediatric Psychology
• Social work journals
• Child Adolescent Social Work Journal
• Social Work
• Social Work in Health Care
• Public health professionals
• Family and Community Health
• American Journal of Health Behavior
• American Journal of Public Health

29
Target Audiences (continued)

• Health education practitioners
• Health Promotion Practice
• Patient Education and Counseling
• Evaluation professionals
• Evaluation Review
• School health journals
• Journal of School Health
• Sociology journals
• Journal of Marriage and the Family

30
Other Venues

• Editorial or comment pieces
• Online journals
• Non-peer review journals
• The Prevention Researcher www.TPRonline.org
• Book chapters
• Textbooks
• Handbooks
• Magazine article for general readership
• Web posting of your paper