Jessica TurpakCarey Masse, B.S.Brandon Schultz, Ed.S.

1
Jessica Turpak Carey Masse, B.S. Brandon Schultz,
Ed.S.
Treatment Fidelity A Look At a School-Based
Academic Intervention for Youth with Attention
Deficit Hyperactivity Disorder (ADHD)
RESULTS
ABSTRACT

• CHP participant response frequencies were
  examined and findings were as follows
• There was 100 agreement between the students,
  mentors, parents, and consultants when asked, Is
  an assignment notebook required?
• When asked Is there a requirement for teacher
  initials? There was high agreement between
  student, mentor, and consultant (ranging from 70
  to 80). Agreement was slightly lower between
  parents and the school consultant with an
  agreement rate of 50, and even lower between the
  parent, mentor, and student with frequencies at
  40 (see table).
• Regarding the question, How often is the
  assignment notebook checked? there was a high
  agreement between the mentors and students (70)
  and a low agreement between the parents and
  students (30), as well as among parents and
  consultants (40).
•      

• Treatment fidelity demonstrates that an
  experimental manipulation is conducted as planned
  and that content and process of the intervention
  are equal across participants (Dumas, Lynch,
  Laughlin, Smith, Prinz, 2001) Various studies
  indicate that treatment fidelity plays a vital
  role in ensuring overall validity in a treatment
  program (Calsyn, 2000). Fidelity was assessed in
  the Challenging Horizons Program (CHP), a school
  based intervention for adolescents with ADHD.
  This poster presents fidelity assessment data on
  one of the variety of interventions in the CHP.
  Methods for assessing fidelity are reviewed and
  the rationale for selecting survey methodology
  for the CHP is explained. Initial data reflected
  high and low agreements among respondents
  regarding key components of the assignment
  tracking method. Implications for the assessment
  of fidelity in school-based mental health
  services are discussed.

Response Rates Among Parents, Mentors, Students
and School Consultant
 
DISCUSSION
INTRODUCTION
Assessing treatment fidelity within the
Challenging Horizons Program serves the practical
and vital role of evaluating the delivery of
school-based treatment interventions. After
completing the fidelity questionnaires, initial
data revealed incongruity among participant
responses. The highest levels of inconsistency
were between parents and all other raters.
Miscommunication, misunderstanding and lack of
knowledge regarding students intervention plans
may play a large role in low agreement rates.
Communication between parents and the mentor may
be difficult to accomplish due to time
constraints and scheduling issues. Poor
communication about specific events at middle
school, between parents and their children, is a
common complaint of parents. Other possible
sources of disagreement include inadequate
knowledge of the intervention procedures. Other
issues that may effect fidelity may include
respondent exaggeration and questionnaire
defects. Possible solutions to communication
problems may include requiring monthly parent and
mentor meetings to review and update the students
interventions. Additional training sessions may
be necessary to increase adherence to specific
details of the intervention. The data presented
in this poster represents only a sub-sample of
the fidelity data being collected in this
project, but it does have implications for the
measurement of fidelity. Parents may be poor
sources of information about the implementation
of procedures at middle schools. In addition,
fidelity questions need to be very focused and
directly ask about specific aspects of the
intervention. Future analyses should attempt to
evaluate fidelity data in relation to treatment
outcomes.
The Challenging Horizons Program is a
school-based psychosocial treatment intervention
for adolescents with Attention Deficit
Hyperactivity Disorder (ADHD). Parents of the
adolescents, mentors and the school consultant
collaboratively determine the appropriate
intervention methods for each student, providing
a tailored treatment program based on established
academic and psychosocial treatment methods (e.g.
Assignment Notebook, Organizational Skills
Interpersonal Skills Group) (Evans, Serpell,
Schultz Williams, 2003). Treatment fidelity is
used to demonstrate if a manipulation was
conducted in an experiment as intended (Dumas et
al., 2001). Moncher and Prinz (1991) state that
monitoring treatment fidelity is critical to the
outcome of a treatment program, explaining if
fidelity goes unchecked, treatment outcomes may
be due to an unknown variable rather than the
treatment. It may appear that the treatment
program is ineffective when in fact it simply has
poor implementation. Treatment fidelity of the
CHP was assessed to determine if the program was
being implemented according to manualized
procedures. Treatment fidelity can be assessed
through multiple methods observation, video or
audiotape, and questionnaires (Moncher Prinz,
1991). Questionnaires were chosen as the primary
form of fidelity measurement for the CHP, in
light of observational measurement constraints.
Multiple respondents were used to obtain data
from various sources and improve the validity of
the assessment. Higher levels of fidelity
positively affect validity, giving the study the
ability to be more successfully generalized to
other settings (Evans, Langberg, Williams
2003).

Note D/K Dont Know response, (--) Missing Data
 
METHODS Continued

• Materials
• Assignment Notebook Questionnaire
• Distributed to parents, students, mentors and the
  school consultant.
• Consisted of multiple questions aimed at
  determining the extent to which core aspects of
  the intervention were being completed as planned.
  
• Some questions consisted of multiple choice
  answers, while others gave the respondent the
  option of choosing yes/no or dont know.
• The number of items on each questionnaire
  reflected each respondents role in the
  assignment notebook intervention.
• 3 questionnaire items that demonstrated the key
  aspects of a successful assignment notebook
  intervention were examined in the initial
  analysis.
• Procedure
• Questionnaires were completed during the first
  half of the academic school year by parents,
  mentors,students and the school consultant. The
  gathered data was analyzed by running frequency
  distributions and comparing those results among
  respondents.

REFERENCES
Calsyn, R.J. (2000). A checklist for critiquing
treatment fidelity studies. Mental Health
Services Research, 2(2), 107-113. Dumas, J.E.,
Lynch, A.M., Laughlin, J.E., Smith E.P., Prinz,
R.J. (2001). Promoting intervention fidelity
Conceptual issues, methods, and preliminary
results from the Early Alliance prevention
trial. American Journal of Preventive Medicine,
20(1S), 38-47. Evans, S.W, Langberg, J.,
Williams, J. (2003). Achieving generalization in
school-based mental health. In M.D. Weist, S.W.
Evans N.A. Lever (Eds.), Handbook of School
Mental Health Advancing Practice and Research
(pp.335-348). New York, NY Kluwer
Academic/Plenum Publishers. Evans, S.W.,
Serpell, Z., Schultz, B., Williams, A.L.
(2003). Developing and transitioning
school-based treatment for ADHD The Challenging
Horizons Program (CHP). Emotional and Behavioral
Disorders in Youth, 3(4), 95-103. Moncher, F.J.,
Prinz, R.J. (1991). Treatment fidelity in
outcome studies. Clinical Psychology Review, 11,
247-266.

METHODS
Participants A portion of the CHP sample (11)
who enrolled earliest in the project were
included in this study. Parents of the students,
mentors (teachers serving as primary counselors),
and the school consultant (research staff),
completed the treatment fidelity assessments.
Ten (10) students received Assignment Notebook
as part of their personalized intervention
program this intervention used to address
difficulties the students have recording and
organizing their assignments.