Title: Evaluating CCTV and Recording of Child Interviews and Testimony
1Evaluating CCTV and Recording of Child Interviews
and Testimony
- M. Christine Kenty, PhD
- Sharon Elstein, MS
- ABA Center on Children and the Law
2KENTY'S CLUES FOR EVALUATION
- 1. Base any evaluation on your own goal pathway
and logic model. - 2. Organizations are social systems with their
own cultures. - 3. Whatever people don't want you to study,
that's the really important thing.
3And those touchy things are likely to be
- Collaboration
- Quality of Forensic Interviewing
- Decision-making about arrest, prosecution and
child protection - Relating to victims and families
- Childrens experiences before and in the grand
jury or courtroom
4Kentys Clues
- 4. It's trouble if only one person is doing all
the thinking about evaluation. - 5. You can't keep partners and stakeholders too
well informed about the evaluation process.
5Kentys Clues
- 6. Any evaluation (or program report) requires
DATA, so decide early what you need to record and
then keep it up. - 7. For any evaluation, there are many good
designs, but no perfect ones. - 8. Dont kill the messenger
if you don't like the news!
6Evaluation
- is a way to improve a program by systematically
examining and analyzing what the program is doing
and what it has accomplished.
7The Ill know it when I see it Rule
- Ill know that our program is working when I
see.
8What evidence can convince us and others that our
program is on target?
- What would tell us that something has happened?
- How can we count it or track it?
9What evaluation can do
- Help improve the program from the beginning
- Provide staff and stakeholders with a much-needed
sense of accomplishment - Guide protocol, policy and law reform
- Assist in developing future funding
10Dont put the evaluation in the hands of just one
individual
Whether that is an internal or external evaluator
11A healthy organization needs to know the program
mission, plan the work, develop enthusiasm, and
bring things to fruition.
12An organization also needs to look at what its
doing, keep what's good and try to jettison what
isn't working.
13Evaluation isn't a separate topic.
- it's just one more piece of the work
14Forces will try to marginalize and minimize an
evaluation
- Dont let that happen
- establish a strong committee!
15Base the evaluation on your agency pathway.
- Logic model and pathway map are popular
phrases with funders.
16Logic Model or Pathway MapEach part should
logically follow from the last
17We all work on underlying assumptions, which
might be ---
- Better technology will improve dispositions
- Technology will make things easier for kids
fewer interviews, less testifying - Our forensic interviewing will stand up to
scrutiny - The defense bar will not limit the potential of
recorded testimony - We will know if this is working well
- TALK ABOUT AND CLARIFY THESE!
18Logic Model or Pathway Map
19You may do a needs assessment to describe your
context
- A needs assessment is a systematic way to
discover - what you need in order to accomplish a goal,
and then - make decisions based on that assessment
20Typical needs assessments
- Estimate how many clients/professionals will
participate in a new program - Determine what resources are already in place and
what has to be put into place - Decide what an agency or community needs to
provide to get a particular result - Envision how technology and products will be used
so that the equipment will be right - Decide what training people need
21You may have already done one kind of needs
assessment
- but you may still want to do another piece as you
begin to implement your program
221. Estimate how many clients/professionals will
be involved
- Count of allegations, investigations,
interviews, arrests, prosecutions, hearings,
dispositions in the last year - List all the professionals who will need to be
trained or familiarized
232. Determine what resources are in place and
needed
- All professionals and what they do
- Adequacy of infrastructure rooms, technology,
wiring, lighting - Relevant state statutes re victims and CCTV
and/or recorded testimony - Policies and procedures for interagency work
- Other available funding
243. Decide what is needed for specific result
(These are suggestions, not requirements)
- What equipment, facilities, personnel, time,
training, systems, policies, statutes do we need,
for example - To videotape all child interviews for children
aged 3-13? - To decrease the number of child victims who
testify in person? - To improve successful disposition rates?
- To make the system more child-friendly?
254. Envision how equipment/product will be used
- Stationary or portable
- Professional technician or multiple users
- Video all children or selected cases or ages
- CCTV according to statutes
- Show videos in what rooms to what audiences
- Who needs a cut-off switch
- Maintenance, upkeep costs, back-ups
265. Determine what training people will need
- Technical skill in equipment use and maintenance
- Scheduling and informing children/caregivers
- Interviewing skills
- Judiciary, Prosecutors and Bar
- Permissibility/use of recording and CCTV
according to all statutes - Awareness of capacity
- Forensic use of recordings at multiple points
- Quality assessment and record-keeping for
recorders and prosecutors
27Design method of data collection
28Design methods of data collection
29Design methods of data collection
30Logic Model or Pathway Map
31Logic Model or Pathway Map
32Logic Model or Pathway Map
33Logic Model or Pathway Map
34Ideas for short term outcomes
- Plans are complete we know who contributed and
what went into planning - Stakeholders are aware and committed
- Equipment is acquired and installed
35Logic Model or Pathway Map
36Ideas for medium term results
- Equipment being used regularly
- People skilled and knowledgeable
- Stakeholders have assessed use
- Data being collected on interviewing, use of
recordings and CCTV, and can be summed - Data establishing a baseline for later comparison
37Logic Model or Pathway Map
38Ideas for long term results (impact)
- Quality of forensic interviewing and recording is
consistently high - Prosecutors use recording and CCTV regularly and
effectively in known ways - Prosecution rates and/or case dispositions have
been improved - Stakeholders, clients and families are satisfied
with the process and use of recordings - Laws and policies have been changed
39TYPES OF EVALUATIONS
- NEEDS ASSESSMENT
- A systematic way to discover what you need to
accomplish a goal - Helps make informed planning decisions
40TYPES OF EVALUATIONS
- FORMATIVE EVALUATION
- Short-term initial feedback on how the program is
working - Helps quickly readjust planned activities to be
more effective
41TYPES OF EVALUATIONS
- PROCESS EVALUATION
- Describes how something happened rather than
outcomes - To understand the internal dynamics of
organizations and relationships, and capture what
activities are actually happening.
42TYPES OF EVALUATIONS
- IMPACT or OUTCOME EVAL
- determines whether a program produced desired
results. - Requires articulated outcomes and targets and
they must be measurable.
43RESEARCH METHODS
44RESEARCH METHODS
- QUANTITATIVE METHODS
- numerical research by collecting data about
pre-selected variables, and studying cause and
effect
- QUALITATIVE METHODS
- naturalistic research by studying participants'
perceptions and experiences in context and the
way they make sense of them
45QUANTITATIVE METHODS
- surveys with pre-determined categories and rating
scales - document review e.g. counting up numbers of
arrests or prosecutions, and comparing them to
other groups or time periods - evaluator attempts to keep at an objective
distance from the people
46QUALITATIVE METHODS
- surveys with open-ended questions
- interviews semi-structured
- observation
- document review e.g. process, attitudes
- case studies
- focus groups
- evaluator gets close to the people to capture
what is actually happening
47Collect both quantitative and qualitative data -
they are equally valid
- Qualitative data help capture changes in
processes and relationships, and some things just
arent countable.
48Sampling how many records or which people to ask
- More important to have a representative sample
than a large sample
49But, every professional might need to be heard so
that no one feels left out and there is no
suspicion of bias
- So sampling may not be acceptable
50Shaping evaluation questions
51Shaping evaluation questions 2
52Shaping evaluation questions 3
53Shaping evaluation questions 4
54Shaping evaluation questions 5
55Shaping evaluation questions 6
56Shaping evaluation questions 7
57Shaping evaluation questions 8
58Shaping evaluation questions 9
59Shaping evaluation questions 10
60Shaping evaluation questions 11
61Shaping evaluation questions 12
62A monitoring system is part of the implementation
of the program, and is also a key element of any
evaluation.
63The monitoring system insures that....
- Each part of the system can get the information
it needs to proceed, i.e. arrest and prosecution,
child protection, treatment, victim advocacy. - You can assess the effectiveness of your program.
64 If you know what is happening to your children
and defendants from start to finish...
- Then you have an integrated data system
65If you dont............................
- Then policies, technology or turf issues are
preventing it, and your evaluation will be
affected.
66Data collection for recorders
- Intake log of all alleged victims demo, perps
- Log of recorded interviews
- Date, Number, Name, Age, Gender, Length
- Forensic Interviewer
- Observers keep copy in central location, not
just in evidence - Referred for prosecution capture prosecutor
and date - Quick assessment of quality of interview by
effectiveness, protocol and forensic value
highlight really good or not so good interviews
for training and peer review library
67Data collection instrument for assistant
prosecutors Keep copies in a central location,
not separately in each file or jacket!!
- ____ Defendant _______Victim________AP________
- Observed the forensic interview
- Reviewed the recording of the interview
- Forensic value of the recorded interview
Poor Good Very Good - Explain
- Used the recording to encourage family
cooperation w/ prosecution - Explain
- Used the recording in diverting the case
- Explain
- Used the recording to obtain a pre-indictment
plea - Explain
68Data collection instrument for assistant
prosecutors 2
- Used the recording or CCTV to obtain an
indictment (GJ, deposition, other) - Explain
- Used the recording to obtain a post-indictment
plea - Explain
- Used the recording or CCTV at pre-trial hearing
or trial - Explain
- The child testified before the grand jury or at
preliminary hearing - Explain
- The child testified at trial
- Explain
- Verdict and sentence
69Other possible evaluation data
- Answers to items on a survey
- Answers to questions in an interview format
- Responses in a focus group
- Observations of group interactions in a
planning/protocol meeting - Observations of forensic interviews or childrens
testimony
70You cant keep everybody too well informed about
the evaluation.
- Too much is never enough despite your best
efforts, somebody may still feel blindsided.
71Dont kill the messenger if you dont like the
news!
- Dont allow the evaluation to be deep-sixed if
there are negative findings.
72And dont ever let the evaluator or committee
twist in the wind!
- If the committee has done it right, everybody
will understand the evaluation process and what
the findings are before a final report comes out.