Grant Writing Workshop for Early-Career Investigators

1
Grant Writing Workshop for Early-Career
Investigators

• Allen Ruby, Ph.D.
• Associate Commissioner for Policy and Systems
• National Center for Education Research
• Amy Sussman, Ph.D.
• Program Officer
• National Center for Special Education Research

2
Todays Purpose

• To provide advice to early career researchers on
  submitting funding applications to the Institute
  of Education Sciences
• To review IES FY 2013 requests for applications
  and their requirements

3
The Big QuestionCan I be the PI on an IES
Grant?

• YES, if . . .
• Your background supports your role as PI
• A strong team can supplement your background
• You write clearly and address the reviewers
• You address the objectives and student outcomes
  of interest for IES grant funded research
• You apply to the right grant program
• Education Research Grants (84.305A) and Special
  Education Research Grants (84.324A)
• Other research grant programs
• You address the key requirements of the Research
  Narrative
• You address the submission requirements

4
Your Challenge

• Your challenge is to convince reviewers that you
  and your team have the knowledge, skills, and
  experience to implement well what you have
  proposed.

5
Background

• Receiving a grant is a process that begins before
  the submission of your application
• Review your past experience as a researcher. Can
  you argue that you have the knowledge/skills to
  be a PI
• Substantive
• Methodological
• Project management
• Productivity
• You will need to document this experience in your
  application more than an experienced researcher
  would

6
Examples of a Strong Background

• Built a record in a substantive topic and/or
  methodology as a graduate student, as a postdoc,
  and in your present position
• Published on own or with well-regarded
  experienced researchers in substantive area you
  propose to do future work in well-regarded
  journals
• Published from dissertation and other projects
• Supports your future work in this topic
• Held major responsibilities within a project
• Provides experience in carrying out project
• New projects can be based on finding of old ones
• Extensive work with a secondary data set
• Provides a foundation for future research using
  this data

7
Building Your Research Record

• Run smaller projects (e.g., internal grants,
  small external grants)
• Take on roles in larger external grants
• Sometimes you have to do work not directly in
  your immediate interest
• Sometimes you can carve out space under another
  PIs project to do your work of interest
• Usually related to the projects goals
• Make sure you get lead/sole authorship

8
Building Research Record (cont.)

• In either case, take on roles that will provide
  you with the experience to lead a future project
• Co-I
• Co-PI
• Project manager
• Data manager
• Build expertise, e.g. substantive area, specific
  design, analytical method, data set
• Build links to school/district/state education
  leadership
• Publish from all these types of projects
• Provide useful information to practitioners

9
Building a Good Team

• Review the expertise needed to carry out your
  proposed project
• Review your expertise
• Recruit people with well-regarded specific
  expertise needed by project, e.g.
• Statistician/econometrician or substantive expert
  for a secondary data analysis
• Survey developer or implementor for a data
  collection
• Curriculum developer or professional development
  expert for a project to develop an intervention
• Experimental design for an evaluation of an
  intervention
• Psychometrician or substantive expert for a
  measurement project
• Project manager

10
Building a Team Synthetic Reviewer Comment

• Given that the PI has not run a large federal
  grant and does not appear to have formal training
  in reading I would recommend that senior
  researchers who have expertise in reading and
  have received funding in the past be added to the
  grant.

11
Building a Team (cont.)

• Be aware that the peer reviewers want to be
  assured that early career researchers have access
  to experienced researchers
• Project management
• Substantive and methodological areas
• Including areas that the early career researcher
  has expertise in

12
Building a Team Synthetic Reviewer Comment

• The lead investigator is junior and has not run a
  study of this scope before. However, this is only
  a moderate weakness given his prior experiences
  with data collection and the excellent
  supervision and support that he will receive.

13
Building a Team (cont.)

• Including experienced researchers can be done in
  various ways
• Co-PI
• Co-I
• Advisory Panel
• Consultants
• They should be on for enough time to be seen as
  able to make a contribution to project
• They should have clear roles, duties, and
  periodic input
• Especially true for advisory panels and
  consultants

14
Building a Team Synthetic Reviewer Comment

• The members of the advisory panel have been
  involved in the design and conduct of many
  experiments and will be of great help to the
  Project Director in the development of the
  interview protocol, in the analysis phase of the
  project, and in reviewing papers stemming from
  the analyses.

15
Write Clearly

• Reviewers often complain about lack of clarity in
  applications
• The significance of the project is written about
  in too general of terms
• There is a lack of detail on a key aspect of the
  work, e.g., the factor or intervention being
  studied, the cycle for developing an
  intervention, or the description of the data
  analysis
• Reviewers have less information about you to
  judge your abilities and so may be less willing
  to give you the benefit of the doubt when writing
  is unclear or lacks detail

16
Write Clearly Synthetic Reviewer Comment

• No information is given regarding the actual
  interventions to be used with the at-risk
  students in the control and experimental
  conditions. This makes it impossible to judge the
  potential impact of the intervention on the
  reading outcome of at-risk children.

17
Write Clearly Synthetic Reviewer Comment

• The data analysis plan is unclear. What kind of
  scores from the measures will be used in what
  specific models to address which specific
  research questions. Many different analyses are
  mentioned but their tie-in with specific research
  questions is unclear.
• In the regression discontinuity model that is
  provided how are variables centered to provide
  the interpretation of the coefficients as
  provided? What is the sample size that will be
  used? How will clustering be handled? Why are
  grade level and time both in the model? More
  information is needed to help understand the
  analysis plan.

18
Write Clearly (cont.)

• Maintain consistency throughout application
• Ensure that sections support one another
• The research plan addresses the original research
  questions that were justified as significant
• Write for both generalist and specialist
• Avoid use of jargon and assumptions of knowledge
• Have both an expert and non-expert do a read
  through

19
Write Clearly Put the Punch Line Upfront

• Opening paragraph sets the scene for the
  reviewers
• Identifies the significance of the work to be
  done and what actually will be done
• Reviewers use it to organize information from
  rest of application
• You can lose your reviewers right off with an
  unclear opening

20
Address the Reviewers

• Reviewers focus on the Research Narrative (more
  on this later)
• Reviewers include both specialists in your area
  and generalists
• Initial reviews are done by a substantive
  reviewers and a methodologist
• Panel has expert in every component of your study
• Show personnel can do the work
• For resubmissions, address the previous reviews

21
Advice From an Early Career PI

• Be a co-PI (or key personnel) on a previous
  project
• Highlight your management role from this project
• Can be small internal grant, state or foundation
  grant, or federal grant
• Have personnel with 'senior expertise' that you
  don't have
• Add Co-PIs who have received federal grants and
  have expertise in critical elements for the
  project and/or
• Have an advisory board with senior people, a
  clearly defined role, and access throughout the
  year not just at the formal meetings
• Reviewer feedback suggested that although I was
  junior, I had built a strong team that could
  support me

22
Advice From an Early Career PI (cont.)

•  Extend off of your prior research work
• Built directly off of dissertation work that had
  been published in a good peer-reviewed journal
• Continued this work with a small internal grant
  from own institution
• IES Application was based on data and results
  from dissertation and the internal grant study
• Try again
• Most of the reviewer feedback was very helpful
  in strengthening the second application which was
  funded

23
Applying for an IES Grant

• Address the objectives and student outcomes of
  interest for IES grant funded research
• Apply to the right grant program
• Education Research Grants (84.305A) and Special
  Education Research Grants (84.324A)
• Other research grant programs
• Address the key requirements of the Research
  Narrative
• Address the submission requirements

24
IES Organizational Structure
Office of the Director
National Board for Education Sciences
Standards Review Office
National Center for Education Evaluation
National Center for Education Statistics
National Center for Education Research
National Center for Special Education Research
25
Overall Research Objectives For Grant-Funded
Research

• Develop or identify education interventions
  (practices, programs, policies, and approaches)
  that enhance academic achievement and that can be
  widely deployed
• Identify what does not work and thereby encourage
  innovation and further research
• Understand the processes that underlie the
  effectiveness of education interventions and the
  variation in their effectiveness

26
Final Outcomes of Interest are for Students
(Student Outcomes)

• Birth through Preschool
• School readiness for both Centers
• Developmental outcomes for infants and toddlers
  with or at-risk for disabilities
• Kindergarten through Grade 12
• Academic outcomes in reading, writing, math and
  science
• Behaviors, interactions, and social skills that
  support learning in school
• High school graduation
• Functional outcomes that improve educational
  results, transitions to employment, independent
  living, and postsecondary education for students
  with disabilities

27
Student Outcomes (cont.)

• Postsecondary
• Enrollment, persistence, completion
• Achievement in gateway math science courses
• Achievement in introductory composition courses
• Adult Education
• Reading, writing, and math for adult basic and
  secondary education and English language learners

28
Applying for an IES Grant (cont.)

• Apply to the right grant program
• Education Research Grants (84.305A) and Special
  Education Research Grants (84.324A)
• Other research grant programs
• Address the key requirements of the Research
  Narrative
• Address the submission requirements

29
Primary Research Grant Programs

• Education Research Grant Programs (84.305A)
• Special Education Research Grant Programs
  (84.324A)
• These grant programs are organized by research
  topic and research goal

30
Education Research Topics (84.305A)

• Cognition and Student Learning
• Early Learning Programs and Policies
• Education Technology
• Effective Teachers Effective Teaching
• English Learners
• Improving Education Systems Policies,
  Organization, Management, and Leadership
• Mathematics and Science Education
• Postsecondary and Adult Education
• Reading and Writing
• Social and Behavioral Context for Academic
  Learning

31
Special Education Research Topics (84.324A)

• Autism Spectrum Disorders
• Cognition and Student Learning in Special
  Education
• Early Intervention and Early Learning in Special
  Education
• Families with Children with Disabilities
• Mathematics and Science Education
• Professional Development for Teachers and Related
  Service Providers
• Reading, Writing, and Language Development
• Social and Behavioral Outcomes to Support
  Learning
• Special Education Policy, Finance, and Systems
• Technology for Special Education
• Transition Outcomes for Secondary Students with
  Disabilities

32
FY2013 Research Goals

• Exploration
• Development Innovation
• Efficacy and Replication
• Effectiveness
• Measurement

33
Exploration Goal

• Explore associations between education outcomes
  and malleable factors
• Identify factors and conditions that may mediate
  or moderate the relations between malleable
  factors and student outcomes
• Possible methodological approaches include
• Analyze secondary data
• Collect and analyze primary data
• Complete a meta-analysis

34
Development Innovation Goal

• Develop an innovative intervention (e.g.,
  curriculum, instructional approach, education
  practice, program, or policy)
• OR improve an existing education intervention

• AND collect data on its feasibility and usability
  in actual education settings
• AND collect pilot data on student outcomes.

35
Efficacy Replication Goal

• Evaluate whether or not a fully developed
  intervention is efficacious under limited or
  ideal conditions
• OR
• Gather follow-up data examining the longer term
  effects of an intervention with demonstrated
  efficacy
• OR
• Replicate an efficacious intervention varying the
  original conditions

36
Effectiveness Goal

• Evaluate whether a fully developed intervention
  that has evidence of efficacy is effective when
  implemented under routine practice through an
  independent evaluation
• Prior to submitting an effectiveness proposal, at
  least two efficacy studies of the intervention
  with beneficial and practical impacts on student
  outcomes must have been completed
• Follow-up projects to determine longer-term
  impacts of intervention that has been found
  effective can also be proposed

37
Measurement Goal

• Development of new assessments or refinement of
  existing assessments, and the validation of these
  assessments
• OR
• Validation of existing assessments for specific
  purposes, contexts and populations

38
Award Maximums (84.305A 84.324A)
Goal Maximum (direct indirect)
Exploration Secondary data With primary data 2 years, 700,000 4 years, 1,600,000
Development Innovation 4 years, 1,500,000
Efficacy Replication Follow-up study 4 years, 3,500,000 3 years, 1,200,000
Effectiveness Follow-up study 5 years, 5,000,000 3 years, 1,500,000
Measurement 4 years, 1,600,000
39
Applying for an IES Grant (cont.)

• Apply to the right grant program
• Education Research Grants (84.305A) and Special
  Education Research Grants (84.324A)
• Other research grant programs
• Address the key requirements of the Research
  Narrative
• Address the submission requirements

40
Early Career Program

• Research Training Program in Special Education
  Early Career Development and Mentoring (CFDA
  84.324B)
• New competition for FY2013
• Considered a research training grant
• Requires research project

41
Early Career Program (cont.)

• Eligibility for Principal Investigator
• Focus of research on infants, toddlers, children,
  or youth with or at risk for disabilities
• Need for additional research training
• Completed doctoral degree or postdoctoral
  training within 3 years of the application due
  date
• Tenure-track position at IHE
• No previous IES award as PI or co-PI

42
Early Career Program (cont.)

• Components of Early Career Program
• Research Plan
• Plan of research that corresponds to NCSER topic
  and goal structure
• Career Development Plan
• Mentoring plan (primary mentor, possible
  co-mentors)
• Additional training plan (e.g., workshops,
  courses, summer institutes)
• Explicitly integrated with research plan

43
Statistical and Research Methodology in Education
(84.305D)

• Research projects intended to expand and improve
  the methodological and statistical tools
  available for education researchers
• These tools will be used to improve the design of
  research studies, analysis of research data, and
  interpretation of research findings

44
Evaluation of State and Local Education Programs
and Policies (84.305E)

• Support for rigorous evaluations of education
  programs or policies that are paid for and
  implemented by State or local education agencies
• Evaluations are to determine both the overall
  impact of the programs/policies and the impact
  across a variety of conditions

45
Researcher-Practitioner Partnerships in Education
Research (84.305H)

• Intended to support the partnering of
  researchers with State and local education
  agencies in the development of joint research
  projects

46
Accelerating the Academic Achievement of Students
with Learning Disabilities Research Initiative
(84.324D)

• To develop and evaluate interventions (e.g.,
  instructional approaches, curricula, technology)
  to accelerate the reading and math achievement of
  students with or at risk for reading and math
  disabilities in grades 3 through 8
• Will create a tightly linked network of
  researchers across a variety of disciplines who
  will work collaboratively to address the problem

47
Award Maximums (84.305B-H 84.324B-D)
Program Maximum Number of Years Maximum Award (direct indirect)
305B Postdoc Training 5 years 687,000
305B Researcher Policymaker Training 3 years 1,000,000
324B Early Career 4 years 400,000
324D A3 5 years 10,000,000
305D Stats/Methods 3 years 900,000
305E State/Local 305H Partnerships 5 years 2 years 5,000,000 400,000
48
Applying for an IES Grant (cont.)

• Address the key requirements of the Research
  Narrative
• Address the submission requirements

49
Research Narrative

• Key portion of your application
• Comprised of four sections
• Significance
• Research Plan
• Personnel
• Resources

50
Significance Section

• Describe the overall project and the Research
  Questions you intend to answer
• Provide a compelling rationale for the project
• Varies by grant program
• For 84.305A and 84.324A varies by research goal

51
Research Plan

• Detail the design, activities, methods, and the
  analysis that you will use to answer the Research
  Questions described in the Significance section
• Show familiarity with secondary data (e.g.,
  missing data and attrition, ability to identify
  subgroups, ability to merge data sets)
• Varies by grant program
• For 84.305A and 84.324A, varies by research goal

52
Personnel Section

• Describe key personnel
• Link each person and their expertise to their
  role in project - show that every aspect of
  project has person with expertise to do it
• Methodologists show expertise in particular
  method to be used
• Substantive person for all issues addressed
• Do not propose to hire a key person with X
  expertise
• Project management skills
• Give time contribution for each person show that
  every aspect of project has enough time from
  expert
• Orient CVs same way specific to project
• 4 pages plus 1 page for other sources of support

53
Resources

• Show the institutions involved have the capacity
  to support the work
• Do not use university boilerplate
• Show that all organizations involved understand
  and agree to their roles
• What will each institution, including schools,
  contribute to the project
• Show strong commitment of schools and districts
  and alternatives in case of attrition
• Detailed Letters of Support from research
  institutions, States, districts, schools placed
  in Appendix C
• Detailed Letters of Support from data providers

54
Appendices

• Appendix A (15 page limit)
• Figures, charts, and tables
• Examples of measures
• 3 pages to address past reviewer comments or to
  argue that a proposal is a new submission
• Appendix B (10 page limit)
• Examples of materials used in an intervention or
  assessment
• Appendix C (no page limit)
• Letters of agreement (districts, schools, data
  providers, other partners, consultants)
• Clearly state responsibilities of the writer

55
Budget and Budget Narrative

• Provide a clear budget and budget narrative for
  overall project and each sub-award
• IES Grants.gov Application Submission Guide
  describes budget categories
• Check RFA for specific budget requirements for
  Research Goals and Grant Programs
• Ensure agreement among Research Narrative,
  Budget, and Budget Narrative

56
Applying for an IES Grant (cont.)

• Address the submission requirements

57
Important Dates and Deadlines
FY 2013 NCER (305) NCSER (324) Grant Programs Application Deadline Letter of Intent Due Date Application Posted www.grants.gov Start Dates
305A Ed Research 324A Sp Ed Research 305D Stats/Methods 6/21/12 4/19/12 4/19/12 3/1/13 to 9/1/13
305A Ed Research 324A Sp Ed Research 305B Research Training 324B Sp Ed Research Training 324D A3 Initiative 305E State/Local 305H Researcher- Practitioner Partnerships 9/20/12 7/19/12 7/19/12 7/1/13 to 9/1/13
58
Review Application Requirements

• Request for Applications
• Currently available at http//ies.ed.gov/funding
• Grants.gov Application Submission Guide
• Will be available 4/19/12 for June deadline at
  http//ies.ed.gov/funding.
• Application Package
• Will be available on Grants.gov on 4/19/12 for
  June deadline

59
Information Sources

• Begin at the IES website (http//ies.ed.gov/)
• Sign up for the IES Newsflash
• Look at the IES funding webpage
• Review current Requests for Applications
• Look at the abstracts of projects funded under a
  research topic or program
• Contact relevant program officer(s) for the grant
  program and topic of interest in the relevant
  Center

60
http//ies.ed.gov
61
(No Transcript)
62
(No Transcript)
63
Contact Relevant Program Officers

• Program officers are associated with competitions
  and/or topics, and their contact information is
  included at the end of each RFA
• Contact information is also available on each of
  the program pages on the IES website

64
Resources for Researchers

• Visit links for faculty and researchers on IES
  website
• Review past webinars, and participate in future
  webinars for the FY 2013 competitions
• Watch methodology videos

65
http//ies.ed.gov/resourcesforresearchers.asp
66
Submit a Letter of Intent

• LOIs are submitted electronically using the
  instructions provided at https//iesreview.ed.gov
  
• We encourage all researchers to submit Letters of
  Intent
• 4/19/12 for June applications (passed, email LOI
  directly to the program officer)
• 7/19/12 for September applications

67
Grant Submission

• Make sure your institution is registered on
  grants.gov
• Complete your online forms and upload PDFs
• Authorized representative completes the process
• Submit by 43000 eastern time on deadline
  earlier is safer
• If problems uploading
• Contact Help Line 1-800-518-4726 and get a case
  number
• You should receive three emails
• Grants.gov will say they have received your email
  and assign you a number that starts with GRANT
• Grants.gov will say your application is validated
  or rejected due to errors. If the latter,
  resubmit until validated.
• Department of Education will assign you a grant
  number starting with R305 or R324

68
Application Review (Standards Review
Office)

• Compliance screening for format requirements
• Responsiveness screening to program/goal
  requirements
• Assigned to review panel
• 2-3 reviewers (substantive and methodology)
• If scored high enough, application is reviewed by
  full panel
• Many panelists will be generalists to your topic
• There will an expert in every procedure you use
• Overall score plus scores on Significance,
  Research Plan, Personnel, and Resources
• So far, all applications with overall score of
  Outstanding and Excellent have been funded
• Resubmissions encouraged talk to program officer
  and address reviewer comments

69
Peer Review Process Information

• http//ies.ed.gov/director/sro/peer_review/index.a
  sp

70
Notification Process

• All applicants will receive email notification of
  the status of their application
• All applicants receive copies of reviewer
  comments via email
• If you are not granted an award the first time,
  plan on resubmitting, and talk to your program
  officer

71
For More Information

• http//ies.ed.gov/funding

Allen Ruby Allen.Ruby_at_ed.gov Amy
Sussman Amy.Sussman_at_ed.gov