Writing Your Research
Plan
Sharon L. Milgram, Director NIH OITE
The Psychology of Grant Review
Reviewers are:
Over-committed, over-worked and tired
Inherently skeptical and critical
Often only peripherally interested in your work
Make their job easier with:
Well-organized, clearly written prose
Lots of section headings and breaks in the writing
Repeat important points at several places in the application
Well designed flow diagrams, charts, figures
And avoid irritating them by:
Exceeding page limits, using small fonts and narrow margins
Putting information in the wrong section
Omitting or mislabeling references/figures
Submitting an application that is sloppy or full of typographical
errors
Three NIH Grant Cycles Per Year
Receipt Dates Review Advisory Potential
(2/5, 6/5, 10/5) Meetings Council Start
Cycle Date
1 Jan Feb Mar Apr May June July Aug Sept Oct Nov Dec
2 May June July Aug Sept Oct Nov Dec Jan Feb Mar Apr
3 Sept Oct Nov Dec Jan Feb Mar Apr May June July Aug
The Organizational Process
“Many of us when confronted by a writing
deadline, skip the organizational phase of
writing. This is akin to leaving on a trip to
unknown parts without a road map, hotel
reservations, or plans of any sort.”
Successful Scientific Writing by Matthews and Mathews (Cambridge University Press)
Decisions to make early
Identify a funding Institute
Where do you want to target your application?
Who are the appropriate Program Officers?
Note that each Institute has a different pay line
Select the appropriate funding mechanism
Career Development/Transition Awards
Research Project Grants (e.g., R01)
Consider if a specific RFA is available in your field
Identify an appropriate Scientific Review Group
Depends on Scientific Area
CSR Rosters are available online
Getting Started: Administration Issues
Download and carefully read all instructions and
deadlines
Important to get latest materials - especially now
Register for government internet based application and
award systems, particularly the eRA Commons
Talk with lab/department administrators about budgeting,
all required approvals, and routing procedures
Begin approvals process well in advance of the deadline
Contact collaborators and arrange for letters as needed
Getting Started: Science Issues
Read the literature broadly - not deeply; save important
papers for a deeper read later
Engage your lab, mentors, collageagues, and
collaborators in the brainstorming process
Find outside experts to talk with - but go prepared
Talk with the relevant NIH Program Officer(s)
Begin early to define, organize and plan the content
NOTE: Early means 6 - 9 months before the deadline
More Science Issues
Look hard at your publications - any that need to be
submitted NOW?
Insufficient publication record is a common concern during review
Identify ‘need to have’ preliminary data
Identify methods you need to learn more about or develop
expertise in
Ideally will will know, or know someone who knows, all the methods
you propose
Make lists of reagent, cell type, animal, or human subject
issues you need to deal with
Critical reagents must me in-hand
What makes a research project
fundable?
Clearly addresses an important problem
Typically not more than one
Driven by a focused and testable hypothesis
Asks questions that prove or disprove a hypothesis rather
than search for a problem or simply collect information.
Lays the foundation for further research in the field, opens
up new fields, or impacts the way we view a problem
All aspects of the project are clearly linked
You seem like the ‘right person’ to do it
Now is not the time to pitch new ideas
Strategies To Keep in Mind
Find information on formats, page limits, and rules in
advance, NOT after writing your first draft
Understand the review criteria from the outset and keep
these in mind as you are writing
Page Limits
Introduction 1 page
Except Intro to Resubmission for Ts, K12 and R25
Specific Aims 1 page
Research Strategy 6 pages
R03, R13/U13, R21, R36, R41, R43, Fs, SC2, SC3
Research Strategy 12 pages
R01, single project U01, R10, R15, R18, U18, R21/R33, R24, R33,
R34, U34, R42, R44, DP3, G08, G11, G13, UH2, UH3, SC1
Candidate Information + Research Strategy 12 pages
Career Development Awards (Ks, except K12)
Research Training Program Plan 25 pages
Including NRSA (Ts), K12 and R25
Restructured Research Plan
Introduction
Specific Aims
Background and Significance Research
Preliminary Studies/Progress Report Strategy
Research Design and Methods
Inclusion Enrollment Report
Progress Report Publication List
Human Subjects Sections
protections, women/minorities, enrollment, children
Other Research Plan Sections
animals, select agent, consortium, support, sharing
Appendix
Restructured Research Plan
Previous Application Current Application
Background and Significance Research Strategy
a. Significance
b. Innovation
Preliminary Studies/Progress
c. Approach
Report
• Preliminary Studies for
New Applications
Research Design and Methods • Progress Report for
Renewal/Revision
What Reviewers Evaluate for Research
Grants
Overall Impact
“Core” Criteria
Significance
Investigators
Innovation
Approach
Environment
Additional Issues (e.g. Human Subjects Protections)
Overall Impact
Reflects the reviewer’s assessment of the likelihood for
the project to exert a sustained, powerful influence on
the research field(s) involved
Based on the five core review criteria
and additional review criteria as applicable for the project
proposed
An application does not need to be strong in all
categories to be judged likely to have major scientific
impact.
For example, a project that by its nature is not innovative may be
essential to advance a field.
Significance
Does the project address an important problem or a
critical barrier to progress in the field?
If the aims of the project are achieved, how will scientific
knowledge, technical capability, and/or clinical practice
be improved?
How will successful completion of the aims change the
concepts, methods, technologies, treatments, services,
or preventative interventions that drive this field?
Investigator(s)
Are the PIs, collaborators, and other researchers well
suited to the project?
If Early Stage Investigators or New Investigators, do they
have appropriate experience and training?
If established, have they demonstrated an ongoing
record of accomplishments that have advanced their field
(s)?
If the project is collaborative or multi-PD/PI, do the
investigators have complementary and integrated
expertise; are their leadership approach, governance
and organizational structure appropriate for the project?
Innovation
Does the application challenge and seek to shift current
research or clinical practice paradigms by utilizing novel
theoretical concepts, approaches or methodologies,
instrumentation, or interventions?
Are the concepts, approaches or methodologies,
instrumentation, or interventions novel to one field of
research or novel in a broad sense?
Is a refinement, improvement, or new application of
theoretical concepts, approaches or methodologies,
instrumentation, or interventions proposed?
Approach
Are the overall strategy, methodology, and analyses
well-reasoned and appropriate?
Are potential problems, alternative strategies, and
benchmarks for success presented?
If the project is in the early stages of development, will
the strategy establish feasibility and how will particularly
risky aspects be managed?
If the project involves clinical research, are the plans for
protection of human subjects, and inclusion of minorities
and members of both sexes/genders, and the inclusion
of children, justified in terms of the scientific goals and
research strategy proposed?
Environment
Will the scientific environment in which the work will be
done contribute to the probability of success?
Are the institutional support, equipment and other
physical resources available to the investigators
adequate for the project proposed?
Will the project benefit from unique features of the
scientific environment, subject populations, or
collaborative arrangements?
Specific Aims
Provides an overview of the details - tells what your
proposal is about, and how you will get there
start with 1 - 2 paragraph general overview
then list AIMS, each clearly defined
end with a brief statement of what you will learn if successful
The reader must finish this section convinced that the
proposed research is significant and that you have a
feasible approach
The aims should be clearly and concisely stated; many also
include sub-aims
Typically 2 - 4 related aims. Later aims should NOT
depend on the success of previous aims
Significance & Innovation
The place to clearly state the importance and innovative-
ness of the proposed research
Not over- or under-hyped
Disease relevance is one, of several approaches
Looks both backward and forward and points out
controversies and discrepancies that your work will
address
Should be appropriately referenced with an honest and
balanced discussion of others’ work
Do not underestimate the value of this section. A proposal
with a strong research plan will generate little enthusiasm if
the problem is not seen as significant.
Approach
Includes both preliminary results/progress report AND your
proposed studies
Organize by specific aims
Useful tool: rationale - approach - possible problems
Clearly discuss controls, both positive and negative
Show you have thought through issues of feasibility, sample
size, patient recruitment, data analysis, etc.
Include a discussion of expected outcomes, data
interpretation, potential problems, and alternate approaches
Preliminary Data or Progress Report
Key pieces of data to generate excitement and
enthusiasm for the proposed studies
Demonstrates feasibility that you can do what you say
you are going to do
Shows you are a careful scientist who does controls and
does not over-interpret data
Figures should have clear legends and should be large
enough for reviewers to easily read
Consider whether to include key pieces of published
data
Keep in Mind
Reviewers generally assume that new investigators are
incapable of conducting experiments if they have not
demonstrated previous competence with the methodology.
Including a timeframe helps provide a framework for
understanding your plan
Reviewers carefully read sections relating to animal use or
human subjects
Strong research plans:
Explicitly state the rationale for the proposed studies
Never assume reviewers will intrinsically appreciate or
understand what you intend
Use flow diagrams for overview, and for complex
experiments and protocols
Include well-designed, easy to follow tables and figures
Address priorities if patients, reagents or resources will
be limited
Include a discussion of how the data will be analyzed
and interpreted
Include realistic discussions of pitfalls and provide
alternate approaches
Important Point
It is your goal to get people excited about your research.
Let your enthusiasm for your research be reflected in
your proposal.
If you are not enthusiastic when writing your proposal, it
is unlikely the reviewers will see anything different
Other important considerations
Biosketch
Indicate your qualifications to carry out the proposed
work
Don’t “pad” with lots of “in preparation” manuscripts
Literature cited/Bibliography
Be thorough, but critical, in citing previous work in the
field
Letters of collaboration
Should be enthusiastically supportive and definitively
state what will be provided
You may need to write these for your collaborators
Common criticisms - Avoid getting
“dinged”
Rationale for hypothesis or methods not sound
Models over-hyped as relevant to the human situation
Diffuse, unfocused or superficial examination of the field
Unexciting science - an incremental advance for the field
Mediocre preliminary data that are over-interpreted
Lack of experience in required methodologies
Unrealistic amount of work
Lack of sufficient experimental detail
Too many irrelevant experimental details
Insufficient discussion of potential pitfalls and alternate strategies
Lack of knowledge of published work
Hard to read - poorly constructed, dense, or filled with typographical/
grammatical errors
What Reviewers Really Say
This is the first of three very long aims that could make its own
proposal. The subaims just go on and on.
An important question and an elegant approach; however there is
no discussion of how many targets are expected, and most
importantly, what criteria will be used to select which targets to
pursue.
The new computational tools are not tested in relevant biological
systems.
This is a horizontal contribution to the field.
The investigator does not pay sufficient attention to feasibility
issues, including the enrollment of research subjects and careful
attention to inclusion issues.
Insufficient information is given to indicate how the CART analysis
will be implemented, and no discussion of power analysis is given.
These omissions are particularly unfortunate.
The role of these very senior scientists needs to be defined.
Options After a Negative Initial Peer
Review
Appeal of the initial review to the Institute Council/Board
Must demonstrate that the initial review was procedurally or
scientifically flawed. Objections to scientific interpretations or
emphasis do not succeed
Generally not a good choice
Discuss with Program Officer first
Submission of an amended application
Amended Applications
Can submit one amended application
Must respond to reviewers’ criticisms
Do not have to agree or make the suggested changes, but must
respond to the comments
Do not attack the reviewers’ competence, abilities, etc. This will
only hurt your cause.
No guarantees that amended application will score better
than previous submission
Different reviewers
Different panel of applications
Revisions
Give yourself the time and space to feel sad and angry, but
appreciate that your colleagues, students, lab members
are watching
Avoid calling or writing your program officer until you have
calmed down
Then read the reviewer's comments CAREFULLY
You will need to decide whether or not the reviewers show
any enthusiasm for your application.
Talk with:
senior scientists with experience reading critiques
your program officer
Revisions (II)
If you decide to re-apply, respond explicitly to the
criticisms, indicating how and where you have revised your
application.
If you disagree with the reviewer on certain points, state
your arguments in a logical manner without challenging
their intelligence or understanding of the research area.
Your rebuttal is limited to 1 page; begin with a short
summary and then address each reviewer’s concerns,
one-by-one or by grouping similar concerns.
An example - absolute agreement
Reviewer 1 accurately pointed out that we had not
sufficiently discussed the detergents used to prepare cell
lysates for pull-down and co-immunoprecipitation assays.
We now expanded this discussion in AIM 3 of the revised
application.
Reviewer 2 pointed out that we lacked a clear way to
address the relevance of these protein interactions in an
animal model. There are no universally accepted animal
models for CF lung disease, but we now include studies in
mouse tissues and/or well-differentiated human primary
airway epithelial (WD-PAE) cell cultures to further explore
the physiological relevance of the interactions we identify.
An Example - Graciously Disagreeing
We wholeheartedly agree with Reviewer 2 that unfocused
research can indeed lead to “a quagmire of interacting
proteins”. However, we have several strategies in place to
ensure that we do not go down such a path. Specifically,
…….. As proof of principle, our progress since June 2004
clearly indicates that we can rapidly identify important
interactions for further analysis. Therefore we have
retained the protein interactions screens described in AIM
3 of the original application. However in response to the
reviewer’s concerns, we have significantly narrowed our
screen.
Conclusion
Unfortunately, only some of the deserving applications
can be funded
Maximize your chances for success by
Planning ahead
Remembering your target audiences
Showing the reviewers that you’ve thought about your project
Preparing a reader-friendly application
Remaining optimistic, and letting your enthusiasm for your
science come through
Exploring all potential funding mechanisms - internal, foundation,
and government
It’s About More Than The Science
Observation I:
Strong writing can not compensate for bad ideas, but weak writing
easily ruins good ideas
Observation II:
Writing well is a learned skill
There are great resources at NIH
Helpful Web Resources
New OER podcasts: http://grants.nih.gov/podcasts/
All_About_Grants/index.htm
NIH Home page http://www.nih.gov/
NIH Grant Application Basics (Includes guides, tips, and
tutorials) http://grants.nih.gov/grants/grant_basics.htm
Information on Study Sections http://cms.csr.nih.gov/
Science magazine GrantsNet
http://sciencecareers.sciencemag.org/funding
