Grant and Manuscript Writing

Abstract

Perhaps nothing is more important to a new investigator than how to properly prepare a grant to request funding for clinical research or how to write a manuscript for publication. In this chapter we will review the basic elements for successful grant and manuscript writing, discuss advantages and disadvantages of K versus R applications for National Institutes of Health (NIH) funding, illustrate the “fundamentals” for each section for a standard NIH R-series application, and describe the key components necessary to transition to a successful NIH research career.

Basic Tenets of Grant Writing

The three fundamental principles involved in the successful preparation of an NIH grant are to understand the mission of the particular NIH branch from which you wish to secure funding, to know the peer review process, and to build the best team possible to accomplish the work proposed. It is very important, particularly to new investigators, to secure collaborators for areas in which you lack experience and training. While this often proves to be challenging for the new investigator since it is difficult to secure the attention of busy senior investigators, it is a critical step toward securing funding for the work you propose. Finally, grant writing like any skill, can only be optimized by doing it repeatedly. You can read all about the physics of learning to ride a bicycle, but until one does it repetitively, one will not be good at it. The same is true with respect to grant writing: writing, editing, and re-writing of the grant should occur on a regular basis.

Having all the tools described above in your “toolbox”, however, will not necessarily lead to a successful grant. The ideas must be presented, or “marketed” in such a way as to show the review team the importance of the proposed work as well as its innovative elements. The grant proposal must be presented in an attractive way and the information placed where reviewers expect to find it. Complex writing styles are also ill advised for grants. It is important to use clear and simple sentence structures, and to avoid complicated words. Also avoid the temptation to use abbreviations to save space since many abbreviations, or unusual abbreviations, make a grant difficult to read. Instead, use a reviewer friendly approach where the formatting is simple and the font is readable. Organize and use subheadings effectively (e.g., like a blueprint to the application), and use topic sentences for each section that build the “story” of your grant in a logical and sequential way. Use spell-checking programs before submission, and also, ask a colleague to read through the final draft before submission. Most importantly, be consistent in specific aims and format throughout the application.

Very importantly, the proposal must convince evaluators that the problem you are addressing is critical and significant, and that the team can deliver. Also, it is important to recognize that when investigators possess knowledge about a subject, it is hard for them to imagine what it is like not to know, and this is referred to as “the curse of expertise” [1]. This “curse” prevents effective communication. VanEkelenberg also points to the “chain of reasoning” to refer to the importance of a “roadmap that guides the reader through the proposal”. He also has developed a table (Table 19.1) that provides a model for the chain of reasoning.

Table 19.1 The PROSANA model for developing the “chain of reasoning”

The Blueprint of a Research Grant

For the scientist, the most important content of the NIH grant for which the proponent is fully responsible consists of the:

• Abstract

• Budget for initial period

• Budget for 5-year period

• Introduction (Revised or Supplemental applications)

• Research Plan, which includes:

  • Specific Aims

  • Background and Significance

  • Preliminary Studies/Progress Report

  • Research Design and Methods

  • Use of Human Subjects

  • Use of Vertebrate Animals

  • Literature Cited

  • Data Sharing Plan

There are many administrative forms that also must be included from your agency (such as the face page and the checklist, to name a few), but the items described above are where you will spend the majority of your time. It is important to carefully read the instructions, and also to check with your agency’s grants and contracts officer to resolve any questions early in the process of preparing your application.

Writing the Research Grant

In writing the research grant, start with strengths by clearly articulating the problem you will address and how it relates to the present state of knowledge. Find the gap in knowledge and show how your study will fill that gap and move the field closer to the desired state of knowledge. Pick the “right” question, knowing that the question should have potential to get society closer to an important scientific answer while at the same time knowing that there are many, more questions than one can answer in an individual career. In other words, get the right question, but don’t spend so much time figuring out what the right question is that you don’t move forward. The questions should lead you to research that have the potential for being fun. While securing NIH funding is an important milestone in your career, remember if your study is funded, you will be doing it for at least the next 2–5 years and it will impact your future area of research. Don’t propose any research question that you really do not think you will enjoy for the “long term”. Aside from the fun aspect (which is an important one), the “right” research question should lead to a hypothesis that is testable, that is based upon existing knowledge and fills and existing gap in specific areas of knowledge. Finally, the “right” research question is a question that can be transformed into a feasible study plan. How does one find the “right” research question? Open your eyes and observe: patients often provide clues into what is known and unknown about clinical practice. This approach formed the basis of one of the authors R01 (“does the variable left ventricular hypertrophy response in the context of hypertension have a genetic basis?”). Another way of coming by the “right” research question is through teaching and through new technologies.

Abstract

The abstract and specific aims (described below) are the two most important components of any grant application and must provide a cohesive framework for the application. The abstract provides an outline of the proposed research for you and the reviewer. Include in the abstract the research question that the study will address with a brief justification to orient the reviewer, the overall hypotheses to be tested, the study population you will recruit, the methods you will use, and the overall research plan (Table 19.2). These details are important so that study section personnel can decide which study section best fits the grant. The final statement in the abstract should indicate how the proposed research, if, successful, will advance your field of research. Always revise the abstract after your complete proposal has been written so that it agrees with what you have written in the research section.

Table 19.2 Components of an abstract

Developing a Research Question and Specific Aims

In developing a research question, one needs to choose a “good” or the “right” question as discussed above (also see Chap. 2). The “right” research question should lead you towards a testable hypothesis about the mechanisms underlying the disease process you are studying. A testable hypothesis will also require a feasible experimental design such that you can test the various predictions of your hypotheses in the most rigorous way so that your study does all that it can to fail to refute the null hypothesis if it is true. Once you have a testable hypothesis and a feasible and rigorous design to translate the research question into the hypothesis, there are certain necessary components that one needs to consider. Certainly, the hypothesis should define the study purpose, but should also address: the patient/subject eligibility (i.e., characterize the study population); the exposure (or the intervention); the comparison group; and the endpoints (outcomes, dependent variable – refer to PI(E)COS in Chap. 3). As described by Hulley et al. the criteria of a good hypothesis is that it is feasible, interesting, novel, ethical, manageable in scope, and relevant. It is helpful to engage colleagues to respond to how novel and interesting the hypothesis is and to address whether the results of your study will confirm extend, or refute prior findings, or provide new knowledge. Arguably, the most common mistake a new investigator makes is to have failed to narrowly focus the question such that it is feasible to answer with the research proposed. That is, avoid having a question that is too broad or vague to be reasonably answered. Finally, include only experiments that you and your colleagues and you’re your institution have the expertise and resources to conduct.

For the NIH grant, the hypotheses are written in Section A of the proposal, named “Specific Aims.” Specific aims are extensions of your research questions and hypotheses, and they should generally be no more than one page and should include (i) a brief introduction that underscores the importance of the proposed research, (ii) the most important findings to date, and (iii) the problem that the proposed research will address. Using the example of the genetic determinants of ventricular hypertrophy mentioned above, the aims section began with “(i) LVH is a common condition associated with cardiovascular morbidity and mortality… (ii) we have shown that LVH is, at least in part, genetically determined…. (iii) we anticipate these strategies will identify genetic variants that play clinically significant roles in LVH (Table 19.3)”. Such knowledge may suggest novel pathways to be explored as targets for preventive or therapeutic interventions.

Table 19.3 Components of specific aims

Even though the specific aims should be comprehensive in terms of the proposed research, the aims should be brief, simple, focused, and limited in number. Draft the specific aims like you would a novel such that you create a story that builds logically (i.e. each aim should flow logically into the next aim). The aims should be “realistic”, that is, they should represent one’s capacity for completing the work you propose and within the budget and the time requested. Use a variety of action verbs, such as characterize, create, determine, establish, delineate, analyze, or identify, to name a few. Most importantly, keep the aims simple, at the appropriate level of your team’s expertise, and where you have supporting preliminary data.

Writing specific aims can take on a variety of models. One model might be to have each aim present a different approach that tests a central hypothesis. Another model may be to have each aim develop or define the next logical step in a disease process. You should avoid a model in which an aim is dependent of the successful completion of an earlier aim. In other words, do not have aims that could only successfully move when and if the earlier aim is successful. Such contingent aims reduce the scientific merit of the grant since reviewers cannot assess their probability of success.

The Background and Significance Section

The background and significance section must convince your reviewers that your research is important; in other words, you must market your idea to reviewers in such a way that it engages them intellectually and excites them in terms of the potential for impact on clinical practice, and ultimately, health. You must also provide the foundation for your research, and show your knowledge of the literature. To provide the reviewer evidence of your ability to critically evaluate existing knowledge, the background and significance section should not only clearly state and justify the hypotheses, but should also justify variables and measurements to be collected, and how the research will extend knowledge when the hypotheses are tested. The wrap-up paragraph should discuss how your proposed research fits into the larger picture and demonstrate how the work proposed fills an important gap in knowledge. Some key questions to address are (Table 19.4):

• What is the current state of knowledge in this field?

• Why is this research important? Does it fill a specific gap in knowledge?

• What gaps in knowledge will this project fill?

• More generally, why is this line of research important?

Captivate the reviewer by emphasizing why the research question is fascinating. For instance, what is known? What question is still unanswered? And why do we want to answer this particular question? Finally, you must address what your proposed project has to do with the public health or clinical medicine.

Table 19.4 What should be in the background and significance section

Background and significance sections will be read by experts in your field since reviewers are selected based on their matched expertise with your project. Therefore, you must be both factual and provide “readable” material. Whenever possible, use cartoons or diagrams to clarify concepts and to visually break up the page. It is also useful to create a “road map” for your application in the introductory paragraph (e.g. in one of the author’s section, the following was used: “in this section, we review (1) the epidemiology of hypertension; (2) the pathophysiology of hypertension; (3) other medical consequences of hypertension; (4) the clinical treatment of hypertension; (5) the genetics of hypertension, and (6) implications for proposed research”. Having this roadmap is particularly important for the reviewer, since often a busy reviewer may only skim headings. Your headings within the background and significance section should lead the reviewer to know fully why that section is in the application. Like the specific aims, it is important to keep the background and significance section simple, to avoid jargon, to define acronyms, to use “sound bites”, and repeatedly use these “sound bites” throughout the application. Finally, engage a colleague from a close but unrelated field to read the background section to test the ease of understanding of its structure and content to a non-expert.

Preliminary Studies Section

The best predictor of what you will do tomorrow is what you did yesterday

The NIH has specific Instructions for the preliminary studies section, and “suggest” this section should provide an account of the principal investigator’s preliminary studies relevant to the work proposed and/or any other information—from the investigator and/or the research team—that will help to establish their experience and competence to pursue the proposed project. Six to eight pages are recommended for this section. Content should include previous research, prior experiments that set the stage for the proposal and build the foundation for the proposed study. The pilot data provided should be summarized using tables and figures. Interpretation is also important so that you demonstrate your ability to articulate accurately the relevance of your pilot data and correctly state the impact of your prior work. In a related way, this section also uses the previous results to demonstrate the feasibility of your proposed project. To convince reviewers of your research feasibility, you should discuss your own work--and that of your collaborators - on reasonably related projects, in order to convince reviewers that you can achieve your research aims. Pilot studies are required for many (but not all) R-series grants, and are extremely important to show your project is “do-able”.

The preliminary study section is particularly important for junior investigators where there may be inadequate investigator experience or training for the proposed research, a limited publication record, and/or a team that lacks the skill set required for the research proposed. The quality of the preliminary study section is critically important for junior investigators as the quality of the presentation of the pilot work is evidence of your ability to complete the work you propose.

Research Design and Methods

The research design and methods section is the place where you cover all the materials and methods needed to complete the proposed research. You must leave adequate time and sufficient space to complete this section. Many applicants run out of time and page requirements before the last aim is addressed in sufficient detail, significantly weakening the application. As concordant with the aims, it is important to not be overly ambitious. In the opening paragraph of this section it is also an important time to re-set “the scene” by refreshing the reviewer regarding the overview for each specific aim. Sometimes, this is the section where reviewers began to read the application. As you progress, use one paragraph to overview each specific aim, and then to deal with each sub-aim separately.

You should be clear, concise, yet detailed regarding how you will collect, analyze, and interpret your data. As stated in the specific aims section, it is important to keep your words and sentence structure simple because if the reviewer is confused and has to read your proposal numerous times, your score will suffer. At the end of this section give your projected sequence or timetable. This is the section to convince reviewers that have the skills, knowledge and resources to carry out the work, and that you have considered potential problems and pitfalls and considered a course of action if your planned methods fail. Finally, by providing data interpretation and conclusions based on the expected outcome, or on the chance that you find different results than expected (a not uncommon occurrence), it demonstrates that you are a thoughtful scientist.

One should provide a bit of detail for each section, such as addressing the design chosen for your research project and why you chose that design rather than another, what population you will study and why, what will be measured and how it will be operationalized in the clinical setting, and on what schedule. Develop each specific aim as a numerical entity by reiterating it, and using BOLDING or a text box in order to highlight it. Briefly re-state the rationale for your each aim.

Patient Enrollment

Convey to the reviewer your appreciation for the challenges in recruiting. Discuss from where the population will be recruited, what the population characteristics (gender, age, inclusion and exclusion criteria) will be, how subjects will be selected and the specific plans for contact and collaboration with clinicians that may assist you. Provide any previous experience you have with recruitment and include some numbers of subjects, and response rates, from previous or preliminary studies. Provide strategies to remedy any slow recruitment that might occur. Be cognizant of NIH policies in order to properly address issues related to gender, minority, and children inclusions and exclusions.

One also needs to consider and address the participant burden for the proposed research in order to properly weigh the benefits and costs of participation. In many studies, research subjects should be paid but not to the degree that it is coercive (See Chap. 8).

Methods

One should provide details for the most important techniques to be used in your research. For commercially available methods you need only to briefly describe or reference the technique; but, for methods crucial to your aims, you need to provide adequate description such as referencing published work, abstracts, or preliminary studies.

In the author’s experience, there are some common weaknesses of the Methods Section. These weaknesses include such issues as an illogical sequence of study aims and experiments; that subsequent aims (also known as contingent aims) rely on previous aims such that if the previous aims fail, the study comes to a halt. Inadequate description of contingency plans, or poorly conceived plans, or plans that are not feasible significantly weaken a proposal. Other weaknesses include not adequately describing or constructing the control groups; and/or underestimating the difficulty of the proposed research.

Tips for Successful Grants

A successful grant proposal generally “tells a story” and engages the reviewer. The proponent should anticipate questions that are likely to occur and present a balanced view for the reviewers. To be successful, you must not take things for granted, and you must deliver a clear, concise, and simply stated set of aims, background, preliminary studies, and experimental methods that has addressed threats to both internal and external validity. You must be able to follow directions precisely and accurately, and target your grant to the expected audience (i.e., your reviewer). Your timeline and budget must align with your aims. As stated earlier, you should obtain an independent review both from your mentors and collaborators, but from external reviewers if possible. And finally, and perhaps most importantly, remember, not every proposal gets FUNDED!, in fact only a minority get funded so it is prudent to submit a number of different proposals, understanding that you won’t get funded unless you submit proposals. When resubmitting proposals you should be careful to revise it based upon the critique and realize that reviewers are attempting to help you make your study better. There is no use getting mad–get funded instead! Every application must be above any level of embarrassment (i.e., do not submit anything that is not your best work). Develop a game face after submission, and be confident about your proposal. To maintain your sanity through the process, convince yourself that your grant won’t get funded while concurrently reminding your colleagues it is tough to get funded.

Types of NIH Research Funding

There are a number of types of NIH research funding, but of most relevance to clinical research are:

• Grant (Investigator Initiated)

• Cooperative Agreement (NIH is a partner; assistance with substantial involvement)

• Contract (purchaser)

• Training Awards

• Research career development awards

• Mentored NIH Career Development Awards

• K01/K08 Research/Clinical Scientist

• K23 and K24 Patient Oriented Research

• Mentored Research Scientist Development Award (K01)

These awards provide support for an intensive, supervised career development experience, leading to research independence for early or mid-career training, as well as to provide for a mechanism for career change (K24). The K24 requires that the applicant have a substantial redirection, appropriate to the candidate’s current background and experience, or that the award provides for a significant career enhancement. “Unlike a postdoctoral fellowship, the investigator must have demonstrated the capacity for productive work following the doctorate, and the institution sponsoring the investigator must treat the individual as a faculty member.”

The characteristics of the ideal candidate may vary. For example, the candidate may have been a past PI on an NIH research or career development award; but, if the proposed research is in a fundamentally new field of study or there has been a significant hiatus because of family or other personal obligations, they may still be a candidate for one of these awards. However, the candidate may not have a pending grant nor may they concurrently apply for any other career development award.

Summary

Remember; logically develop your aims, background, preliminary studies and research design and methods into a cohesive whole. Clearly delineate what will be studied, why it is important, how you will study it, who(m) you will study, and what the timeline is to complete the research. When writing, say what you’re going to say, then say it, and finally summarize what you said. Write a powerful introduction, particularly if you are constructing a revised application. Develop your “take-home messages” and reiterate them throughout your application. Finally, be tenacious: learn from your mistakes, pay careful attention to critiques, collaborate with smart people and find a good mentor. And, above all, keep it simple.

Manuscript Preparation

Many manuscripts follow after work is presented in abstract form at a major medical meeting. But, Fosbol has noted “while conferences allow abstracts public airing and media attention, we find it perplexing that two-thirds of these abstracts will not be published within a 2-year period” [2], and only 40 % will be published at 5 years. Fosbol also pointed out that abstracts rejected for presentation still had a 1 in 4 chance of being published; and, Winnik et al. found that among abstracts accepted to the European Society of Cardiology the subsequent publication of a manuscript reached 38 % and for rejected abstracts 24 % [3]. To analyze this issue further, Krzyzanowska et al. [4] reported on identifying factors associated with time to publication. They found that of 510 randomized trials, 26 % were not published in full within 5 years after presentation at a meeting. Eighty-one percent of the studies with significant results had been published but only 68 % with non-significant results were published in this same time period. They stated “non-publication breaks the contract that investigators make with trial participants, funding agencies, and ethics boards”.

The quality of reporting of abstracts is another issue that has been examined. Krzyzanowska et al. evaluated 510 abstracts and reported deficiencies in almost all [5]. For example 22 % of the abstracts failed to provide explicit identification of the primary endpoint. The general recommendations for abstract content are shown in Table 19.5.

Table 19.5 Suggested guidelines for what should be included in an abstract

There are many areas of overlap between writing a grant and writing a manuscript, but many differences as well. Irrespective of whether one is writing a grant or a manuscript (or anything else for that matter) it is important to remember that your writing is a reflection of your thinking, and as such, it should be clear and concise. If you want to be taken seriously, one must become a better writer (and that applies to all of us). Kerpan [6] outlined five steps to become a better writer as follows: practice, practice, practice; say it out loud; make it more concise; work on your headlines, and read great works.

The basic outline of a manuscript and a grant are the same: Title, Abstract, Introduction, Methods (to include patient recruitment, characteristics of the study population, study design etc.), and Statistical Analysis. Unlike a grant, however, save for preliminary study results, the actual results of the study are then presented followed by a focused Discussion, which should include study limitations, and finally, conclusions. Obviously budgetary data, data sharing considerations, and a few other issues peculiar to grants are not part of the manuscript preparation; but, funding sources and potential conflicts of interest (see below) should be listed.

In general, formulating relevant and precise questions that can be answered can be complex and time consuming. A structured approach for framing questions that uses five components may help facilitate the process. This approach is commonly known by the acronym “PICOS or PECOS” (Table 19.6): the patient population or the disease being addressed (P), the interventions (I) or exposure (E), the comparator group (C), the outcome or endpoint (O), and, the study design chosen (S). Providing information about the population requires a precise definition of a group of participants (often patients), such as men over the age of 65 years, their defining characteristics of interest (often disease), and possibly the setting of care considered, such as an acute care hospital. The interventions (exposures) under consideration in the manuscript need to be transparently reported. For example, if the reviewers answer a question regarding the association between a woman’s prenatal exposure to folic acid and subsequent offspring’s neural tube defects, reporting the dose, frequency, and duration of folic acid used in different studies is likely to be important for readers to interpret the review’s results and conclusions. Other interventions (exposures) might include diagnostic, preventative, or therapeutic treatments, arrangements of specific processes of care, lifestyle changes, psychosocial or educational interventions, or risk factors. Clearly reporting the choice of the comparator (control) group, and the intervention(s), such as usual care, drug, or placebo, is essential for readers to fully understand the reasons for one’s choice. Comparator groups are often very poorly described as are the reason(s) for that choice. Clearly reporting what the intervention or exposure is compared with is very important and may sometimes have implications. The outcomes of the intervention being assessed, such as mortality, morbidity, symptoms, or quality of life improvements, should be clearly specified as they are required to interpret the validity and generalizability of the studies results.

Table 19.6 Pneumonic for helping to remember a structured approach for framing questions

Guideline Statements for Manuscript Preparation (Table 19.7)

Table 19.7 Examples of some guideline statements for what should be included in manuscripts

Guideline statements for manuscripts include:

• CONSORT (Consolidated Standards of Reporting Trials)

• STROBE (Strengthening the Reporting of Observational Studies in Epidemiology)

• PRISMA (Preferred Reporting items for Systematic Reviews and Meta-Analyses) that is an extension of QUOREM (Quality of Reporting of Meta-Analyses for meta-analyses of RCTs)

• MOOSE (Meta-analysis of Observational Studies in Epidemiology, for meta-analyses of observational trials)

• STREGA (Strengthening the Reporting of Genetic Association Studies - an extension of STROBE)

A checklist has been formulated for each, and these guidelines have been accepted by many (most) Journals. These checklists vary somewhat from each other but there are many areas in common as well.

CONSORT [7]

The CONSORT 2010 Statement is a 25-item checklist and a flow (exclusionary cascade) diagram (see Fig. 19.1). It provides guidance for reporting all randomized controlled trials, but focuses on the most common design type—individually randomized, two-group, parallel trials. Other trial designs, such as cluster randomized trials and non-inferiority trials, require varying amounts of additional information.

Fig. 19.1

Flow diagram of the progress through the phases of a parallel randomized trial of two groups (that is, enrolment, intervention allocation, follow-up, and data analysis)

Title

Often, the title of the manuscript is added just before the manuscript is submitted to a journal for their consideration for publication, and yet it is the first thing that the editor and reviewers will see. Therefore, the title for the manuscript should be given some thought. A catchy title might grab the interest of the potential reader, but it should be accurate and reflect what was actually addressed by the study. It is often surprising how frequently the two are disparate. An example of this disparity (a true example with some words changed to protect the author) is a manuscript entitled “Discrepancy of Drug X and Drug Y Between the Blood Pressure Lowering Effect and Effect on Endothelial Function”, and the studies conclusion which stated “in conclusion, our results suggest that Drug X is recommended as second-line treatment despite the failure to lower blood pressure as much as Drug Y.” Finally, whereas the title of an oral presentation might include alliterations, these should be avoided in manuscript submissions (in the authors opinion Editors have little sense of humor).

Abstract

After the title, the Abstract is the next thing editors, reviewers, and ultimately readers will see. In fact, sometimes it is the only thing about the manuscript that will be read. It frequently is also what will be electronically accessible. Thus, like the title, considerable thought should be given to its content. Most journals are now suggesting and even requiring a structured abstract. This begins with the Background of the Study or Study Objectives depending on the specific journal. The Methods Section is usually next followed by a Results Section, and finally Study Conclusions. For most journals the word count for Abstracts ranges from 200 to 350 so one needs to carefully read the Instructions to Authors Section for details. Many (most) journals are now using an online submission format, so if the word count is in excess of what the journal allows it will automatically prevent submission. Thus, the abstract needs to be on the one hand concise, but on the other hand include all pertinent aspects of the design and results. Frequently it takes longer to write the abstract than the manuscript which brings to mind a quote by the French philosopher and mathematician Blaise Pascal who is quoted as saying “I am sorry to have wearied you with so long a letter but I did not have time to write you a short one” [8].

Keywords

Often, not much thought is given to keyword selection, and yet it is these words that will allow for future searches to identify the appropriate studies for literature searches and meta-analyses. One should, in fact, give due thought to these words by considering what you would want to enter into your search engine to find the data included in one’s manuscript.

Introduction

The manuscript introduction should be compared to the grant’s background and significance section but briefer. It should set the stage for the aims and/or hypothesis for the study one is writing about, thus, again it should be relatively brief and focused, that is, it should not be a literature review. It should also state the aim, hypothesis, and/or objective of the study about which one is reporting. The main function of the introduction though, is to “motivate the audience to read the paper and care about its results” [9].

Methods

The Methods section includes discussions of the trial design (e.g. parallel, factorial, crossover etc.) including the allocation ratio, eligibility criteria, the settings and location of the study population, and intervention details (e.g. how and when administered) with enough detail to allow replication. In addition, outcome(s) should be completely defined, pre-specified, and include both primary and secondary outcomes and how and when they were assessed. A statistical section should include sample size calculations and any planned interim analyses and stopping guidelines, randomization methods, type, etc., blinding (method and who was blinded), and statistical methods. In addition, the analytical approach used (e.g. intention to treat, etc.), should be included. If subgroup analyses are performed, the numbers of such analyses and whether they were pre-specified or post hoc should be mentioned. In addition, a discussion should be included of how patients/subjects who are lost to follow-up were handled.

Results

The results section should include a participant flow diagram (so that the reader can assess the studies generalizability, and other potential biases, recruitment strategies, baseline data (i.e. a description of the baseline features important to the study), almost always this includes demographics (e.g. comparing age, sex, socioeconomic differences between the groups studied). This baseline table is frequently called “table 1” of most manuscripts. The collection of baseline data has at least four main purposes:

• To characterize the patients included in the trial, i.e. to determine how successful randomization was

• To allow assessment of how well the different treatment groups are balanced,

• To allow for analysis per treatment group,

• To allow for subgroup analysis in order to assess whether treatment differences depend on certain patient characteristics

Some questions raised by baseline data analysis are; how is it measured? What does it mean if there is or is not statistically significant differences? And, does sample size matter? [10] An argument that exists is over whether to use statistical testing of baseline differences or to rely on a subjective comparison of baseline variables. One side of the argument is that on the one-hand, just because there is a difference in a baseline variable it doesn’t mean that it influences the outcome(s); and, on the other hand, just because there is no statistical difference doesn’t mean that there is not a baseline variable that does influence outcome. Furthermore, if the sample size is large, small differences that may not be clinically meaningful might show very significant statistical differences. Irrespective, it is generally agreed that statistically significant differences or lack thereof should not be completely relied upon, statistically significant differences are less of an issue for sample sizes over 500, and that baseline variables give some measure to assess comparability between the groups under study. Table 19.8 is an example of the “table 1” baseline comparability’s. This table can also be used to illustrate the issue of “column vs. row’ percentages and how data is displayed. If one is interested in emphasizing how a variable is distributed over outcomes (i.e. the percentages of each outcome per group) the data would be portrayed one way. On the other hand, if the interest is in emphasizing the percentages of groups that have the outcome, the data would be portrayed in another way (See Table 19.9).

Table 19.8 An example of a baseline variables table

Table 19.9 Compares the presentation of column vs. row data

In addition, the numbers analyzed for each group, estimated effect size and its precision (both absolute and relative effect size) any ancillary analyses, and any safety or analyses for harm.

Discussion

The discussion section should begin with a summary of results presented in general terms. Next should be a focused discussion of the study results in terms of what is already in the literature, to include similarities and differences. If mechanisms have been explored or suggested by the study, a discussion should include that as well.

Every study has limitations, so a frank discussion of those limitations, and the degree to which they might alter the results is appropriate. This should include any sources of potential bias and the generalizability of the study results.

Finally the manuscript should end with a focused conclusion that is again reflective of the study title and aims, followed by any acknowledgements, potential conflicts of interest, and funding sources. Welch [9, 11] have published outlines of what should go into the discussion and in what order and this is reproduced in Table 19.10.

Table 19.10 An outline of how to construct the discussion section of a manuscript

In a humorous but appropriate list of rules developed by Frank l. Vasco entitled “How to Write Good” [12], there are 23 tips provided as follows: avoid alliterations always; prepositions are not words to end a sentence with; avoid clichés like the plague (they are old hat); employ the vernacular; eschew ampersands & abbreviations etc.; parenthetical remarks (however relevant) are unnecessary; it is wrong to ever split an infinitive; contractions aren’t necessary; foreign words and phrases are not apropos; one should never generalize; eliminate quotations (as Ralph Waldo Emerson once said, “I hate quotations. Tell me what you know”); comparisons are as bad as clichés; don’t be redundant, don’t use more words than necessary, it’s highly superfluous; profanity sucks; be more or less specific; understatement is always best; exaggeration is a billion times worse than understatement; one word sentences…: eliminate analogies in writing, they are like feathers on a snake; the passive voice is to be avoided; go around the barn at noon and avoid colloquialisms; even if a mixed metaphor sings, it should be derailed; who reads rhetorical questions. Added to this is the proper choice of words to realistically reflect what you as the author is really saying. Accad [13] makes this point by suggesting that authors have embraced the activity of fortunetelling with the increasing use of the word “predicts” in medical writing which he sites a hyperbole. His point is that the use of the much mis-understood P value (to provide a sense of objectivity) refers to a group effect and not an individual patient. As an example of this latter concept he points out that when one “…is told that cardiac troponin predicts death because its elevation in the postsurgical setting is more prevalent among those who later died” when the actual results were when elevated values were identified 21 % died vs. 6 % who lived (and this ignores the fact that in this particular group, elevated levels could foretell a fatal outcome in only 32 %).

STROBE [14]

The STROBE statement defines the scope of the recommendations that cover three main study designs: cohort, case-control, and cross-sectional studies. A checklist of 22 items has been developed that relate like the CONSORT statement to the title, abstract, introduction, methods, results, and discussion Section. 18 items are common to all three observational study designs, and four are specific for cohort, case-control, or cross-sectional studies.

Otherwise, the same or similar principles hold for STROBE and CONSORT. Some differences between STROBE and CONSORT relate to the study designs. For example, for cohort studies the matching criteria and number of exposed and unexposed subjects should be mentioned, while for a case-control study the matching criteria and the number of controls per case should be emphasized. In terms of statistical analyses those used for control of confounding should be described as well as how missing data was addressed (see Chap. 3) along with a description of any sensitivity analyses.

PRISMA [15]

The PRISMA Statement for Reporting Systematic Reviews and Meta-Analyses of RCTs again has many common features of the other guideline statements and consists of a 27-item checklist and a four-phase flow diagram. Some differences between the guidelines include the mention in the title that identifies the report as a systematic review, meta-analysis, or both; and a mention of the synthesis and search methods in both the abstract and methods sections. In the methods section one should indicate if a review protocol exists, if and where it can be accessed (e.g., web address), and, if available, provide registration information including registration numbers. Eligibility in the context of meta-analyses specifies the characteristics of the studies included in the analysis (e.g., length of follow-up) and report characteristics (e.g., years considered, language, publication status) used as criteria for eligibility, giving rationale. Information sources should be described (e.g., databases with dates of coverage, contact with study authors to identify additional studies) the date last searched, and one should present a full electronic search strategy for at least one database, including any limits used, such that it could be repeated. State the process for selecting studies (i.e., screening, eligibility, included in systematic review, and, if applicable, included in the meta-analysis). More specifically, the method of data extraction from reports (e.g., piloted forms, independently, in duplicate) and any processes for obtaining and confirming data from investigators should be mentioned. The risk of bias in the individual studies that make up the meta-analysis and the methods used for assessing those risks (including specification of whether this was done at the study beginning, or outcome level), and how this information was to be used in any data synthesis. One should present the numbers of studies screened, assessed for eligibility, and included in the review, with reasons for exclusions at each stage, ideally with a flow diagram (Fig. 19.2). The results section should include for each study, the characteristics for which data were extracted (e.g., study size, follow-up period) and the results of individual studies. For all outcomes considered (benefits or harms), there should be a presentation for each study that includes: (a) simple summary data for each intervention group and (b) effect estimates and confidence intervals, ideally with a forest plot, including confidence intervals and measures of consistency.

Fig. 19.2

The flow of identifying and choosing studies to be included in the meta-analysis

For meta-analyses of observational trials the Meta-analysis of Observational Studies in Epidemiology (MOOSE) guidelines are suggested [16].

STREGA [17]

The STREGA statement for reporting genetic association studies builds on the STROBE guidelines and provides additions to 12 of the 22 items on the STROBE checklist. The components of the title, abstract, study design and population are similar to STROBE. But, things unique to genetic studies (e.g. whether the Hardy-Weinberg equilibrium was considered, methods used for genotypes or haplotypes, reporting of the numbers of individuals in whom genotyping was attempted and the numbers in whom it was successful) are obviously necessary for this specialized field. The interested reader can refer to the guideline document for more details and Chap. 11.

Conflicts of Interest, Authorship, Coercive Citation, and Disclosures in Industry-Related Associations

The International Committee of Medical Journal Editors (ICMJE) have published authorship criteria and to summarize “authorship credit should be based upon (1) substantial contributions to the conception and design, acquisition of data, or analysis and interpretation of data; (2) drafting the article or revising it critically for important intellectual content; and (3) final approval of the version to be published”. Authors should meet all three criteria. Regarding authorship, there has been a war on so called “ghost authorship”. According to one report, in 2008, honorary authors were attached to 25 % of research reports, 15 % of review articles, and 11 % or Editorials published in six major journals; [18] while in another report Mowatt et al. 39 % had evidence of honorary authorship [19] Greenland and Fontanarosa note that many times honorary authorship amounts to “coercive authorship” in which a senior person insists on being listed as an author even though they did not contribute substantially to the work; while in other cases the senior author is added in the hopes of increasing the chance of a manuscript being accepted [20]. Ghostwriting is defined as a person who writes books, articles, stories, reports, or other texts that are officially credited to another person-the opposite of honorary authorship. However, in the medical arena, what really happens is a ghostwriter submits their work to a medical investigator who then has the chance to edit, delete or add to the text as they see fit. None-the-less ghostwriting is highly discouraged in the scientific literature. Honorary and ghost authorship are frequently lumped together as “inappropriate authorship”.

Coercive citation is the practice in which an editor “forces” an author to add citations to an article (usually from that Editor’s Journal) before they will agree to publish it. This is done to inflate the journal’s impact factor (IF). Wilhite and Fong noted that despite the IFs shortcomings they continue to be a means by which the quality of science is weighed [21]. The Impact Factor of a journal was devised as a way to rank scientific journals, and is a measure of how often, on average, papers published in a journal are cited in other academic publications. IFs are used in some institutions as a promotional tool, but more recently IFs have become a source of increasing controversy, and Franck has criticized the practice where “success in science is rewarded with attention” [22].

The Institute of Medicine defines conflict of interest (COI) as “a set of circumstances that creates a risk that professional judgment or actions regarding a primary interest will be unduly influenced by a secondary interest”. The term COI has taken on an almost presumption of guilt, partially the result of a few highly publicized incidents in which there was an attempt to manipulate clinical research by blocking publications, withholding data, and falsely reporting results of a 12-month study as a 6 month trial. These events led in 2004 to the ICMJE’s call for mandatory clinical trial registration [23] –(this reference also serves as an excellent in-depth review of the subject).

There is a good deal of variation between journals in what information they require before accepting manuscripts for publication. One journal requires a 17-page questionnaire to be filled out. This has resulted in attempts to develop a uniform disclosure form, but with little success. In this regard, the authorship issue involved with industry-supported studies highlights the conflicts between academia and Industry. The general view is such funded studies particularly those with industry authors would be more biased and of lesser quality that studies funded through other sources. The increasing number of clinical trials that have full or partial industry funding has been increasing, and industry employees are increasingly appearing as coauthors of clinical trials that adds fuel to this belief, and yet there is little proof to support that belief. Booth et al. evaluated reports of RCTs evaluating systemic therapy of breast, colorectal and non-small cell lung cancer [24] and found that for-profit sponsorship and statistically significant results are independently associated with the endorsement of the experimental arm, even though authors who perform key roles in the conception, design, analysis, and interpretation of oncology trials are likely to have financial ties to industry [25]. Kaiser et al. published a non-industry supported study entitled “Is Funding Source Related to Study Design Quality in Obesity or Nutrition Supplement Randomized Control Trials (RCTs)?” The purpose of that study was to examine systematic quality differences amongst obesity and nutrition RCTs based on funding status in top tier journals. Thirty-eight obesity or nutrition intervention RCT articles were selected from high-profile journals (Lancet, Annals of Internal Medicine, JAMA, British Medical Journal) published between 2000 and 2007. Paired papers were selected from the same journal published in the same year, one not reporting industry funding and the other reporting industry funding. Papers had the following identifying information redacted: journal, title, authors, funding source and institution(s). Three raters independently and blindly rated each paper according to the Chalmers Method [26]. Total quality scores were calculated. The Wilcoxon signed ranks test and paired-samples t-test were used to compare Chalmer’s Index score between industry-funded versus non-industry funded studies. Inter-rater reliability using an intraclass correlation coefficient = 0.82 (95 % C.I. = .80 − .84). Mean quality score for industry-funded studies = 13.7, SD = 3.01; for non-industry funded studies mean score = 13.2, SD = 4.09. The Wilcoxon Signed Ranks test statistic, Z = −.523, p = .601 (two-tailed) indicated no categorical difference in study quality. Paired-samples t-test also indicated no significant mean difference in total quality scores between funding categories, t(18) = .587, p = .564 (two-tailed). They concluded that recently published RCTs in nutrition and obesity that appear in top-tier journals seem to be equivalent in quality of design, regardless of funding source (Table 19.11).

Table 19.11 Descriptive and test statistics for total and subscale scores for each funding category

In terms of conflict of interest, attention has been focused on whether financial ties to one drug company are associated with favorable results or conclusions. These ties have been questioned both as it relates to authors but also to journals. This has led the Cochrane Collaboration to put out a statement of its current policy that states “the sponsorship of a Cochrane review by any commercial source or sources…is prohibited” [28]. However, this area has been dominated by perceptions and not necessarily fact. Yank et al. [29] attempted to study financial ties by evaluating 124 meta-analyses that evaluated the effects of antihypertensive drugs in adults that compared any comparator on clinical endpoints. They concluded that “meta-analyses on antihypertensive drugs and with financial ties to one drug company are not associated with favorable results but are associated with favorable conclusions” (a so-called “spin” on the interpretation of the results) and that this discordance was not apparent in studies supported by non-profit groups. In an effort to address the financial conflict of interest and the impact that it has on the results of trials, Aneja et al. studied this question with respect to major cardiovascular trials. In their analysis they found that “self declared financial conflict of interest and source of funding do not seem to impact outcomes…” [30] and that a sub-analysis based upon the type of funding, or the selection of a surrogate over a clinical endpoint also did not seem to increase the likelihood of favorable trial results. In an accompanying editorial by Califf [30] some limitations of Aneja’s results was pointed out (e.g. three major journals were selected and how representative these journals were compared to all the literature was pointed out, along with the fact that self-reported financial conflict of interest could be inaccurate).

One major concern about conflicts of interest revolves around the development of clinical practice guidelines, since these guidelines are being increasingly used in malpractice cases and for forming the basis of many pay-for-performance initiatives. For example a study published in 2011 reported that more than half of the participants involved in writing recent American College of Cardiology/American Heart Association clinical practice guidelines reported some financial conflict of interest [31]. Rochan et al. developed a financial conflict of interest checklist for clinical research studies and invited comments, but there is still wide variation in requirements [32]. Controversy even exists about the term “conflict” which Weber points out “…almost implies that in order to receive the funding to do the research, the physician had to do something that had an adversarial or negative impact on the patients he was caring for.” [33] Indeed, Stossel states in that same article that “medicine is incomparably better than when I started out practicing about 40 years ago,” it is not because doctors are now somehow more ethical or have been more heavily regulated — rather, it is because of the products that they have developed and gotten through their collaborations with industry.

Another trend that is occurring with regard to manuscript publication is the increased frequency of open-access journals. Part of the justification for open access journals is the flawed peer-review process. Horton (Richard Horton, FRCP FMedSci, editor-in-chief of The Lancet) has opined, “… we know that the system of peer review is biased, unjust, unaccountable, incomplete, easily fixed, often insulting, usually ignorant, occasionally foolish, and frequently wrong.” We also know that the agreement between reviewers is often low, reviewers miss many mistakes, and reviewers can be biased against certain institutions and work that disagrees with what they have published. Peer review has resulted in the rejection of at least two papers that ultimately led to Nobel prizes; and that part of the reason for this is that there is little reward for the time spent in peer review, either monetarily of towards promotion.

Open access journals are scholarly journals that are available online to the reader “without financial, legal, or technical barriers other than those inseparable from gaining access to the internet itself” [34, 35]. Open access got its start about a decade ago and quickly won widespread acclaim with the advent of well-regarded, peer-reviewed journals like those published by the Public Library of Science, known as PLoS. Such articles were listed in databases like PubMed, which is maintained by the National Library of Medicine, and selected for their quality.

Some open access journals are subsidized, and some require payment on behalf of the author. Subsidized journals are financed by an academic institution, learned society or a government information center; those requiring payment are typically financed by money made available to researchers for the purpose from a public or private funding agency, as part of a research grant. There have also been several modifications of open-access journals that have considerably different natures: hybrid open-access journals and delayed open-access journals.

Open-access journals may be considered as:

• Journals entirely open access

• Journals with research articles open access (hybrid open-access journals)

• Journals with some research articles open access (hybrid open-access journals)

• Journals with some articles open access and the other delayed access

• Journals with delayed open access (delayed open-access journals)

  • Journals permitting self-archiving of articles

Advantages and disadvantages of open access journals are the subject of much discussion amongst scholars and publishers. A few obvious advantages of open access journals include the free access to scientific papers regardless of affiliation with a subscribing library, lower costs for research in academia and industry, in addition to improved access for the general public and higher citation rates for the author. The argument for open access is that peer review has many problems by itself, and it has become increasingly difficult to find qualified peer reviewers willing to spend uncompensated time for that task. For open access journals, it is expected that the reader will act as the peer reviewer, but some researchers are now raising the alarm about what they see as the proliferation of online journals that will print seemingly anything for a fee. They warn that non-experts doing online research will have trouble distinguishing credible research from junk. In fact Jeffrey Beall, a librarian at Auraria Library, University of Colorado Denver, in Denver, Colorado, has been posting frequently updated lists of potential predatory open access journals [36] and Nissan [37] cites an example reported in the New Science Magazine of a hoax designed to test the legitimacy of a certain publisher.

Another consideration in manuscript preparation is the expense of publishing, thus, manuscripts must be as brief as possible. And many journals are moving toward “open access” publications where the cost of the publication is borne by the author. To emphasize the brevity that manuscripts must strive for, a rather humorous exchange has been published in;

THE JOURNAL OF APPLIED BEHAVIOR ANALYSIS 1974, 7, 497 NUMBER 3, entitled THE UNSUCCESSFUL SELF-TREATMENT OF A CASE OF “WRITER’S BLOCK” by DENNIS UPPER, VETERANS ADMINISTRATION HOSPITAL, BROCKTON, MASSACHUSETTS

• Abstract, None

• Introduction, none

• Methods and Results, None

• Discussion, Blank

• References, 0

• Portions of this paper were not presented at the 81st Annual American Psychological Association Convention, Montreal, Canada, August 30, 1973. Reprints may be obtained from Dennis Upper, Behavior Therapy Unit, Veterans Administration Hospital, Brockton, Massachusetts 02401.

• Received 25 October 1973. (Published without revision.)

• COMMENTS BY REVIEWER A

• I have studied this manuscript very carefully with lemon juice and X-rays and have not detected a single flaw in either design or writing style. I suggest it be published without revision. Clearly it is the most concise manuscript I have ever seen-yet it contains sufficient detail to allow other investigators to replicate Dr. Upper’s failure. In comparison with the other manuscripts I get from you containing all that complicated detail, this one was a pleasure to examine.

• Surely we can find a place for this paper in the Journal-perhaps on the edge of a blank page.

A follow-up manuscript was published some years later in the same Journal (JOURNAL OF APPLIED BEHAVIOR ANALYSIS 2007, 40, 773 NUMBER 4 (WINTER 2007)) entitled A MULTISITE CROSS-CULTURAL REPLICATION OF UPPER’S (1974) UNSUCCESSFUL SELF-TREATMENT OF WRITER’S BLOCK by ROBERT DIDDEN RADBOUD UNIVERSITY NIJMEGEN JEFF SIGAFOOS UNIVERSITY OF TASMANIA MARK F. O’REILLY UNIVERSITY OF TEXAS AT AUSTINGIULIO E. LANCIONI UNIVERSITY OF BARI PETER STURMEY QUEENS COLLEGE, CITY UNIVERSITY OF NEW YORK

• ABSTRACT: None

• INTRODUCTION: None

• METHODS and RESULTS: None

• DISCUSSION: None

• Reviewers Comments

• The Consistency Between the Findings of This Multisite Cross-cultural Replication by Didden, Sigafoos, O’Reilly, Lancioni, and Sturmey and those reported in Upper’s classic paper on writer’s block (Upper, 1974) are remarkable and serve to substantially extend the generality of Upper’s findings.

• The consistency between the editorial opinion of the action editor, Linda LeBlanc, whose reviewer comments are enclosed verbatim parenthetically here

• (     ) and this paper is equally remarkable.

• This kind of symmetry is rare in any science and particularly rare in behavior analysis, and because of it I was compelled to accept the Didden et al. paper without revision. I did not change one word, and this is a first in my tenure as editor. Another virtue of the paper is its awe-inspiring brevity. It is my hope that it will one day serve as the model for Brief Reports in JABA.

• Preparation of this article was supported by a grant of $2.50 from the first author’s personal funds. We hope to submit a version of this paper at the next international conference in St. Tropez. Received July 2, 2007 Final acceptance July 5, 2007.