Manuscript review:  Pompano Lab Template and Tips

Top 4 questions to answer in your review (by Andrew Kinman & RRP)

1. What is the impact (a.k.a. significance) of the paper -- i.e. if they achieved their purpose, what did they achieve and why does it matter.

1. Does this fit the scope of the journal (some journals require work that impacts society directly; others accept advances in a specific field; others accept small incremental or technical progress)

2. What is the innovation claim

1. Is it actually new/ innovative? - answering this question requires a literature search
2. Does this fit the scope of the journal (some journals require completely new innovative work; others accept work that is useful even if not innovative)

3. What are the authors’ conclusions, and does the data support the conclusion?

1. If the conclusions are not supported by the data, this is a major flaw in the manuscript and must be addressed by revising the conclusions or adding data to support them.

4. Are the experiments accurate/meaningful

1. Are the proper controls included
2. Was the data interpreted accurately
3. Were the experiments sufficient to support all aspects of the conclusions, or should x,y,z samples/experiments have been tested too.
4. Were any major questions left unanswered, whose answer would change the impact or conclusion of the paper

Please follow the template below:

Internal record keeping info:

Manuscript Number:

Journal:

Title:  

Date of Review:

Reviewed by:

Comments to Author:

First paragraph:  Start with a short summary of your understanding of the main achievements of the paper (like a mini abstract, but in just a sentence or two).  Highlight the innovative parts and explicitly comment on any features of the work that are done especially well (if any… e.g. device design, experimental design).  If there are any fundamental flaws (conclusions not supported by data, many missing controls, etc), summarize those here -- but remember that many/most papers are not fundamentally flawed. End with a comment on whether the article would or would not be of interest to the readers of the journal, and why [you should look up the scope of the journal online, and tailor your comments to the scope].  Be polite and respectful no matter what.  Imagine you are receiving this review -- would you be offended?  If so, change the language.

Transition to major concerns (again respectful) and suggestions (if any). Some papers are excellent and no or few concerns are listed.  Others are weak in one or more areas.  If there are many concerns, separate them into Major and Minor as noted below.  Major is any issue that should prevent publication if not addressed AND are not easily addressable.  Minor are issues that it is okay if they are left unaddressed, or are easy to do.  “The paper would be improved by considering the following revisions:”

Major

1. List suggested revisions here. List in order of decreasing importance… most important first. 

2. If the manuscript is fundamentally flawed (the main conclusions not supported by data, many missing controls) and definitely needs to be rejected, then focus your critiques on only the most major flaws.  Do not bother with the minor critiques in this case, as the manuscript will need to be heavily rewritten anyway.  If the same flaw appears repeatedly, save yourself time by writing it as a single critique and then just listing the most egregious examples.  “The manuscript suffered from a lack of adequate controls, making it impossible to assess the validity of the conclusions.  For example, Figure 1a was missing XX control, Figure 2c was missing XX control, ….”

3. Maintain a polite demeanor.  Examples:

1. The conclusion ABC was not supported by the data in Figure ###.  It seems that the data actually show XYZ.  The authors should clarify how they came to the stated conclusion and provide XXX data to support it.

Minor

        Maintain a polite demeanor.  Examples:

2. In Paragraph 2 of the introduction/discussion/conclusion (page 5), XYZ point-of-confusion should be clarified, because…
3. Topic ABC has been studied previously.  Appropriate citations should be included, such as …
4. The axes in Figure 3 should be relabelled to say AA and BB.
5. The data in Figure Z / Table Y  or the prediction from Equation Z suggests that … . The text should explain whether this is true / how would you test this prediction / what would this result mean if it were true.
6. The manuscript would be much stronger/clearer if … ABC data were included in Figure 1 / experiments were compared to theoretical predictions / XYZ confounding factor were addressed on page 10.
7. More details are needed to describe the methods used in XXX experiments to ensure reproducibility.  For example…
8. The entire manuscript would benefit from grammatical editing by an experienced scientific writer.  
9. All variables should be defined.

Recommendation:  Accept / Accept after minor revisions / Accept after Major Revisions / Reject

Confidential Comments to Editor:  Any comments that the editor needs to see to make a decision about the paper, but that the author should not see.  Often there are none.  I will include a comment with your names, telling the editors that reviewers-in-training assisted with the review.

----------------

Writing style:

• Avoid use of “I” and “you” or “they” pronouns in reviews.  They make it too personal.  Instead, write in the third person (as in examples a-f above).

• Example:  Instead of “I think it would be helpful to...” just say “It would be helpful to...”.
• Example:  Instead of “You should do X”, say, “It would be helpful to do X” or “X would improve the paper”

• Be specific.

• Do not write:  “Please reorganize Figure 2 to make it easier to read.”
• In response to this comment, the authors are left unsure what to do.  Never assume that they will be able to look holistically at the figure to make it “easier to read”, because likely they already made it as clear as they know how.  Therefore if, as a reviewer, you want to see something changed, you need to list each change individually.
• Instead, suggest specific changes:  “In Figure 2a, the font size should be larger to improve legibility, and it is worth considering reversing the order of images in Figure 2d to match the order in which they are discussed in the text”.
• If you cannot figure out what specific changes are needed, just don’t say anything.

Philosophy on asking for additional work:

Here we come to a philosophical issue in reviewing:  To what extent to ask for new experiments, versus just asking for discussion of possible extensions.  There is much debate about this question and the answer differs between fields.  However, the general consensus is that a reviewer should only ask for new experiments that would be needed to support the conclusions that the author initially set forth.  

• For example, it is appropriate to request a new experiment to add a missing control condition, or to substantiate a claim that is currently unsupported.
• Avoid demanding that the authors should have done it in the way you would have done, e.g. use this assay instead of that assay, or this cell marker instead of that one.  Such demands are a common junior-reviewer mistake and indicate lack of appreciation that there are multiple ways to ask and answer a question.  Unless the choice impeded the ability to draw conclusions from the data, doing things a different way is fine.
• It is bad for the whole field if reviewers just ask for “more”… e.g. more readouts for a particular experiment, or just more experiments because they seem interesting.  If the readouts shown already were sufficient to support the conclusions of the paper, then a reviewer could ask the author to discuss potential additional readouts that would be useful.  That way they can just add some text to the discussion and still publish the work in a reasonable amount of time.
• Sometimes, the experiments shown are sufficient to support the authors conclusions, but more experiments would be needed to “elevate” the paper to the level of impact needed for the journal in question.  In that case, a reviewer could suggest the additional experiments, noting explicitly that they would be needed to raise the impact, but that the paper would stand as it is in a different more specialized journal.

Helpful resources for when you are new to reviewing:

Wiley publisher’s guide to peer review:

https://authorservices.wiley.com/Reviewers/journal-reviewers/how-to-perform-a-peer-review/step-by-step-guide-to-reviewing-a-manuscript.html 

Frederic G. Hoppin, Jr. "How I Review an Original Scientific Article", American Journal of Respiratory and Critical Care Medicine, Vol. 166, No. 8 (2002), pp. 1019-1023.  doi: 10.1164/rccm.200204-324OE

http://www.atsjournals.org/doi/full/10.1164/rccm.200204-324OE#.VRQUG-GxXwA 

How to become good at peer review: A guide for young scientists, by Jennifer Raff http://violentmetaphors.com/2013/12/13/how-to-become-good-at-peer-review-a-guide-for-young-scientists/ 

SEE NEXT PAGE FOR ACS REVIEWER LAB LINK