Peer review tips for students

Peer Reviewing Tips
How to write a good peer review report

In scientific publishing, researchers are both authors and reviewers, but while many graduate students will be taught how to write an article, not all PhD students and postdocs will be taught how to write a peer review report. Since all peer review reports on F1000Research articles are public, they form a collective resource of peer review examples. Below, experienced peer reviewers share their tips for writing a good peer review report. In addition, we have selected several peer review reports from papers published in F1000Research that can be used as examples.

  • Stay In Scope
    Keep comments within the scope of the paper.
    Sheila McCormick, University of California, Berkeley
  • Be Constructive
    Be constructive, view your reviewer role as an opportunity to help improve the paper you are reviewing.
    Bruce MacIver, Stanford University
    A helpful review with advice for improvement
    John Banks says:

    This article addresses the links between habitat condition and an endangered bird species in an important forest reserve (ASF) in eastern Kenya. It addresses an important topic, especially given ongoing anthropogenic pressures on this and similar types of forest reserves in eastern Kenya and throughout the tropics. Despite the rather small temporal and spatial extent of the study, it should make an important contribution to bird and forest conservation.

    Read full report

    This article addresses the links between habitat condition and an endangered bird species in an important forest reserve (ASF) in eastern Kenya. It addresses an important topic, especially given ongoing anthropogenic pressures on this and similar types of forest reserves in eastern Kenya and throughout the tropics. Despite the rather small temporal and spatial extent of the study, it should make an important contribution to bird and forest conservation.

    There are a number of issues with the methods and analysis that need to be clarified/addressed however; furthermore, some of the conclusions overreach the data collected, while other important results are given less emphasis that they warrant. Below are more specific comments by section:

    Abstract:

    The conclusion that human-driven tree removal is an important contributor to the degradation of ASF is reasonable given the data reported in the article. Elephant damage, while clearly likely a very big contributor to habitat modification in ASF, was not the focus of the study (the authors state clearly in the Discussion that elephant damage was not systematically quantified, and thus no data were analyzed) ‐ and thus should only be mentioned in passing here ‐ if at all.

    Introduction:

    More information about the life history ecology of A. Sokokensis would provide welcome context here. A bit more detail about breeding sites as well as dispersal behavior etc. would be helpful and especially why these and other aspects render the Pipit a good indicator species/proxy for habitat condition. This could be revisited in the Discussion as links are made between habitat conditions and occurrence of the bird (where you discuss the underlying mechanisms for why it thrives in some parts of ASF and not others, and why its abundance correlate strongly with some types of disturbance and not others). Again, you reference other studies that have explored other species in ASF and forest disturbance, but do not really explicitly state why the Pipit is a particularly important indicator of forest condition.

    Methods:
    • Bird Survey: As described, all sightings and calls were recorded and incorporated into distance analysis but it is not clear here whether or not distances to both auditory and visual encounters were measured the same way (i.e., with the rangefinder). Please clarify.
    • Floor litter sampling: Not clear here whether or not litter cover was recorded as a continuous or categorical variable (percentage). If not, please describe percentage categories used.
    Results:
    • Mean litter depth graph (Figure 2) and accompanying text reports the means and sd but no post-hoc comparison test (e.g. Tukey HSD) need to report the stats on which differences were/were not significant.
    • Figure 3 you indicate litter depth was better predictor of bird abundance than litter cover, but r-squared is higher for litter cover. Need to clarify (and also indicate why you chose only to shown depth values in Figure 3.
    • The linear equation can be put in Figure 3 caption (not necessary to include in text).
    • Figure 4 stats arent presented here; also, the caption states that tree loss and leaf litter are inversely correlated this might be taken to mean, given discussion (below) about pruning, that there could be a poaching threshold below which poaching may pay dividends to Pipits (and above which Pipits are negatively affected). This warrants further exploration/elaboration.
    • The pruning result is arguably the most important one here this suggests an intriguing trade-off between poaching and bird conservation (in particular, the suggestion that pruning by poachers may bolster Pipit populations or at the very least mitigate against other aspects of habitat degradation). Worth highlighting this more in Discussion.
    Discussion:
    • Last sentence on p. 7 suggests causality (That is because) but your data only support correlation (one can imagine that there may have been other extrinsic or intrinsic drivers of population decline).
    • P. 8: discussion of classification of habitat types in ASF is certainly interesting, but could be made much more succinct in keeping with focus of this paper.
    • P. 9, top: first paragraph could be expanded as noted before, tradeoff between poaching/pruning and Pipit abundance is worth exploring in more depth. Could your results be taken as a prescription for understory pruning as a conservation tool for the Sokoke Pipit or other threatened species? More detail here would be welcome (and also in Conclusion); in subsequent paragraph about Pipit foraging behavior and specific relationship to understory vegetation at varying heights could be incorporated into this discussion. Is there any info about optimal perch height for foraging or for flying through the understory? Linking to results of other studies in ASF, is there potential for positive correlations with optimal habitat conditions for the other important bird species in ASF in order to make more general conclusions about management?
    View Article & Reports
    Show less
  • Manage Your Time
    Dont underestimate the time it takes to carefully analyze a manuscript and write a constructive review.
    Hugues Abriel, University of Bern
  • Organize Your Comments
    When listing your specific concerns, separate them into major and minor points and, if your list is very long, consolidate the most minor points.
    Robert Fisher, Mount Sinai School of Medicine
    This well-organized review helped the authors improve their article
    Lisa Locatello says:

    Bierbach and co-authors investigated the topic of the evolution of the audience effect in live bearing fishes, by applying a comparative method. They specifically focused on the hypothesis that sperm competition risk, arising from male mate choice copying, and avoidance of aggressive interactions play a key role in driving the evolution of audience-induced changes in male mate choice behavior.

    Read full report

    Bierbach and co-authors investigated the topic of the evolution of the audience effect in live bearing fishes, by applying a comparative method. They specifically focused on the hypothesis that sperm competition risk, arising from male mate choice copying, and avoidance of aggressive interactions play a key role in driving the evolution of audience-induced changes in male mate choice behavior.

    The authors found support to their hypothesis of an influence of SCR on the evolution of deceptive behavior as their findings at species level showed a positive correlation between mean sexual activity and the occurrence of deceptive behavior. Moreover, they found a positive correlation between mean aggressiveness and sexual activity but they did not detect a relationship between aggressiveness and audience effects.

    The manuscript is certainly well written and attractive, but I have some major concerns on the data analyses that prevent me to endorse its acceptance at the present stage.

    I see three main problems with the statistics that could have led to potentially wrong results and, thus, to completely misleading conclusions.

    • First of all the Authors cannot run an ANCOVA in which there is a significant interaction between factor and covariate Tab. 2 (a). Indeed, when the assumption of common slopes is violated (as in their case), all other significant terms are meaningless. They might want to consider alternative statistical procedures, e.g. JohnsonNeyman method.
    • Second, the Authors cannot retain into the model a non significant interaction term, as this may affect estimations for the factors Tab. 2 (d). They need to remove the species x treatment interaction (as they did for other non significant terms, see top left of the same page 7).
    • The third problem I see regards all the GLMs in which species are compared. Authors entered the 'species' level as fixed factor when species are clearly a random factor. Entering species as fixed factors has the effect of badly inflating the denominator degrees of freedom, making authors conclusions far too permissive. They should, instead, use mixed LMs, in which species are the random factor. They should also take care that the degrees of freedom are approximately equal to the number of species (not the number of trials). To do so, they can enter as random factor the interaction between treatment and species.

    Data need to be re-analyzed relying on the proper statistical procedures to confirm results and conclusions.

    A more theoretical objection to the authors interpretation of results (supposing that results will be confirmed by the new analyses) could emerge from the idea that male success in mating with the preferred female may reduce the probability of immediate females re-mating, and thus reduce the risk of sperm competition on the short term. As a consequence, it may be not beneficial to significantly increase the risk of losing a high quality and inseminated female for a cost that will not be paid with certainty. The authors might want to consider also this for discussion.

    Lastly, I think that the scenario generated from comparative studies at species level may be explained by phylogenetic factors other than sexual selection. Only the inclusion of phylogeny, that allow to account for the shared history among species, into data analyses can lead to unequivocal adaptive explanations for the observed patterns. I see the difficulty in doing this with few species, as it is the case of the present study, but I would suggest the Authors to consider also this future perspective. Moreover, a phylogenetic comparative study would be aided by the recent development of a well-resolved phylogenetic tree for the genus Poecilia (Meredith 2011).

    Minor comments:

    Page 3: the authors should specify that also part of data on male aggressiveness (3 species from Table 1) come from previous studies, as they do for data on deceptive male mating behavior.

    Page 5: since data on mate choice come from other studies is it so necessary to report a detailed description of methods for this section? Maybe the authors could refer to the already published methods and only give a brief additional description.

    Page 6: how do the authors explain the complete absence of aggressive displays between the focal male and the audience male during the mate choice experiments? This sounds curious if considering that in all the examined species aggressive behaviors and dominance establishment are always observed during dyadic encounters.

    View Article & Reports
    Show less
  • Focus On The Science
    If the paper is in English, but not written by a native speaker, please be tolerant and just point out anything which changes the meaning: they have done a fantastic job in the first place.
    Sue Malcolm, Institute of Child Health, University College London
  • Start At The End
    Look at the conclusion paragraph first, thats going to tell you how exciting the science is. A good introduction tells you how a paper advances science; a good conclusion tells you how its going to change the world!
    Vincent Rotello, The University of Massachusetts Amherst
  • Consider The Statistics
    Its helpful if you comment on the number of replicates, the controls, and the statistical analyses. This information is crucial for understanding how robust the outcome is.
    Christine Mummery, Leiden University Medical Center
    An example of addressing statistical analysis
    Chris Baker says:

    In their response to my previous comments, the authors have clarified that only the data from the Experimental phase were used to calculate prediction accuracy. However, if I now understand the analysis procedure correctly, there are serious concerns with the approach adopted.

    Read full report

    In their response to my previous comments, the authors have clarified that only the data from the Experimental phase were used to calculate prediction accuracy. However, if I now understand the analysis procedure correctly, there are serious concerns with the approach adopted.

    First, let me state what I now understand the analysis procedure to be:

    • For each subject the PD values across the 20 trials were converted to z-scores.
    • For each stimulus, the mean z-score was calculated.
    • The sign of the mean z-score for each stimulus was used to make predictions.
    • For each of the 20 trials, if the sign of the z-score on that trial was the same as for the mean z-score for that stimulus, a hit (correct prediction) was assigned. In contrast, if the sign of the z-score on that trial was the opposite as for the mean z-score for that stimulus, a miss (incorrect prediction) was assigned.
    • For each stimulus the total hits and misses were calculated.
    • Average hits (correct prediction) for each stimulus was calculated across subjects.

    If this is a correct description of the procedure, the problem is that the same data were used to determine the sign of the z-score that would be associated with a correct prediction and to determine the actual correct predictions. This will effectively guarantee a correct prediction rate above chance.

    To check if this is true, I quickly generated random data and used the analysis procedure as laid out above (see MATLAB code below). Across 10,000 iterations of 100 random subjects, the average prediction accuracy was ~57% for each stimulus (standard deviation, 1.1%), remarkably similar to the values reported by the authors in their two studies. In this simulation, I assumed that all subjects contributed 20 trials, but in the actual data analyzed in the study, some subjects contributed fewer than 20 trials due to artifacts in the pupil measurements.

    If the above description of the analysis procedure is correct, then I think the authors have provided no evidence to support pupil dilation prediction of random events, with the results reflecting circularity in the analysis procedure.

    However, if the above description of the procedure is incorrect, the authors need to clarify exactly what the analysis procedure was, perhaps by providing their analysis scripts.

    MATLAB code:
    nTrials = 10; % 10 trials for each stimulus/condition for boot = 1:10000 % 10,000 iterations for bootstrapping for i = 1:100 % 100 subjects data = randn(nTrials,2); % generate random values for each trial meandata = squeeze(mean(data(:))); % calculate mean stddata = std(data(:)); % calculate standard deviation zdata = (data - meandata)/(stddata); % convert to z-scores meancond1 = mean(zdata(:,1)); % calculate mean for each stimulus/condition meancond2 = mean(zdata(:,2)); if meancond1 > 0 % evaluate sign of the mean values conscore = 1; % conscore indicates for which condition, positive z-values will indicate correctness conscoreB = 2; % conscoreB indicates for which condition, negative z-values will indicate correctness elseif meancond2 > 0 conscore = 2; conscoreB = 1; else error = 'They are equal' % if mean z-values are equal, arbitrarily assign correctness conscore = 1; conscoreB = 2; end accScores(i) = sum(squeeze(zdata(:,conscore))>0)./nTrials; %calculate average correct for each condition for each subject accScoresB(i) = sum(squeeze(zdata(:,conscoreB))<0)./nTrials; end mAcc(boot) = mean(accScores); % calculate average correct for each condition across subjects for each iteration mAccB(boot) = mean(accScoresB); end meanBoot = mean(mAcc)% calculate mean correct for each condition across iterations meanBootB = mean(mAccB) stdBoot = std(mAcc)% calculate standard deviation for each condition across iterations stdBootB = std(mAccB)
    View Article & Reports
    Show less
  • If Its Good, Say So
    Dont be afraid to be positive. If a paper that you are asked to review is really good, say so!
    Anthony Imbalzano, University of Massachusetts Medical School
    Positive comments in a detailed review
    Sue Griffin says:

    I think this paper excellent and is an important addition to the literature. I really like the conceptualization of a self-replicating cycle as it illustrates the concept that the problem starts with the neuron, i.e., due to one or more of a variety of insults, the neuron is negatively impacted and releases H1, which in turn activates microglia with over expression of cytokines that may, when limited, foster repair but when activated becomes chronic (as is demonstrated here with the potential of cyclic H1 release) and thus facilitates neurotoxicity. I hope the authors intend to measure cytokine expression soon, especially IL-1 and TNF in both astrocytes and microglia, and S100B in astrocytes.

    Read full report

    I think this paper excellent and is an important addition to the literature. I really like the conceptualization of a self-replicating cycle as it illustrates the concept that the problem starts with the neuron, i.e., due to one or more of a variety of insults, the neuron is negatively impacted and releases H1, which in turn activates microglia with over expression of cytokines that may, when limited, foster repair but when activated becomes chronic (as is demonstrated here with the potential of cyclic H1 release) and thus facilitates neurotoxicity. I hope the authors intend to measure cytokine expression soon, especially IL-1 and TNF in both astrocytes and microglia, and S100B in astrocytes.

    02/08/2013 ‐ additional comments

    In more detail, Gilthorpe and colleagues provide novel experimental data that demonstrate a new role for a specific histone proteinthe linker histone, H1in neurodegeneration. This study, which was originally designed to identify axonal chemorepellents, actually provided a previously unknown role for H1, as well as other novel and thought provoking results. Fortuitously, as sometimes happens, the authors had a pleasant surprise: their results set some old dogmas on their respective ears and opened up new avenues of approach for studying the role of histones in self-amplification of neurodegenerative cycles. In point, they show that H1 is not just a nice little partner of nuclear DNA as previously thought. H1 is released from damaged (or leaky) neurons, kills adjacent healthy neurons, and promotes a proinflammatory profile in both microglia and astrocytes.

    Interestingly, the authors conceptualization of a damaged neuron H1 release healthy neuron killing cycle does not take into account the H1-mediated proinflammatory glial response. This facet of the study opens for these investigators a new avenue they may wish to follow: the role of H1 in stimulation of neuroinflammation with overexpression of cytokines. This is interesting, as neuronal injury has been shown to set in motion an acute phase response that activates glia, increases their expression of cytokines (interleukin-1 and S100B), which, in turn, induce neurons to produce excess Alzheimer-related proteins such as βAPP and ApoE (favoring formation of mature Aβ/ApoE plaques), activated MAPK-p38 and hyperphosphorylated tau (favoring formation of neurofibrillary tangles), and α synuclein (favoring formation of Lewy bodies). To date, the neuronal response shown responsible for stimulating glia is neuronal stress related release of sAPP, but these H1 results from Gilthorpe and colleagues may contribute to or exacerbate the role of sAPP.

    View Article & Reports
    Show less