Something that I realised when I had my undergrad in the lab was that looking at a paper critically doesn’t come naturally for many people. It can be really difficult to see the good and bad points, and if the results actually support the conclusions drawn. With that in mind, here is how I go about it.
First of all, I don’t read papers in the order they’re laid out in, which is why it’s mixed up. If you read it in order, how are you supposed to know if the methods were the best way to reach the conclusions when you haven’t read the discussion or results yet?
Disclaimer: there are many different factors to consider, and this is by no means exhaustive. I’m assuming the more general overview could be applied to many different disciplines, but I’m a biologist so can’t say for sure.
If you’re familiar with the subject, the introduction shouldn’t be too difficult. A quick skim read should suffice, it’s mainly to get a jist of what question the authors are asking. Unless it’s an area you don’t have much prior knowledge of, in which case reading this bit well can really help to understand why the authors are even asking this question in the first place.
Most of the time this will be nicely laid out into different subheadings, depending on the journal. What you’re looking for is the 1 or 2 sentences towards the end of each section concluding the relevant findings. Any bold, sweeping statements are a warning flag at this stage, until we get to the results and see if they can fully support these conclusions.
The better papers will also have a few sentences stating what the limitations of the study were; how it could be further improved and what future studies would be useful to further confirm their findings. If the suggestions include using a wider variety of techniques than used in the paper, that’s a big plus.
Now we’re getting to the (hopefully) good bit. The aim here is to try to see if the conclusions the authors have made match the actual results. For each figure, read the figure legend carefully to work out a) what they’re trying to get across, and b) if the results are convincing and thorough enough to support their theories.
This is going to vary massively depending on your subject area, and as I’m sure you’re aware, different researchers favour different methods. Some might prefer microscopy, whereas others might like to do techniques like proteomics to support their work. However, the key points are:
- Have they used suitable controls?
- Have they confirmed the results with different methods?
If you don’t think the proper controls have been used, what do you think they could have included to make it more reliable? Would you use any additional techniques if you were doing the same thing? If at this point you’re unsure of how they’ve done something, now is the time to scour the methods and find out the nitty-gritty of it all.
Is it a good paper?
The best way to critique a paper is to pick every little thing apart and inspect it very carefully. Whether you think a paper is good or not is entirely up to you. The more you take the time to read papers, the easier it will be to spot the good, the bad and the just plain ugly papers out there. Something that is published doesn’t necessarily make it good, and although the journal’s impact factor may give an indication, it is certainly not a fool proof method of telling whether a paper is good or bad. Good papers can be in bad journals, and, perhaps less often, bad papers can be in really good journals.
What are your top tips for looking at papers?