by Nisha Shah
As a junior researcher in a well-known academic centre, I often have to review the literature to: familiarise myself with a topic area; search the literature to support the writing of a paper; and, critically evaluate previous literature for quality and find gaps in a research area.
The latter is often the most difficult: critiquing others’ work, especially if it’s a prominent academician from a big academic institution, can be daunting as I am still trying to develop my analytical abilities. So where do you begin to know where to start critically evaluating literature?
Ask a friend
One place to start is to ask a friend, a peer or a supervisor about how they read academic literature in a critical manner. You can have them read the same paper, then discuss what you both thought, even if you identify with or contradict each other. What do others consider that you may not have thought about and vice versa? Journal clubs can be very good for picking up critical appraisal skills too.
Be methodical
You only have to google ‘how to critically evaluate literature’ to be flooded with lists upon lists of some very good methods to help review the quality of different types of studies. Firstly, however, you need to know what type of study is being reported in the paper, then use the relevant quality tool to outline what is reported. The NHLBI (National Heart, Lung, and Blood Institute) has a list of quality assessment tools for various study types. These are a good place to start to assess the paper(s) you are reviewing: https://www.nhlbi.nih.gov/health-pro/guidelines/in-develop/cardiovascular-risk-reduction/tools
Whatever your tool of choice, it does not have to be prescriptive, especially if you are not conducting an actual systematic review, but just developing your critical thinking skills. You can adapt these quality tools to suit the needs of your evaluation, or even create your own set of quality questions.
Don’t be too critical! All research will have bias
Once you have mastered all the things you should look out for in a paper, you can appraise much quicker. However, bear in mind, that whilst it is important to judge whether the claim(s) authors make against the presented evidence is valid or not, getting bogged down by details can deter from seeing the wider relevance of the work. Particularly as a junior researcher, it is easy to be fussy about whether 500 participants in a study are better than 50 (well, the more the better, but for qualitative work you might be pushing that statement a bit).
The usual cases in healthcare research involve real world data, constrained to the limitations of design and analysis methodologies employed, it can be easy to be too critical and not see the bigger picture. The advice is to be sensible, by understanding the work in the context of what it is trying to say and within its limitations.
In conclusion: practice, practice, practice!
Dissecting research and appraising pitfalls can be difficult, but you do get better by speaking to others and learning from each other, and over time as your project and experience progresses.