We need to be mindful of the quality of evidence. Being actively open minded is part of good professional practice, but there is another level of more active critical appraisal. For example, before changing an organisational policy to accord with a new systematic review, we need to be sure of the underlying quality of that review. Are the results valid? Are they reliable? When and how should it be implemented? Critical appraisal has been defined as: the process of assessing and interpreting evidence by systematically considering its validity, results, and relevance. [1]
Sometimes these questions are easy to answer. For example, imagine you are considering surgery for your 82 year old patient. You might find a systematic review of randomised controlled trials examining the outcomes of this surgery. But on closer reading, you note that the authors actually exclude any studies with patients aged 65+. Surgical outcomes could be vastly different between the two different age groups. In such a case it is clear that you cannot automatically generalise the findings to your patient. However the appraisal process can be far more complex in other cases. There are some tools to help.
You can access others' published critical appraisals of studies. Reviewers commonly critically appraise a published research study, and create a structured abstract and commentary on the findings. Two of the most well known freely available databases of such reviews include the Database of Abstracts of Reviews of Effects, and the NHS Economic Evaluations Database. These databases are automatically included when you search The Cochrane Library. Finding such a published appraisal gives you an immediate critique of the quality of a piece of a research, usually by experts in the field.
Another strategy is to check printed or online responses or 'Letters to the Editor' in the subsequent few journal issues that follow the issue publishing an article of interest. These letters can give you an immediate expert opinion of the study.
Other published critical appraisals form the basis of a new type of journal that has emerged over the last few years. These are known as summary journals. Some, such as Bandolier, and ACP Journal Club are free to access. Most, to date, still require a paid subscription, such as Evidence Based Medicine, Evidence Based Nursing etc.
Because different research designs can have different potential sources of bias, critical appraisal tools, such as Critical Appraisal Skills Programme (CASP) Appraisal Tools Public Health Resource Unit NHS Vic and JAMA Users Guides to Evidence Based Practice Centre for Health Evidence, have been developed. These provide you with lists of the most important questions which should be asked of different research designs. Palliative care (as with other fields) uses a range of study methodologies from qualitative through to quantitative designs, and it is important to match the choice of tool to the study's basic design. Using an appropriate tool to critically appraise research in your field, as a group, may be a useful journal club activity.
Finding out more
Glossary of Cochrane Collaboration and Research Terms
Aggregated list of Critical Appraisal Tools
References
- Green S, Higgins J, editors. Glossary. Cochrane Handbook for Systematic Reviews of Interventions 4.2.5 [updated May 2005]. http://www.cochrane.org/resources/glossary.htm/ (accessed 31 March 2008).
This page was created on 30 April 2008 and is due for review in April 2010