Assessing statistics

Statistical analysis of quantitative research relates to the certainty of findings. You may have heard of statistical significance. However, this is not the same as clinical significance and it is important that you understand these concepts when making decision about patient care. [1] Here we introduce some of these to help you analyse quantitative palliative care research data.

The following terms are commonly used and when you come across them in research articles you will need to know what they mean or where you can find out more:

ANOVA - Analysis of variance
AR - Absolute risk
ARR - Absolute risk reduction
ARI - Absolute risk increase
CI - Confidence interval
HR - Hazard ratio
NNH - Number needed to harm
NNT - Number needed to treat
OR - Odds ratio
P - Value
RR - Relative risk
RRI - Relative risk increase risk
RRR - Relative risk reduction
SD - Standard deviation
SE - Standard error
SEM - Standard error of the mean
SMD - Standardised mean difference
WMD - Weighted mean difference

P-values and confidence interval

Along with p-values, the 95% confidence interval (CI) is one of the most frequently used statistical interpretations of data. The p-value reflects the probability that there is no difference between the groups being compared (null hypothesis). So, the smaller p-value is the less likely it is that any differences are due to chance. However, CI informs us of the lower and upper limits surrounding a parameter such as mean effect or odds ratio. In palliative care variability between patients or study participants can be considerable. Therefore, in many cases the CI can be more useful, and it may be large. You will need to decide whether the lower end of effect is acceptable. The video from NCCMT explains this from a pragmatic viewpoint of CI.

Clinical significance

Statistical confidence (p value) tells you if an effect is real but it does not inform you whether it is sufficient clinically to prompt a change in practice. Clinical significance (CS) is about the practical impact of an intervention in the real world, and it requires clinical judgement based on the context of care. Analysis of the CI can help you to determine CS.

Forest plot

A meta-analysis is used to statistically compare and quantitatively combine data from a number of studies (often RCTs). In this way outcomes from palliative care studies with very few participants might be combined to increase the cohort size and hence study power to arrive at a conclusion. This comparison is generally represented as a forest plot that graphically summarises the findings and any uncertainty surrounding the outcome. To help you interpret a forest plot the Cochrane blog Reading a Forest plot or the NCCMT video provide a starting point. If the studies are very different e.g. study design, participants, outcomes reported, then it may not be possible to conduct a meta-analysis.

Attrition bias

The high mortality rates and symptom burden among people with palliative care needs is challenging for research. [2] Because of this, patient attrition, or loss to follow up is highly likely and can affect study outcomes. The MORECare statement was developed in response to this to ensure best practice reporting of the reason for study attrition (death, illness, or random loss). [3] Generally, an attrition rate of 20% is considered acceptable. Attrition rates in palliative care trials vary but average 29%. [2] To account for this as well as reporting the reasons you should check that the study authors use a suitable imputation method such as Area under the curve analysis, last observation carried forward, or intention to treat analysis.

AI for bias

RobotReviewer is an artificial intelligence approach to extracting PICO information and assessing bias within RCT reports. This free to use software is now utilised by TRIPdatabase as part of their article description and is being trialled by Cochrane for use in their systemic review process. You can add pdfs of your RCT articles to the Demo RobotReviewer to gain a first impression of potential bias within a paper or series of papers and then read the paper to compare your judgment.

The bias outputs for RobotReviewer are:

  • Random sequence generation
  • Allocation concealment
  • Blinding of participants and personnel
  • Blinding of outcome assessment

  1. Scott IA. Statistical and clinical significance. Med J Aust. 2017 Aug 4;207(5):187-189. doi: 10.5694/mja16.01148.
  2. Oriani A, Dunleavy L, Sharples P, Perez Algorta G, Preston NJ. Are the MORECare guidelines on reporting of attrition in palliative care research populations appropriate? A systematic review and meta-analysis of randomised controlled trials. BMC Palliat Care. 2020 Jan 9;19(1):6. doi: 10.1186/s12904-019-0506-6.
  3. Higginson IJ, Evans CJ, Grande G, Preston N, Morgan M, McCrone P, et al. Evaluating complex interventions in end of life care: the MORECare statement on good practice generated by a synthesis of transparent expert consultations and systematic reviews. BMC Med. 2013 Apr 24;11:111. doi: 10.1186/1741-7015-11-111.

Page created 16 May 2022