Exploring the magnitude of verification bias in diagnostic accuracy studies




Poster session 1


Sunday 16 September 2018 - 12:30 to 14:00

All authors in correct order:

Llewellyn A1, Simmonds M1
1 Centre for Reviews and Dissemination, University of York, United Kingdom
Presenting author and contact person

Presenting author:

Alexis Llewellyn

Contact person:

Abstract text
When assessing a new diagnostic test, patients with negative test results are less likely to receive a 'gold standard' test to confirm the diagnosis if that test is invasive or potentially harmful. This may result in misclassifying undetected disease as a true negative, so overestimating estimates of diagnostic accuracy. This is known as verification bias. The QUADAS-2 quality assessment tool currently does not include an assessment of the magnitude of verification bias.

To investigate the magnitude of verification bias and its implications for patients undergoing colposcopy tests.

We conducted a systematic review and meta-analysis of the diagnostic accuracy of adjunctive colposcopy for detecting precancerous and cancerous lesions in women at risk of cervical cancer. We assessed risk of bias using a modified version of the QUADAS-2 tool. We incorporated the predicted direction of bias in the QUADAS-2 tool, and explored the predicted magnitude of verification bias. We used sensitivity analyses to compare the effect of studies at high versus low risk of verification bias on sensitivity and specificity. Through simulations we explored the likely effect of verification bias on diagnostic accuracy estimates by varying the likely rate of misclassified undetected lesions based on published epidemiological evidence.

We included 11 diagnostic accuracy studies. Eight studies were at high risk of verification bias, and only three studies conducted biopsies (the 'gold standard') in all women analysed. Both sensitivity analyses and simulations confirmed that verification bias could overestimate sensitivity and specificity, but the magnitude and significance of bias varied depending on the method and assumptions we used.

Cochrane Reviews of diagnostic accuracy should incorporate considerations of the direction and magnitude of bias. The impact of verification bias on diagnostic accuracy should ultimately be weighed against the potential benefits of limiting the use of unnecessary testing.

Patient or healthcare consumer involvement:
We involved the main relevant UK patient group to define the scope and key outcomes of the review and gave them the opportunity to comment on the study report. The accuracy of colposcopy for detecting precancerous and cancerous lesions was important for this patient group.

Relevance to patients and consumers: 

The main relevant UK charity for women affected by cervical cancer and cervical abnormalities was involved when defining the scope and key outcomes for the review and was given the opportunity to comment on the study report. The main outcome in this study was important to the patient group.