Development and evaluation of a text analytics algorithm for automated application of national COVID-19 shielding criteria in rheumatology patients
Authors' conclusion
An automated algorithm for risk stratification has several advantages including reducing clinician time for manual review to allow more time for direct care, improving efficiency and increasing transparency in individual patient communication.
Study context
- Population
- Rheumatology outpatients of a large UK foundation trust, risk-stratified against national COVID-19 shielding criteria13,665 free-text medications
- Participants
- 803 outpatients
- Comparison
- Automated shielding algorithm vs clinical review
- Outcome
- Algorithm performance against clinical review (gold standard)
Methodological detailstext-mining algorithm development and evaluation, clinical review as gold standard, single foundation trust
- Text-mining algorithm development and evaluation
- Clinical review as gold standard
- Single foundation trust
Performance
As reported in the abstract
Identification of patients meeting shielding criteria
| Test or model | Sensitivity | Specificity | Positive likelihood ratio | Negative likelihood ratio | F1 score |
|---|---|---|---|---|---|
| Automated algorithm | 80% (95% CI: 75%, 85%) | 92% (95% CI: 90%, 94%) | 10 (95% CI: 8, 14) | 0.21 (95% CI: 0.16, 0.28) | 0.81 |
Automated algorithm
Sensitivity80% (95% CI: 75%, 85%)
Specificity92% (95% CI: 90%, 94%)
Positive likelihood ratio10 (95% CI: 8, 14)
Negative likelihood ratio0.21 (95% CI: 0.16, 0.28)
F1 score0.81
Further findings
Diagnoses extracted and mapped to SNOMED-CT
- Free-text diagnoses extracted and mapped
- 5942
Evaluation of mismatches
- Algorithm performed correctly against the gold standard in most cases
Deployment time on additional patients
- Additional patients
- 15 865
- Data extraction time
- 18 hours
- Deployment time
- 1 hour
Original publication
Development and evaluation of a text analytics algorithm for automated application of national COVID-19 shielding criteria in rheumatology patients. Ann Rheum Dis. Published 15 July 2024. doi:10.
Composed by AI from the published abstract and checked against it. How Briefs are made →