Fibromyalgia & musculoskeletal pain
Large language model-driven sentiment analysis for facilitating fibromyalgia diagnosis
Authors' conclusion
This proof-of-concept study suggests LLM-driven sentiment analysis, especially with prompt engineering, may facilitate FM diagnosis by detecting subtle differences in pain expression.
Study context
- Population
- 40 patients with fibromyalgia (FM) (2016 ACR criteria) and 40 non-FM chronic pain controls referred to rheumatology clinics; transcribed responses to questions on pain and sleep40 FM, 40 controls
- Comparison
- Prompt-engineered vs ablated LLM sentiment analysis for FM diagnosis
- Outcome
- Accuracy, precision, recall, specificity and AUROC for distinguishing FM, with rheumatologist diagnosis as ground truth
Methodological detailsproof-of-concept, ablation comparison
- Proof-of-concept
- Ablation comparison
Performance
As reported in the abstract
Diagnostic performance for distinguishing FM
| Test or model | AUROC | Specificity | Accuracy | Precision | Recall |
|---|---|---|---|---|---|
| Prompt-engineered | 0.86 | 0.82 | 0.87 | 0.92 | 0.84 |
| Ablated | 0.76 | 0.75 | 0.76 | 0.75 | 0.77 |
AUROC
Prompt-engineered0.86
Ablated0.76
Specificity
Prompt-engineered0.82
Ablated0.75
Accuracy
Prompt-engineered0.87
Ablated0.76
Precision
Prompt-engineered0.92
Ablated0.75
Recall
Prompt-engineered0.84
Ablated0.77
- Prompt-engineered vs Ablated: McNemar's test p<0.001 — accuracy superior to ablated approach
Original publication
Large language model-driven sentiment analysis for facilitating fibromyalgia diagnosis. RMD Open. Published 28 June 2024. doi:10.
Composed by AI from the published abstract and checked against it. How Briefs are made →