Artificial Intelligence‐Driven Extraction of Patient‐Reported Pain Outcomes in Osteoarthritis Using Large Language Models
Authors' conclusion
LLMs accurately extract explicitly documented PROMs from clinical notes and reliably identify documentation artifacts. Inference from narrative documentation was less precise when notes lacked sufficient contextual information and detailed pain descriptions.
Study context
- Population
- Patients in a prospective Osteoarthritis (OA) registry with routine internal medicine and orthopedic clinic notespaired internal medicine and orthopedic clinic notes
- Participants
- 163 patients
- Index test or model
- LLM-extracted or inferred pain scores vs abstracted scores and registry-collected pain scores (reference standard)
- Target condition
- pain scores on a 0-10 scale
Methodological detailspaired internal medicine and orthopedic clinic notes, prospective OA registry
- Paired internal medicine and orthopedic clinic notes
- Prospective OA registry
Performance
As reported in the abstract
| Test or model | n | Value | p |
|---|---|---|---|
| IM notes | 132 | τ 0.76 | p < 0.0001 |
| Orthopedic notes | 65 | τ 0.95 | p < 0.0001 |
n
IM notes132
Orthopedic notes65
Value
IM notesτ 0.76
Orthopedic notesτ 0.95
p
IM notesp < 0.0001
Orthopedic notesp < 0.0001
Further findings
| Outcome | IM notes | Orthopedic notes |
|---|---|---|
| LLM-based pain inference vs registry scores, notes without explicit pain scoresτ | 0.17 | 0.11 |
| LLM-physician agreement | ||
| Within ±1 point | 54.2%-88.0% | 54.8%-70.5% |
| Within ±2 points | 66.7%-100% | 84.5%-97.7% |
IM notes after excluding EMR-generated default zero scores
- EMR-generated default zero scores identified by the LLM in IM notes
- 15
- Agreement after excluding them (τ)
- 1.0
- p
- P < 0.0001
LLM-based pain inference vs registry scores, notes without explicit pain scores
- Performed comparably to physician estimation
Original publication
Artificial Intelligence‐Driven Extraction of Patient‐Reported Pain Outcomes in Osteoarthritis Using Large Language Models. Arthritis Care Res (Hoboken). Published 4 November 2026. doi:10.
Composed by AI from the published abstract and checked against it. How Briefs are made →