You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/index.md
+15-21Lines changed: 15 additions & 21 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,8 +1,5 @@
1
-
---
2
-
hide:
3
-
- navigation
4
-
---
5
-
# ER-Reason: A Benchmark Dataset for LLM-Based Clinical Reasoning in the Emergency Room
1
+
2
+
## ER-Reason: A Benchmark Dataset for LLM-Based Clinical Reasoning in the Emergency Room
6
3
7
4
**Advancing the evaluation of clinical reasoning in emergency care**
8
5
@@ -15,36 +12,33 @@ hide:
15
12
</div>
16
13
---
17
14
18
-
Large language models (LLMs) are increasingly applied to the field of medicine, but most benchmarks focus on narrow, exam-style medical question-answering rather than the evaluation of full clinical reasoning. ER-REASON captures the end-to-end ER decision-making pipeline, and is designed to test how well LLMs can approximate the complex, real-world reasoning of clinicians in a high-stakes settings.
15
+
Large language models (LLMs) are increasingly applied to the field of medicine, but most benchmarks focus on narrow, exam-style medical question-answering rather than full clinical reasoning. ER-Reason captures the end-to-end emergency room (ER) decision-making pipeline, designed to test how well LLMs can approximate the complex, real-world reasoning of clinicians in high-stakes settings.
19
16
20
17
---
21
18
22
19
## Key Components
23
20
24
21
### Dataset
25
-
- Contains de-identified longitudinal notes from **3,984 patients** across **3,984 ER encounters**, totaling **25,174 clinical notes**.
26
-
- Diverse note types: discharge summaries, progress notes, imaging reports, consult notes, echocardiography reports, and ER provider documentation.
22
+
-**3,984 patients** across **3,984 ER encounters**
23
+
-**25,174 de-identified clinical notes**
24
+
- Diverse note types: discharge summaries, progress notes, imaging reports, consult notes, echocardiography reports, and ER provider documentation
27
25
28
26
### Workflow-Aligned Tasks
29
-
This benchmark is structured around **five key stages of the emergency room workflow**, allowing realistic evaluation of reasoning at each stage:
27
+
This benchmark is structured around **five key stages of the ER workflow**:
30
28
31
-
1.**Triage Intake:**Initial review of patient presentation and acuity scoring
32
-
2.**Assessment:**Summarization of patient history in longitudinal notes
33
-
3.**Treatment Planning:**Assessment of differential diagnoses and clinical decision factors
34
-
4.**Disposition:**Determination of whether the patient should be discharged, admitted, observed, or transferred
35
-
5.**Final Diagnosis:**Formulation of the final patient diagnosis
29
+
1.**Triage Intake:**initial review of patient presentation and acuity scoring
30
+
2.**Assessment:**summarization of patient's medical history in longitudinal notes
31
+
3.**Treatment Planning:**assessment of differential diagnoses and clinical decision factors
32
+
4.**Disposition:**determination of whether the patient should be discharged, admitted, observed, or transferred
33
+
5.**Final Diagnosis:**formulation of the patient's final diagnosis
36
34
37
35
### Physician-Authored Rationales
38
-
- Includes **72 expert-authored rationales** explaining their clinical decision-making process.
39
-
- Provides a unique "gold standard" for evaluating reasoning in medical decisions, which is often missing from traditional EHR data.
36
+
- Includes **72 expert-authored rationales** explaining their clinical decision-making process
37
+
- Provides a rare "gold standard" for evaluating reasoning in medical decisions, often a missing component in traditional EHR data
40
38
41
39
---
42
40
43
-
This benchmark is a significant step forward in evaluating the real-world applicability of LLMs in emergency care by moving beyond factual recall to assess **true clinical reasoning, pattern recognition, and decision-making** under realistic conditions.
44
-
45
-
46
-
47
-
41
+
ER-Reason represents a major step forward in benchmarking the real-world applicability of LLMs in emergency care. By moving beyond factual recall, it evaluates models on **true clinical reasoning, pattern recognition, and decision-making** under realistic conditions.
0 commit comments