You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
|**Time Features**| Explicit time encodings showed no statistically significant benefit. | Sequence order in transformers may be sufficient for the tasks studied. |
16
16
|**Value Features**| Importance is task-dependent (critical for mortality, less so for readmission). | Code sequences alone can carry significant predictive signal for some tasks. |
17
-
|**Frozen Encoders**| Dramatically outperform trainable encoders with far fewer parameters. | Pretrained knowledge acts as a powerful, regularized feature extractor. |
18
-
|**Code Information**| Emerges as the most critical predictive signal across all experiments. | The quality of code representations is paramount for model performance. |
17
+
|**Frozen Encoders**| Tend to outperform trainable encoders with far fewer parameters. | Pretrained knowledge may serve as an effective regularized feature extractor. |
18
+
|**Code Information**| Appears to be the strongest predictive signal across the experiments studied. | Code representation quality may be a key driver of model performance. |
19
+
20
+
<palign="center">
21
+
<imgsrc="docs/waterfall_all_tasks_grouped.png"width="85%"alt="Waterfall plot of ablation results across all tasks" />
22
+
</p>
19
23
20
24
## Repository Structure
21
25
@@ -43,6 +47,26 @@ This work presents a systematic evaluation of tokenization approaches for clinic
43
47
-**Framework**: MEDS-Torch with transformer encoders
44
48
-**Evaluation**: AUROC with 10 random seeds, statistical significance testing
45
49
50
+
<palign="center">
51
+
<imgsrc="docs/efficiency_frontier_textcode.png"width="75%"alt="Efficiency frontier: performance vs. parameter count for TextCode variants" />
52
+
</p>
53
+
46
54
---
47
55
48
-
*This research demonstrates that simpler, more parameter-efficient tokenization approaches can achieve competitive performance in clinical time series modeling, challenging assumptions about the necessity of complex temporal encodings while clarifying the task-dependent role of value features.*
56
+
*This research suggests that simpler, more parameter-efficient tokenization approaches may achieve competitive performance in clinical time series modeling, raising questions about the necessity of complex temporal encodings and highlighting the task-dependent role of value features.*
57
+
58
+
---
59
+
60
+
## Citation
61
+
62
+
If you use this code or build on this work, please cite:
63
+
64
+
```bibtex
65
+
@misc{attrach2025ehrtokenization,
66
+
title = {Rethinking Tokenization for Clinical Time Series: When Less is More},
67
+
author = {Al Attrach, Rafi and Fani, Rajna and Restrepo, David and Jia, Yugang
68
+
and Celi, Leo Anthony and Sch\"{u}ffler, Peter},
69
+
year = {2025},
70
+
note = {Machine Learning for Health (ML4H) 2025 - Findings Track}
0 commit comments