Initial commit: Code and experiments for 'Rethinking Tokenization for Clinical Time Series'

Author: rafiattrach

.github/workflows/code-quality-main.yaml

@@ -1,32 +1,21 @@
 # Rethinking Tokenization for Clinical Time Series (ML4H Findings 2025)
 
 ## Acknowledgements
-This repository contains the code for the experiments in our paper, "Rethinking Tokenization for Clinical Time Series: When Less is More." The codebase is adapted from the `meds-torch` library. We thank the original authors for their foundational work. For the maintained, production-ready version of the library, please see the official `meds-torch` repository.
+This repository contains the code for the experiments in our paper, "Rethinking Tokenization for Clinical Time Series: When Less is More." The codebase is adapted from the [`meds-torch` library](https://github.com/Oufattole/meds-torch). We thank the original authors for their foundational work. For the maintained, production-ready version of the library, please see the official repository.
 
 ---
 
 ## Overview
+This work presents a systematic evaluation of tokenization approaches for clinical time series modeling. We compare Triplet and TextCode strategies across four prediction tasks on MIMIC-IV to investigate the roles of time, value, and code representations. Our findings suggest that for transformer-based models, tokenization can often be simplified without sacrificing performance.
 
-This work presents a systematic evaluation of tokenization approaches for clinical time series modeling, comparing Triplet and TextCode tokenization strategies across four clinical prediction tasks using the MIMIC-IV dataset.
+## Key Findings Summary
 
-## Key Research Contributions
-
-### 1. Triplet Tokenization Ablations
-- **Time2Vec Implementation**: Advanced time encoding using sinusoidal basis functions
-- **LeTE Implementation**: Learnable time embeddings with Fourier and spline components  
-- **Component Ablations**: Systematic removal of time and value features to isolate predictive signals
-- **Code-only Variants**: Minimal tokenization using only medical codes
-
-### 2. TextCode Tokenization Improvements
-- **Flexible TextCode Encoder**: Support for both trainable and frozen language model encoders
-- **Enhanced Code Mappings**: Complete coverage of medical code descriptions (100% vs 25% baseline)
-- **Multi-scale Encoders**: Evaluation across 15M to 600M parameter language models
-- **Domain Comparison**: Clinical vs general-domain pretrained encoders
-
-### 3. Experimental Framework
-- **Controlled Comparisons**: Systematic variation along mapping coverage, training approach, encoder scale, and domain axes
-- **Statistical Rigor**: Paired Wilcoxon tests with Bonferroni correction across 10 random seeds
-- **Reproducible Pipeline**: Standardized MEDS-Torch framework with transformer encoders
+| Component            | Finding                                                                              | Implication                                                                    |
+| -------------------- | ------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------ |
+| **Time Features**    | Explicit time encodings showed no statistically significant benefit.                 | Sequence order in transformers may be sufficient for the tasks studied.        |
+| **Value Features**   | Importance is task-dependent (critical for mortality, less so for readmission).      | Code sequences alone can carry significant predictive signal for some tasks.   |
+| **Frozen Encoders**  | Dramatically outperform trainable encoders with far fewer parameters.                | Pretrained knowledge acts as a powerful, regularized feature extractor.        |
+| **Code Information** | Emerges as the most critical predictive signal across all experiments.               | The quality of code representations is paramount for model performance.        |
 
 ## Repository Structure
 
@@ -47,24 +36,13 @@ This work presents a systematic evaluation of tokenization approaches for clinic
 - `experiment_no_value.sh` - No-value ablation experiments
 - `experiment_flexible_textcode.sh` - TextCode optimization experiments
 
-## Key Findings
-
-1. **Time Features**: Explicit time encodings provide no consistent statistically significant benefit across clinical tasks
-2. **Value Features**: Show task-dependent importance, affecting mortality but not readmission prediction
-3. **Frozen Encoders**: Dramatically outperform trainable counterparts while requiring fewer parameters
-4. **Code Information**: Emerges as the most critical predictive signal in clinical time series
-
 ## Dataset and Framework
 
 - **Dataset**: MIMIC-IV processed into MEDS format
 - **Tasks**: In-hospital mortality, ICU mortality, post-discharge mortality, 30-day readmission
 - **Framework**: MEDS-Torch with transformer encoders
 - **Evaluation**: AUROC with 10 random seeds, statistical significance testing
 
-## Reproducibility
-
-All experiments use the standardized MEDS-Torch pipeline with consistent hyperparameters and evaluation protocols. The code preserves the original research branches as separate implementations to enable direct comparison of different tokenization approaches.
-
 ---
 
 *This research demonstrates that simpler, more parameter-efficient tokenization approaches can achieve competitive performance in clinical time series modeling, challenging assumptions about the necessity of complex temporal encodings while clarifying the task-dependent role of value features.*