Pretraining EHR Foundation Models with Patient-Aware Sampling

arXiv cs.LG5d4 min read

arXiv:2607.22114v1 Announce Type: new Abstract: Autoregressive foundation models for electronic health records (EHRs) typically inherit pretraining methods from language modeling, where patient trajectories are concatenated into a single token stream and windows are sampled from that stream. In EHR data, this choice is consequential: windows may mix multiple patients, and patients with longer records contribute more optimization updates, potentially introducing bias. We propose Patient Sampling, a pretraining sequence-construction method that allows us to control how training signal is distrib

Read the full story at arXiv cs.LG →

Pretraining EHR Foundation Models with Patient-Aware Sampling

More in Machine Learning

Reducing Human Annotation with ML Active Learning

Learning What Matters: Supervising Sparse Attention Routing with Causal Evidence Sets

On the Depth Scalability of Logic Gate Networks

From Perturbation Correction to Geometry-Aware Sampling: Sharpness-Guided Equilibrium Sampling for Balanced Flat Minima in Long-Tailed Learning

Parameter-free Adaptive Sparse Attention via Compression-Based Content Selection

Cross-Domain Off-Policy Evaluation and Learning for Contextual Bandits