Gül Sena Altıntaş

Hello, I am Gül Sena (/ɡuːl se’na/) ! I am a third (oh, wow) year Computer Science PhD student at UofT working with Colin Raffel. I am currently a visiting researcher at Antoine Bosselut’s NLP group. My current research interests lie in the intersection of multilinguality and modularity in language models. I am particularly interested in how we can move away from one-big-size-fits-all architectures toward systems that adapt efficiently across diverse languages and domains. These days I am thinking a lot about tokenization and input representations in language models; how they relate to multilingual, cross-domain and cross-modal abilities. I am also increasingly thinking about base model evaluations. Feel free to reach out if any of this resonates!

Even though I am no longer working in this area, I am also fascinated by the literature on linear mode-connectivity and loss landscapes especially their implications for learning theory, decentralization and modularity.

Prior to my PhD, I completed my masters at ETH Zürich and my bachelors at Koç University. I enjoy teaching. In my free time I like rowing, running, and cooking.

news

Sep 01, 2026	I will be spending the Fall semester at the NLP Lab in EPFL.
May 30, 2026	Our paper TokSuite: Measuring the Impact of Tokenizer Choice on Language Model Behavior is accepted to ICML 2026 as an oral paper, you can access the recording of our presentation here. Check toksuite/leaderboard for how models behave under our benchmark.
Dec 30, 2025	Our preprint TokSuite: Measuring the Impact of Tokenizer Choice on Language Model Behavior is on ArXiV.

selected publications

TokSuite: Measuring the Impact of Tokenizer Choice on Language Model Behavior

Gül Sena Altıntaş, Malikeh Ehghaghi, Brian Lester, and 4 more authors

In Proceedings of the 43rd International Conference on Machine Learning (ICML), 2026

Abs Bib PDF Video Code

Tokenizers provide the fundamental basis through which text is represented and processed by language models (LMs). Despite the importance of tokenization, its role in LM performance and behavior is poorly understood due to the challenge of measuring the impact of tokenization in isolation. To address this need, we present TokSuite, a collection of models and a benchmark that supports research into tokenization’s influence on LMs. Specifically, we train fourteen models that use different tokenizers but are otherwise identical using the same architecture, dataset, training budget, and initialization. Additionally, we curate and release a new benchmark that specifically measures model performance subject to real-world perturbations that are likely to influence tokenization. Together, TokSuite allows robust decoupling of the influence of a model’s tokenizer, supporting a series of novel findings that elucidate the respective benefits and shortcomings of a wide range of popular tokenizers.
@inproceedings{altintas2026toksuite, author = {Alt{\i}nta\c{s}, G{\"u}l Sena and Ehghaghi, Malikeh and Lester, Brian and Liu, Fengyuan and Zhao, Wanru and Ciccone, Marco and Raffel, Colin}, title = {{TokSuite}: Measuring the Impact of Tokenizer Choice on Language Model Behavior}, booktitle = {Proceedings of the 43rd International Conference on Machine Learning (ICML)}, year = {2026}, eprint = {2512.20757}, archiveprefix = {arXiv}, url = {https://arxiv.org/abs/2512.20757}, }
The Butterfly Effect: Neural Network Training Trajectories Are Highly Sensitive to Initial Conditions

Gül Sena Altıntaş, Devin Kwok, Colin Raffel, and 1 more author

In Forty-second International Conference on Machine Learning, 2025

Abs Bib Open Review PDF Code

Neural network training is inherently sensitive to initialization and the randomness induced by stochastic gradient descent. However, it is unclear to what extent such effects lead to meaningfully different networks, either in terms of the models’ weights or the underlying functions that were learned. In this work, we show that during the initial “chaotic” phase of training, even extremely small perturbations reliably causes otherwise identical training trajectories to diverge—an effect that diminishes rapidly over training time. We quantify this divergence through (i) L^2 distance between parameters, (ii) the loss barrier when interpolating between networks, (iii) L^2 and barrier between parameters after permutation alignment, and (iv) representational similarity between intermediate activations; revealing how perturbations across different hyperparameter or fine-tuning settings drive training trajectories toward distinct loss minima. Our findings provide insights into neural network training stability, with practical implications for fine-tuning, model merging, and diversity of model ensembles.
@inproceedings{altintas2025butterfly, title = {The Butterfly Effect: Neural Network Training Trajectories Are Highly Sensitive to Initial Conditions}, author = {Alt{\i}nta{\c{s}}, G{\"u}l Sena and Kwok, Devin and Raffel, Colin and Rolnick, David}, booktitle = {Forty-second International Conference on Machine Learning}, year = {2025}, url = {https://openreview.net/forum?id=L1Bm396P0X}, open_review = {https://openreview.net/forum?id=L1Bm396P0X}, }