Hi, this is Songtao. I'm currently a Ph.D. candidate at Penn State advised by Prof. Peng Liu. I received my B.S. in Computer Science and Data Science from Fudan University.
My research focuses on efficient architectures and expressive algorithms for language modeling.
I am on the job market!
Selected Publications
* denotes equal contribution (first, second, or senior)
-
Multi-Head Low-Rank Attention
Songtao Liu, Hongwu Peng, Zhiwei Zhang, Zhengyu Chen, Yue Guo
International Conference on Learning Representations (ICLR), 2026
[arXiv] [Blog] [Code] [Weights&Data]
-
High-Layer Attention Pruning with Rescaling
Songtao Liu, Peng Liu
Transactions on Machine Learning Research (TMLR), 2026
[arXiv] [Code]
-
Preference Optimization for Molecule Synthesis with Conditional Residual Energy-based Models
Songtao Liu, Hanjun Dai, Yue Zhao, Peng Liu
International Conference on Machine Learning (ICML), 2024
Oral Presentation
[arXiv] [Code]
-
FusionRetro: Molecule Representation Fusion via In-Context Learning for Retrosynthetic Planning
Songtao Liu, Zhengkai Tu*, Minkai Xu*, Zuobai Zhang*, Lu Lin, Rex Ying, Jian Tang, Peilin Zhao, Dinghao Wu
International Conference on Machine Learning (ICML), 2023
[arXiv] [Code]
Professional Services
Conference/Journal Reviewer: ICML 2022/2024/2025, NeurIPS 2022/2023/2024/2025, ICLR 2024/2025/2026, AISTATS 2024, TMLR 2024