What doesn’t kill you makes you stronger.

I am Weiyao Luo (罗巍耀), a master student at the Institute of Computational Linguistics in the School of SS, Peking University, advised by Dr. Zhifang Sui and Dr. Sujian Li. Prior to this, I earned my Bachelor’s degree at Northeastern University advised by Dr. Shi Feng.

My research interests mainly lie in natural language processing, especially 1) Large-scale Pre-training Language Model, and 2) Multimodality

I am actively seeking for engineer or researcher position (graduate in 2025). Please drop me an email (A@B,A=luoweiyao993,B=gmail.com) if you want to chat.

Education

Peking University, Sep. 2022 - Jun. 2025 (expected)

Northeastern University, Sep. 2018 - Jun. 2022

Experience

Tencent WXG, Channels Algorithm（视频号）, June. 2024 – Now

  • Intern.
  • Topic: LLM For Recommendation System.

MiHoYo AI, March. 2024 – June. 2024

  • LLM Intern.
  • Topic: Role-play Large Language Model.

Tencent TEG, Hunyuan LLM, Sep. 2023 – March. 2024

  • Research Intern advised by Suncong Zheng and Di Wang.
  • Topic: Large Language Model.

Microsoft Research Asia, NLC Group, Dec. 2022 – Aug. 2023

  • Research Intern advised by Tengchao Lv, and Furu Wei.
  • Topic: Large Language Model, DocumentAI.

Publications and Manuscripts

* means equal contribution.

FaGANet: A Fact-Checking Model with Integrated Encoder Leveraging Graph Attention Network and Contextual Information. [PDF]
Weiyao Luo, Junfeng Ran, Zailong Tian, Sujian Li, Zhifang Sui.
COLING2024(CCF-B).

PeriodicLoRA: Breaking the Low-Rank Bottleneck in LoRA Optimization. [PDF]
Xiangdi Meng, Damai Dai, Weiyao Luo, Zhe Yang, Shaoxiang Wu, Xiaochen Wang, Peiyi Wang, Qingxiu Dong, Liang Chen, Zhifang Sui.
Preprint.

Taking a Deep Breath: Enhancing Language Modeling of Large Language Models with Sentinel Tokens. [PDF]
Weiyao Luo, Suncong Zheng, Heming Xia, Weikang Wang, Yan Lei, Tianyu Liu, Shuang Chen, Zhifang Sui
EMNLP2024(CCF-A).

Leveraging Two-Stream Cause-Effect Relation for Emotion-Cause Analysis
Junfeng Ran, Zailong Tian, Weiyao Luo, Sujian Li
NLPCC2024(CCF-C).

Kosmos-2.5: A Multimodal Literate Model. [PDF]
Tengchao Lv, Yupan Huang, Jingye Chen, Lei Cui, Shuming Ma, Yaoyao Chang, Shaohan Huang, Wenhui Wang, Li Dong, Weiyao Luo, Shaoxiang Wu, Guoxin Wang, Cha Zhang, Furu Wei.
Preprint.

BiSAGA: A Novel Bidirectional Sparse Graph Attention Adapter for Evidence-Based Fact-Checking
Junfeng Ran, Weiyao Luo, Guangxiang Zhao, Dawei Zhu, Zailong Tian, Longyun Wu, Hailiang Huang, Zhifang Sui, Sujian Li
Under review.

Hornors & Awards

  • Merit Student of Peking University. 2023.
  • National scholarship. 2021.
  • Outstanding Graduates in Northeastern University. 2022
  • A-class scholarship in Northeastern University. 2021.
  • B-class scholarship in Northeasterng University. 2020, 2019.
  • Meritorious Winner in Interdisciplinary Contest In Modeling (top 8% of 10053 teams). 2020

Contact Me

  • Email: luoweiyao993 AT gmail DOT com
  • Links: GitHub