Kun Zhou (周昆)

Kun Zhou is a final year Ph.D student at School of Information, Renmin University of China, under the supervision of Wayne Xin Zhao and Ji-Rong Wen. His research interests are in natural language processing (NLP), with focuses on large language model (LLM) and their applications in complex scenarios. He obtained the master degree from Peking University, and bachelor from Hohai University.

News

I am seeking for Postdoc Position about Large Language Models.

Our four papers are accepted by EMNLP 2023. Thanks for my comrades! See you in Singapore!

Our one paper is accepted by NeurIPS 2023 dataset Track. Thanks for my comrades!

Research Interests

Currently, my primary research lies in large language models (LLMs). I am devoting to answer the following research problems:

  • How to Build Stronger General Large Language Models?
    • Background: Crafting A Comprehensive Survey about LLMs: Arxiv
    • Training: Improving LLMs via High-quality Instructions: YuLan-Chat
    • What is stronger? Studying Benchmark Leakage in LLM Evaluation Fairness: Arxiv
  • How to Create Effective and Efficient LLM-based Agents for Complex Scenarios?
    • Tool-Augmented Math Agent: Tool-Augmented Reasoning Agent EMNLP 2023, Iterative Deliberation for Error Check NeurIPS 2023
    • Reasoning Agent over Structured Data: Toolization with Prompting Method EMNLP 2023
    • Multimodal LLM-based Agent: Visual Instruction Tuning Arxiv, Evaluation of Object Hallucination EMNLP 2023

Before the era of large language models, I have investigated pre-trained language models (PLMs), and their application on sentence representation learning and complex reasoning.

  • Effective Fine-tuning PLMs
    • General Fine-tuning Strategy: Visual-Augmentation ACL 2023, Over-Parameterization ACL 2023
  • PLM-based Sentence Representations
    • Unsupervised Sentence Representations: Debiased Contrastive Learning ACL 2022
    • Dense Text Retrieval: Ambiguous Negatives Sampling EMNLP 2022, Multi-task Bottlenecked MAE ECML-PKDD 2023
  • PLM-based Complex Reasoning
    • Math-specific PLMs: Pre-training with Memory Network ACL 2022, Curriculum Pre-training KDD 2022, Unified MoE Pre-training KDD 2023
    • PLM+KG for Knowledge Reasoning: Simple Rule-based KG Encoder NAACL 2022, Unified Relation Learning ICLR 2023

I also have done interesting research works about conversational recommender systems, sequential recommendation and dialog system. My whole publication list is in Publication

Open-Source Projects

Most of my research work are open-source. Here are some my preferable projects!

  • YuLan-Chat
    • YuLan-Chat-2 is developed by continually-pretraining and instruction-tuning LLaMA-2 with high-quality English and Chinese data. It can support 8k maximum length now, and outperforms other llama-2-based bilingual LLM on MMLU, C-Eval and AGIEval-Gaokao.
    • YuLan-Chat is a chat-based large language model developed based on fine-tuning LLaMA. YuLan-Chat can chat with users to well follow English or Chinese instructions, and can be deployed on an GPU (A800-80G or RTX3090) after quantization. YuLan (Simplified Chinese 玉兰), is the campus flower of Renmin University of China.
  • LLMSurvey
    • A collection of papers and resources related to Large Language Models. The organization of papers refers to our survey “A Survey of Large Language Models”.

Experiences

2022.4 - 2023.6, Research Intern, NLC Group, MSRA.

Mentor: Yeyun Gong, Nan Duan

2021.9 - 2022.4, Research Intern, iFLYTEK Research.

Mentor: Jing Sha, Shijin Wang

2019.12 - 2021.5, Research Intern, NLP center, Meituan Dianping.

Mentor: Sirui Wang, Fuzheng Zhang

2018.8 - 2019.6, Research Intern, XiaoIce, Microsoft Asia.

Mentor: Kai Zhang, Yu Wu

Rewards and Honors

  • Five Highly-cited Papers are selected as the most influential KDD/WWW/CIKM papers by PaperDigest:
  • 2022 Baidu Scholarship (10 PhD Students) Link
  • 2022 Bytedance Scholarship (10 PhD Students in China Mainland) Link
  • 2022 MSRA Fellowship (12 PhD Students in Asia-Pacific-region) Link
  • 2022 Baosteel Scholarship (12 Students in RUC).
  • 2022 National Scholarship.
  • LIC2021 Multi-Skill Dialog Challenge Link
    • Ranked 1st in Automatic Metrics Track, 3rd in Human Evaluation Track
  • LIC2020 Conversational Recommendation Challenge Link
    • Ranked 2st in Automatic Metrics Track, 4rd in Human Evaluation Track
  • 2018 Jane Street Electronic Trading Challenges
    • Ranked 1st
  • 2018 The Data Open Challenges Citadel Datathen
    • Ranked 2nd
  • 2016 American Mathematical Contest in Modeling
    • Honorable Mention
  • 2015 China Undergraduate Mathematical Contest in Modeling
    • National Second Prize
  • 2015 National Zhou Peiyuan college student mechanics competition
    • National Third Prize
  • 2015 Jiangsu Province Undergraduate Mechanical Competition
    • The First Prize

Service

  • IJCAI 2021
    • Senior PC Reviewer
  • AAAI, IJCAI, KDD, SIGIR, WWW, WSDM, ACL, EMNLP, COLING, TOIS, TORS
    • PC Reviewer

Zhihu Posts

I love writing blogs in my free time. Here are some my favorite posts!

BERT meet Knowledge Graph:预训练模型与知识图谱相结合的研究进展

BERT meet KG第二弹:新训练方式,新问题视角