CV

Education

• B.S. in Physics, University of Science and Technology of China (USTC), Aug. 2010 - Jun. 2014
• M.S. in Physics, University of Science and Technology of China (USTC), Aug. 2014 - Jun. 2017
• Ph.D. in Computer Science, University of Technology Sydney (UTS), Aug. 2017 - Nov. 2021

Work experience

• Feb. 2024-Present: Research Scientist
  • RIEKN AIP
  • Supervisor: Taiji Suzuki
• June. 2022-Jan 2024: Postdoctoral Researcher
  • RIEKN AIP
  • Supervisor: Taiji Suzuki
• Nov. 2022-Feb. 2023: Visiting Researcher
  • Mohamed bin Zayed University of Artificial Intelligence
  • Supervisor: Zhiqiang Xu

Mentees

I am fortunately co-supervising several Ph.D. students:

• Zixu Zhao (Ph.D. student, UNSW)
• Siqing Li (Ph.D. student, UNSW)
• Yilan Chen (Ph.D. student, UCSD)

Publications

Adaptive multi-GPU exchange Monte Carlo for the 3D random field Ising model

Critical percolation clusters in seven dimensions and on a complete graph

Mean field theory for deep dropout networks: digging up gradient backpropagation deeply

On the neural tangent kernel of deep networks with orthogonal initialization

Gaussian process latent variable model factorization for context-aware recommender systems

On the Equivalence between Neural Network and Support Vector Machine

Towards Deepening Graph Neural Networks: A GNTK-based Optimization Perspective

Auto-scaling Vision Transformers without Training

Pruning graph neural networks by evaluating edge properties.

Deep Architecture Connectivity Matters for Its Convergence: A Fine-Grained Analysis.

Interpreting Operation Selection in Differentiable Architecture Search: A Perspective from Influence-Directed Explanations.

Deep Active Learning by Leveraging Training Dynamics.

Weighted Mutual Learning with Diversity-Driven Model Compression.

Analyzing Deep PAC-Bayesian Learning with Neural Tangent Kernel: Convergence, Analytic Generalization Bound, and Efficient Hyperparameter Selection.

No Free Lunch in Neural Architectures? A Joint Analysis of Expressivity, Convergence, and Generalization.

Single-Pass Contrastive Learning Can Work for Both Homophilic and Heterophilic Graph.

Understanding and Improving Feature Learning for Out-of-Distribution Generalization.

Fed-CO2: Cooperation of Online and Offline Models for Severe Data Heterogeneity in Federated Learning.

Analyzing Generalization of Neural Networks through Loss Path Kernels.

Understanding Convergence and Generalization in Federated Learning through Feature Learning Theory.

Graph Lottery Ticket Automated.

Global and Local Prompts Cooperation via Optimal Transport for Federated Learning.

The Heterophilic Snowflake Hypothesis: Training and Empowering GNNs for Heterophilic Graphs.

Provably Neural Active Learning Succeeds via Prioritizing Perplexing Samples.

Diffusion Models Demand Contrastive Guidance for Adversarial Purification to Advance.

Mesa-Optimization in Autoregressively Trained Transformers: Emergence and Capability.

SLTrain: a sparse plus low rank approach for parameter and memory efficient pretraining

Provable and Efficient Dataset Distillation for Kernel Ridge Regression

Federated Learning from Vision-Language Foundation Models: Theoretical Analysis and Method.

Provably Transformers Harness Multi-Concept Word Semantics for Efficient In-Context Learning.

Unveil Benign Overfitting for Transformer in Vision: Training Dynamics, Convergence, and Generalization.

On the Comparison between Multi-modal and Single-modal Contrastive Learning.

Talks

An Introduction to the Neural Tangent Kernel.

July 01, 2020

Talk at Renmin University of China, hosted by A/Prof. Yong Liu, Beijing, China

Understanding deep neural networks through over-parameterization.

June 01, 2021

Talk at Monash University, hosted by Prof. Reza Haffari, Melbourne, Australia

Closing the Gap between Theory and Applications in Deep Learning.

December 01, 2021

Seminar Talk at RIKEN AIP, hosted by A/Prof. Taiji Suzuki, Tokyo, Japan

Towards Deepening Graph Neural Networks: A GNTK-based Optimization Perspective.

June 12, 2022

Invited Talk at AI TIME, Sydney, Australia

Understanding Deep Learning through Over-parameterization: from Kernel Regime to Feature Learning.

February 15, 2023

Invited Talk at The Chinese University of Hong Kong, hosted by Dr. Fenglei Fan, Hong Kong, China

Explainable deep learning: an over-parameterization perspective.

March 23, 2023

Invited Talk at Vector Institute ＆ RIKEN AIP Joint Symposium on Machine Learning and Artificial Intelligence, Nihonbashi, Tokyo

Explainable deep learning: an over-parameterization perspective.

March 26, 2023

Invited Talk at MBZUAI & RIKEN-AIP Joint Workshop, Nihonbashi, Tokyo

Graph neural networks provably benefit from structural information: a feature learning perspective.

July 28, 2023

Contributed Talk at ICML 2023 workshop on High-Dimensional Learning Dynamics, Honolulu, Hawaii

Feature Learning for Out-of-Distribution Generalization

March 05, 2024

Invited Talk at The Machine Learning Summer School in Okinawa 2024, Okinawa Institute of Science and Technology (OIST), Japan

Understanding Convergence and Generalization in Federated Learning through Feature Learning Theory.

May 23, 2024

Invited Talk at HKBU-RIKEN AIP Joint Workshop on Artificial Intelligence (AI) and Machine Learning (ML), Hong Kong

Understanding Convergence and Generalization in Federated Learning through Feature Learning Theory.

May 23, 2024

Invited Talk at The Third RIKEN AIP & A*STAR-CFAR Joint Workshop on Machine Learning and Artificial Intelligence, A*STAR, Singapore

Benign Overfitting of Vision Transformers.

December 20, 2024

Invited Talk at The 14th American Institute of Mathematical Sciences, NYU Abu Dhabi

Teaching

Thermodynamics and Statistical Physics

Statistical Physics

Service and leadership

• Conference reviewer: NeurIPS 2021-2022, ICLR 2022-2023, ICML 2021-2022, KDD 2022, CVPR 2022, IJCAI 2022, SIGIR 2022, AISTATS 2022-2023, AAAI 2022 (Meta Reviewer).
• Journal reviewer: IEEE Transactions on Cybernetics, IEEE TKDE.