Yiqi "Viscent" Zhang. (张懿麒)
A non-typical geek.
From neurons to neural networks.
From microscopes to supercomputers.
The future awaits.
Hi, my name is
About Me
I am a PhD candidate at HPC-AI Lab in National University of Singapore, currently focusing in high-performance machine learning systems.
Meanwhile, I have 4 years of experience in Neuroscience. Details are available in my CV.
My expertise include but not limited to below:- High Performance Computing
- Machine Learning Systems
- Heterogeneous Computing
- Neuroscience
- Medical Imaging
Experience
My research here mainly to propose a novel RL training framework that redefines the post-training process of LLMs. We endeavor to make the post-training scripting natural and easy to use for algorithm designer, without fuzzing with parallelisms and other low-level details. Meanwhile, our revolutionary infrastructure design enables the maximum cluster-level utilization among heterogeneous requests.
My research here focused on the optimizing the entire RL training workflow, mainly focusing on enhancing the training efficiency from both infrastructure and algorithm perspectives. By batching-centric optimization, we eliminated ~70% bubble in rollout computation and reduced the total training steps over 50%.
HPC-AI Tech is a cutting-edge lab that integrates high performance computing seamlessly with deep learning. Topics of research interests encompass Machine Learning, High Performance Computing, Machine Learning Systems, Parallel and Distributed Systems, and AI Applications (e.g. CV, NLP, Biology).
Institute of Psychiatry, Psychology & Neuroscience is a world-leading research group aiming at decoding the brain. We proposed and implemented a Bayesian U-Net for MRI super resolution, which yields high resolution images as well as a reference-free metric indicating model-data fitness. Through comprehensive testing, we showed that the model successfully performed SR in a variety of data including young adult brains, neonatal brains, and glioblastoma-affected brains, in which the B-UNet exhibited state-of-the-art performance. Based on the reference-free metric, we also designed an automated Bayesian model selector to ensure the optimal output quality without referencing to a ground truth image. By virtue of strong connection with the industry, this work is expected to largely facilitate the diagnosis in areas with poor access to high-field scanners.
The lab mainly focuses on ischemic stroke and Alzheimer’s disease, their mechanism and treatment using MCAO and APP/PS1 mouse model. During the research, I conducted various types of experiments, including electrophysiological data acquisition and analysis (LFP/EEG), immunohistochemistry assay, animal behavioural experiments and cell experiments. The skills and experience gained from both physiological and molecular experiments make me ready for a variety of challenges in wet lab. Apart routine biological works in wet lab, I also developed various stimulation delivery devices from whisker stimulator to rhythmic light flicker with synchronization capability to psychology software.
The project I led, 40 Hz light flicker alters human brain EEG microstates and complexity with implications in brain diseases (Yiqi Zhang et al., 2021), is published on Frontiers in Neuroscience. We investigated the EEG features including microstates and connectivity on 20 healthy young adults exposed to 40 Hz flicker and reported unprecedented findings on the EEG microstate complexity.
Education
2018 - 2022
BSc in Biological Science
Southern University of Science and Technology
Grade:
Summa Cumme Laude
Publications
SortedRL: Accelerating RL Training for LLMs through Online Length-aware Scheduling
We propose a RL rollout scheme that leverages the length distribution of the training data to dynamically gather trajectories for efficient on-policy training. By constructing micro-curriculum on length basis, we cut the training steps over 50% and eliminated ~70% bubbles in rollout computation.
SpeedLoader: An I/O efficient scheme for heterogeneous and distributed LLM operation
The 38th Annual Conference on Neural Information Processing Systems
In this work, we redesign the data flow of heterogeneous hardware and sharded model training to minimize the excessive communication overhead. Our proposed scheme significantly enhances training and inference throughput of large language models under restrictive computational resources.
AADG: Automatic Augmentation for Domain Generalization on Retinal Images
IEEE Transactions on Medical Imaging
Our AADG framework can effectively sample data augmentation policies that generate novel domains and diversify the training set from an appropriate search space. Specifically, we introduce a novel proxy task maximizing the diversity among multiple augmented novel domains as measured by the Sinkhorn distance in a unit sphere space, making automated augmentation tractable.
40 Hz light flicker alters human brain EEG microstates and complexity with implications in brain diseases
Frontiers in Neuroscience
Previous studies showed that entrainment of light flicker at low gamma frequencies provided neuroprotection in mouse models of Alzheimer’s disease (AD) and stroke. We studied healthy adult's EEG response to 40 Hz Flicker.
Achievements
Overall Champion of 4th APAC HPC-AI Competition
HPC-AI Advisory Council
AI Special Prize of 4th APAC HPC-AI Competition
HPC-AI Advisory Council
First prize winner of ASC20-21
ASC20-21 Student Supercomputer Challenge (ASC committee)
Highest Linpack Benchmark Winner
SC21 Virtual Student Cluster Competition (ACM SIGHPC/IEEE)
Accelerated Computing C++/Python
Certificated by NVIDIA DLI
Third Place Winner of ISC22
ISC22 Student Cluster Competition (HPC-AI Advisory Council)
President's Graduate Fellowship
National University of Singapore
Get In Touch
My inbox is always open. Whether you have a question or just want to say hi, I’ll try my best to get back to you!