Lianyu Hu is a PhD graduating from Tianjin University, China, supervised by Prof Wei Feng. During his graduate student period, he worked closely with Prof Shenglan Liu. His research interest includes Video Understanding, Sign Lnaguage Understanding and Multimodal Learning.

🔥 News

• We release LightVLM, an highly efficient method for large vision language models with a two-stage design. It improves model efficiency by first conducting visual token merging in the encoding stage and then adopt KV Cache compression in the decoding stage. It could achieve about 2× throughput across diffferent benchmarks and 3.21× throughput boost when outputting longer sequences.

• We release iLLaVA, an efficient method for large vision language models by merging visual tokens. It could achieve about 2× throughput and 1.7× - 2× memory reduction with comparable performance through merging redundant visual tokens in some certain layers.

• We release CorrNet+, an unified model with superior performance on both continuous sign language recognition and sign language translation tasks by using only RGB inputs.

• We release DSTA-SLR, which performs sign language recognition (SLR) with pure skeleton inputs but ahcieves comparable accuracy and much faster speed than recognition with RGB inputs.

📝 Publications

📖 PrePrint

• LightVLM: Acceleraing Large Multimodal Models with Pyramid Token Merging and KV Cache Compression, Lianyu Hu, Fanhua Shang, Liang Wan, Wei Feng. 2025.09.

• iLLaVA: An Image is Worth Fewer Than 1/3 Input Tokens in Large Multimodal Models, Lianyu Hu, Fanhua Shang, Liang Wan, Wei Feng. 2024.12. [code].

• CorrNet+: Sign Language Recognition and Translation via Spatial-Temporal Correlation, Lianyu Hu, Wei Feng, Liqing Gao, Zekang Liu, Liang Wan. 2024.04. [code].

• Improving Continuous Sign Language Recognition with Adapted Image Models, Lianyu Hu, Tongkai Shi, Liqing Gao, Zekang Liu, Wei Feng. 2024.04. [code].

🖊️ Selected Publications ($\dagger$ denotes Correspding Author)

• GReg: Geometry-Aware Region Refinement for Sign Language Video Generation. Tongkai Shi, Lianyu Hu^$\dagger$, Fanhua Shang, Liqing Gao, Wei Feng. ICCV 2025.

• Deep Correlated Prompting for Visual Recognition with Missing Modalities. Lianyu Hu, Tongkai Shi, Wei Feng, Fanhua Shang, Liang Wan. NeurIPS 2024. [code].

• Pose-Guided Fine-Grained Sign Language Video Generation. Tongkai Shi, Lianyu Hu, Fanhua Shang, Jichao Feng, Peidong Liu, Wei Feng. ECCV 2024. [code].

• Spatial Temporal Aggregation for Efficient Continuous Sign Language Recognition. Lianyu Hu, Liqing Gao, Zekang Liu, Wei Feng. IEEE Transactions on Emerging Topics in Computational Intelligence.

• Dynamic Spatial-Temporal Aggregation for Skeleton-Aware Sign Language Recognition. Lianyu Hu, Liqing Gao, Zekang Liu, Wei Feng. COLING 2024. [code].

• COMMA: Co-Articulated Multi-Modal Learning. Lianyu Hu, Liqing Gao, Zekang Liu, Chi-Man Pun, Wei Feng. AAAI 2024. [code].

• Scalable Frame Resolution for Efficient Continuous Sign Language Recognition. Lianyu Hu, Liqing Gao, Zekang Liu, Wei Feng. Pattern Recognition.

• AdaBrowse: Adaptive Video Browser for Efficient Continuous Sign Language Recognition. Lianyu Hu, Liqing Gao, Zekang Liu, Chi-Man Pun, Wei Feng. ACMMM 2023 (Oral). [code].

• Skeleton-Based Action Recognition with Local Dynamic Spatial-Temporal Aggregation. Lianyu Hu, Shenglan Liu, Wei Feng. Expert Systems with Applications. [code]. (Previous name: Spatial Temporal Graph Attention Network for Skeleton-Based Action Recognition)

• Continuous Sign Language Recognition with Correlation Network. Lianyu Hu, Liqing Gao, Zekang Liu, Wei Feng. CVPR 2023. [code].

• Self-Emphasizing Network for Continuous Sign Language Recognition. Lianyu Hu, Liqing Gao, Zekang Liu, Wei Feng. AAAI 2023 (Oral). [code].

• Temporal Lift Pooling for Continuous Sign Language Recognition.Lianyu Hu, Liqing Gao, Zekang Liu, Wei Feng. ECCV 2022. [code].

• HFNet: A Novel Model for Human Focused Sports Action Recognition.Lianyu Hu, Liqing Gao, Zekang Liu, Wei Feng. ACMMM 2020 Workshop.

🎖 Honors and Awards

• 2025.06, Outstanding Graduate
• 2024.12, 优秀学生标兵（ten per year）
• 2024.10, National Scholarship
• 2023.10, National Scholarship

📖 Educations

• 2021-2025, PhD in Computer Science and Technology, Tianjin Univerisity
• 2018-2021, MEng in Computer Science and Technology, Dalian University of Technology
• 2014-2018, BSc in Electronics and Information Engineering, Dalian University of Technology