Lianyu Hu is a 3rd-year PhD candidate in Tianjin University, China, supervised by Prof Wei Feng. During his graduate student period, he worked closely with Prof Shenglan Liu. His research interest includes Video Understanding, Sign Lnaguage Understanding and Multimodal Learning.

🔥 News

• We release CorrNet+, an unified model with superior performance on both continuous sign language recognition and sign language translation tasks by using only RGB inputs.

• We release AdaptSign, a continuous sign language recognition (CSLR) model powered by frozen pretrained image models with 18.5% & 18.8% WER on phoenix2014, 18.6% & 19.8% WER on phoenix2014-T, 26.7% & 26.3% WER on CSL-Daily.

• We release DSTA-SLR, which performs sign language recognition (SLR) with pure skeleton inputs but ahcieves comparable accuracy and much faster speed than recognition with RGB inputs.

📝 Publications

📖 PrePrint

• CorrNet+: Sign Language Recognition and Translation via Spatial-Temporal Correlation, Lianyu Hu, Wei Feng, Liqing Gao, Zekang Liu, Liang Wan. 2024.04. [code].

• Improving Continuous Sign Language Recognition with Adapted Image Models, Lianyu Hu, Tongkai Shi, Liqing Gao, Zekang Liu, Wei Feng. 2024.04. [code].

🖊️ Selected Publications

• Spatial Temporal Aggregation for Efficient Continuous Sign Language Recognition. Lianyu Hu, Liqing Gao, Zekang Liu, Wei Feng. IEEE Transactions on Emerging Topics in Computational Intelligence.

• Dynamic Spatial-Temporal Aggregation for Skeleton-Aware Sign Language Recognition. Lianyu Hu, Liqing Gao, Zekang Liu, Wei Feng. COLING 2024. [code].

• COMMA: Co-Articulated Multi-Modal Learning. Lianyu Hu, Liqing Gao, Zekang Liu, Chi-Man Pun, Wei Feng. AAAI 2024. [code].

• Scalable Frame Resolution for Efficient Continuous Sign Language Recognition. Lianyu Hu, Liqing Gao, Zekang Liu, Wei Feng. Pattern Recognition.

• AdaBrowse: Adaptive Video Browser for Efficient Continuous Sign Language Recognition. Lianyu Hu, Liqing Gao, Zekang Liu, Chi-Man Pun, Wei Feng. ACMMM 2023 (Oral). [code].

• Skeleton-Based Action Recognition with Local Dynamic Spatial-Temporal Aggregation. Lianyu Hu, Shenglan Liu, Wei Feng. Expert Systems with Applications. [code]. (Previous name: Spatial Temporal Graph Attention Network for Skeleton-Based Action Recognition)

• Continuous Sign Language Recognition with Correlation Network. Lianyu Hu, Liqing Gao, Zekang Liu, Wei Feng. CVPR 2023. [code].

• Self-Emphasizing Network for Continuous Sign Language Recognition. Lianyu Hu, Liqing Gao, Zekang Liu, Wei Feng. AAAI 2023 (Oral). [code].

• Temporal Lift Pooling for Continuous Sign Language Recognition.Lianyu Hu, Liqing Gao, Zekang Liu, Wei Feng. ECCV 2022. [code].

• HFNet: A Novel Model for Human Focused Sports Action Recognition.Lianyu Hu, Liqing Gao, Zekang Liu, Wei Feng. ACMMM 2020 Workshop.

🎖 Honors and Awards

• 2023.10, National Scholarship

📖 Educations

• 2021-now, PhD candidate in Computer Science and Technology, Tianjin Univerisity
• 2018-2021, MEng in Computer Science and Technology, Dalian University of Technology
• 2014-2018, BSc in Electronics and Information Engineering, Dalian University of Technology