On 14 July, the IEEE/CVF International Conference on Computer Vision announced the acceptance results and the research group of Prof. Fan Zhang from our institute had a paper selected. This paper has made innovative contributions to the field of long-tailed image recognition.
Paper Information: Qihao Zhao, Chen Jian, Wei Hu, Fan Zhang, Jun Liu: “MDCS: More Diverse Experts with Consistency Self-distillation for Long-tailed Recognition (IEEE/CVF International Conference on Computer Vision, 2023) ”
Abstract: Recently, multi-expert methods have led to significant improvements in long-tail recognition (LTR). We summarize two aspects that need further enhancement to contribute to LTR boosting: (1) More diverse experts; (2) Lower model variance. However, the previous methods didn't handle them well. To this end, we propose More Diverse experts with Consistency Self-distillation (MDCS) to bridge the gap left by earlier methods. Our MDCS approach consists of two core components: Diversity Loss (DL) and Consistency Self-distillation (CS). In detail, DL promotes diversity among experts by managing their focus on different categories. To reduce the model variance, we employ KL divergence to distill the stable and richer knowledge of weakly augmented instances for the experts' self-distillation. In particular, we design Confident Instance Sampling (CIS) to select the correctly classified instances for CS to avoid biased knowledge. In the analysis and ablation study, we demonstrate that our method compared with previous work can effectively increase the diversity of experts, significantly reduce the variance of the model, and improve recognition accuracy. Moreover, the roles of our DL and CS are mutually reinforcing and coupled: the diversity of experts benefits from the CS, and the CS cannot achieve remarkable results without the DL. Experiments show our MDCS outperforms the state-of-the-art by 1% ~ 2% on five popular long-tailed benchmarks, including CIFAR10-LT, CIFAR100-LT, ImageNet-LT, Places-LT, and iNaturalist 2018. The code is available at: https://github.com/fistyee/MDCS
International Conference on Computer Vision (ICCV), organised by IEEE, is held every two years and is one of the world's leading academic conferences in the field of computer science, and is rated as the top tier academic conference by the China Computer Federation (CCF A). ICCV is ranked 17th among all academic journals/conferences by Google Scholar's conference impact rankings of academic journals/conferences published in 2023.
The first author of the paper is Qihao Zhao, a Ph.D. student in the School of Information Science and Technology, and the second author is Chen Jiang, a senior undergraduate student, under the supervision of Prof. Fan Zhang and Associate Prof. Wei Hu, and in collaboration with the Singapore University of Technology and Design (SUTD), with the Beijing University of Chemical Technology (BUCT) as the first completing institution. This work was supported by the National Natural Science Foundation of China and other projects.