Abstract: Knowledge distillation conducts an effective model compression method while holding some limitations: (1) the feature based distillation methods only focus on distilling the feature map but ...