Shypd.aiThe largest open index of AI toolsTeacher-free-Knowledge-Distillation is an academic research tool that implements knowledge distillation via label smoothing regularization. It allows for improving model performance without requiring a separate teacher model.
Trending
About
Teacher-free-Knowledge-Distillation provides an implementation for a novel approach to knowledge distillation, as detailed in a CVPR2020 Oral paper. This method, titled "Revisiting Knowledge Distillation via Label Smoothing Regularization," enables significant improvements in model accuracy without the need for a stronger teacher model or extensive computational resources. The framework supports "self-training" and "manually-designed regularization" strategies. For instance, it can enhance powerful student models like ResNeXt101-32x8d by 0.48% on ImageNet or ResNeXt29 by over 1.0% on CIFAR100. The repository includes code for environment setup, dataset handling (CIFAR100, CIFAR10, Tiny_ImageNet), training baseline models, and conducting exploratory experiments like Reversed KD and Defective KD, alongside the core Teacher-free KD implementations.
Capabilities
Pricing & Plans
Open Source
Free
FAQs
Trending
Also listed in
