|
Quoc Le
@quocleix
|
12. stu |
|
Want to improve accuracy and robustness of your model? Use unlabeled data!
Our new work uses self-training on unlabeled data to achieve 87.4% top-1 on ImageNet, 1% better than SOTA. Huge gains are seen on harder benchmarks (ImageNet-A, C and P).
Link: arxiv.org/abs/1911.04252 pic.twitter.com/0umSnX7wui
|
||
|
||
|
|
Quoc Le
@quocleix
|
12. stu |
|
Example predictions on robustness benchmarks ImageNet-A, C and P. Black texts are correct predictions made by our model and red texts are incorrect predictions by our baseline model. pic.twitter.com/eem6tlfyPX
|
||
|
|
||
|
|
Quoc Le
@quocleix
|
12. stu |
|
Full comparison against state-of-the-art on ImageNet. Noisy Student is our method. Noisy Student + EfficientNet is 11% better than your favorite ResNet-50 😉 pic.twitter.com/BhwgJvSOYK
|
||
|
|
||
|
|
Quoc Le
@quocleix
|
13. stu |
|
I also highly recommend this nice video that explains the paper very well:
youtube.com/watch?v=Y8YaU9…
|
||
|
|
||
|
Rishabh Agarwal
@rishabh_467
|
12. stu |
|
I feel that Q-learning in RL does something similar (predicting a guess from a guess) although we don't have any labels to start with. It would be interesting if developments here turn out to be useful for RL.
|
||
|
|
||
|
Dhruv Shah
@_prieuredesion
|
13. stu |
|
That's an interesting point! My first thought reading this was the classic DYNA-style training routine (dl.acm.org/citation.cfm?i…)
Really excited to see ideas re-spawning! :D
|
||
|
|
||
|
Abhishek Srivastav
@abhisri14
|
12. stu |
|
Any quick intuition on what is the source of new information that the model is picking up?
|
||
|
|
||
|
Qizhe Xie
@QizheXie
|
13. stu |
|
Adding noise to the student model is important so that the student can outperform the teacher.
|
||
|
|
||
|
彡Sαι彡
@saikrishnaklu
|
13. stu |
|
|
||
|
olti
@battle8500
|
13. stu |
|
Whereas people usually tend to make look difficult what is easy, you say it's easy. 👏👏👏
|
||
|
|
||
|
Cinjon Resnick
@cinjoncin
|
13. stu |
|
Is this equivalent to ensembling?
|
||
|
|
||
|
Mohamed Hamed
@hamed_mo7amed
|
13. stu |
|
Simple and effective, great job :)
|
||
|
|
||
|
Andrew Davison
@AjdDavison
|
13. stu |
|
|
||
|
Shikun Liu
@liu_shikun
|
13. stu |
|
Thanks Andy. It would be quite useful to evaluate my Adobe project on a new/ much difficult Imagenet test set. But tbh, I read this abstract and felt that...the power of making this level of compute could already feed a small country in Africa for 3 months...
|
||
|
|
||