|
@quocleix | |||||
|
Pretrained checkpoints in Pytorch: github.com/rwightman/gen-…
h/t to @wightmanr
|
||||||
|
||||||
|
|
Quoc Le
@quocleix
|
25. stu |
|
AdvProp: One weird trick to use adversarial examples to reduce overfitting.
Key idea is to use two BatchNorms, one for normal examples and another one for adversarial examples.
Significant gains on ImageNet and other test sets. twitter.com/tanmingxing/st…
|
||
|
||
|
|
Quoc Le
@quocleix
|
25. stu |
|
Many of us tried to use adversarial examples as data augmentation and observed a drop in accuracy. And it seems that simply using two BatchNorms overcomes this mysterious drop in accuracy.
|
||
|
|
||
|
|
Quoc Le
@quocleix
|
25. stu |
|
As a data augmentation method, adversarial examples are more general than other image processing techniques. So I expect AdvProp to be useful everywhere (language, structured data etc.), not just image recognition.
|
||
|
|
||
|
|
Quoc Le
@quocleix
|
26. stu |
|
AdvProp improves accuracy for a wide range of image models, from small to large. But the improvement seems bigger when the model is larger. pic.twitter.com/13scFaoQzB
|
||
|
|
||
|
Ross Wightman
@wightmanr
|
26. stu |
|
I'm not a Google team, just one dude who likes PyTorch and enjoys making good models available to other PyTorch users. So, thanks to @quocleix and all team members for making lots of good models available.
|
||
|
|
||