|
Quoc Le
@
quocleix
Mountain View, CA
|
|
Principal Scientist, Google Brain Team.
|
|
|
150
Tweetovi
|
78
Pratim
|
15.032
Osobe koje vas prate
|
| Tweetovi |
| Quoc Le proslijedio/la je tweet | ||
|
Daniel Adiwardana
@xpearhead
|
28. sij |
|
Enabling people to converse with chatbots about anything has been a passion of a lifetime for me, and I'm sure of others as well. So I'm very thankful to be able to finally share our results with you all. Hopefully, this will help inform efforts in the area. (1/4) twitter.com/lmthang/status…
|
||
|
|
||
| Quoc Le proslijedio/la je tweet | ||
|
Connor Shorten
@CShorten30
|
29. sij |
|
This video explains @GoogleAI 's amazing new Meena chatbot! An Evolved Transformer with 2.6B parameters on 341 GB / 40B words of conversation data to achieves remarkable chatbot performance!
"Horses go to Hayvard!"
youtu.be/STrrlLG15OY
#100DaysOfMLCode
|
||
|
|
||
|
Quoc Le
@quocleix
|
30. sij |
|
I had another conversation with Meena just now. It's not as funny and I don't understand the first answer. But the replies to the next two questions are quite funny. pic.twitter.com/lpOZpsvDck
|
||
|
|
||
|
Quoc Le
@quocleix
|
29. sij |
|
We cited another Riloff paper but thanks for the reference to Riloff 1996. We will look into it and cite in the next revision.
|
||
|
|
||
|
Quoc Le
@quocleix
|
29. sij |
|
"steer" is a type of cows:
britannica.com/animal/steer
|
||
|
|
||
|
Quoc Le
@quocleix
|
29. sij |
|
My favorite conversation is below. The Hayvard pun was funny but I totally missed the steer joke at the end until it was pointed out today by @Blonkhart pic.twitter.com/AmTobwf9A0
|
||
|
|
||
|
Quoc Le
@quocleix
|
28. sij |
|
You can find some sample conversations with the bot here:
github.com/google-researc…
|
||
|
|
||
|
Quoc Le
@quocleix
|
28. sij |
|
New paper: Towards a Human-like Open-Domain Chatbot. Key takeaways:
1. "Perplexity is all a chatbot needs" ;)
2. We're getting closer to a high-quality chatbot that can chat about anything
Paper: arxiv.org/abs/2001.09977
Blog: ai.googleblog.com/2020/01/toward… pic.twitter.com/5SOBa58qx3
|
||
|
|
||
|
Quoc Le
@quocleix
|
19. pro |
|
We have a few data points that suggest such improvements are meaningful:
1. Better ImageNet models transfer better to other datasets: arxiv.org/abs/1805.08974
2. Better accuracy on ImageNet gives vast improvements in out-of-distro generalization: arxiv.org/abs/1911.04252
|
||
|
|
||
| Quoc Le proslijedio/la je tweet | ||
|
Connor Shorten
@CShorten30
|
3. pro |
|
This video explains AdvProp from @GoogleAI! This technique leverages Adversarial Examples for ImageNet classification by using separate Batch Normalization layers for clean and adversarial mini-batches.
youtu.be/KTCztkNJm50
#100DaysOfMLCode
|
||
|
|
||
| Quoc Le proslijedio/la je tweet | ||
|
Jeff Dean
@JeffDean
|
26. stu |
|
Some nice case studies about how @GCPcloud's AutoML products can help tackle real-world problems in visual inspection across a number of different manufacturing domains, being used by companies like Global Foundries and Siemens.
cloud.google.com/blog/products/…
|
||
|
|
||
|
Quoc Le
@quocleix
|
26. stu |
|
Yes, you're probably right. We see similar results with other data augmentation methods: we need bigger models to learn from the extra augmented data.
|
||
|
|
||
|
Quoc Le
@quocleix
|
26. stu |
|
Pretrained checkpoints in Pytorch: github.com/rwightman/gen-…
h/t to @wightmanr
|
||
|
|
||
|
Quoc Le
@quocleix
|
26. stu |
|
AdvProp improves accuracy for a wide range of image models, from small to large. But the improvement seems bigger when the model is larger. pic.twitter.com/13scFaoQzB
|
||
|
|
||
|
Quoc Le
@quocleix
|
25. stu |
|
As a data augmentation method, adversarial examples are more general than other image processing techniques. So I expect AdvProp to be useful everywhere (language, structured data etc.), not just image recognition.
|
||
|
|
||
|
Quoc Le
@quocleix
|
25. stu |
|
Many of us tried to use adversarial examples as data augmentation and observed a drop in accuracy. And it seems that simply using two BatchNorms overcomes this mysterious drop in accuracy.
|
||
|
|
||
|
Quoc Le
@quocleix
|
25. stu |
|
AdvProp: One weird trick to use adversarial examples to reduce overfitting.
Key idea is to use two BatchNorms, one for normal examples and another one for adversarial examples.
Significant gains on ImageNet and other test sets. twitter.com/tanmingxing/st…
|
||
|
|
||
|
Quoc Le
@quocleix
|
22. stu |
|
You have a good point. Please take a look at my additional tweet above for latency on CPU and GPU (Figure 4 in the paper).
|
||
|
|
||
|
Quoc Le
@quocleix
|
22. stu |
|
And latency on CPU and GPU: pic.twitter.com/u0itlVZqP6
|
||
|
|
||
|
Quoc Le
@quocleix
|
22. stu |
|
Architecture of EfficientDet pic.twitter.com/8ZbS7JfEGZ
|
||
|
|
||