|
Thang Luong
@
lmthang
United States
|
|
Senior research scientist at Google Brain, learning with unlabeled data (NoisyStudent, Electra, #MeenaBot). PhD @StanfordNLP, thesis NMT. Co-founder @vietaiorg.
|
|
|
1.202
Tweetovi
|
97
Pratim
|
14.026
Osobe koje vas prate
|
| Tweetovi |
|
Thang Luong
@lmthang
|
17 h |
|
You got it right. We only minimize cross-entropy during training & SSA was used during evaluation.
|
||
|
|
||
|
Thang Luong
@lmthang
|
17 h |
|
Sorry that Meena failed you! Maintaining personality and keeping the facts right are attributes we wish Meena to have as highlighted in the blog post. But try having similar conversations with other bots, maybe you will like Meena better ;) pic.twitter.com/0vuUNcz9SH
|
||
|
|
||
| Thang Luong proslijedio/la je tweet | ||
|
Buster Keaton Gifs
@BusterKeatonGif
|
23 h |
|
Happy #PalindromeDay
02 02 2020
This is the first time in over 900 years (since 11/11/1111) the date is palindromic no matter the date format.
It’s also the 33rd day of the year with 333 days left! pic.twitter.com/0hYP0p6mEa
|
||
|
|
||
|
Thang Luong
@lmthang
|
2. velj |
|
Congrats Russ!
|
||
|
|
||
|
Thang Luong
@lmthang
|
1. velj |
|
yes, we don't encode personality and exactly as you said, the sampling strategy will lead to different conversation flows. And these are definitely not Meena's true "beliefs".
|
||
|
|
||
|
Thang Luong
@lmthang
|
1. velj |
|
Good point! Many evaluations use 5-point Likert scale & look at other aspects, e.g., diversity, relevance, humanlikeness, etc. We use binary evaluation and think SSA
is basic to human quality & easy for crowdworkers to rate. also in paper, SSA correlates with humanlikeness. pic.twitter.com/ptrX4Ofs7m
|
||
|
|
||
| Thang Luong proslijedio/la je tweet | ||
|
Tyler Roost
@TylerRoost
|
1. velj |
|
|
||
|
|
||
|
Thang Luong
@lmthang
|
1. velj |
|
We are hard at work on this though we also need to be careful regarding safety and bias challenges as highlighted at the end of the blog post. Stay tune for more updates! ai.googleblog.com/2020/01/toward…
|
||
|
|
||
|
Thang Luong
@lmthang
|
1. velj |
|
Yes, Meena maintains its state pretty well: it gets stronger as the conversation progresses and sometimes a bit too strong that repetition happens (in a sensible way though), which is why we need to handle cross-turn repetition as detailed in the paper.
|
||
|
|
||
|
Thang Luong
@lmthang
|
31. sij |
|
Yes, we had those fun tests during the development of Meena and they were very hard at times!
|
||
|
|
||
|
Thang Luong
@lmthang
|
31. sij |
|
I chatted with #MeenaBot about the #coronavirus and her advice is to see a doctor sooner rather than later. I guess it's not a bad one & hope everyone is well! On the other hand, Meena is also excited about technology, especially VR! pic.twitter.com/pKRxfFxp38
|
||
|
|
||
|
Thang Luong
@lmthang
|
30. sij |
|
Highly recommend watching this 8-minute video youtu.be/STrrlLG15OY on #MeenaBot & the paper, with details not included the blog such as SSA vs humanlikeness correlation, sample-and-rank, removing cross-turn repetition. (Blog: ai.googleblog.com/2020/01/toward…) twitter.com/CShorten30/sta…
|
||
|
|
||
|
Thang Luong
@lmthang
|
29. sij |
|
We created that table but didn't include because we felt like the specific details here do not matter a lot. what matters is showing the strong correlation exists & the community can verify with their own settlings.
|
||
|
|
||
|
Thang Luong
@lmthang
|
29. sij |
|
It's on subwords (with 8K units).
|
||
|
|
||
|
Thang Luong
@lmthang
|
29. sij |
|
Good question. We optimize for standard cross-entropy. Perplexity is simply the exponentiation of the per-word cross-entropy.
|
||
|
|
||
|
Thang Luong
@lmthang
|
29. sij |
|
That was our very early motivation at the beginning of the project. We have then shifted the focus of optimizing
for deception detection, i.e., passing the Turing test, to optimizing for human-like qualities, which was the goal of Meena.
|
||
|
|
||
|
Thang Luong
@lmthang
|
29. sij |
|
Perplexity for a language model, by definition, is computed by first averaging all neg log predictions and then exponentiating. Does that help explain? towardsdatascience.com/perplexity-int…
|
||
|
|
||
|
Thang Luong
@lmthang
|
29. sij |
|
We are actively looking into this, stay tune for more updates!
|
||
|
|
||
|
Thang Luong
@lmthang
|
29. sij |
|
We are actively looking into this, stay tune for more updates!
|
||
|
|
||
|
Thang Luong
@lmthang
|
29. sij |
|
We are actively looking into this. Stay tune for more updates!
|
||
|
|
||