Twitter | Pretraživanje | |
Thang Luong
Introducing , a 2.6B-param open-domain chatbot with near-human quality. Remarkably, we show strong correlation between perplexity & humanlikeness! Paper: Sample conversations:
Reply Retweet Označi sa "sviđa mi se" More
Thang Luong 28. sij
Odgovor korisniku/ci @quocleix @xpearhead
is based on the Evolved Transformer (ET, an improved Transformer) & trained to minimize perplexity, the uncertainty of predicting the next word in a conversation. We built a novel "shallow-deep" seq2seq architecture: 1 ET block for encoder & 13 ET blocks for decoder.
Reply Retweet Označi sa "sviđa mi se"
Thang Luong 28. sij
Odgovor korisniku/ci @quocleix @xpearhead
We design a new human evaluation metric, Sensibleness & Specificity Average (SSA), which captures key elements of natural conversations. SSA is also shown to correlate with humanlikeness while being easier to measure. Human scores 86% SSA, 79%, other best chatbots 56%.
Reply Retweet Označi sa "sviđa mi se"
Thang Luong 28. sij
Odgovor korisniku/ci @quocleix @xpearhead
Implications from the project: 1. Perplexity might be "the" automatic metric that the field's been looking for. 2. Bots trained on large-scale social conversations & pushed hard for low perplexity will be good. 3. Safety layer is needed for respectful conversations!
Reply Retweet Označi sa "sviđa mi se"
M Akash Kumar 28. sij
Odgovor korisniku/ci @lmthang @quocleix @xpearhead
Nice work! Will the models be publicly released?
Reply Retweet Označi sa "sviđa mi se"
Thang Luong 29. sij
Odgovor korisniku/ci @akashkm99 @quocleix @xpearhead
We are actively working on this, so stay tune for more updates!
Reply Retweet Označi sa "sviđa mi se"
Asa Cooper Stickland 29. sij
Odgovor korisniku/ci @lmthang @quocleix @xpearhead
Awesome! I would love an appendix or similar with the ppl and exact architecture of the variants you tried out (e.g. ET vs vanilla Transformer)
Reply Retweet Označi sa "sviđa mi se"
Thang Luong 29. sij
Odgovor korisniku/ci @AsaCoopStick @quocleix @xpearhead
We created that table but didn't include because we felt like the specific details here do not matter a lot. what matters is showing the strong correlation exists & the community can verify with their own settlings.
Reply Retweet Označi sa "sviđa mi se"
Rajat Monga 29. sij
Odgovor korisniku/ci @lmthang @quocleix @xpearhead
Congratulations and the others!
Reply Retweet Označi sa "sviđa mi se"
Thang Luong 29. sij
Odgovor korisniku/ci @rajatmonga @quocleix @xpearhead
Thanks Rajat!
Reply Retweet Označi sa "sviđa mi se"
Jonathan Fly 👾 29. sij
Odgovor korisniku/ci @lmthang @quocleix @xpearhead
>near-human quality
Reply Retweet Označi sa "sviđa mi se"
Jonathan Fly 👾 29. sij
Odgovor korisniku/ci @lmthang @quocleix @xpearhead
This one is really impressive
Reply Retweet Označi sa "sviđa mi se"