Twitter | Pretraživanje | |
Thang Luong 28. sij
Introducing , a 2.6B-param open-domain chatbot with near-human quality. Remarkably, we show strong correlation between perplexity & humanlikeness! Paper: Sample conversations:
Reply Retweet Označi sa "sviđa mi se"
Thang Luong 28. sij
Odgovor korisniku/ci @quocleix @xpearhead
is based on the Evolved Transformer (ET, an improved Transformer) & trained to minimize perplexity, the uncertainty of predicting the next word in a conversation. We built a novel "shallow-deep" seq2seq architecture: 1 ET block for encoder & 13 ET blocks for decoder.
Reply Retweet Označi sa "sviđa mi se"
Thang Luong 28. sij
Odgovor korisniku/ci @quocleix @xpearhead
We design a new human evaluation metric, Sensibleness & Specificity Average (SSA), which captures key elements of natural conversations. SSA is also shown to correlate with humanlikeness while being easier to measure. Human scores 86% SSA, 79%, other best chatbots 56%.
Reply Retweet Označi sa "sviđa mi se"
Thang Luong
Implications from the project: 1. Perplexity might be "the" automatic metric that the field's been looking for. 2. Bots trained on large-scale social conversations & pushed hard for low perplexity will be good. 3. Safety layer is needed for respectful conversations!
Reply Retweet Označi sa "sviđa mi se" More
Sanuj 2. velj
Odgovor korisniku/ci @lmthang @quocleix @xpearhead
Meena is pretty impressive. Was minimizing perplexity (or cross-entropy) the only training objective? Or was there something more to ensure things like consistency during a conversation? It seems like SSA was only used for evaluation. Forgive me if I overlooked any paper details!
Reply Retweet Označi sa "sviđa mi se"
Thang Luong 2. velj
Odgovor korisniku/ci @sanuj_sharma @quocleix @xpearhead
You got it right. We only minimize cross-entropy during training & SSA was used during evaluation.
Reply Retweet Označi sa "sviđa mi se"
Andrew Sears 29. sij
Odgovor korisniku/ci @lmthang @quocleix @xpearhead
I've never heard "Dolphin power!" in a conversation before... perhaps training it on '90s sitcoms might improve the "Perplexity", or just keep that level of awesomeness in the model. It also appears to have a propensity for saying cool! Set this up as a Google Home action!
Reply Retweet Označi sa "sviđa mi se"
Danny Iskandar 29. sij
Odgovor korisniku/ci @lmthang @quocleix @xpearhead
any time/plan to release as opensource?
Reply Retweet Označi sa "sviđa mi se"