|
deepset
@
deepset_ai
Berlin, Deutschland
|
|
Bringing cutting-edge NLP to the industry via open-source. Checkout: github.com/deepset-ai/FARM and github.com/deepset-ai/hay…
|
|
|
113
Tweetovi
|
98
Pratim
|
236
Osobe koje vas prate
|
| Tweetovi |
|
deepset
@deepset_ai
|
3. velj |
|
The new release also includes many other exciting features: Checkpointing & caching, AMP, SageMaker integration, flexible LR schedules, early stopping, cross-validation, windows support and many more!
Thanks to all contributors!
Details: github.com/deepset-ai/FAR… (4/N)
|
||
|
|
||
|
deepset
@deepset_ai
|
3. velj |
|
#FARM is built on top of the great #transformers by @huggingface. With today's release of v 0.4.1, we go a huge step towards framework compatibility by allowing users to convert models seamlessly between FARM <-> transformers and load models from @huggingface's model hub. (3/N)
|
||
|
|
||
|
deepset
@deepset_ai
|
3. velj |
|
That's why #opensource #NLP frameworks should be compatible to each other instead of building borders. While there are good reasons to have different tooling for different user groups/use cases, we should build an ecosystem rather than silos. (2/N)
|
||
|
|
||
|
deepset
@deepset_ai
|
3. velj |
|
#opensource is more than just public code. It's a mindset of sharing, being transparent and collaborating across organizations.
It's about building on the shoulders of other projects and advancing together the state of technology (1/N)
@huggingface, @spacy_io, @fastdotai, #NLP pic.twitter.com/YkGjDU6sHu
|
||
|
|
||
|
deepset
@deepset_ai
|
29. sij |
|
Today's #NLP is heavily fueled by the power of #GPUs. Glad to announce that we are now a member of @NVIDIA's Inception program! Looking forward to even more GPU power and acceleration of our models via #apex & co
@NvidiaAI #NLP #cuda #amp #deeplearning pic.twitter.com/IHG90MzMbr
|
||
|
|
||
|
deepset
@deepset_ai
|
28. sij |
|
It's based on the nice work by Zhengyan Zhang & Xiaozhi Wang 🧡
|
||
|
|
||
|
deepset
@deepset_ai
|
28. sij |
|
As we believe in #opensource, you can find the public google slides here: docs.google.com/presentation/d… Feel free to use it in your own slides & comment missing LMs! #openslides
|
||
|
|
||
|
deepset
@deepset_ai
|
28. sij |
|
It's challenging to keep track of all the latest #languagemodels out there. What was again the difference between #RoBERTa and #BERT? What's the core idea behind #T5? Here's a little (not comprehensive) #cheatsheet that we use for workshops
#nlp #NLProc #deeplearning pic.twitter.com/KxBSWv8OTd
|
||
|
|
||
|
deepset
@deepset_ai
|
27. sij |
|
We can also recommend the multilingual XLM-R. Gives impressive results on German
towardsdatascience.com/xlm-roberta-th…
|
||
|
|
||
|
deepset
@deepset_ai
|
27. sij |
|
Maybe the next version :). Our main goal was to make sure people find it. We actually considered ALBERT back then. Good that we didn't pick that one. Would have been a mess :D
|
||
|
|
||
| deepset proslijedio/la je tweet | ||
|
Sebastian Ruder
@seb_ruder
|
27. sij |
|
Transfer learning is increasingly going multilingual with language-specific BERT models:
- 🇩🇪 German BERT deepset.ai/german-bert
- 🇫🇷 CamemBERT arxiv.org/abs/1911.03894, FlauBERT arxiv.org/abs/1912.05372
- 🇮🇹 AlBERTo ceur-ws.org/Vol-2481/paper…
- 🇳🇱 RobBERT arxiv.org/abs/2001.06286
|
||
|
|
||
| deepset proslijedio/la je tweet | ||
|
Sebastian Ruder
@seb_ruder
|
27. sij |
|
New NLP News: NLP Progress, Restrospectives and look ahead, New NLP courses, Independent research initiatives, Interviews, Lots of resources (via @revue)
newsletter.ruder.io/archive/217744
|
||
|
|
||
| deepset proslijedio/la je tweet | ||
|
Ivan Bilan
@DemiourgosUA
|
20. sij |
|
GitHub Repo Spotlight №3:
Transfer Learning library for NLP called FARM:: github.com/deepset-ai/FARM
With FARM you easily use BERT, XLNet, and others easily for any downstream NLP tasks. FARM is great for fast prototyping too.
#NLP #DataScience #AI
|
||
|
|
||
|
deepset
@deepset_ai
|
17. sij |
|
Are you doing #NLP in a non-english language? Try the multilingual XLM-R model! It gave us amazing results in German (for the SOTA chasers: yes, it's also outperforming previous results with BERT & Co).
Blog:
towardsdatascience.com/xlm-roberta-th…
|
||
|
|
||
| deepset proslijedio/la je tweet | ||
|
PyTorch
@PyTorch
|
15. sij |
|
v1.4: customizable mobile builds, Distributed Model Parallelism via experimental RPC API, Java Bindings, Chaining LRSchedulers
Summary: pytorch.org/blog/pytorch-1…
Release Notes: github.com/pytorch/pytorc…
Last release for Python 2 (bye bye!)
|
||
|
|
||
| deepset proslijedio/la je tweet | ||
|
Google AI
@GoogleAI
|
16. sij |
|
Introducing Reformer, an efficiency optimized #ML architecture, based on the Transformer model for language understanding, that can handle context windows of up to 1 million words, all on a single accelerator with only 16GB of memory. Read all about it ↓ goo.gle/2treP7r
|
||
|
|
||
| deepset proslijedio/la je tweet | ||
|
Alon Talmor
@AlonTalmor
|
1. sij |
|
We present our new year special: “oLMpics - On what Language Model pre-training captures״, arxiv.org/abs/1912.13283, Exploring what
symbolic reasoning skills are learned from an LM objective. We introduce 8 oLMpic games and controls for disentangling pre-training from fine-tuning. pic.twitter.com/ECQ7ZpcKlg
|
||
|
|
||
| deepset proslijedio/la je tweet | ||
|
Sebastian Ruder
@seb_ruder
|
18. pro |
|
Great to see VCs being excited about NLP. Recent examples:
- @Lux_Capital's investment in @huggingface: medium.com/@brandon.reeve…
- @Accel's investment in @Rasa_HQ: accel.com/interests/OurI…
- "Entering the Golden Age of NLP" by @thresholdvc: medium.com/@thresholdvc/n…
|
||
|
|
||
| deepset proslijedio/la je tweet | ||
|
DeepMind
@DeepMind
|
16. pro |
|
What does it mean to understand language? We argue that human-like understanding requires complementary memory systems and rich representations of situations.
A roadmap for extending ML models toward human-level language understanding: arxiv.org/abs/1912.05877 pic.twitter.com/eKBOekfmgj
|
||
|
|
||
|
deepset
@deepset_ai
|
15. pro |
|
As promised: here are the slides from Malte's talks in Warsaw!
- Keynote at @pydatawarsaw: drive.google.com/file/d/1V81Vn5…
- Talk at HumanTech: drive.google.com/file/d/1uQM3nE…
Reach out to us if you have some large polish text data set (> 10GB) and want to train a polish BERT or ALBERT. pic.twitter.com/noyQpfqzeh
|
||
|
|
||