Twitter | Pretraživanje | |
Matthew Honnibal
Computational linguist from Sydney and Berlin. 💫 Author of the NLP tools. 💥 Founder
2.681
Tweetovi
730
Pratim
9.591
Osobe koje vas prate
Tweetovi
Matthew Honnibal proslijedio/la je tweet
Ines Montani 〰️ 2 h
Travelling to Medellín today for ✈️🐍 My keynote will be about the what the future holds for NLP, and how we can enable a new level of developer experience to make the technology available to everyone 🙌
Reply Retweet Označi sa "sviđa mi se"
Matthew Honnibal 2. velj
Odgovor korisniku/ci @rasbt @CodeBug88 i 5 ostali
The model definition is functional, but the training loop is imperative.
Reply Retweet Označi sa "sviđa mi se"
Matthew Honnibal 2. velj
Odgovor korisniku/ci @BramVanroy @spacy_io @stanfordnlp
Nice work! I think there's a spacy-udpipe package as well? I haven't used it, but the UDPipe models are pretty good. It's possible the wrapper will be better. I never solved the batching thing in the spacy-stanfordnlp wrapper, so the speed is much worse than it should be.
Reply Retweet Označi sa "sviđa mi se"
Matthew Honnibal 1. velj
Odgovor korisniku/ci @rasbt @jeremyphoward i 4 ostali
It's actually a bit different from how Keras had multiple backends. Keras needed to touch quite a big API surface, because it was delegating all the numeric computation. Thinc's shims are just wrappers that have to pass around the inputs, outputs and gradients.
Reply Retweet Označi sa "sviđa mi se"
Matthew Honnibal 1. velj
Odgovor korisniku/ci @rasbt @jeremyphoward i 4 ostali
Well the idea is that the Model API is pretty simple, and it doesn't specify the internals. Internally a Model can call into another library to do its work, so you could mix models that call into libraries freely. That's not always an awesome approach, but you could.
Reply Retweet Označi sa "sviđa mi se"
Matthew Honnibal 1. velj
Odgovor korisniku/ci @rasbt @jeremyphoward i 4 ostali
The pluggability can be an advantage. MXNet has some great sparse models, and many do use Tensorflow. A lot of NLP researchers use DyNet. There's also many internal custom codebases. An interface layer isn't something everyone needs, but I think some people will.
Reply Retweet Označi sa "sviđa mi se"
Matthew Honnibal 1. velj
Odgovor korisniku/ci @rasbt @jeremyphoward i 4 ostali
Well more broadly, ML is pretty varied, and there are a lot of potential spaCy models that aren't NNs. I've always wanted easy support for Vowpal Wabbit and XGBoost for instance. So we didn't want to code for PyTorch, we wanted an interface layer. And actually even in NNs.. 1/...
Reply Retweet Označi sa "sviđa mi se"
Matthew Honnibal 1. velj
Odgovor korisniku/ci @ionekr
I'm sorry about that but it never should have appeared on amazon. It's been cancelled for ages because the field changed faster than we could write it. O'Reilly have said they can't get it taken down. I dont know what to do.
Reply Retweet Označi sa "sviđa mi se"
Matthew Honnibal 31. sij
Odgovor korisniku/ci @crosspacific1 @Smerity i 2 ostali
Yes absolutely. You can easily make just the loops do what you want, while keeping the rest in python and raising proper python errors. For an lstm it wouldn't matter much...but sometimes it does, and you don't want to have both c++ and cython
Reply Retweet Označi sa "sviđa mi se"
Matthew Honnibal 31. sij
Odgovor korisniku/ci @Smerity @NvidiaAI @PyTorch
I'd say CUDA and cython for gpu and cpu, jax for tpu. Did you see this?
Reply Retweet Označi sa "sviđa mi se"
Matthew Honnibal proslijedio/la je tweet
Natalie Schluter 27. sij
Come join my lab as a PhD. Less than a week left to apply! 🏃‍♀️⌚️
Reply Retweet Označi sa "sviđa mi se"
Matthew Honnibal 29. sij
Odgovor korisniku/ci @phi_nate
Yes they can. It maybe helps to read about the backprop strategy, and then the pytorch shim source
Reply Retweet Označi sa "sviđa mi se"
Matthew Honnibal 29. sij
Odgovor korisniku/ci @pommedeterre33
Thanks, I do hope you find it useful! For the record though TF on Swift has pretty different aims! I do think it's sensible for people to ask "is doing ML in python a local maximum?" , and explore alternatives
Reply Retweet Označi sa "sviđa mi se"
Matthew Honnibal 29. sij
Odgovor korisniku/ci @texttheater @explosion_ai i 3 ostali
Uuuh no good reason. I should fix that thanks! For some reason I always wrote ReLu? Maybe i'll just make it Relu.
Reply Retweet Označi sa "sviđa mi se"
Matthew Honnibal 28. sij
Odgovor korisniku/ci @deliprao @explosion_ai @GoogleColab
The trick we've used for Cupy+PyTorch is to route cupy memory allocations via PyTorch. The same trick will work for other combinations, maybe we can even make PyTorch request memory from TF instead of directly from the device. I'd be happy to dig into PyTorch's C++ API. TF...No.
Reply Retweet Označi sa "sviđa mi se"
Matthew Honnibal 28. sij
Odgovor korisniku/ci @deliprao @explosion_ai @GoogleColab
Ah we should be clearer about this...TF is a huge pain in the ass. Cupy/MXNet/PyTorch work great. DyNet soon. TF...Only sort of. We put a lot of work in but they still don't support array interchange. It does work for testing and debugging, but GPU hogging will be an issue.
Reply Retweet Označi sa "sviđa mi se"
Matthew Honnibal 28. sij
Odgovor korisniku/ci @devonzuegel
After doing this for several years, I can report: yep, it works great, just like you imagine. Also walkable cities, less time stuck in traffic.
Reply Retweet Označi sa "sviđa mi se"
Matthew Honnibal 28. sij
Odgovor korisniku/ci @deliprao @explosion_ai @GoogleColab
Yes, definitely: the only thing we need is support for array interchange in JAX. It's coming soon. Once it lands you'd be able to combine a stitch a Flax layer onto a PyTorch model. I'm hoping this makes it easy to port models between frameworks layer by layer.
Reply Retweet Označi sa "sviđa mi se"
Matthew Honnibal proslijedio/la je tweet
Delip Rao 28. sij
Something tells me this won’t be yet another library. I love this massive upgrade to Thinc from its older incarnation.
Reply Retweet Označi sa "sviđa mi se"
Matthew Honnibal proslijedio/la je tweet
Stephen Reuben 28. sij
Prodigy built on Spacy is a godsend for anyone working in data poor domains. It allows you to annotate your data and generate a more expressive dataset or just a less sparse one. Currently works for NLP and a few separate use cases but still Check it out. Thanks
Reply Retweet Označi sa "sviđa mi se"