|
Matthew Honnibal
@
honnibal
Berlin, Germany
|
|
Computational linguist from Sydney and Berlin. 💫 Author of the @spacy_io NLP tools. 💥 Founder @explosion_ai
|
|
|
2.681
Tweetovi
|
730
Pratim
|
9.591
Osobe koje vas prate
|
| Tweetovi |
| Matthew Honnibal proslijedio/la je tweet | ||
|
Ines Montani 〰️
@_inesmontani
|
2 h |
|
Travelling to Medellín today for @PyConColombia ✈️🐍
My keynote will be about the what the future holds for NLP, and how we can enable a new level of developer experience to make the technology available to everyone 🙌 pic.twitter.com/XsgNXS9osA
|
||
|
|
||
|
Matthew Honnibal
@honnibal
|
2. velj |
|
The model definition is functional, but the training loop is imperative.
|
||
|
|
||
|
Matthew Honnibal
@honnibal
|
2. velj |
|
Nice work! I think there's a spacy-udpipe package as well? I haven't used it, but the UDPipe models are pretty good. It's possible the wrapper will be better.
I never solved the batching thing in the spacy-stanfordnlp wrapper, so the speed is much worse than it should be.
|
||
|
|
||
|
Matthew Honnibal
@honnibal
|
1. velj |
|
It's actually a bit different from how Keras had multiple backends. Keras needed to touch quite a big API surface, because it was delegating all the numeric computation. Thinc's shims are just wrappers that have to pass around the inputs, outputs and gradients.
|
||
|
|
||
|
Matthew Honnibal
@honnibal
|
1. velj |
|
Well the idea is that the Model API is pretty simple, and it doesn't specify the internals. Internally a Model can call into another library to do its work, so you could mix models that call into libraries freely. That's not always an awesome approach, but you could.
|
||
|
|
||
|
Matthew Honnibal
@honnibal
|
1. velj |
|
The pluggability can be an advantage. MXNet has some great sparse models, and many do use Tensorflow. A lot of NLP researchers use DyNet. There's also many internal custom codebases.
An interface layer isn't something everyone needs, but I think some people will.
|
||
|
|
||
|
Matthew Honnibal
@honnibal
|
1. velj |
|
Well more broadly, ML is pretty varied, and there are a lot of potential spaCy models that aren't NNs. I've always wanted easy support for Vowpal Wabbit and XGBoost for instance. So we didn't want to code for PyTorch, we wanted an interface layer. And actually even in NNs.. 1/...
|
||
|
|
||
|
Matthew Honnibal
@honnibal
|
1. velj |
|
I'm sorry about that but it never should have appeared on amazon. It's been cancelled for ages because the field changed faster than we could write it. O'Reilly have said they can't get it taken down. I dont know what to do.
|
||
|
|
||
|
Matthew Honnibal
@honnibal
|
31. sij |
|
Yes absolutely. You can easily make just the loops do what you want, while keeping the rest in python and raising proper python errors.
For an lstm it wouldn't matter much...but sometimes it does, and you don't want to have both c++ and cython
|
||
|
|
||
|
Matthew Honnibal
@honnibal
|
31. sij |
|
I'd say CUDA and cython for gpu and cpu, jax for tpu. Did you see this? github.com/lmnt-com/haste
|
||
|
|
||
| Matthew Honnibal proslijedio/la je tweet | ||
|
Natalie Schluter
@natschluter
|
27. sij |
|
Come join my lab as a PhD. Less than a week left to apply! 🏃♀️⌚️ twitter.com/natschluter/st…
|
||
|
|
||
|
Matthew Honnibal
@honnibal
|
29. sij |
|
Yes they can. It maybe helps to read about the backprop strategy, and then the pytorch shim source thinc.ai/docs/concept
|
||
|
|
||
|
Matthew Honnibal
@honnibal
|
29. sij |
|
Thanks, I do hope you find it useful! For the record though TF on Swift has pretty different aims! I do think it's sensible for people to ask "is doing ML in python a local maximum?" , and explore alternatives
|
||
|
|
||
|
Matthew Honnibal
@honnibal
|
29. sij |
|
Uuuh no good reason. I should fix that thanks! For some reason I always wrote ReLu? Maybe i'll just make it Relu.
|
||
|
|
||
|
Matthew Honnibal
@honnibal
|
28. sij |
|
The trick we've used for Cupy+PyTorch is to route cupy memory allocations via PyTorch. The same trick will work for other combinations, maybe we can even make PyTorch request memory from TF instead of directly from the device. I'd be happy to dig into PyTorch's C++ API. TF...No.
|
||
|
|
||
|
Matthew Honnibal
@honnibal
|
28. sij |
|
Ah we should be clearer about this...TF is a huge pain in the ass. Cupy/MXNet/PyTorch work great. DyNet soon. TF...Only sort of. We put a lot of work in but they still don't support array interchange. It does work for testing and debugging, but GPU hogging will be an issue.
|
||
|
|
||
|
Matthew Honnibal
@honnibal
|
28. sij |
|
After doing this for several years, I can report: yep, it works great, just like you imagine. Also walkable cities, less time stuck in traffic.
|
||
|
|
||
|
Matthew Honnibal
@honnibal
|
28. sij |
|
Yes, definitely: the only thing we need is support for array interchange in JAX. It's coming soon. Once it lands you'd be able to combine a stitch a Flax layer onto a PyTorch model. I'm hoping this makes it easy to port models between frameworks layer by layer.
|
||
|
|
||
| Matthew Honnibal proslijedio/la je tweet | ||
|
Delip Rao
@deliprao
|
28. sij |
|
Something tells me this won’t be yet another #deeplearning library. I love this massive upgrade to Thinc from its older incarnation. twitter.com/explosion_ai/s…
|
||
|
|
||
| Matthew Honnibal proslijedio/la je tweet | ||
|
Stephen Reuben
@StephenReuben13
|
28. sij |
|
Prodigy built on Spacy is a godsend for anyone working in data poor domains. It allows you to annotate your data and generate a more expressive dataset or just a less sparse one. Currently works for NLP and a few separate use cases but still Check it out. Thanks @explosion_ai
|
||
|
|
||