Twitter | Search | |
Jeremy Howard
The fastai paper (with ) is now available on arXiv and on our site! It's been peer reviewed and will appear in the journal Information soon. The paper covers v2, which is a from-scratch rewrite that focuses on usability and hackability. 1/n
fastai is a modern, open source, deep learning library which is organized around two main design goals: to be approachable and rapidly productive, while also being deeply hackable and configurable
fast.ai fast.ai @fastdotai
Reply Retweet Like More
Jeremy Howard Feb 13
Replying to @jeremyphoward
The paper explains that by carefully crafting a layered API, fastai v2 has lots of new functionality (both new to fastai, and new to deep learning libraries in general), yet contains <10k lines of code!
Reply Retweet Like
Jeremy Howard Feb 13
Replying to @jeremyphoward
A high-level API powers ready-to-use functions to train models in various applications, offering customizable models with sensible defaults. It is built on top of a hierarchy of lower level APIs which provide composable building blocks
Reply Retweet Like
Jeremy Howard Feb 13
Replying to @jeremyphoward
With the high-level API it requires fewer lines of code from the user to re-specify information or merely to connect components. As a result, every line of user code tends to be more likely to be meaningful, and easier to read
Reply Retweet Like
Jeremy Howard Feb 13
Replying to @jeremyphoward
The mid-level API provides the core deep learning and data-processing methods for each of these applications, and low-level APIs provide a library of optimized primitives and functional and object-oriented foundations, which allows the mid-level to be developed and customised
Reply Retweet Like
Jeremy Howard Feb 13
Replying to @jeremyphoward
The lines of code to create a segmentation model are almost identical to those for classification, except for those necessary for the differences in input data. The same code that for displaying image classes example also displays the segmentation data
Reply Retweet Like
Jeremy Howard Feb 13
Replying to @jeremyphoward
The same code (with minor changes for input data formats) can also train a language model, NLP classifier, tabular model, collaborative filtering, etc. And each supports the same methods to display data and results
Reply Retweet Like
Jeremy Howard Feb 13
Replying to @jeremyphoward
fastai v2 makes deployment very convenient--a saved model serializes everything needed for deployment, including all the data processing steps and the details of each step. For instance, normalization statistics are stored and used automatically for inference
Reply Retweet Like
Jeremy Howard Feb 13
Replying to @jeremyphoward
fastai v2 supports incrementally adding fastai features to plain PyTorch code, without requiring extensive rewrites
Reply Retweet Like
Jeremy Howard Feb 13
Replying to @jeremyphoward
Many libraries, including fastai v1, provide a high-level API to users, and a low-level API used internally, but nothing in between. This has 2 problems: it becomes harder and harder to create additional high-level functionality, as the system becomes more sophisticated...
Reply Retweet Like
Jeremy Howard Feb 13
Replying to @jeremyphoward
...and for users of the system who want to customize and adapt it, they often have to rewrite significant parts of the high-level API, and understand the large surface area of the low-level API in order to do so. fastai v2's new mid-tier API solves this.
Reply Retweet Like
Jeremy Howard Feb 13
Replying to @jeremyphoward
For instance, v2 provides a new generic optimizer that allows recent optimization techniques to be implemented in a handful of lines of code, by refactoring out the common functionality using optimizer callbacks. The paper uses the LAMB optimizer as an example
Reply Retweet Like
Jeremy Howard Feb 13
Replying to @jeremyphoward
The mid-tier API is built on a new low-level API foundation, without which this new functionality wouldn't be possible
Reply Retweet Like
Jeremy Howard Feb 13
Huge thanks to for commissioning the paper and editing the special edition it's appearing in, and for being deeply involved in the paper writing process - the paper would have been much less strong without him. And thanks to the for everything!
Reply Retweet Like
Jeremy Howard Feb 13
Replying to @ProjectJupyter
The paper includes 64 citations, so we can't thank everyone we relied on here in this thread! Suffice to say: we built on the work of many, many people. In particular, and provided an amazing foundation for this work and we're really grateful the exist!
Reply Retweet Like
Jeremy Howard Feb 13
Replying to @clattner_llvm
Special thanks also to for all his support. Porting much of fastai to inspired many of the ideas that are now in the Python library--especially the new type dispatch system and more functional style
Reply Retweet Like
Jeremy Howard Feb 13
Replying to @clattner_llvm @rasbt
Thanks also to the open-access journal "Information" and edition editor for inviting a couple of folks without PhDs write a paper, and putting up with our desire to write it in plain English, rather than in the usual turgid academic style!
Reply Retweet Like