Twitter | Search | |
Gabriele Svelto
Over the past few months we worked on improving stability on . This effort quickly became an example of good synergies between projects. A short thread 👇
Reply Retweet Like More
Gabriele Svelto Oct 6
Replying to @gabrielesvelto
When Firefox crashes the user can send us a crash report which we use to analyze the problem and hopefully fix it. However to extract useful information from the report we need the debug information from the affected build and its dependencies
Reply Retweet Like
Gabriele Svelto Oct 6
Replying to @gabrielesvelto
Most users on use the build that comes packaged with their favorite distribution rather than our builds. Because of that we couldn't do much with their reports as we lacked the debug information for those builds (with some exceptions)
Reply Retweet Like
Gabriele Svelto Oct 6
Replying to @GTKtoolkit @FFmpeg
To make matters worse we didn't have the debug information for dependencies which include a lot of different projects: , , , , etc...
Reply Retweet Like
Gabriele Svelto Oct 6
Replying to @ubuntu @debian and 4 others
To address this issue we started scraping debug information for builds and its dependencies from the package repositories of multiple distrubtions: , , , , and
Reply Retweet Like
Gabriele Svelto Oct 6
Replying to @gabrielesvelto
With that information now available we were able to analyze >99% of the crash reports we received from users, up from less than 20%
Reply Retweet Like
Gabriele Svelto Oct 6
Replying to @gabrielesvelto
This greatly sped up our response time to -specific stability issues, and often allowed us to identify problems in and versions of before they reached users on the release channel
Reply Retweet Like
Gabriele Svelto Oct 6
Replying to @gabrielesvelto
We could also quickly identify issues in bleeding-edge components such as , , and video acceleration; oftentimes providing a fix within days of the change that triggered the issue
Reply Retweet Like
Gabriele Svelto Oct 6
Replying to @gabrielesvelto
But we didn't stop there: we could now identify distro-specific stability issues and regressions. We started informing package maintainers of the problems and often had them resolved quickly
Reply Retweet Like
Gabriele Svelto Oct 6
Replying to @gabrielesvelto
Some distributions which were using their own crash reporting system for decided to switch to ours given the improved results
Reply Retweet Like
Gabriele Svelto Oct 6
Replying to @gabrielesvelto
We could also identify regressions and issues in our dependencies. We communicated the issues upstream and sometimes contributed fixes, benefiting both our users and theirs
Reply Retweet Like
Gabriele Svelto Oct 6
Replying to @gabrielesvelto
Additionally we identified a number of elusive security-sensitive issues in upstream projects thanks to large user base and crash analysis tools such as our probabilistic heap checker
Reply Retweet Like
Gabriele Svelto Oct 6
Replying to @getsentry
This also exposed some limitations in the tools we use for crash analysis, so we decided to rewrite them in largely relying on the excellent crates developed by
Reply Retweet Like
Gabriele Svelto Oct 6
Replying to @gabrielesvelto
The resulting tools were dramatically faster than our old ones, used a fraction of the memory and produced more accurate results
Reply Retweet Like
Gabriele Svelto Oct 6
Replying to @gabrielesvelto
Patches flowed both ways: we contributed improvements to their crates (and their dependencies) while they expanded their APIs to address our new use-cases and fixed the issues we discovered
Reply Retweet Like
Gabriele Svelto Oct 6
Replying to @gabrielesvelto
This goes on to show how collaboration between projects not only benefits their users but ultimately improves the whole ecosystem and the broader community that relies on it
Reply Retweet Like
Gabriele Svelto Oct 6
Special thanks to and all the others that contributed to this effort!
Reply Retweet Like
Gabriele Svelto Oct 7
Replying to @mozthunderbird
Yesterday I forgot something important: all of the above also applies to the Linux version of /
Reply Retweet Like