There are many crash reporting services and all claim to provide the best crash reports. So we started wondering which service actually holds up to their claims?
CrashProbe is a crash reporting test suite presented by HockeyApp that benchmarks and compares popular iOS and Mac OS X crash reporting services. In our effort to ensure the best possible and most accurate crash reports, we created a set of test cases that we use to verify PLCrashReporter and improve HockeyApp.
Two components are involved in the crash reporting process:
A crash reporting framework that can be embedded into apps to capture crashes and exceptions.
A server-side process that symbolicates the crash reports to add details like class names, filenames, line numbers, and additional information like the reason.
Both components in this process are important, so even if a service is using the same framework as another service, the crash report can still be quite different.
CrashProbe is open source, so you can perform all tests yourself with any crash reporting framework. Compare the quality of the symbolicated crash reports yourself on iOS and OS X. The source code is available under the MIT license on GitHub.