Hacker News new | past | comments | ask | show | jobs | submit login
Finding UI crashes by fuzzing input events with american fuzzy lop (caolanm.blogspot.com)
99 points by buovjaga on Oct 25, 2015 | hide | past | favorite | 15 comments



For those of you writing Android apps, this is essentially what the monkey[1] tool is for.

[1]: http://developer.android.com/tools/help/monkey.html


The difference here is that AFL is not just banging on the keyboard at random--it's doing a guided search to find inputs that change the program state in interesting ways. This is more likely to find bugs in a finite amount of time.


This article kind of makes me wonder if there's much sense into finding (and fixing) bugs in the high level parts of a program (where no other code depends on) if it normally doesn't occur.


For something as LibreOffice I think it absolutely makes sense. The most frustrating thing when working with such software is a crash when you haven't saved your work. Auto-save may help, but it's still annoying.

What this gives you is a reliable way to reproduce obscure input scenarios that trigger a crash, which is often the only thing that's missing for fixing long-tail crashes. Of course, if there were now hundreds of such crashes you'd need to start triaging to avoid devoting hours of bug fixing to crashes no one has ever seen or will ever see, but if it's just a dozen rare ones and one high-profile frequent one ... I wouldn't think about triaging yet.


According to this blog post he found 11 bugs that are unlikely to occur and one that's happening quite often.

Fixing the other 11 would already make sense for the plain reason that it makes it easier to find more of the severe bugs (because your fuzzing output won't be cluttered by them).

But in general I think an office application is one of the apps where I think a high amount of stability and not crashing even in exotic situations is a clear win. Even if it only crashes for one out of 10.000 users - that's still a lot of crashes and potentially lost work.


Fixing these kinds of bugs also improve developer productivity in my experience.

Code that can trigger crashes like this often has problems with ownership and/or concurrent access. At least I often get stuck on such code when doing modifications, asking myself how on earth they guard against null ptr derefs etc. In many cases that sneaking suspicion is founded and there really are bugs lurking in the code I'm supposed to extend in some way.


Yeah, it's a great way to find smelly bits of code.


The thing here is that the LO codebase is huge and they don't have enough devs to maintain it.

So they deal with this by (1) turning all the automated code quality tools they can on it (2) getting top-notch open source devs to fix the problems the tools turn up.

Result: a more stable application for the users - remember that most of LO's users are nontechnical people relying on it for real work - and a more maintainable code base for the devs.

The reason this is important is that huge important smelly codebases with too few devs will increasingly be the norm in the future, and LO is developing important tools of attack.


The issue, I feel, is with VCL. However, there has been an incredible amount of refactoring - for instance, they have done some awesome things around Lifecycle management [1] and idle tuners have been improved put of sight.

The code is getting better all the time :-)

1. http://opengrok.libreoffice.org/xref/core/vcl/README.lifecyc...


Oh yeah. This is why 5.0 feels ridiculously more responsive than 4.4: they finally addressed timer issues.


After the last AFL article I ran it on a side project with a good input test suite where I was not the original author. Really easy to integrate, and I was impressed with the bugs it found. Running the generated input through valgrind made fixing problems really simple.

Add in the clang static analyzer and there are a lot of really nice tools for reasoning about memory issues.


I would love to have a general purpose Qt input event fuzzing library.


Cool, would this also work for testing websites with phantomjs?


tools like metasploit and arachni scanner are designed for that task


Fuzzy Dunlop?




Consider applying for YC's first-ever Fall batch! Applications are open till Aug 27.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: