New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fuzzer script #1534
Comments
Original Redmine Comment Sweet. Always wanted to try AFL but never got to it. Ran it for 2 minutes (which know is nothing the the grand scheme). Perhaps you can set it up to use only ASCII to speed things up - as any control chars are unlikely to be interesting. |
Original Redmine Comment As far as I know there isn't an option to keep the inputs ASCII. There is an option to use a collection of interesting strings. That could be used to make the fuzzer know the whole set of valid keywords for Verilog and the set of valid symbols. I would expect that this would make things way more efficient. I've never used this option though and someone would have to write this list. A different thing that can be done to speed up the fuzzer is to give it a larger corpus of examples to start with. In my example script, I created a single starting point, but to find the bugs that I filed I started with a set of around 1000 short files that I had around for other reasons. A good place to start might be to point it at the existing files in the test_regress/t directory. You're right that two minutes is nowhere near how long things are expected to take. Assuming that you're seeing the number of paths found increase, it's not a bad idea to leave it to run overnight although I was seeing some results within the first half hour. |
Original Redmine Comment This is much appreciated.
How far do you want to take this? Not sure if you are willing to do more work, certainly would be helpful. Here's some suggestions based on your ideas:
|
Original Redmine Comment The contributor agreement is fine. At this point, I don't have a real bird's eye view of the how the project works to be able to how exactly things should be integrated. For example, where would you want this in the tree? And where would this be expected to be run? Is there a set of platforms that I should make sure things work on? From reading the bug about using Travis CI, it seems like there isn't a set of dedicated machines anywhere to run things on. Is there a document somewhere that lays out some development guidelines? For example, one of the scripts that I attached is Python and that might be the first piece of Python in the tree. |
Original Redmine Comment I'd suggest putting the files under the "nodist" directory, this is for things that are developer only and don't need to go into a tarball. I personally ran the fuzzing from the $VERILATOR_ROOT directory (where the .git file is.) I don't think we'd fuzz on Travis-CI, basically if if works on a standard Linux distro like Ubuntu 18.04 we're good. There's some information in docs/CONTRIBUTING.adoc and docs/internals.adoc that might be of interest. (Make sure you pull first as these were recently cleaned up.) Thanks for looking into this. |
Original Redmine Comment Scripts pushed to git. Know there will be minor changes to e.g. ignore some internal errors, feel free to send as pull requests, or post a new issue. |
Author Name: Eric Rippey
Original Redmine Issue: 1534 from https://www.veripool.org
Original Assignee: Eric Rippey
In this bug: https://www.veripool.org/issues/1529-Verilator-Sefault-on-short-testcase
There was a request to file a bug with info about the fuzzing the produced issues 1529-1533 was run. This is that bug.
These were done with American Fuzzy Lop (http://lcamtuf.coredump.cx/afl/)
The attached script should be do this when run on a Debian or Ubuntu based machine.
This script assumes that it is run as root. This is only needed for some of the steps. In general, running programs that are expected to have unusual behavior as root is probably not a great idea - you may want to edit this. This should get things started though.
Note that the tool's idea of what's an "interesting crash" is not perfect. There are both cases where something looks fine to the tool but it actually produced the "buffer overflow" error, and also cases where what it classifies as a crash do not have any obvious problem. So I also have a script which I was using to rerun the cases that it found. This is in "actual_fail.py". This assumes a couple of things about paths that will need to be edited.
The text was updated successfully, but these errors were encountered: