Tuesday, December 22, 2009

What is Fuzz testing

On weekend, one of my friends talked about Fuzzy testing, which pushed me to do some "googling" about fuzzy testing. I am sharing this information however, you can read about fuzzy testing more on wikipedia.
Fuzz testing or fuzzing is a software testing technique that provides invalid, unexpected, or random data to the inputs of a program. If the program fails (for example, by crashing or failing built-in code assertions), the defects can be noted.
File formats and network protocols are the most common targets of fuzz testing, but any type of program input can be fuzzed. Interesting inputs include environment variables, keyboard and mouse events, and sequences of API calls. Even items not normally considered "input" can be fuzzed, such as the contents of databases, and shared memory.
However, fuzz testing is not a substitute for exhaustive testing or formal methods: it can only provide a random sample of the system's behavior, and in many cases passing a fuzz test may only demonstrate that a piece of software handles exceptions without crashing, rather than behaving correctly. Thus, fuzz testing can only be regarded as a bug-finding tool rather than an assurance of quality.The oldest and simplest form of fuzzing, sending a stream of random bits to software, continues to find bugs in command-line applications. Another common technique that is easy to implement is mutating existing input (e.g. files from a test suite) by flipping bits at random or moving blocks of the file around. But the most successful fuzzers have detailed understanding of the format or protocol being tested.
Types of bugs found: Straight-up failures such as crashes, assertion failures, and memory leaks are easy to detect. The use of a memory debugger can help find bugs too subtle to always crash.
Fuzz testing is especially useful against large C or C++ applications, where any bug affecting memory safety is likely to be a severe vulnerability. It is these security concerns that motivate the development of most fuzzers.
Since fuzzing often generates invalid input, it is especially good at testing error-handling routines, which are important for software that does not control its input. As such, simple fuzzing can be thought of as a way to automate negative testing. More sophisticated fuzzing tests more "main-line" code, along with error paths deep within it.

Reproduction and isolation: As a practical matter, developers need to reproduce errors in order to fix them. For this reason, almost all fuzz testing makes a record of the data it manufactures, usually before applying it to the software, so that if the computer fails dramatically, the test data is preserved. If the fuzz stream is pseudo-random number generated it may be easier to store the seed value to reproduce the fuzz attempt.
Once a bug found through fuzzing is reproduced, it is often desirable to produce a simple test case to make the issue easier to understand and debug. A simple testcase may also be faster and therefore more suitable for inclusion in a test suite that is run frequently
Enjoy testing :)

4 comments:

  1. Hey Kashif,

    Fuzzing is very often used for security testing. Some security testers make a specific test of it. Fuzzing is useful as you described to explore the unexpected part.
    Some news site for example have already article without posting them. By fuzzing the parameter in the URL for example you can find new articles, new pages or unexpected behaviour that gives your more access.

    There are two ways of fuzzing, replacive and recursive. Each tester should know this way of testing because it`s easy and you can add this simple to existing tests.

    Open source tools like WebScarab (from owasp) and Burp Suite (from portzwigger.net) can do it for you. If you want to know more please let me know.

    ReplyDelete
  2. Thanks andreas,

    I really appreciate your positive feedback, specially sharing information about open source tools.

    ReplyDelete
  3. First of all kashif the writing is good, BUT :)
    "Fuzz testing can only be regarded as a bug finding tool rather then an assurance of quality" why?
    see with fuzz testing we check that what is the behavior of the application in any unexpected situation, i.e. whether it handles exceptional cases or not, then how you can say that it is just a bug finding tool not assuring the quality....
    Plus what is the diffrence between Fuzz Testing and Negative Testing? if both are same then why this buzz word "Fuzz Testing" is used?

    And Andreas Prins
    plz share something more about these tools

    ReplyDelete
  4. Thanks for the article, excellent stuff.
    You can get info on Web Testing as well with some guidelines with different.

    ReplyDelete