02:00 - 02:30

• Hercules: Reproducing Crashes in Real-World Application Binaries

  Binary analysis is a well-investigated area in software engineering and security. Given real-world program binaries, generating test inputs which cause the binaries to crash is crucial. Generation of crashing inputs has many applications including off-line analysis of software prior to deployment, or online analysis of software patches as they are inserted. In this work, we present a method for generating inputs which reach a given "potentially crashing" location. Such potentially crashing locations can be found by a separate static analysis (or by gleaning crash reports submitted by internal / external users) and serve as the input to our method. The test input generated by our method serves as a witness of the crash. Our method is particularly suited for binaries of programs which take in complex structured inputs. Experiments on real-life applications such as the Adobe Reader and the Windows Media Player demonstrate that our Hercules tool built on selective symbolic execution engine S2E can generate crashing inputs within few hours, where symbolic approaches (as embodied by S2E) or black box fuzzing approaches (as embodied by the commercial tool Peach Fuzzier) failed.

   

  Preprint available:

Van-Thuan Pham; Wei Boon Ng; Konstantin Rubinov; Abhik Roychoudhury

National University of Singapore, Singapore

02:30 - 03:00

• CARAMEL: Detecting and Fixing Performance Problems That Have Non-Intrusive Fixes

  Performance bugs are programming errors that slow down program execution. While existing techniques can detect various types of performance bugs, a crucial and practical aspect of performance bugs has not received the attention it deserves: how likely are developers to fix a performance bug? In practice, fixing a performance bug can have both benefits and drawbacks, and developers fix a performance bug only when the benefits outweigh the drawbacks. Unfortunately, for many performance bugs, the benefits and drawbacks are difficult to assess accurately. This paper presents CARAMEL, a novel static technique that detects and fixes performance bugs that have non-intrusive fixes likely to be adopted by developers. Each performance bug detected by CARAMEL is associated with a loop and a condition. When the condition becomes true during the loop execution, all the remaining computation performed by the loop is wasted. Developers typically fix such performance bugs because these bugs waste computation in loops and have non-intrusive fixes: when some condition becomes true dynamically, just break out of the loop. Given a program, CARAMEL detects such bugs statically and gives developers a potential source-level fix for each bug. We evaluate CARAMEL on real-world applications, including 11 popular Java applications (e.g., Groovy, Log4J, Lucene, Struts, Tomcat, etc) and 4 widely used C/C++ applications (Chromium, GCC, Mozilla, and My SQL). CARAMEL finds 61 new performance bugs in the Java applications and 89 new performance bugs in the C/C++ applications. Based on our bug reports, developers so far have fixed 51 and 65 performance bugs in the Java and C/C++ applications, respectively. Most of the remaining bugs are still under consideration by developers.

   

  Preprint available:

 

Adrian Nistor; Po-Chun Chang; Cosmin Radoi; Shan Lu

Chapman University, USA; University of Wisconsin-Madison, USA; University of Illinois at Urbana-Champaign, USA; University of Chicago, USA

03:00 - 03:30

• An Empirical Study on Real Bug Fixes

  Software bugs can cause significant financial loss and even the loss of human lives. To reduce such loss, developers devote substantial efforts to fixing bugs, which generally requires much expertise and experience. Various approaches have been proposed to aid debugging. An interesting recent research direction is automatic program repair, which achieves promising results, and attracts much academic and industrial attention. However, people also cast doubt on the effectiveness and promise of this direction. A key criticism is to what extent such approaches can fix real bugs. As only research prototypes for these approaches are available, it is infeasible to address the criticism by evaluating them directly on real bugs. Instead, in this paper, we design and develop BUGSTAT, a tool that extracts and analyzes bug fixes. With BUGSTAT's support, we conduct an empirical study on more than 9,000 real-world bug fixes from six popular Java projects. Comparing the nature of manual fixes with automatic program repair, we distill 15 findings, which are further summarized into four insights on the two key ingredients of automatic program repair: fault localization and faulty code fix. In addition, we provide indirect evidence on the size of the search space to fix real bugs and find that bugs may also reside in non-source files. Our results provide useful guidance and insights for improving the state-of-the-art of automatic program repair.

   

  Preprint available:

Hao Zhong; Zhendong Su

Shanghai Jiao Tong University, China; University of California at Davis, USA