Kristen Walcott-Justice, University of Virginia, will be presenting "Testing in Resource Constrained Environments" on Thursday, March 8th at Noon in Tome 115. Free pizza - everyone welcome.
Abstract: Testing is a critical and expensive part of the software development life cycle. Often, test suites grow in size to the extent that they may take days, weeks, or months to excute. Because the time to test is limited, test cases are selected and prioritized. More effective prioritizations can be created when the constrained resources, such as time or power, are considered. By prioritizing with constraints in mind, the final test case prioritization will yield a higher average percent of faults detected than more naive prioritization approaches.
One challenge of prioritizing and selecting test cases is in estimating the fault finding ability of the tests. This is traditionally performed through test coverage analysis.
The overhead of test coverage analysis is dominated by monitoring the application, which is traditionally performed using instrumentation. However, instrumentation can prohibitively increase the time overhead and code growth of an application. As an alternative to instrumentation, we explore how recent hardware advances can be leveraged to improve the overheads of test coverage analysis. These hardware advances include hardware performance monitors and multicore technology.
In this talk, she will be presenting our system, THeME (Testing by Hardware Monitoring Events), a testing framework that replaces instrumentation with hardware monitoring. THeME consists of a runtime system that takes advantage of hardware mechanisms and multiple cores and a static component to further extend the coverage derived from hardware event sampling. The results show that up to 90% of the actual coverage can be determined with less time overhead and negligible code growth compared to instrumentation, making it particularly applicable for use in resource constrained environments.