Causal Testing Artifact (ICSE 2020)

This is the artifact for the paper "Causal Testing: Understanding Defects' Root Causes". This artifact replicates the user study setup used to evaluate our technique, including the supplementary data collection materials and prototype source code.

What is included in the artifact?

Our artifact includes the following:

1. A virtual machine file (CausalTesting_Artifact.ova) that contains the user study setup and source code for the version of Holmes used.

2. Instructions for obtaining and running Holmes on the virtual machine (INSTALL.md).

3. Contact information for the authors (CONTACT.md).

4. The license for our materials and prototype (LICENSE.md).

5. A description of what artifact badges we believe we support and why (STATUS.md).

6. The form participants completed as they worked through each defect in the user study (Cause-Identification-Form.pdf).

7. The post-questionnaire participants filled out as a form of debriefing following the study (Evaluation-Questionnaire.pdf)

8. The demographic questionnaire used to gather information about our participants background and experience (Demographic-Questionnaire.pdf).

9. The script used to explain to participants what they would be doing during the user study (User-Study-Script.pdf).

10. The pdf of our technical research paper "Causal Testing: Understanding Defects' Root Causes (ICSE2020-paper.pdf).

Where can I obtain the artifact components?

All the above listed artifacts, with the exception of the virtual machine file, are located in this repository. A link to the virtual machine file can be found in INSTALL.md.