Replication Instructions

To replicate the statistical test results presented in the paper follow the following instructions.

Clone this repository using the command git clone https://github.com/LASER-UMASS/JavaRepair-replication-package.git

NOTE:

• This repository requires >13GB space so make sure you have enough space before cloning this repository
• The scripts are written using R version 3.4.4 so make sure you have R installed along with the required R packages before running the following commands.

The following commands assume that this repository is cloned in your $HOME directory. Update the paths in the commands depening on where you clone this repository (if needed).

RQ1 (Do G&V techniques produce patches for real-world defects?)

1. cd ~/JavaRepair-replication-package/src/statistical-tests-execution/RQ1/
2. bash reproduce-rq1.sh ../../../results/RQ1/patch-quality-assessment/

The precomputed results are available here.

RQ2(How often and how much do the patches produced by G&V techniques overfit to the developer-written test suite and fail to generalize to the evaluation test suite, and thus ultimately to the program specification?)

1. cd ~/JavaRepair-replication-package/src/statistical-tests-execution/RQ2/
2. bash reproduce-rq2.sh ../../../results/RQ2/patch-quality-assessment/

The precomputed results are available here.

RQ3(How do the coverage and size of the test suite used to produce the patch affect patch quality?)

1. cd ~/JavaRepair-replication-package/src/statistical-tests-execution/RQ3/
2. bash reproduce-rq3.sh ../../../results/RQ2/patch-quality-assessment/

The precomputed results are available here.

RQ4(How does the number of tests that a buggy program fails affect the degree to which the generated patches overfit?):

1. cd ~/JavaRepair-replication-package/src/statistical-tests-execution/RQ4/
2. bash reproduce-rq4.sh ../../../results/RQ4/patch-quality-assessment/

The precomputed results are available here.

RQ5(How does the test suite provenance (whether it is written by developers or generated automatically) influence patch quality?)

1. cd ~/JavaRepair-replication-package/src/statistical-tests-execution/RQ5/
2. bash reproduce-rq5.sh ../../../results/RQ5/patch-quality-assessment/

The precomputed results are available here.

RQ6(Can overfitting be mitigated by exploiting randomness in the repair process? Do different random seeds overfit in different ways?)

1. cd ~/JavaRepair-replication-package/src/statistical-tests-execution/RQ5/
2. bash reproduce-rq6.sh ../../../results/RQ6/patch-quality-assessment/

The precomputed results are available here.