Lab 7: Delta

Fall Semester 2019

Objective
In this lab you will implement the Delta Debugging algorithm to find 1-minimal failing test cases
on a program using both line and character granularity.

Resources
1. Delta Debugging webpage: ​http://www.st.cs.uni-sb.de/dd/

Caution
Dr. Zeller, the author of the Delta Debugging technique, and several other people have published
implementations of the Delta Debugging algorithm. While we encourage you to research and
read more about the algorithm, looking at someone else’s implementation, will almost certainly
cause your implementation to be similar to theirs. Therefore, make sure that you do not view or
reference any implementations or code of the algorithm.

Setup
1. Download ​Delta.zip​ from Canvas and decompress it. It will produce the directory
Delta​ where you should find:
a. DeltaDebug.java​ - skeleton file for you to implement your algorithm in
b. DeltaDebugTest.java​ - test class for you to test running the algorithm
c. SecretCoder.java​ - a buggy encoder program
d. long_failing_text.txt​ - an input text file on which the buggy encoder
program fails.
e. min_failing_text_line.txt​: the correct minimal test input that should be
produced by your Delta Debugging program on S​ ecretCoder​ for part 1 (line
granularity)
f. min_failing_text_char.txt​: the correct minimal test input that should be
produced by your Delta Debugging program on S ​ ecretCoder​ for part 2
(character granularity)
 2. Compile the DeltaDebug.java and DeltaDebugTest.java programs (each time you change
one of these files, you will need to recompile before running it)
○ javac DeltaDebug.java
○ javac DeltaDebugTest.java

Part 1: Line Granularity

Program Description
SecretCoder​ implements a Caesar (or shift) cipher with a right shift of 98 places. Instead of
the regular alphabet, this program is intended to encode the 128 characters in the ASCII
character set (value 0 to 127). When the program runs successfully, it creates a file
output.txt​ with the encoded contents of the input file.

Setup
In the ​Delta​ directory, execute the command ​javac SecretCoder.java​ to build the
program. To run S ​ ecretCoder​, execute the command:
java SecretCoder <input file>
where ​<input file>​ is the name of a text file to be encoded.

Problem Summary
The bug in the encoding program ​SecretCoder​ occurs when ​long_failing_text.txt
contains a non-ASCII character. The program is using the UTF-8 character set to determine the
number of characters in the file and creates a char array to store the encoded values for printing.
It does not take into account that non-ASCII characters in the UTF-8 character set could be
included into the file and these will take up more than one byte. When a non-ASCII character is
read into the file as two or more bytes, the char array runs out of room and throws a
java.lang.ArrayIndexOutOfBoundsException​.

Your task is to implement the delta debugging algorithm presented in the lecture to obtain a
minimal test case on which ​SecretCoder​ fails for this same reason -
java.lang.ArrayIndexOutOfBoundsException​.

To accomplish this task, we have provided a framework and you will need to implement the
deltaDebug​ method in D ​ eltaDebug.java​. For part one of the lab, we will use line
​ eltaDebugTest.java​ class that you will use to run
granularity. We have provided a D
DeltaDebug​ to test your changes by running the following command:
java DeltaDebugTest line
Note that ​DeltaDebugTest​ calls the ​deltaDebug​ method in ​DeltaDebug​ with 5
parameters:
deltaDebug(Boolean char_granularity, String program,
String failing_file, String error_msg,
String final_minimized_file)

● char_granularity - If false, implement line granularity only

The grading script will call the ​deltaDebug​ method in ​DeltaDebug.java​ the same way the
provided ​DeltaDebugTest.java​ file does. ​Therefore, it is very important that you do not
make any changes to the type or method signature of d ​ eltaDebug ​in D
​ eltaDebug.java.

Ensure you are using the ​failing_file​ and ​final_minimized_file​ parameters for your file
names in ​DeltaDebug.java​ instead of hardcoding files because the grader will be using
​ eltaDebugTest.java​ .
additional files besides the ones used in D

We have provided the outline for some helper methods in DeltaDebug.java that you may find
useful (there are comments in the code to explain what these do). You can modify these methods
or add new helper methods in D​ eltaDebug.java​ in any way you see fit as long as your
​ eltaDebug​ with the original parameters.
algorithm is invoked by calling d

If ​deltaDebug​ is written correctly, executing the test program will create a 1-minimal test case
named ​my_min_failing_text_line.txt​ that is identical to the provided
min_failing_text_line.txt.

Verify they are identical (​including whitespace & new lines)​ using the diff command:
​diff my_min_failing_text_line.txt min_failing_text_line.txt

For grading, we will also be testing your algorithm on hidden input files. We encourage you to
come up with different inputs to test how your algorithm handles different cases. Feel free to edit
DeltaDebugTest.java​ to pass in different file names for testing - just remember you do not
turn in this file so none of your actual logic can be here.
The hidden test cases will be similar in format to the provided one, but will have a different file
name, different number of lines, different failing character, different position for failing
character, or other changes appropriate for a text file. You can assume that there will be a single
failing character in the file.

Your algorithm should run in less than 5 minutes on the course VM.

If you add print statements for debugging, make sure to remove these before submission.

Part 2: Character Granularity

Now you will extend your algorithm to perform minimization at the character granularity level
when the flag is set. Make sure to continue to minimizing the file by lines as much as possible,
and then switch to character granularity when line granularity has reached its minimum.

To test your changes, you should now run ​DeltaDebugTest​ with the following command:
java DeltaDebugTest char

If ​deltaDebug​ is written correctly, executing the test program will create a 1-minimal test case
named ​my_min_failing_text_char.txt​ that is identical to the provided
min_failing_text_char.txt.

Verify they are identical (​including whitespace)​ using the diff command:
diff my_min_failing_text_char.txt min_failing_text_char.txt

The same grading notes and 5 minute timeout limit apply as in part 1.

Items to Submit
We expect your submission to conform to the standards specified below. To ensure that your
submission is correct, you should run the provided file validator. You should not expect
submissions that have an improper folder structure, incorrect file names, or in other ways do not
conform to the specification to score full credit. The file validator will check folder structure and
names match what is expected for this lab, but won’t (and isn't intended to) catch everything.

The command to run the tool is: ​python3 zip_validator.py lab7 lab7.zip

Submit the following files in a single compressed file (​.zip​ format) named ​lab7.zip​. For full
credit, there must not be any subfolders or extra files contained within your zip file.
 ● DeltaDebug.java

Upload your zip file to Canvas. Make sure the spelling of the filenames and the extensions are
exactly as indicated above. Misspellings in filenames may result in a deduction to your grade.
Also double check that you are not accidentally submitting DeltaDebugTest.java.

Grading
Part 1:
● 20 points: DeltaDebug.java correctly minimizes ​long_failing_text.txt​ to
min_failing_test_case_line.txt
● 30 points: Other test cases
Part 2:
● 20 points: DeltaDebug.java correctly minimizes ​long_failing_text.txt​ to
min_failing_test_case_char.txt
● 30 points: Other test cases