As with earlier posts for Matlab, I'm putting together a set of exercises for automated student testing in introductory programming classes, but this time for Java. The idea is to integrate this into Virtual Programming Lab or secure lab tests. In a future post I'll do a VPL version of this, but for now this is just a proof-of-concept that will run inside your Java IDE -- IntelliJ in my case.
As with the Matlab example, the idea is to serve up a flowchart to the student and to ask the student to implement the flowchart. I'm again using "ASCII art" flowcharts because they can be embedded in text files and can be generated using pseudo-code within Diagon (either from the command line or on the web). The Diagon system isn't limited to flowcharts, so this could easily be converted to have students implement equations, etc.
There are four java files:
- Main.java
- MyTest.java
- StudentSolution.java
- TeacherReferenceSolutions.java
as well as two data files, of which only one is used within Java:
- diagon_flowcharts.txt
- diagon_scripts.txt
We use the flowcharts file in the Java program but the diagon_scripts.txt file is used to generate more flowcharts if you want to scale this up.
In this example, each flowchart has two input variables and one output result. All are of type Integer. When testing this in IntelliJ it should alternate between finding that the student solution is correct and that it's not correct, like this:
Main.java
There's not much here because we're relying on the jUnit testing in MyTest.java:
public class Main { public static void main(String[] args) { System.out.println("Nothing much happens in Main!"); System.out.println("Check out MyTest for the jUnit testing."); } }
MyTest.java
This is where all the work happens. It's where we run the jUnit testing. We'll
- Print out the desired flowchart (chosen at random)
- Load up the teacher solution that matches the chosen flowchart
- Compare the teacher solution to the student solution (which is fixed in StudentSolution.java)
- Run the comparison in a series of "sub-tests" for various pairs of input variable values.
Here is the MyTest.java file:
/* MyTest.java : * the jUnit testing file. * We compare the student's solution to the teacher solution here. */ import org.junit.Assert; import org.junit.Test; import java.io.IOException; import java.nio.file.Files; import java.nio.file.Paths; import java.util.ArrayList; import java.util.List; import java.util.Random; import java.util.function.BiFunction; public class MyTest { static private final String STARTSTRING = "% ----- start ----"; static private final String ENDSTRING = "% ----- end ----"; static private final String FLOWCHARTFILE = "diagon_flowcharts.txt"; @Test // method public void testOne() { TeacherReferenceSolutions theReference = new TeacherReferenceSolutions(); StudentSolution theStudentAttempt = new StudentSolution(); Integer studentAnswer = null; Integer teacherAnswer = null; Integer inputOne = null; Integer inputTwo = null; Integer minInput = -100; // Boundaries on inputs (max, min) Integer maxInput = +400; final Integer TESTTOTALNUMBER = 10; // Ten sub-tests run on student vs. teacher. /* Put all the teacher solutions into an ArrayList */ ArrayList<BiFunction<Integer, Integer, Integer>> methodList = new ArrayList<>(); methodList.add(TeacherReferenceSolutions::method1); methodList.add(TeacherReferenceSolutions::method2); methodList.add(TeacherReferenceSolutions::method3); methodList.add(TeacherReferenceSolutions::method4); /* more than 4 flowcharts? Add here.. */ /* Choose the specific method RANDOMLY, print the flowchart and start the test */ Integer chosenMethodIndex = new Random().nextInt(methodList.size() - 0) + 0; /* read the diagon_flowcharts.txt file so that, later, we can print out one of the flowcharts. */ List<String> allFlowcharts; try { allFlowcharts = Files.readAllLines(Paths.get(FLOWCHARTFILE)); } catch (IOException e) { System.out.println("Can't read flowchart file."); throw new RuntimeException(e); } /* Get the indices for both flowchart start and end. */ ArrayList<Integer> indexStart = new ArrayList<>(); ArrayList<Integer> indexEnd = new ArrayList<>(); for (Integer i = 0; i < allFlowcharts.size(); i++) { // Start indices if (allFlowcharts.get(i).equals(STARTSTRING)) { indexStart.add(i); } // End indices if (allFlowcharts.get(i).equals(ENDSTRING)) { indexEnd.add(i); } } /* Choose which flowchart to aim for as the target */ Integer maxFlowcharts = indexStart.size(); if (maxFlowcharts != indexEnd.size()){ System.out.println("****** Warning! Flowchart file is wrong. Mismatch in start vs. end. *******"); } var indexFlowchart = new Random().nextInt(maxFlowcharts - 0) + 0; /* print out the flow chart and pretend to ask student to implement it. */ System.out.println("Dear Student: create a method that implements this flowchart:"); for (Integer i = indexStart.get(indexFlowchart); i < indexEnd.get(indexFlowchart); i++) { System.out.println(allFlowcharts.get(i)); } // First loop. Print to screen messages that students can more easily decode. for (Integer testIteration = 0; testIteration < TESTTOTALNUMBER; testIteration++) { inputOne = new Random().nextInt(maxInput - minInput) + minInput; inputTwo = new Random().nextInt(maxInput - minInput) + minInput; /* Run the teacher solution once. Get the reference output. */ teacherAnswer = methodList.get(indexFlowchart).apply(inputOne, inputTwo); System.out.println("Subtest " + testIteration + " : testing against inputs : " + inputOne + " and " + inputTwo); // Run the student's answer. Get the student's output. */ studentAnswer = theStudentAttempt.studentMethod(inputOne, inputTwo); try { Assert.assertEquals(teacherAnswer, studentAnswer); System.out.println("Sub-Test " + testIteration + " passed."); System.out.println("Teacher solution: " + teacherAnswer + " ... versus student solution: " + studentAnswer); System.out.println("------------------------------------------------------"); System.out.println(""); } catch (AssertionError e) { System.out.println(""); System.out.println("****************************************************"); System.out.println("vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv"); System.out.println("\n\t\t\t Sub-Test " + testIteration + " + DID NOT PASS!! \t\t\t\t"); System.out.println("For inputs " + inputOne + " and " + inputTwo + " we got the following output:"); System.out.println(e.getMessage()); System.out.println(""); System.out.println("^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^"); System.out.println("****************************************************"); System.out.println(""); } } // Second loop. Allows Junit to register the error in VPL or in the IDE. for (Integer testIteration = 0; testIteration < TESTTOTALNUMBER; testIteration++) { inputOne = new Random().nextInt(maxInput - minInput) + minInput; inputTwo = new Random().nextInt(maxInput - minInput) + minInput; /* Run the teacher solution once. Get the reference output. */ teacherAnswer = methodList.get(indexFlowchart).apply(inputOne, inputTwo); System.out.println("Subtest " + testIteration + " : testing against inputs : " + inputOne + " and " + inputTwo); // Run the student's answer. Get the student's output. */ studentAnswer = theStudentAttempt.studentMethod(inputOne, inputTwo); // no try-catch here. This will register with the IDE or VPL Assert.assertEquals(teacherAnswer, studentAnswer); } } // end method }
The "student solution" is one that we're assuming that the student wrote. In a real exercise we would have to give the student an opportunity to write the solution. That mechanism is entirely dependent on the manner in which you deploy the exercise. One of my favourites is Virtual Programming Lab (VPL) on Moodle, but it could also be PrairieLearn or the local (to YorkU) WebSubmit system. In any event, here is the pretend student solution:
/* StudentSolution.java * An example of a file that a student might submit */ public class StudentSolution { public Integer studentMethod(Integer s, Integer t) { /* s and t are inputs */ Integer a = 0; Integer i = 1; for(i=1; i < 10; i++){ // Version 2: For loop // while (i < 10) { // Version 1: While loop. if (s >= 0) { if (t < 10) { a = a + 2; } else if (t < 20) { a = a + 4; } else { a = a + 5; } } else { if (t < 5) { a = a + 3; } else if (t < 10) { a = a + 2; } else { a = a + 1; } } // i = i + 1; // Use if using a while loop. (Version 1) } return a; } }
For the student solution, we're using a For loop. But that's just one way to do it. A While loop could also work. The Diagon app doesn't support For loops (for now) so the original flowchart and reference solutions are While-based, but it shouldn't matter what the student writes, as long as it is equivalent and compiles in Java.
That leaves us with two more files: the reference solutions in TeacherReferenceSolutions.java and the corresponding flowcharts in diagon_flowcharts.txt. Here are the Java methods in TeacherReferenceSolutions.java:
/* TeacherReferenceSolutions.java. * For each method in this file there will be a corresponding flowchart * in the diagon_flowcharts.txt file. (there is also a corresponding * bit of pseudo-code in diagon_scripts.txt. */ public class TeacherReferenceSolutions { /* Method 1 (will be index 0) */ public Integer method1(Integer s, Integer t) { /* s and t are inputs */ Integer a = 0; Integer i = 1; while (i < 10) { if (s >= 0) { if (t < 10) { a = a + 2; } else if (t < 20) { a = a + 4; } else { a = a + 5; } } else { if (t < 5) { a = a + 3; } else if (t < 10) { a = a + 2; } else { a = a + 1; } } i = i + 1; } return a; } /* Method 2 (will be index 1) */ private Integer method2(Integer s, Integer t) { /* s and t are inputs */ Integer a = 0; Integer i = 1; while(i < 5) { if(s <= 10){ if(t < 5) { a = a+1; } else if(t < 10) { a = a+5; } else { a = a+2; } } else{ if(t > 20) { a = a+32; } else if(t > 13) { a = a+20; } else { a = a+11; } } i = i+1; } return a; } /* Method 3 (will be index 2) */ public Integer method3(Integer s, Integer t) { Integer a = 0; Integer i = 1; while(i < 10) { if(s <= 5){ if(t < 2) { a = a+3; } else if(t < 8) { a = a+2; } else { a = a+1; } } else{ if(t > 10) { a = a+3; } else if(t > 5) { a = a+2; } else { a = a+1; } } i = i+1; } return a; } /* Method 4 (will be index 3) */ public Integer method4(Integer s, Integer t) { /* s and t are inputs */ Integer a = 0; Integer i = 1; while(i < 2) { if(s == 10){ if(t > 5) { a = a+0; } else if(t > 2) { a = a+2; } else { a = a+4; } } else{ if(t > 5) { a = a+32; } else if(t > 1) { a = a+20; } else { a = a+11; } } i = i+1; } return a; } /* Method 5 (will be index 4) */ public Integer method5(Integer s, Integer t) { Integer a = 0; Integer i = 1; /* s and t are inputs */ a = 0; i = 1; while(i < 6) { if(t > 10){ if(s > 5) { a = a+2; } else { a = a+40; } } else if( t > 5 ){ if(s == 5) { a = a+3; } else { a = a+1; } } else{ if(s < 5) { a = a+2; } else { a = a+1; } } i = i+1; } return a; } /* Add more methods here, as needed. Make sure to update the 1. diagon_flowcharts.txt file and 2. the MyTest.java file, too. */ }
and, finally, we have all of the ASCII art flowcharts that correspond to the methods above:
% diagon_flowcharts.txt % first % ----- start ---- ┌─────┐ │START│ └──┬──┘ ┌──▽──┐ │a = 0│ └──┬──┘ ┌─▽─┐ │i=1│ └─┬─┘ ▽___ ╱ ╲ ____╱ i<10 ╲____________________________________________________ │ no ╲ ╱ │ │ ╲____╱ │ │ │yes │ │ __▽___ ______ │ │ ╱ ╲ ╱ ╲ ┌───────┐ │ │ ╱ s >= 0 ╲____________________╱ t < 10 ╲___________│a = a+2│ │ │ ╲ ╱yes ╲ ╱yes └───┬───┘ │ │ ╲______╱ ╲______╱ │ │ │ │no │no │ │ │ __▽__ _▽__ │ │ │ ╱ ╲ ┌───────┐ ╱ ╲ ┌───────┐ │ │ │ ╱ t < 5 ╲____________│a = a+3│ ╱ t<20 ╲___│a = a+4│ │ │ │ ╲ ╱yes └───┬───┘ ╲ ╱yes└───┬───┘ │ │ │ ╲_____╱ │ ╲____╱ │ │ │ │ │no │ │no │ │ │ │ _▽__ │ ┌───▽───┐ │ │ │ │ ╱ ╲ ┌───────┐ │ │a = a+5│ │ │ │ │ ╱ t<10 ╲___│a = a+2│ │ └───┬───┘ │ │ │ │ ╲ ╱yes└───┬───┘ │ │ │ │ │ │ ╲____╱ │ │ │ │ │ │ │ │no │ │ │ │ │ │ │ ┌───▽───┐ │ │ │ │ │ │ │ │a = a+1│ │ │ │ │ │ │ │ └───┬───┘ │ │ │ │ │ │ │ └───────────┴─┬──────┴────────┴───────────┴────────┘ │ │ ┌───▽───┐ │ │ │i = i+1│ │ │ └───┬───┘ │ │ └──────────────────────────────────────────┘ ┌─▽─┐ │END│ └───┘ % ----- end ---- % second % ----- start ---- ┌─────┐ │START│ └──┬──┘ ┌──▽──┐ │a = 0│ └──┬──┘ ┌──▽──┐ │i = 1│ └──┬──┘ _▽___ ╱ ╲ ____╱ i < 5 ╲________________________________________________________ │ no ╲ ╱ │ │ ╲_____╱ │ │ │yes │ │ ___▽___ _____ │ │ ╱ ╲ ╱ ╲ ┌───────┐ │ │ ╱ s <= 10 ╲_______________________╱ t < 5 ╲_____________│a = a+1│ │ │ ╲ ╱yes ╲ ╱yes └───┬───┘ │ │ ╲_______╱ ╲_____╱ │ │ │ │no │no │ │ │ __▽___ __▽___ │ │ │ ╱ ╲ ┌────────┐ ╱ ╲ ┌───────┐ │ │ │ ╱ t > 20 ╲_____________│a = a+32│╱ t < 10 ╲___│a = a+5│ │ │ │ ╲ ╱yes └────┬───┘╲ ╱yes└───┬───┘ │ │ │ ╲______╱ │ ╲______╱ │ │ │ │ │no │ │no │ │ │ │ __▽___ │ ┌───▽───┐ │ │ │ │ ╱ ╲ ┌────────┐ │ │a = a+2│ │ │ │ │ ╱ t > 13 ╲___│a = a+20│ │ └───┬───┘ │ │ │ │ ╲ ╱yes└────┬───┘ │ │ │ │ │ │ ╲______╱ │ │ │ │ │ │ │ │no │ │ │ │ │ │ │ ┌────▽───┐ │ │ │ │ │ │ │ │a = a+11│ │ │ │ │ │ │ │ └────┬───┘ │ │ │ │ │ │ │ └─────────────┴┬────────┴────────┴────────────┴────────┘ │ │ ┌───▽───┐ │ │ │i = i+1│ │ │ └───┬───┘ │ │ └─────────────────────────────────────────────┘ ┌─▽─┐ │END│ └───┘ % ----- end ---- % third % ----- start ---- ┌─────┐ │START│ └──┬──┘ ┌──▽──┐ │a = 0│ └──┬──┘ ┌──▽──┐ │i = 1│ └──┬──┘ _▽____ ╱ ╲ ____╱ i < 10 ╲___________________________________________________ │ no ╲ ╱ │ │ ╲______╱ │ │ │yes │ │ __▽___ _____ │ │ ╱ ╲ ╱ ╲ ┌───────┐ │ │ ╱ s <= 5 ╲____________________╱ t < 2 ╲____________│a = a+3│ │ │ ╲ ╱yes ╲ ╱yes └───┬───┘ │ │ ╲______╱ ╲_____╱ │ │ │ │no │no │ │ │ __▽___ __▽__ │ │ │ ╱ ╲ ┌───────┐ ╱ ╲ ┌───────┐ │ │ │ ╱ t > 10 ╲___________│a = a+3│╱ t < 8 ╲___│a = a+2│ │ │ │ ╲ ╱yes └───┬───┘╲ ╱yes└───┬───┘ │ │ │ ╲______╱ │ ╲_____╱ │ │ │ │ │no │ │no │ │ │ │ __▽__ │ ┌───▽───┐ │ │ │ │ ╱ ╲ ┌───────┐ │ │a = a+1│ │ │ │ │ ╱ t > 5 ╲___│a = a+2│ │ └───┬───┘ │ │ │ │ ╲ ╱yes└───┬───┘ │ │ │ │ │ │ ╲_____╱ │ │ │ │ │ │ │ │no │ │ │ │ │ │ │ ┌───▽───┐ │ │ │ │ │ │ │ │a = a+1│ │ │ │ │ │ │ │ └───┬───┘ │ │ │ │ │ │ │ └───────────┴─┬──────┴────────┴───────────┴────────┘ │ │ ┌───▽───┐ │ │ │i = i+1│ │ │ └───┬───┘ │ │ └──────────────────────────────────────────┘ ┌─▽─┐ │END│ └───┘ % ----- end ---- % fourth % ----- start ---- ┌─────┐ │START│ └──┬──┘ ┌──▽──┐ │a = 0│ └──┬──┘ ┌──▽──┐ │i = 1│ └──┬──┘ _▽___ ╱ ╲ ____╱ i < 2 ╲______________________________________________________ │ no ╲ ╱ │ │ ╲_____╱ │ │ │yes │ │ ___▽___ _____ │ │ ╱ ╲ ╱ ╲ ┌───────┐ │ │ ╱ s == 10 ╲______________________╱ t > 5 ╲____________│a = a+0│ │ │ ╲ ╱yes ╲ ╱yes └───┬───┘ │ │ ╲_______╱ ╲_____╱ │ │ │ │no │no │ │ │ __▽__ __▽__ │ │ │ ╱ ╲ ┌────────┐ ╱ ╲ ┌───────┐ │ │ │ ╱ t > 5 ╲_____________│a = a+32│╱ t > 2 ╲___│a = a+2│ │ │ │ ╲ ╱yes └────┬───┘╲ ╱yes└───┬───┘ │ │ │ ╲_____╱ │ ╲_____╱ │ │ │ │ │no │ │no │ │ │ │ __▽__ │ ┌───▽───┐ │ │ │ │ ╱ ╲ ┌────────┐ │ │a = a+4│ │ │ │ │ ╱ t > 1 ╲___│a = a+20│ │ └───┬───┘ │ │ │ │ ╲ ╱yes└────┬───┘ │ │ │ │ │ │ ╲_____╱ │ │ │ │ │ │ │ │no │ │ │ │ │ │ │ ┌────▽───┐ │ │ │ │ │ │ │ │a = a+11│ │ │ │ │ │ │ │ └────┬───┘ │ │ │ │ │ │ │ └────────────┴─┬───────┴────────┴───────────┴────────┘ │ │ ┌───▽───┐ │ │ │i = i+1│ │ │ └───┬───┘ │ │ └───────────────────────────────────────────┘ ┌─▽─┐ │END│ └───┘ % ----- end ---- % fifth % ----- start ---- ┌─────┐ │START│ └──┬──┘ ┌──▽──┐ │a = 0│ └──┬──┘ ┌──▽──┐ │i = 1│ └──┬──┘ _▽___ ╱ ╲ ____╱ i < 6 ╲_________________________________________________________ │ no ╲ ╱ │ │ ╲_____╱ │ │ │yes │ │ __▽___ _____ │ │ ╱ ╲ ╱ ╲ ┌───────┐ │ │ ╱ t > 10 ╲__________________________________╱ s > 5 ╲___│a = a+2│ │ │ ╲ ╱yes ╲ ╱yes└───┬───┘ │ │ ╲______╱ ╲_____╱ │ │ │ │no │no │ │ │ ___▽___ ______ ┌────▽───┐ │ │ │ ╱ ╲ ╱ ╲ ┌───────┐│a = a+40│ │ │ │ ╱ t > 5 ╲___________╱ s == 5 ╲___│a = a+3│└────┬───┘ │ │ │ ╲ ╱yes ╲ ╱yes└───┬───┘ │ │ │ │ ╲_______╱ ╲______╱ │ │ │ │ │ │no │no │ │ │ │ │ __▽__ ┌───▽───┐ │ │ │ │ │ ╱ ╲ ┌───────┐│a = a+1│ │ │ │ │ │ ╱ s < 5 ╲___│a = a+2│└───┬───┘ │ │ │ │ │ ╲ ╱yes└───┬───┘ │ │ │ │ │ │ ╲_____╱ │ │ │ │ │ │ │ │no │ │ │ │ │ │ │ ┌───▽───┐ │ │ │ │ │ │ │ │a = a+1│ │ │ │ │ │ │ │ └───┬───┘ │ │ │ │ │ │ │ └───────────┴───┬────┴────────────┴─────────┴───────────┘ │ │ ┌───▽───┐ │ │ │i = i+1│ │ │ └───┬───┘ │ │ └─────────────────────────────────────────────┘ ┌─▽─┐ │END│ └───┘ % ----- end ----
And then there are the scripts that were used to generate the flowcharts in Diagon, diagon_scripts.txt. These aren't used directly by the unit tester. Here is the script:
% diagon_scripts.txt % first % ----- start ---- /* s and t are inputs */ "START"; "a = 0"; "i=1"; while("i<10") { if("s >= 0"){ if("t < 10") { "a = a+2"; } else if("t<20") { "a = a+4"; } else { "a = a+5"; } } else{ if("t < 5") { "a = a+3"; } else if("t<10") { "a = a+2"; } else { "a = a+1"; } } "i = i+1"; } "END"; % ----- end ---- % second % ----- start ---- /* s and t are inputs */ "START"; "a = 0"; "i = 1"; while("i < 5") { if("s <= 10"){ if("t < 5") { "a = a+1"; } else if("t < 10") { "a = a+5"; } else { "a = a+2"; } } else{ if("t > 20") { "a = a+32"; } else if("t > 13") { "a = a+20"; } else { "a = a+11"; } } "i = i+1"; } "END"; % ----- end ---- % third % ----- start ---- "START"; "a = 0"; "i = 1"; while("i < 10") { if("s <= 5"){ if("t < 2") { "a = a+3"; } else if("t < 8") { "a = a+2"; } else { "a = a+1"; } } else{ if("t > 10") { "a = a+3"; } else if("t > 5") { "a = a+2"; } else { "a = a+1"; } } "i = i+1"; } "END"; % ----- end ---- % fourth % ----- start ---- /* s and t are inputs */ "START"; "a = 0"; "i = 1"; while("i < 2") { if("s == 10"){ if("t > 5") { "a = a+0"; } else if("t > 2") { "a = a+2"; } else { "a = a+4"; } } else{ if("t > 5") { "a = a+32"; } else if("t > 1") { "a = a+20"; } else { "a = a+11"; } } "i = i+1"; } "END"; % ----- end ---- % fifth % ----- start ---- /* s and t are inputs */ "START"; "a = 0"; "i = 1"; while("i < 6") { if("t > 10"){ if("s > 5") { "a = a+2"; } else { "a = a+40"; } } else if(" t > 5 "){ if("s == 5") { "a = a+3"; } else { "a = a+1"; } } else{ if("s < 5") { "a = a+2"; } else { "a = a+1"; } } "i = i+1"; } "END"; % ----- end ----