In another post, I set up a basic Virtual Programming Lab exercise for RISC-V simulations with Assembler coding and the Kite simulator. That example is vulnerable to students hard-coding the register values without actually writing the necessary code. But I wrote it in order to provide a solid first example.
Here, we're going to write an exercise in VPL that is harder to fool. We'll do two things to accomplish this. We will...
- Initialize all register values to random values, and
- Allow for specific registers to have specific initial values.
(In a future blog post I'll deal with running the simulation multiple times, each time with a different set of randomized register values.)
This works because of two important elements:
- The VPL system typically relies on two bash shell scripts to work: vpl_run.sh and vpl_evaluate.sh.
- The Kite simulator loads a pair of text files when it starts up. The first file provides the starting state of the registers and the second file provides the starting state of a bank of memory. (reg_state and mem_state, respectively)
So, we'll create a bash script that will initialize that register file to random values. Now, I'm not a bash scripting person, so I turned to Bing to help me out. Here is the basic code for generating 30 registers (x0 to x29). I'll modify this later to permit all 32 registers within vpl_run.sh and vpl_evaluate.sh.
# File created with Bing/ChatGPT
#!/bin/bash
# Define the output file. (the register state file, reg_state)
output_file="reg_state"
# Define an associative array for specific values
declare -A specific_values
# Example of setting specific values (uncomment and modify as needed)
# specific_values[5]=12345 # Assign 12345 to the 6th line (index 5)
# specific_values[10]=54321 # Assign 54321 to the 11th line (index 10)
# Function to generate a random 16-bit integer
generate_random_16bit() {
echo $((RANDOM % 32768))
}
# Create or clear the output file
> "$output_file"
# Loop to create 30 lines
for i in {0..29}; do
if [[ -n "${specific_values[$i]}" ]]; then
value="${specific_values[$i]}"
else
value=$(generate_random_16bit)
fi
echo "x$i = $value" >> "$output_file"
done
echo "File created: $output_file"This code will generate a file called file.txt with 30 lines in it. Each line will be of the form A = B, where A is a value like x0, x1, x2, etc. and B will be some random value. Now, there are situations where an instructor might want to hard code a register value. The instructor can then write something like specific_values[5]=12345 to set register x5 to 12345.
Here's a student problem:
Student task: Take the negative value found in register x6, invert all the bits in register x6 and then add one to the answer. Store that final value in x5.
(this is a variation of existing lab problem, aex3a: " Convert -5 to +5 by negating its bits and adding 1. Save your solution as a file named aex3a.asm for possible future use."
You code up the secret teacher solution as this:
xori x7, x6, -1
addi x5, x7, 1Call this file teacherAssemblerSolution.asm and upload it to the Executable Files section of the VPL exercise.
Now, in VPL, we need to set the vpl_run.sh file:
#!/bin/bash
#
# vpl_run.sh script (KITE RISCV version)
# The output of the student program is tested against the expected output file.
#
# James Andrew Smith; drsmith@yorku.ca August 30, 2024.
#
# Based on example by D. Thiebaut at Smith College
# http://www.science.smith.edu/dftwiki/index.php/Tutorial:_Moodle_VPL_--_Testing_a_C_Program
# and
# https://www.mapleprimes.com/posts/38232-Scripting-With-Maple
#
# Modified for Maple and described here: https://www.yorku.ca/professor/drsmith/2021/08/01/vpl-using-maple-for-math-assignments/
#
# Updated in August 2024 for Kite simulation and RISC-V.
# A small section of this used Bing/ChatGPT to generate random values for the registers.
# -----------------------------------------------------------------
cat > vpl_execution <<EEOOFF
#!/bin/bash
# Variables (make changes here)
teacherSolution=teacherAssemblerSolution.asm
studentProgram=StudentSubmission.asm
maxGrade=1
minGrade=0
consolationGrade=0
init_register_file="reg_state"
# --- randomize the register state file, reg_state ------
# this section of the bash script was generated used Bing/ChatGPT
# vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
# Generate an array of 32 random 16-bit unsigned integers
declare -a arr
for i in {0..31}; do
arr[i]=\$((RANDOM % 65536))
done
# Override specific elements with constant values (example: arr[0] = 1234, arr[15] = -5678)
arr[0]=1234 # x1 is fixed to 1234
arr[15]=-5678 # x15 is fixed to -5678
arr[6]=\$(( \$RANDOM - 32768 )) # x6 is a random negative number.
# Create the string with 32 lines in the form A = B
output=""
for i in {0..31}; do
output+="x\$i = \${arr[i]}\n"
done
# ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
# -------------------------------------------------------------------
# --- create test (Teacher Solution) ---
# Write the string to reg_state.txt. (to initialise registers)
echo -e "\$output" > "\$init_register_file"
# run Kite on the teacher solution. Output to solution.txt. Extra text to /dev/null
echo "running KITE on teacher solution..."
/eecs/local/pkg/kite/bin/kite \${teacherSolution} > solutionTemp.txt
# erase all lines prior to the string
sed -i '1,/Register state:/d' solutionTemp.txt
# erase all lines at the end of the file after "elapsed":
sed -n '/elapsed/q;p' solutionTemp.txt > solution.txt
# Default grade is zero.
grade=\$minGrade
for i in 1 ; do
echo "TEST \$i"
# ==============================================
# TEST i
# ==============================================
# Write the string to reg_state.txt. (to initialise registers)
echo -e "\$output" > "\$init_register_file"
echo "Running KITE on the student solution..."
/eecs/local/pkg/kite/bin/kite \${studentProgram} > userTemp.out
# erase all lines prior to the string
sed -i '1,/Register state:/d' userTemp.out
# erase all lines at the end of the file after "elapsed":
sed -n '/elapsed/q;p' userTemp.out > user.out
# all processing of whitespace to remove it has been, itself removed.
# want to see how that's done? Go to
# http://www.science.smith.edu/dftwiki/index.php/Tutorial:_Moodle_VPL_--_Testing_a_C_Program
#--- compute difference, with exact whitespace ---
diff -y user.out solution.txt > diff.out
# debug:
# echo "----- diff.out ------"
# cat diff.out
# echo "---------------------"
#--- reject if different ---
if ((\$? > 0)); then
echo "Your output is NOT correct."
echo "---------------"
echo "Your output:"
echo "---------------"
cat user.out
echo ""
echo "---------------"
echo "Expected output "
echo "---------------"
cat solution.txt
grade=\$( expr \$minGrade )
#--- consolation grade ("thanks for coming out")---
#grade=\$( expr \$consolationGrade )
# --------------------- REWARD IF CORRECT OUTPUT -----------------
else
#--- good output ---
echo "Congrats, your output is correct."
echo " --------------------------------"
cat user.out
echo "--------------------------------"
grade=\$( expr \$maxGrade )
fi
done
# Uncomment this for the vpl_run.sh
# Comment it for vpl_evaluate.sh
echo "Possible Grade : \$grade (you need to evaluate for official grade)"
# Uncomment this for vpl_evaluate.sh. Otherwise comment it.
#echo "The following grade is being sent to eClass: \$grade"
#echo "Grade :=>> \$grade"
exit 0
EEOOFF
chmod +x vpl_execution
exit 0We also need to define the vpl_evaluation.sh script:
#!/bin/bash
#
# vpl_evaluate script (KITE RISCV version)
# The output of the student program is tested against the expected output file.
#
# James Andrew Smith; drsmith@yorku.ca August 30, 2024.
#
# Based on example by D. Thiebaut at Smith College
# http://www.science.smith.edu/dftwiki/index.php/Tutorial:_Moodle_VPL_--_Testing_a_C_Program
# and
# https://www.mapleprimes.com/posts/38232-Scripting-With-Maple
#
# Modified for Maple and described here: https://www.yorku.ca/professor/drsmith/2021/08/01/vpl-using-maple-for-math-assignments/
#
# Updated in August 2024 for Kite simulation and RISC-V.
# A small section of this used Bing/ChatGPT to generate random values for the registers.
# -----------------------------------------------------------------
cat > vpl_execution <<EEOOFF
#!/bin/bash
# Variables (make changes here)
teacherSolution=teacherAssemblerSolution.asm
studentProgram=StudentSubmission.asm
maxGrade=1
minGrade=0
consolationGrade=0
init_register_file="reg_state"
# --- randomize the register state file, reg_state ------
# this section of the bash script was generated used Bing/ChatGPT
# vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
# Generate an array of 32 random 16-bit unsigned integers
declare -a arr
for i in {0..31}; do
arr[i]=\$((RANDOM % 65536))
done
# Override specific elements with constant values (example: arr[0] = 1234, arr[15] = -5678)
arr[0]=1234 # x1 is fixed to 1234
arr[15]=-5678 # x15 is fixed to -5678
arr[6]=\$(( \$RANDOM - 32768 )) # x6 is a random negative number.
# Create the string with 32 lines in the form A = B
output=""
for i in {0..31}; do
output+="x\$i = \${arr[i]}\n"
done
# ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
# -------------------------------------------------------------------
# --- create test (Teacher Solution) ---
# Write the string to reg_state.txt. (to initialise registers)
echo -e "\$output" > "\$init_register_file"
# run Kite on the teacher solution. Output to solution.txt. Extra text to /dev/null
echo "running KITE on teacher solution..."
/eecs/local/pkg/kite/bin/kite \${teacherSolution} > solutionTemp.txt
# erase all lines prior to the string
sed -i '1,/Register state:/d' solutionTemp.txt
# erase all lines at the end of the file after "elapsed":
sed -n '/elapsed/q;p' solutionTemp.txt > solution.txt
# Default grade is zero.
grade=\$minGrade
for i in 1 ; do
echo "TEST \$i"
# ==============================================
# TEST i
# ==============================================
# Write the string to reg_state.txt. (to initialise registers)
echo -e "\$output" > "\$init_register_file"
echo "Running KITE on the student solution..."
/eecs/local/pkg/kite/bin/kite \${studentProgram} > userTemp.out
# erase all lines prior to the string
sed -i '1,/Register state:/d' userTemp.out
# erase all lines at the end of the file after "elapsed":
sed -n '/elapsed/q;p' userTemp.out > user.out
# all processing of whitespace to remove it has been, itself removed.
# want to see how that's done? Go to
# http://www.science.smith.edu/dftwiki/index.php/Tutorial:_Moodle_VPL_--_Testing_a_C_Program
#--- compute difference, with exact whitespace ---
diff -y user.out solution.txt > diff.out
# debug:
# echo "----- diff.out ------"
# cat diff.out
# echo "---------------------"
#--- reject if different ---
if ((\$? > 0)); then
echo "Your output is NOT correct."
echo "---------------"
echo "Your output:"
echo "---------------"
cat user.out
echo ""
echo "---------------"
echo "Expected output "
echo "---------------"
cat solution.txt
grade=\$( expr \$minGrade )
#--- consolation grade ("thanks for coming out")---
#grade=\$( expr \$consolationGrade )
# --------------------- REWARD IF CORRECT OUTPUT -----------------
else
#--- good output ---
echo "Congrats, your output is correct."
echo " --------------------------------"
cat user.out
echo "--------------------------------"
grade=\$( expr \$maxGrade )
fi
done
# Uncomment this for the vpl_run.sh
# Comment it for vpl_evaluate.sh
# echo "Possible Grade : \$grade (you need to evaluate for official grade)"
# Uncomment this for vpl_evaluate.sh. Otherwise comment it.
echo "The following grade is being sent to eClass: \$grade"
echo "Grade :=>> \$grade"
exit 0
EEOOFF
chmod +x vpl_execution
exit 0
Then, we need to set VPL "local execution server".
Specify the requested file. Call it StudentSubmission.asm
Copy the two memory files: mem_state and reg_state, either from the web simulator or from the GitHub respository. Upload these to the Execution Files folder. In the Files to Keep when Running page, check off reg_state, mem_state and teacherAssemblerSolution.asm.
In Execution Options, make sure that run script is on autodetect. Click yes for Run and yes for Evaluate. Click no for "evaluate just on submission" and yes for "automatic grade".
Conclusion
So, we've now got a proof-of-concept working for Kite simulations with RISC-V assembler student programs working within Virtual Programming Lab. I'll have to clean this up, make a better tutorial and intro video, but I wanted to post this while it was still on my mind.
Acknowledgements
I would like to thank Mr. Paul Griffith and Mr. Jason Keltz for helping get the RISC-V tools set up on our EECS department machines. Colleagues like Mr. Griffith and Keltz are true anchors for departments, like ours, that are so dependent on technology for both teaching and research.
James Andrew Smith is a Professional Engineer and Associate Professor in the Electrical Engineering and Computer Science Department of York University’s Lassonde School, with degrees in Electrical and Mechanical Engineering from the University of Alberta and McGill University. Previously a program director in biomedical engineering, his research background spans robotics, locomotion, human birth and engineering education. While on sabbatical in 2018-19 with his wife and kids he lived in Strasbourg, France and he taught at the INSA Strasbourg and Hochschule Karlsruhe and wrote about his personal and professional perspectives. James is a proponent of using social media to advocate for justice, equity, diversity and inclusion as well as evidence-based applications of research in the public sphere. You can find him on Twitter. Originally from Québec City, he now lives in Toronto, Canada.