{"id":3178,"date":"2023-11-14T21:45:55","date_gmt":"2023-11-15T02:45:55","guid":{"rendered":"https:\/\/www.yorku.ca\/professor\/drsmith\/?p=3178"},"modified":"2023-11-15T09:51:02","modified_gmt":"2023-11-15T14:51:02","slug":"automated-grading-of-python-in-vpl-using-unit-tests","status":"publish","type":"post","link":"https:\/\/www.yorku.ca\/professor\/drsmith\/2023\/11\/14\/automated-grading-of-python-in-vpl-using-unit-tests\/","title":{"rendered":"Automated Grading of Python in VPL using Unit Tests"},"content":{"rendered":"\n<p>A few of my colleagues here at YorkU have expressed an interest in using Python with Virtual Programming Lab on our Moodle-based eClass Learning Management System.  I don't have a lot of Python experience, having only really used it for some data processing and toy embedded system demonstrations in the past.<\/p>\n\n\n\n<p>So I adapted the Unit Test examples that I did earlier this summer for C (blog posts <a href=\"https:\/\/www.yorku.ca\/professor\/drsmith\/2023\/07\/31\/automated-student-evaluations-in-c-part-1\/\">1<\/a>, <a href=\"https:\/\/www.yorku.ca\/professor\/drsmith\/2023\/08\/19\/automated-student-evaluations-in-c-part-2\/\">2<\/a>, <a href=\"https:\/\/www.yorku.ca\/professor\/drsmith\/2023\/08\/19\/automated-student-evaluations-in-c-part-3\/\">3<\/a>, <a href=\"https:\/\/www.yorku.ca\/professor\/drsmith\/2023\/08\/19\/automated-student-evaluations-in-c-part-4\/\">4<\/a> and<a href=\"https:\/\/www.yorku.ca\/professor\/drsmith\/2023\/08\/19\/automated-student-evaluations-in-c-part-5\/\"> 5<\/a>; original source <a href=\"https:\/\/web.archive.org\/web\/20190208191644\/http:\/\/www.science.smith.edu\/dftwiki\/index.php\/Tutorial:_Moodle_VPL_--_Testing_a_C_Program\">D. Thi\u00e9baut<\/a>) and had Bing \/ ChatGPT write up a sample function and a unit testing Python program for it.  I then adapted it for VPL and the result is here.  I'm going to leave out important details like:<\/p>\n\n\n\n<ol class=\"wp-block-list\"><li>How to set up a VPL exercise on eClass<\/li><li>How to point the VPL exercise to our local VPL server<\/li><\/ol>\n\n\n\n<p>as I've discussed those elsewhere and I just want to get the main scripts and Python files listed here.  Again, I used ChatGPT to create the Python files, so I don't claim any authorship there.  I'm sure that someone with actual Python experience can do a better job and will use the baseline here to get started.<\/p>\n\n\n\n<p>So, what's important here is to have two Python files:<\/p>\n\n\n\n<ol class=\"wp-block-list\"><li>primaryFile.py<ul><li>This is what the teacher runs to test the student's submission.  It contains the unit test and outputs \"PASS\" or \"FAIL\" that get captured by the VPL script files.<\/li><\/ul><\/li><li>StudentSubmission.py <ul><li>This is the file that the student is supposed to write.  Here, it just adds to input parameters and outputs the result (c = a+b)<\/li><\/ul><\/li><\/ol>\n\n\n\n<p>In addition, we have two scripts in VPL:<\/p>\n\n\n\n<ol class=\"wp-block-list\"><li>vpl_run.sh<ul><li>This script will run primaryFile.py and will let the student know if the test passed or failed.  No grade recorded.<\/li><\/ul><\/li><li>vpl_evaluate.sh<ul><li>This script does the same as vpl_run.sh but will record a grade (1 or 0) and send it to the eClass grade book.<\/li><\/ul><\/li><\/ol>\n\n\n\n<h2 class=\"wp-block-heading\">Video Overview<\/h2>\n\n\n\n<p>Here is a video overview of how to set up this Python example in VPL.<\/p>\n\n\n\n<figure class=\"wp-block-embed is-type-video is-provider-youtube wp-block-embed-youtube wp-embed-aspect-16-9 wp-has-aspect-ratio\"><div class=\"wp-block-embed__wrapper\">\n<iframe loading=\"lazy\" title=\"Virtual Programming Lab: Python with Unit Tests\" width=\"1778\" height=\"1000\" src=\"https:\/\/www.youtube.com\/embed\/oRcdxmyogpE?feature=oembed\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share\" allowfullscreen><\/iframe>\n<\/div><figcaption>Video on how to set up Python with Unit Tests in VPL. <a href=\"https:\/\/youtu.be\/oRcdxmyogpE\">https:\/\/youtu.be\/oRcdxmyogpE<\/a><\/figcaption><\/figure>\n\n\n\n<h2 class=\"wp-block-heading\">Files<\/h2>\n\n\n\n<p>Here are the four important files for creating this example in VPL.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">primaryFile.py<\/h2>\n\n\n\n<p>This i<span style=\"font-size: revert;\">s what the teacher runs to test the student's submission.  It contains the unit test and outputs \"PASS\" or \"FAIL\" that get captured by the VPL script files.<\/span><\/p>\n\n\n\n<pre class=\"wp-block-code\"><code># generated in Bing \/ ChatGPT\n# Import the unittest module\nimport unittest\n\n# Import the StudentSubmission.py file\nimport StudentSubmission\n\n# Define a class that inherits from unittest.TestCase\nclass TestAdd(unittest.TestCase):\n\n    # Define a method that starts with test_\n    def test_add(self):\n        # Use self.assertEqual to compare the expected and actual results\n        self.assertEqual(StudentSubmission.add(1, 2), 3) # Pass\n        self.assertEqual(StudentSubmission.add(-1, 1), 0) # Pass\n        self.assertEqual(StudentSubmission.add(2, 2), 4) # Pass\n\n# Run the unit test\nif __name__ == '__main__':\n    # Create a test suite\n    suite = unittest.TestSuite()\n    # Add the test method to the suite\n    suite.addTest(TestAdd('test_add'))\n    # Create a test runner\n    runner = unittest.TextTestRunner()\n    # Run the test suite and capture the result\n    result = runner.run(suite)\n    # Check if the result was successful\n    if result.wasSuccessful():\n        # Output the word pass\n        print(\"PASS\")\n    else:\n        # Output the word fail\n        print(\"FAIL\")\n<\/code><\/pre>\n\n\n\n<h2 class=\"wp-block-heading\">StudentSubmission.py<\/h2>\n\n\n\n<p>Put this in \"requested files\" and pretend the student submitted it.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code># Generated in ChatGPT \/ Bing.\n# Define a function named add that takes two numbers as parameters and returns their sum\ndef add(a, b):\n    # Add the two numbers and store the result in a variable named c\n    c = a + b\n    # Return the value of c\n    return c<\/code><\/pre>\n\n\n\n<h2 class=\"wp-block-heading\">vpl_run.sh<\/h2>\n\n\n\n<p>This is the first script that tests the student code.  It uses the rocket button in VPL.  No grade is assigned.  It will show if the student code in StudentSubmission.py passes or fails the unit tests.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>#!\/bin\/bash\n#\n# vpl_run.sh script \n# An approach for Unit Tested Python programs in VPL, based on similar work for C.\n# Uses bash script and looks for PASS or FAIL from the unit test. \n#\n# \n# James Andrew Smith; drsmith@yorku.ca July 2021 (updated August 2023)\n#\n#\n# Based on an approach to unit testing for C (July &amp; August 2023)\n# 1. https:\/\/www.yorku.ca\/professor\/drsmith\/2023\/07\/31\/automated-student-evaluations-in-c-part-1\/\n# 2. https:\/\/www.yorku.ca\/professor\/drsmith\/2023\/08\/19\/automated-student-evaluations-in-c-part-2\/\n# 3. https:\/\/www.yorku.ca\/professor\/drsmith\/2023\/08\/19\/automated-student-evaluations-in-c-part-3\/\n# 4. https:\/\/www.yorku.ca\/professor\/drsmith\/2023\/08\/19\/automated-student-evaluations-in-c-part-4\/\n# 5. https:\/\/www.yorku.ca\/professor\/drsmith\/2023\/08\/19\/automated-student-evaluations-in-c-part-5\/\n#\n# That C work was based on example by D. Thi\u00e9baut at Smith College\n# http:\/\/www.science.smith.edu\/dftwiki\/index.php\/Tutorial:_Moodle_VPL_--_Testing_a_C_Program\n# now at\n# https:\/\/web.archive.org\/web\/20190208191644\/http:\/\/www.science.smith.edu\/dftwiki\/index.php\/Tutorial:_Moodle_VPL_--_Testing_a_C_Program\n#\n# August 2023 update: removed the SED whitespace processing as (1) it messes with\n# the ASCII art from the flowchart and (2) it's not necessary with the intoduction\n# of the Unity unit tester.\n# -----------------------------------------------------------------\n\n\ncat &gt; vpl_execution &lt;&lt;EEOOFF\n#!\/bin\/bash\n\n# the number of times we need to see a unit test pass.\nNUM_PASSES=1    # TO-DO : update this (in vpl_run and _evaluate) \n\n# output file from execution of the main program\nOUTFILE=\"user.out\"\n\n# Variables (don't add whitespace)\nmaxGrade=1\nminGrade=0\nconsolationGrade=0\n \n# Set the minimum grade.  (slashes due to this script writing vpl_executable)\ngrade=\\$( expr \\$minGrade )\n\n# --- program tested  ---\n# \npython3 primaryFile.py &amp;&gt; \\$OUTFILE\n\n# If the file contains the string \"PASS\" a certain number of times (set above)\n# but also check to see that \"FAIL\" is not returned.\nif &#91; \\$(grep -o \"PASS\" \\$OUTFILE | wc -l) -eq \\$NUM_PASSES ]; then\n    # Check if the file does not contain the string \"FAIL\"\n    if ! grep -q \"FAIL\" \\$OUTFILE; then\n        echo \"The test on your code succeeded.\"\n    else\n        echo \"The test on your code did _not_ succeed.\"\n    fi\nelse\n    echo \"The test on your code did _not_ succeed.\"\nfi\n\necho \"***********************\"\necho \"The test on the student's code returned the following message:\"\ncat \\$OUTFILE\necho \"***********************\"\n\necho \"***********************\"\necho \"No grade has been saved.\"\necho \"Use the evaluation button for grading.\"\necho \"***********************\"\n\nEEOOFF\n\n \nchmod +x vpl_execution<\/code><\/pre>\n\n\n\n<h2 class=\"wp-block-heading\">vpl_evaluate.sh<\/h2>\n\n\n\n<p>This is the script that students execute by clicking on the checkmark icon.  It will submit a grade (0 or 1) to eClass based on whether the student's submission met the requirements of the unit tests.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>#!\/bin\/bash\n#\n# vpl_run.sh script \n# An approach for Unit Tested Python programs in VPL, based on similar work for C.\n# Uses bash script and looks for PASS or FAIL from the unit test. \n#\n# \n# James Andrew Smith; drsmith@yorku.ca July 2021 (updated August 2023)\n#\n#\n# Based on an approach to unit testing for C (July &amp; August 2023)\n# 1. https:\/\/www.yorku.ca\/professor\/drsmith\/2023\/07\/31\/automated-student-evaluations-in-c-part-1\/\n# 2. https:\/\/www.yorku.ca\/professor\/drsmith\/2023\/08\/19\/automated-student-evaluations-in-c-part-2\/\n# 3. https:\/\/www.yorku.ca\/professor\/drsmith\/2023\/08\/19\/automated-student-evaluations-in-c-part-3\/\n# 4. https:\/\/www.yorku.ca\/professor\/drsmith\/2023\/08\/19\/automated-student-evaluations-in-c-part-4\/\n# 5. https:\/\/www.yorku.ca\/professor\/drsmith\/2023\/08\/19\/automated-student-evaluations-in-c-part-5\/\n#\n# That C work was based on example by D. Thi\u00e9baut at Smith College\n# http:\/\/www.science.smith.edu\/dftwiki\/index.php\/Tutorial:_Moodle_VPL_--_Testing_a_C_Program\n# now at\n# https:\/\/web.archive.org\/web\/20190208191644\/http:\/\/www.science.smith.edu\/dftwiki\/index.php\/Tutorial:_Moodle_VPL_--_Testing_a_C_Program\n#\n# August 2023 update: removed the SED whitespace processing as (1) it messes with\n# the ASCII art from the flowchart and (2) it's not necessary with the intoduction\n# of the Unity unit tester.\n# -----------------------------------------------------------------\n\n\ncat > vpl_execution &lt;&lt;EEOOFF\n#!\/bin\/bash\n\n# the number of times we need to see a unit test pass.\nNUM_PASSES=1    # TO-DO : update this (in vpl_run and _evalute) \n\n# output file from execution of the main program\nOUTFILE=\"user.out\"\n\n# Variables (don't add whitespace)\nmaxGrade=1\nminGrade=0\nconsolationGrade=0\n \n# Set the minimum grade.  (slashes due to this script writing vpl_executable)\ngrade=\\$( expr \\$minGrade )\n\n# --- program tested (no extension) ---\n# \npython3 primaryFile.py &amp;> \\$OUTFILE\n\n# If the file contains the string \"PASS\" a certain number of times (set above)\n# but also check to see that \"FAIL\" is not returned.\nif &#91; \\$(grep -o \"PASS\" \\$OUTFILE | wc -l) -eq \\$NUM_PASSES ]; then\n    # Check if the file does not contain the string \"FAIL\"\n    if ! grep -q \"FAIL\" \\$OUTFILE; then\n      echo \"Comment :=>>- Congratulations: your function is CORRECT.\"\n      echo \"Comment :=>> -----------------\"\n      echo \"Comment :=>>- The result of the testing program, with the student function included:\"\n      echo \"&lt;|--\"\n      cat user.out\n      echo \"\"\n      echo \"--|>\"\n      grade=\\$( expr \\$maxGrade )\n    else\n      echo \"Comment :=>>- Your program is NOT correct. Fix it.\"\n      echo \"Comment :=>> ---------------\"\n      echo \"Comment :=>>- The result of the testing program, with the student function included:\"\n      echo \"Comment :=>> ---------------\"\n      echo \"&lt;|--\"\n      cat user.out\n      grade=\\$( expr \\$minGrade )\n    fi\nelse\n      echo \"Comment :=>>- Your program is NOT correct. Fix it.\"\n      echo \"Comment :=>> ---------------\"\n      echo \"Comment :=>>- The result of the testing program, with the student function included:\"\n      echo \"Comment :=>> ---------------\"\n      echo \"&lt;|--\"\n      cat user.out\n      grade=\\$( expr \\$minGrade )\nfi\n   \n\n# Send the grade to eClass via a special-formatted comment.\necho \"The following grade is being sent to eClass: \\$grade\"\necho \"Grade :=>> \\$grade\"\n\nEEOOFF\n\n \nchmod +x vpl_execution\n\n<\/code><\/pre>\n\n\n\n<p>Edits:<\/p>\n\n\n\n<p>November 15, 2023: added the <a href=\"https:\/\/youtu.be\/oRcdxmyogpE\">YouTube video<\/a> and some support text.<\/p>\n\n\n\n<hr class=\"wp-block-separator\"\/>\n\n\n\n<div class=\"wp-block-image\"><figure class=\"alignleft size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"128\" height=\"128\" src=\"https:\/\/www.yorku.ca\/professor\/drsmith\/wp-content\/uploads\/sites\/444\/2021\/12\/noun-signature-1720818-2.png\" alt=\"a pen\" class=\"wp-image-2116\"\/><\/figure><\/div>\n\n\n\n<p>James Andrew Smith is a Professional Engineer and Associate Professor in the Electrical Engineering and Computer Science&nbsp;<a href=\"http:\/\/eecs.lassonde.yorku.ca\">Department<\/a>&nbsp;of York University's&nbsp;<a href=\"http:\/\/lassonde.yorku.ca\">Lassonde School<\/a>, with degrees in Electrical and Mechanical Engineering&nbsp;from the University of Alberta and McGill University.&nbsp;&nbsp;Previously a program director in biomedical engineering, his research background spans robotics, locomotion, human birth and&nbsp;engineering&nbsp;education. While on sabbatical in 2018-19 with his wife and kids he lived in Strasbourg, France and&nbsp;he&nbsp;taught at the&nbsp;<a href=\"https:\/\/www.insa-strasbourg.fr\/en\/\">INSA Strasbourg<\/a>&nbsp;and&nbsp;<a href=\"https:\/\/www.hs-karlsruhe.de\">Hochschule Karlsruhe<\/a>&nbsp;and wrote about his&nbsp;<a href=\"https:\/\/twitter.com\/search?q=(%23sabbaticallife)%20(from%3Aonnimikki)&amp;src=typed_query\">personal<\/a>&nbsp;and&nbsp;<a href=\"https:\/\/twitter.com\/search?q=insa%20(from%3Ajasmith_yorku)&amp;src=typed_query\">professional&nbsp;<\/a><a href=\"https:\/\/twitter.com\/search?q=karlsruhe%20(from%3Ajasmith_yorku)&amp;src=typed_query\">perspectives<\/a>.&nbsp;&nbsp;James is a proponent of using social media to advocate for justice, equity, diversity and inclusion as well as evidence-based applications of research&nbsp;in the public sphere.&nbsp;You can find him on&nbsp;<a href=\"https:\/\/twitter.com\/jasmith_yorku\">Twitter<\/a>. Originally from Qu\u00e9bec City, he now lives in Toronto, Canada.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>A few of my colleagues here at YorkU have expressed an interest in using Python with Virtual Programming Lab on our Moodle-based eClass Learning Management System. I don't have a lot of Python experience, having only really used it for some data processing and toy embedded system demonstrations in the past. So I adapted the [&hellip;]<\/p>\n","protected":false},"author":762,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_kad_blocks_custom_css":"","_kad_blocks_head_custom_js":"","_kad_blocks_body_custom_js":"","_kad_blocks_footer_custom_js":"","footnotes":""},"categories":[47,53,55,294],"tags":[617,219,480,290,261],"class_list":["post-3178","post","type-post","status-publish","format-standard","hentry","category-eclass","category-moodle","category-programming","category-vpl","tag-bash-scripts","tag-python","tag-unit-test","tag-unit-testing","tag-virtual-programming-lab"],"taxonomy_info":{"category":[{"value":47,"label":"eClass"},{"value":53,"label":"Moodle"},{"value":55,"label":"programming"},{"value":294,"label":"VPL"}],"post_tag":[{"value":617,"label":"bash scripts"},{"value":219,"label":"python"},{"value":480,"label":"unit test"},{"value":290,"label":"unit testing"},{"value":261,"label":"virtual programming lab"}]},"featured_image_src_large":false,"author_info":{"display_name":"drsmith","author_link":"https:\/\/www.yorku.ca\/professor\/drsmith\/author\/drsmith\/"},"comment_info":"","category_info":[{"term_id":47,"name":"eClass","slug":"eclass","term_group":0,"term_taxonomy_id":47,"taxonomy":"category","description":"","parent":0,"count":32,"filter":"raw","cat_ID":47,"category_count":32,"category_description":"","cat_name":"eClass","category_nicename":"eclass","category_parent":0},{"term_id":53,"name":"Moodle","slug":"moodle","term_group":0,"term_taxonomy_id":53,"taxonomy":"category","description":"","parent":0,"count":27,"filter":"raw","cat_ID":53,"category_count":27,"category_description":"","cat_name":"Moodle","category_nicename":"moodle","category_parent":0},{"term_id":55,"name":"programming","slug":"programming","term_group":0,"term_taxonomy_id":55,"taxonomy":"category","description":"","parent":0,"count":30,"filter":"raw","cat_ID":55,"category_count":30,"category_description":"","cat_name":"programming","category_nicename":"programming","category_parent":0},{"term_id":294,"name":"VPL","slug":"vpl","term_group":0,"term_taxonomy_id":294,"taxonomy":"category","description":"","parent":0,"count":25,"filter":"raw","cat_ID":294,"category_count":25,"category_description":"","cat_name":"VPL","category_nicename":"vpl","category_parent":0}],"tag_info":[{"term_id":617,"name":"bash scripts","slug":"bash-scripts","term_group":0,"term_taxonomy_id":617,"taxonomy":"post_tag","description":"","parent":0,"count":1,"filter":"raw"},{"term_id":219,"name":"python","slug":"python","term_group":0,"term_taxonomy_id":219,"taxonomy":"post_tag","description":"","parent":0,"count":3,"filter":"raw"},{"term_id":480,"name":"unit test","slug":"unit-test","term_group":0,"term_taxonomy_id":480,"taxonomy":"post_tag","description":"","parent":0,"count":6,"filter":"raw"},{"term_id":290,"name":"unit testing","slug":"unit-testing","term_group":0,"term_taxonomy_id":290,"taxonomy":"post_tag","description":"","parent":0,"count":5,"filter":"raw"},{"term_id":261,"name":"virtual programming lab","slug":"virtual-programming-lab","term_group":0,"term_taxonomy_id":261,"taxonomy":"post_tag","description":"","parent":0,"count":14,"filter":"raw"}],"_links":{"self":[{"href":"https:\/\/www.yorku.ca\/professor\/drsmith\/wp-json\/wp\/v2\/posts\/3178","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.yorku.ca\/professor\/drsmith\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.yorku.ca\/professor\/drsmith\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.yorku.ca\/professor\/drsmith\/wp-json\/wp\/v2\/users\/762"}],"replies":[{"embeddable":true,"href":"https:\/\/www.yorku.ca\/professor\/drsmith\/wp-json\/wp\/v2\/comments?post=3178"}],"version-history":[{"count":4,"href":"https:\/\/www.yorku.ca\/professor\/drsmith\/wp-json\/wp\/v2\/posts\/3178\/revisions"}],"predecessor-version":[{"id":3182,"href":"https:\/\/www.yorku.ca\/professor\/drsmith\/wp-json\/wp\/v2\/posts\/3178\/revisions\/3182"}],"wp:attachment":[{"href":"https:\/\/www.yorku.ca\/professor\/drsmith\/wp-json\/wp\/v2\/media?parent=3178"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.yorku.ca\/professor\/drsmith\/wp-json\/wp\/v2\/categories?post=3178"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.yorku.ca\/professor\/drsmith\/wp-json\/wp\/v2\/tags?post=3178"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}