This may not answer your question but it may provide an alternative
solution.
I had the same challenge that you an year ago so may be my solution will
work for you too.
Imagine that you have a Markdown file that *documents* the expected
results.
--8<---------------cut here---------------start------------->8---
This is the final exam, good luck!
First I'm going to load your code (the student's code):
```python
import student
```
Let's see if you programmed correctly a sort algorithm
```python
data = [3, 2, 1, 3, 1, 9]
student.sort_numbers(data)
[1, 1, 2, 3, 3, 9]
```
Let's now if you can choose the correct answer:
```python
t = ["foo", "bar", "baz"]
student.question1(t)
"baz"
```
--8<---------------cut here---------------end--------------->8---
Now you can run the snippets of code with:
byexample -l python the_markdown_file.md
What byexample does is to run the Python code, capture the output and
compare it with the expected result.
In the above example "student.sort_numbers" must return the list sorted.
That output is compared by byexample with the list written below.
Advantages? Each byexample run is independent of the other and the
snippet of codes are executed in a separated Python process. byexample
takes care of the IPC.
I don't know the details of your questions so I'm not sure if byexample
will be the tool for you. In my case I evaluate my students giving them
the Markdown and asking them to code the functions so they return the
expected values.
Depending of how many students you have you may considere to complement
this with INGInious. It is designed to run students' assignments
assuming nothing on the untrusted code.
Links:
https://byexamples.github.io/byexample/
https://docs.inginious.org/en/v0.7/
On Sun, Aug 15, 2021 at 12:09:58PM -0300, Hope Rouselle wrote:
Hope Rouselle <hrouse...@jevedi.com> writes:
[...]
Of course, you want to see the code. I need to work on producing a
small example. Perhaps I will even answer my own question when I do.
[...]
Here's a small-enough case. We have two students here. One is called
student.py and the other is called other.py. They both get question 1
wrong, but they --- by definition --- get question 2 right. Each
question is worth 10 points, so they both should get losses = 10.
(*) Student student.py
--8<---------------cut here---------------start------------->8---
def question1(t): # right answer is t[2]
return t[1] # lack of attention, wrong answer
--8<---------------cut here---------------end--------------->8---
(*) Student other.py
--8<---------------cut here---------------start------------->8---
def question1(t): # right answer is t[2]
return t[0] # also lack of attention, wrong answer
--8<---------------cut here---------------end--------------->8---
(*) Grading
All is good on first run.
Python 3.5.2 [...] on win32
[...]
reproducible_problem()
student.py, total losses 10
other.py, total losses 10
The the problem:
reproducible_problem()
student.py, total losses 0
other.py, total losses 0
They lose nothing because both modules are now permanently modified.
(*) The code of grading.py
--8<---------------cut here---------------start------------->8---
# -*- mode: python; python-indent-offset: 2 -*-
def key_question1(t):
# Pretty simple. Student must just return index 2 of a tuple.
return t[2]
def reproducible_problem(): # grade all students
okay, m = get_student_module("student.py")
r = grade_student(m)
print("student.py, total losses", r) # should be 10
okay, m = get_student_module("other.py")
r = grade_student(m)
print("other.py, total losses", r) # should be 10
def grade_student(m): # grades a single student
losses = question1_verifier(m)
losses += question2_verifier(m)
return losses
def question1_verifier(m):
losses = 0
if m.question1( (0, 1, 2, 3) ) != 2: # wrong answer
losses = 10
return losses
def question2_verifier(m):
m.question1 = key_question1
# To grade question 2, we overwrite the student's module by giving
# it the key_question1 procedure. This way we are able to let the
# student get question 2 even if s/he got question 1 incorrect.
losses = 0
return losses
def get_student_module(fname):
from importlib import import_module
mod_name = basename(fname)
try:
student = import_module(mod_name)
except Exception as e:
return False, str(e)
return True, student
def basename(fname): # drop the the .py extension
return "".join(fname.split(".")[ : -1])
--8<---------------cut here---------------end--------------->8---
--
https://mail.python.org/mailman/listinfo/python-list
--
https://mail.python.org/mailman/listinfo/python-list
