Feature request: pre-checker script

[alexandrujuncu]

In the current architecture, each assignment is associated with a virtual machine and a checker. There are situations where this modes doesn't match the needs of the assignment.

For example we want a mechanism where a submitted homework needs to run on a virtual machine and then run the same on different virtual machine. Or another case where the virtual machine is not on the checker machine and the vm should first be copied on the tester.

I am proposing a rather large feature request: a sort of a script before the checker script for the setup before the run. A pseudoscript for this could be something like:

upload assesment on vm1
result1 = vm1.runchecker()
upload assesment on vm2
result2 = vm2.runchecker()
return (result1+result2)/2

Or:

get vm1 from ssh://admin@server/home/$student/vm.zip
unzip vm1.zip
upload assesment on vm1
result = vm1.runchekcer()
return result

[mihaimaruseac]

I'd also add the fact that we also need to only run a simple script checking some MD5 for some assignments, without needing special VMs for homework grading.

[cojocar]

@mihaimaruseac : there is a SubmitOnly feature implemented. It was briefly tested. See #59.
@alexandrujuncu : Do you have a concrete example with running on two machines -- I'm not sure if I understood? Also, large submission feature is already implemented.

[alexandrujuncu]

@cojocar: the first example was for a SO1 homework: the student has to submit one archive and the code needs to be tested both on Linux and on Windows. There is only one assignment so the grade is calculated following runs on both machines.

[cojocar]

@alexandrujuncu I find it too specific for having it as feature.

[alexandrujuncu]

@cojocar that's why the feature was not specificity for that usecase, but for a flexible scripting interface (maybe a python wrapper) to cover many scenarios.
And I remembered the 2nd scenario (example). It was for USO. The students had some VMs on a cluster and they worked on their homework there. So each virtual machine WAS the assignment. And vmchecker needed to test the entire vm.

[calin-iorgulescu]

Pull request #73 adds support for custom submissions runners. It allows TAs to write their own runner which extends a base class Runner that defines exactly how a submission is executed. It provides access to the Host and VM APIs, which allow the starting, restarting and stopping of VMs. For example, you could reboot your VM and run tests in different phases, depending on the kernel being used. Interacting with GRUB or similar can even allow you to reboot into another OS (e.g., Windows).

It could be, potentially, extended to use multiple VMs as well. It would require a bit of hacking in vmchecker-vm-executor, but most of the functionality should be there already now.