RudderVirt

#Troubleshooting

When the autograder experiences a failure, it can be frustrating. We have built the autograder to be resilient, but as it's interacting with a real virtual machine, there are always things that can go wrong. This guide provides a set of actionable steps to diagnose and resolve the most common issues.

The autograder talks to the VM over the serial console, not over the network. It never uses SSH, and it does not need the VM to have a working IP address, DNS, or reachable network interface. If grading fails, the VM's NIC is almost never the cause, so reconfiguring or reinstalling network adapters is not a productive place to start. Focus on the serial console and on ruddergrade instead.

#Windows VMs

#First Steps

  • Ensure the VM is on, and booted into Windows
  • Restart the VM
  • Open a command prompt with cmd, and check that C:\Windows\System32\ruddergrade.exe exists and can be executed.

#Advanced Troubleshooting

  1. Run bcdedit in an elevated command prompt and ensure that under the "Windows Boot Loader" section for the current boot entry, ems is set to Yes.
  2. Open services.msc and ensure the "Special Administration Console Helper" service is enabled and running.

#Linux VMs

#First Steps

  • Ensure the VM is on, and booted into the operating system
  • Restart the machine
  • Open a terminal, and check that /bin/ruddergrade exists and can be executed.

#Advanced Troubleshooting

  1. Ensure the serial console is enabled on the VM. systemctl status serial-getty@ttyS0.service

#How the Autograder Works

The autograder is designed to be a flexible and robust system for evaluating student work within a virtual machine. Understanding its basic operation can help in diagnosing problems.

flowchart LR A[Autograder] -- serial console --> B[VM] B --> C[ruddergrade] C --> D{Step passes?} D -- yes --> E[Record success] D -- no --> F[Record failure]

At its core, we send a list of commands to be executed inside the VM, and evaluate their output against expected results.

The autograder communicates with the VM over the serial console, and only over the serial console. Through this connection, it sends commands to be executed in either a Bash shell (on Linux and macOS VMs) or a PowerShell session (on Windows VMs).

This is a deliberate design choice. Grading works on a VM with no network configuration at all, which means students can be graded on lessons where the network is intentionally broken, unconfigured, or the subject of the exercise itself. It also means that a grading failure carries no information about the VM's network stack. If a step fails, the problem is with the serial console, with ruddergrade, or with the command being run, and not with the NIC.

Inside the VM, a program called ruddergrade is responsible for executing the steps of your grading script. When you see output from the grader, it's coming from ruddergrade as it runs through the checks you've defined. On Windows, this program is located at C:\Windows\System32\ruddergrade.exe, and on Linux/macOS, it can be found at /bin/ruddergrade.

#Still having trouble?

If you've followed all the steps and are still stuck, submit a ticket on the help page.