The Lucy Way

Lucy has a particular way of working with student assignment repositories which is not always clear to new users (both course convenors and tutors).

The Lucy Way has three tenets:

1. We don’t do any work in student repositories, we mark from a copy.
2. Each assessment item exists in a separate repository.
3. Marks are not returned to students until approved by the course convenor.

These tenets exist because when students submit assignments through GitLab they are allowed to change their repositories at any time, before or after the deadline. When we mark an assignment, we usually have a deadline after which we no longer accept changes. Lucy is designed to copy the student’s repository at their last push before the deadline and then we (tutors and lecturers) mark the copy.

So tenet (1) exists because we want to take a snapshot of what the student had submitted before the deadline and only look at that copy (the student doesn’t have access to it). The copy is saved GitLab under a “marker user” which so we have a convenient archive of student submissions.

Tenet (2) is related to (1). Lucy is designed so that each repository is only related to one assessment task (that is, we don’t have multiple deadlines for one repository). Changing this would involve changing the logic for choosing commits which is an important piece of code (TM).

Tenet (3) is enabled by (1). Because we don’t mark the student’s work directly in their repositories, they can’t see our marks or feedback until the course convenor chooses to return it.

Here’s some illustrations of The Lucy Way to help understand how it works in practice.

Setup#

In this example the course is “COMP1720”, the year is 2021, and the assignment is called “assignment-1”. The student forks an assignment template called “comp1720-2021-assignment-1”. GitLab automatically gives the marker user (“comp1720-2021-s2-marker”) access to this fork via a server-side hook: the fork comes out private, and the hook adds the marker to it as a Maintainer. See the fork hook for what that hook does and how to get one set up for your course.

Marking#

When it’s time to mark, Lucy inspects the student’s fork of the assignment template and determines which commit was the last to be pushed to GitLab before the deadline.

The state of the repository at that commit is copied to a new repo, named after the student, owned by the marker user. The assignment’s marking file templates (and a pam.md for recording potential academic misconduct) are inserted into the copied repo.

The tutors are able to clone the marker user assignment repositories under their allocation¹ (usually matching students in their labs), and put marks and feedback into the template. Alternatively, the convenor can share the marking repositories with a GitLab group of tutors, who then mark directly in the GitLab web UI. They also have a file for documenting any potential academic integrity issues.

The tutors push back their marks, and the convenor is able to “check” which student repos have been marked and which tutors have yet to complete their marking!

Feedback#

When all the marking is complete and has been approved by the convenor, a feedback.md built from each marking file is sent back to the student’s original repository. Note that we still don’t touch the student’s work-branch, feedback goes into its own branch.

Lucy also collates the marks into a gradebook .CSV, uploads the marks and comments to Canvas, and presents useful statistics on the marks for the convenor.

Steps we skipped#

The above is the simplest case. Lucy also supports a “moderation” step, where all tutors mark a selection of assignments to moderate their standards (see the moderation guide.

Lucy makes it very easy to give individual students extensions, or to change the due date for whole class. It’s also easy to check which students have forked, cloned, and made a commit to their assignment repositories before the due date. Course plugins can go further, e.g. sending customised reminders to students through a course forum.

It’s also possible to automatically send student assignments to MOSS for automatic code similarity checking if plagiarism amongst convergent assessment tasks is a problem.