Autograding web user interfaces with Selenium and Jest in CodeGrade

Web applications are a booming business nowadays. It is no surprise that web development courses are a core part of most current computer science programs, in addition to being present in most multimedia and design programs too. CodeGrade itself is a web application that was initially developed as part of a software engineering course at the University of Amsterdam.

Autograding websites is a little less straightforward, however. Often, the main aspect to grade is the user experience or user interface that the students have created using languages like HTML, CSS and JavaScript or using one of the many web frameworks available. For this, we cannot rely on (command line) input and output tests or unit testing like we usually do. Instead, we will have to look into ways to automatically interact with their website UI and test that.

This is where the two powerful tools come into play: Jest and Selenium. Jest is one of the most popular (unit) testing frameworks for JavaScript (and installed by default on CodeGrade) and Selenium is an open source tool that automates browser testing.

In this blog, we will discuss how you can set up a basic Jest and Selenium configuration in CodeGrade to autograde web development assignments. We will use some information gathered by Johan Holmberg from Malmö University, who has shared his steps to set up Selenium in CodeGrade in a public GitHub repository.

Installing Jest and Selenium in CodeGrade

To start autograding the UI of your web development assignments, we will need two main tools: Jest and Selenium.

• Jest is a testing framework for JavaScript. It currently is one of the most popular testing frameworks, it is easy to set up, fast, has great documentation and is installed by default on CodeGrade! Read more here.
• Selenium is an open source tool to allow web browser automation. Selenium supports automation across different browsers, platforms and programming languages and thus is the perfect tool for autograding webdev courses! Read more here.
  

It is now time to install both these tools in AutoTest. For Jest, CodeGrade has a wrapper script available and thus it can be very easily installed with the `cg-jest install` command. For Selenium, and all additional required packages, installing can be done using `npm` and `apt`. First we can install the required JavaScript packages: `npm install -g selenium-webdriver geckodriver`. Secondly, we will install firefox (the browser we chose to run the students’ website in) and `xvfb`, a tool to allow for virtual framebuffers as we have no real display in CodeGrade and we thus need to run in a headless web browser: `sudo apt install firefox xvfb`.

To make it easier to run and maintain this setup, we can create a simple `setup.sh` script, upload it as a fixture to CodeGrade and run that in our Global Setup Script as `$FIXTURES/setup.sh`. The full script will become (from Johan Holmberg’s GitHub):

-!- CODE language-sh -!-#!/bin/sh
# Installs Jest as per https://help.codegrade.com/user-reference/autotest-general/unit-test
cg-jest install

# Updates the Ubuntu repo settings
sudo apt update

# Installs Firefox and a headless X window manager
sudo apt install firefox xvfb

# Installs the neccessary javascript libraries
npm install -g selenium-webdriver geckodriver

# Makes the run script executable
chmod +x $FIXTURES/run.sh