Autograding C# with xUnit
December 28, 2020

Setting up automated grading for C# with unit testing in CodeGrade

Creating C# assignments in CodeGrade is easy, either using mono, or by actually using .NET Core, which is available for Linux and MacOS as well nowadays. The thing is, C#projects require students to create a complete project, which makes it harder to focus on the important things: the actual code that they write. It is much easier and less error prone, especially for basic assignments, if students can just hand in the .cs files they have created instead of the whole Visual Studio project.

Now comes the interesting part: using those code files to automatically create the project structure around them. The reason we want to create the project files, is so that we can use the .NET tools to test the code and write unit tests using xUnit, one of the most popular unit testing frameworks for C# and pre-installed on CodeGrade. The .NET Core framework provides the "dotnet" command to run projects and tests. But, for the "dotnet" command to work it has to be a valid project. Since we do not want students to hand in full projects, we will have to craft the project and insert the tests automatically. This might sound difficult, but it is actually not that hard and quite fast to set up.

Before we start setting up the assignment in CodeGrade, it is usually a good idea to start locally on your own computer. There we can easily figure out how we can create the structure, test if it works, and then port it to CodeGrade when we have a clear and clean idea.

For this particular example, we will be following most of the structure from this excellent guide from Microsoft. In this example, students will create a small program that will check if a number is prime. Following the earlier guideline, students will only hand in the PrimeService.cs file, which is a simple function to determine if a number is a prime. To make sure students only hand in this file, we can use CodeGrade's hand-in instructions. The Microsoft guide walks you through setting up a whole solution, but for our purposes, that's not necessary. In the end, the simple structure we would like to end up with when running our students’ code in AutoTest is as follows:

C# folder structure

This gives us the following tasks:

Generate the project files:

  • PrimeService.csproj
  • PrimeService.Tests.csproj
  • Other files in the obj/ directory.

Write the unit test file:

  • PrimeService_IsPrimeShould.cs

Zip the project files and upload it as a fixture to CodeGrade.

Create a simple script that generates the folder structure:


Port it to CodeGrade.

Keep in mind that `PrimeService.cs` is submitted by our students and inserted into this folder structure in our AutoTest setup. So, make sure not to upload this file as a fixture.

Generating the .csproj files

Let's start with the first step, and that's creating the project files. To reiterate what was said above: We need to generate the full .NET project using our solution file, `PrimeService.cs`, so that we can run .NET commands on our students’ submissions in CodeGrade. Firstly, we can create the project files for our solution using a simple dotnet command:

-!- CODE language-console -!-dotnet new classlib -o PrimeService

This command will create the PrimeService.csproj file and insert it into a new directory called “PrimeService”. Within this directory dotnet will also create a number of other files which it needs to be recognized as a complete project. However, the command also generates its own cs file within this folder called Class1.cs which we don’t need and can be deleted.

Write the unit tests

Now let's quickly take a look at the solution file that students will hand in:

-!- CODE language-cs -!-using System;
namespace Prime.Services
   public class PrimeService
       public bool IsPrime(int candidate)
           if (candidate < 2)
               return false;
            for (var divisor = 2; divisor <= Math.Sqrt(candidate); divisor++)
               if (candidate % divisor == 0)
                   return false;
           return true;

Then we can start writing some unit tests for this. First, we'll have to create the tests project and link them to the solution:

-!- CODE language-console -!-dotnet new xunit -o PrimeService.Tests
dotnet add ./PrimeService.Tests/PrimeService.Tests.csproj reference ./PrimeService/PrimeService.csproj

The first command will generate a new xUnit project folder for our tests. In it will be a PrimeService.Tests.csproj file, an obj/ directory with other necessary project files, and an empty UnitTest1.cs file which we can populate with our tests. The second command adds a reference between PrimeService.Tests.csproj and PrimeService.csproj so that our unit tests know what class to look for when run.

Now we can rename `UnitTest1.cs` to `PrimeService_IsPrimeShould.cs` and start writing our tests. For this example let's just create one simple test:

-!- CODE language-cs -!-using Xunit;
using Prime.Services;
namespace Prime.UnitTests.Services
   public class PrimeService_IsPrimeShould
       public void IsPrime_InputIs1_ReturnFalse()
           var primeService = new PrimeService();
           bool result = primeService.IsPrime(1);
           Assert.False(result, "1 should not be prime");

We can already test this on our solution locally by simply running the "dotnet test" command.

Want to start using CodeGrade in your C# course?

Generating the folder structure

Once we are done with writing tests, we can start porting this to CodeGrade, but first we will have to write a simple script that creates the desired directory structure from the student files. This will look like this:

-!- CODE language-bash -!-#!/usr/env/bin bash
mv $FIXTURES/PrimeService/ $FIXTURES/PrimeService.Tests .
mv PrimeService.cs PrimeService/

In this script, we are moving our fixtures (which we will soon upload in the autograder configuration) to the student directory. This will create our folder structure. Then we move our students’ submission into the PrimeService/ folder where it can be tested.

Creating the AutoTest in CodeGrade

Now we are ready to create our CodeGrade assignment. First we need to compress our two folders, PrimeService/ and PrimeService.Tests/, into a zip file so that the folder structure is preserved. 

We can then simply upload this zip file along with the script as your AutoTest fixtures. Next, in the Global setup script, we need to do two things: We need to decompress our folders and we need to pass an extra argument to our xUnit wrapper, cg-xunit. We do that with the commands:

`unzip && cg-xunit install_extra PrimeService.Tests/`

Then, we need to execute the script in the "Per-student setup script" with:

`bash $FIXTURES/`

Finally, create a new test category, choose CodeGrade's Unit Test step and select xUnit from the list. You can then select your test using the following file as Extra Argument:


Now the tests will automatically run as soon as a student hands in. Make sure to upload a test submission, so that you can check if the AutoTest works as planned.

The most asked question I get is how long it takes to set up the autograding. So let me answer this question for you for this example. It took me just under an hour to figure out what kind of directory structure was necessary, writing the setup script and testing and debugging it locally and on AutoTest. The upside is that this is work I only have to do once, and for all other assignments I can use the same scripts and easily import this part of the AutoTest. The only thing that has to change are the actual unit tests that I will have to write.

If you have any questions about this example in AutoTest or about CodeGrade in general, feel free to email me at and I am more than happy to help you.

Youri Voet

Youri Voet

Co-founder and CEO
Youri Voet is co-founder and CEO at CodeGrade. During his studies Computer Science and work as a Teaching Assistant at the University of Amsterdam, he developed CodeGrade together with his co-founders to make their own life easier. Youri works together with many educational institutions to continue to offer them the best CodeGrade experience possible.

Continue reading

Watch our ChatGPT and Coding Education webinar!

Watch CodeGrade's webinar on using ChatGPT in coding courses to help students work with this new tool and even use it in your grading worklfow yourself!

Join our webinar on ChatGPT in Coding Education!

Join CodeGrade’s CEO, Youri Voet, for a webinar on the impact of ChatGPT on computer science education. Learn how to make ChatGPT-proof coding assignments, teach AI literacy and how to use ChatGPT to set up automatic testing for your coding assignments.

New features: Assignment Schedules and Asynchronous Assignments

On May 8th and May 22nd, respectively, CodeGrade will launch two exciting new features. These features are Assignment Schedules and Asynchronous Assignments.

5 ways CS instructors can benefit from ChatGPT in their workflow

Learn about 5 ways computer science teachers can use OpenAI's ChatGPT or other AI assistants for their code grading and teaching workflows.

Sign up to our newsletter

Book a quick 30-minute demo and see how it works for you!