Changelog

This sprint we have added a new test block to AutoTest v2 for code structure testing! The "Code Structure Test" block uses Semgrep, a popular code structure testing framework. Using this test block allows you to select a version of semgrep to use, specify the file(s) to test and the specify a pattern to match against. Using the Positive match block allows you to check for code structures that your students should have used in their assignment while the Negative match block lets you search for code that should be avoided.

• You can now specify a default severity for comments generated by the Flake8 block who's severity isn't recognised by CodeGrade. Additionally, you can add a mapping to unknown flake8 error codes.
• The Pytest block now shows a red cross when a submission achieves zero points.
• It's now possible to choose how many submissions are shown on the submission list page.
• We now make sure that the locale of the VM in AutoTest v2 is set correctly.

This sprint we we made three key changes focused on bringing more usability to the UI. We have improved the submission list firstly by making it possible to view the assignment description directly rather than having to go to the settings, and secondly, by making it possible to sort submissions by section. Additionally we have improved the plagiarism checker by making it possible to filter cases using the search bar and by making it possible to load more than 100 cases.

• We now show a warning that the test crashed for IO Tests and hide the diff viewer to prevent confusion about the cause of the test failing.
• We have now made it possible to import an entire assignment setup into assignments that have existing submissions.
• It is no longer necessary to refresh the page when importing an AutoTest v2 configuration.
• AutoTest v2 steps that generate comments on students' code now have a dedicated viewer for a user-friendly output.
• Institutions with a Brightspace LTI1.3 integration can now adjust their assignment names in CodeGrade.
• All new staff users will be granted access to the community library by default.

This update includes the most recent addition to the "cg" commands suite, "cg checkstyle". This new command improves the the usage of checkstyle in CodeGrade by offering a simple way to install checkstyle and, run and parse the output of checkstyle tests. More information about the use of this command can be found by running "cg checkstyle --help" in a Script block in AutoTest v2.

• You can now set test descriptions in AutoTest v2 to automatically expand or collapse given the result of the test.
• The "cg comments generic" and "cg unit-tests junitxml" commands have been simplified to "cg comments" and "cg junitxml". The old commands will be preserved for backwards compatibility.
• We have introduced the "caching" state to the AutoTest v2 output viewer to provide a clearer indication of when AutoTest v2 steps are being cached.

This update expands on the "cg" command in AutoTest v2 (a pre-installed command for parsing test output). The update adds the option to parse JUnit XML files directly to render the output of unit tests by using the command "cg unit-tests junitxml".

Additionally, the "cg comments generic" command can now generate a score based on the comments that were generated. This feature is enabled simply by setting the --score flag. The impact each comment will have on the score can be customized with the --deduction-<severity> PERCENTAGE flag.

• We now support installation blocks for Clang, GCC, and .NET framework in AutoTest v2
• Submissions are now marked late depending on the deadline of the section the student is in.
• The submission will remain marked as late even if the deadlines are shifted after submission, since it was late at the time of handing in
• The specific deadline and lock date used for the student are also available in the AutoTest v2 environment within the $CG_INFO variable.

We are making it easier to use custom test frameworks in AutoTest V2 For Code Quality tests, we have created the “cg comments generic” command. You can provide a regex to this command to match any line in the test output and use that to write comments easily to your students’ submitted files. You can read more about how to use this feature here!

Additionally, code quality tests in ATv2 now also place comments on students’ code while they are working in the editor.

• You can now bulk export plagiarism cases
• You can now set a private Github repo as a template repository in your assignments
• We fixed an issue where AutoTest v2 could not find files in subdirectories in the editor
• We made user role mapping better for clients using SURF SSO
• We fixed a cosmetic issue when dragging test steps in AutoTest v2

The new "Run if" block in AutoTest v2 allows you to determine if your test cases run or not depending on whether tests pass or fail. Select a rubric category and if the percentage of points achieved meets your set criteria, the tests wrapped inside this block will be run, otherwise they will be skipped.

‍

• We improved unit test suite readability by limiting the number of columns rendered in the output.
• We have created a separate permission for creating test submissions, splitting it from the "Create submissions for others" permission.
• We have improved the plagiarism export by having cases from the same file presented together and including more metadata about the submissions and the JPlag report.
• We fixed the API documentation for the Schedules feature.

You can now export plagiarism reports in PDF format for ease-of-sharing. The report is beautifully rendered with LaTeX formatting and soon we’ll add more information to this!

• We increased the rate limit for network connection in ATv2 from 1 MB/s to 70 MB/s.
• We fixed a bug in AutoTest v2 that made it impossible to upload large files.
• We fixed a bug in AutoTest v2 where multiple output viewers for pytest blocks were shown before the test was run
• We fixed a bug that made it impossible to delete sections copied over from another course.

When you set up a lock date in CodeGrade's general settings, a pop-up banner will now appear that allows you to grant students the permission to make submissions after the deadline. This makes the process much quicker as you no longer need to find the permission settings in the course management page.

• The name of the author will now always be shown in the header when the editor is being used.
• We made it possible to access global packages in the pytest step, fixing an issue where users had to install the packages in the pytest block itself.
• You can now change the name of Flake8 and Pytest steps in your AutoTest v2 configuration so that you can give clearer instructions to students.
• We fixed an issue where users using Firefox could not scroll through assignment configurations while importing from the Community Library.
• We fixed an issue where Flake8 would show an error if its config was hidden with a Hide block.

You can now share your assignments with other CodeGrade teachers or import assignments from the community into your own course! The Community Library makes it simpler for new users or users without the time to set up courses to set up their own course in CodeGrade.

You can access the Community Library at the bottom of the general settings of an assignment in the “Danger Zone” next to the assignment importer. Soon you will also be able to publish your assignments to the Community Library. If you already want to share assignments, just reach out to us at support@codegrade.com.

You can now run Flake8 on your students' Python submissions and automatically generate comments on their code.

• We have updated AutoTest v2 to be accessibility compliant according to the WCAG 2.1 guidelines.
• We have added support for Python 3.11 and Python 3.12 for the CodeGrade Python API package.
• We have fixed a bug that meant newer versions of the API libs weren't being published on Pypi.
• We have improved the payment UI for students.

It is no longer required to set a deadline for your students to create submissions. Previously, students would not be able to create a submission for an assignment without a deadline, whereas now they will instead be able to create submissions indefinitely. Effectively, this change makes it possible to run asynchronous assignments. This is especially useful if your course allows students to make submissions for any assignment throughout the duration of the course.

• You can now access submission metadata through the "CG_INFO" environmental variable.
• Other small changes and bug fixes.

You can now connect a template GitHub repository to CodeGrade. Enabling this feature will force your students to create a new GitHub repository from the template you have selected rather than selecting an existing repo. You can use this feature by enabling GitHub/GitLab as a submission method and then toggling the "Enable template repository" option. You will then be prompted to provide the URL of the template repo.

Note: You must make sure the GitHub repo you select as the template must be a template repository. This can be set in GitHub by going to settings and checking the box called "Template Repository" under the General Settings.

• Added a PyTest block in ATv2 allowing you to write your Pytest scripts directly in AutoTest. This removes the need to upload test files or wrapper scripts.
• Added a regex match block for the IO test in ATv2.
• Other glitch and bug fixes

You can now set different availability, deadline and lock dates in your assignments for different sets of students (sections) with Schedules.

To use this feature, go to the general settings of your assignment and scroll to the Schedules settings. If you have created sections in your course, you can add a new schedule. By default your assignment will only have a single deadline for everyone. Schedules, like deadlines, can be set up for each assignment.

With the new feature you can:

• Have multiple sections / labs doing the same assignment at different times.
• Grant deadline extensions to a student or group of students.‍

‍

‍Note:

• When using Schedules, the availability, deadline and lock date will no longer be automatically synchronized from your LMS. In other words, you will have to set the deadline for everyone in CodeGrade.
• If a user is present in multiple sections, for each option (availability, deadline, lock date), they will be granted the most lenient setting applied to one of their sections. For example, if a student is in two sections (A and B) and the section A has a later deadline but section B has an earlier available at date, then the student will be assigned the deadline of section B and the available at of section A.

We've made an improvement to the way you can import assignments from one course to another. You are now first prompted to select which course you would like to import an assignment from and then you can select an assignment from that course. This makes it much easier and fast to find the assignment you are looking for.

• Override GitHub repository - An issue that some users were encountering was that students would connect the wrong GitHub repo to their CodeGrade assignment. This would result in them having difficulty decoupling the repo and connecting a new one. We've now added a feature that allows you to easily override the currently connected GitHub repo with a new one.
• Section division API - This new public API service allows teachers to divide students into sections in their course.

We've made a lot of progress on improving our newly released AutoTest v2. Below are some of the key updates from the last two months:

• We added setup blocks that allow you to install specific versions of Python, Java or JavaScript in the setup.
• We added a timeout wrapper block to set a custom timeout on steps.
• We added a diff viewer to the IO test step for students.
• We added the option to ignore whitespace to IO test match blocks.
• Submission metadata is now available in the AutoTest VM with the environmental variable “CG_INFO”.
• You can now place comments in the code viewer via AutoTest V2 by writing to file descriptor 3 (Structured Output).
• We now allow for descriptions on test steps.

• AutoTest v2 steps no longer unnecessarily collapse while tests are running.
• Fixed a scrolling bug in the AutoTest V2 output viewer in the editor.
• Copying AutoTest v2 configurations should no longer give timeout errors.
• Large amounts of text in the Expected output field no longer causes saving issues.

We have improved how you can publish (and synchronize) the grades and feedback to your students. There is now a dedicated toggle on the Assignment Management page. Furthermore, the availability of an assignment is now managed through the "Available" field, which has three options: "Unavailable", "Always Available" and "Available at [DATE]".

This update replaces the Assignment State. Existing assignments are automatically migrated to use the new fields.

• When you copy or import an assignment, the test submission is now also copied!
• Made it possible for teachers to access the editor to adjust an existing submission at any time via the "Edit" button in the code viewer.
• Made it possible for teachers to upload a test submission even when the "File uploader" option hasn't been selected.
• Added a substring match block for IO tests in AutoTest v2.
• Added a case sensitivity option to IO test match blocks in AutoTest V2.
• Several minor improvements.

You can now use our completely new automatic grader which runs on a new infrastructure and has a completely redesigned user interface. Read more about AutoTest V2!

• Added an announcement center.
• CodeGrade now supports the Test Student role in LTI 1.3 in various LMSs including Canvas.
• You can now view editor templates.
• You can now customize the maximum number of points an assignment has.

You can now bulk enroll users into CodeGrade if you are using CodeGrade without an LMS. Simply upload a CSV file with the name, username and email of the students you wish to enroll and your students are enrolled.

If your students don't have an account yet, they will receive an email to register for an account with the details you specified.

• Fix a bug where the name of a Canvas CodeGrade assignment would sometimes change to the name of the course.
• Fix a bug where assignment descriptions would sometimes be overwritten when copying assignments.

This release brings the highly anticipated start of the private beta for the CodeGrade Editor! Furthermore, we have improved experience of CodeGrade on Apple devices, and made setting up your unit tests for .NET and PHP even easier.

• This release marks the start of the private beta for the CodeGrade Editor. This editor allows you to work on your submissions and create them right inside CodeGrade, without having to open any local editor. If you want to join this private beta please contact us.

This release allows you to personalize CodeGrade even more: Dark Mode can now be toggled automatically and you can select your default Submission Page tab. Additionally, we added many of your small feature request to make your experience even better!

This release improves our peer feedback feature: it is now possible to let groups peer review other groups. Happy new year from all of us at CodeGrade!

‍

‍

This release brings revamped general feedback! It now supports the same awesome features as the inline feedback, such as snippets, markdown and discussions.

We've redesigned our dark mode for this release, making it both prettier and easier on the eyes! But that's not all, it is now also possible to grade submissions in percentages.

‍

API Changes

As this is a major release some API routes have been changed and are not backwards compatible.

• The API introduced last release to retrieve GitHub and Gitlab repositories (GET:/git_providers/<uuid:provider_id>/tokens/<uuid:token_id>/repositories/) now no longer returns an array, nor does it use page pagination. It now returns a GitRepositoriesPage object, which contains the data (under the key data) and a cursor (under the key cursor) that can be used to retrieve the next page.

• All routes that upload files to later be downloaded using the GET:/api/v1/files/ method have been changed. They now all return a MirrorFileResult object. For the documented API this change is backwards compatible, but for some undocumented routes the change is backwards incompatible.

• The undocumented route to export the metadata of an assignment has been changed. It now no longer returns the CSV directly but returns the representation of a background job that will export the assignment.

This release improves the integration between CodeGrade and GitHub and Gitlab, making it easier than ever to setup git assignments. It is now also easier to setup unit-testing in CodeGrade.

‍

‍

This release brings an updated continuous rubric category, where you can now define a lower and upper boundary. Besides that, the assignment state design has been updated and it's been moved to the general settings.

‍

‍

This release introduces anonymous grading, which hides the student name for teachers while grading. Besides that, you can now also copy entire courses and assignments and we have improved the submissions list with more information.

‍

CodeGrade Orchid brings a big update to AutoTest: setup caching. Caching will make autograding quicker, even for configurations with a performance heavy setup. We have also published a new Filesystem version with this release.

‍

‍

This release makes it easier to allow late submissions using the lock date, making it possible to allow late submissions up to a certain date. We’ve also updated various software packages in AutoTest, and made it easier to use the CG_INFO information.

‍

This release kicks off our push to make AutoTest even better this year. You can now give even more accurate feedback by customizing AutoTest messages, and the workflow for uploading fixtures has been optimized.

• Provide more accurate feedback by customizing the messages in AutoTest.

‍

Never hyphenate code. When wrapping lines it could happen that code was hyphenated in the Code Viewer.

API Removals

• The option to get an entire course when getting an assignment has been removed. You should now request the course using the /api/v1/courses/<course_id> route go retrieve this course. The course_id is given when requesting an assignment.

• A plagiarism cases will no longer contain the two linked assignments. They do contain the two linked assignment ids (under the assignment_ids key), and the plagiarism run contains a lookup from assignment id to an assignment like object.

• When requesting all courses using the /api/v1/courses/ route we will no longer provide the name of the role that the current user has in this course. Please either use the /api/v1/permissions/ route to retrieve your own permissions, or the /api/v1/courses/<course_id>/users/ to retrieve the role of a user.

UI Removals

• Linters have been moved to AutoTest (AutoTest Quality Comments). The new integration allows for more flexibility and grading based on linter output.

This release brings many small fixes and improvements, and is the last release on our old infrastructure. We will migrate CodeGrade to AWS on the 28th and 29th of December, during which CodeGrade might not be available.

‍

‍

The Rubric Editor has a new design, which allows you to reorder categories and use markdown. The new Code Quality step in AutoTest allows you to grade and add inline comments to submissions based on code style. This will replace the current Linters integration.

‍

UI Deprecations

• The linters integration has been deprecated. The new AutoTest Quality Comments bring all the advantages, and also allow you to give a grade based on the output of the linter.

CodeGrade now has an Exam mode in which students will be able to log in to CodeGrade and only have access to the course of the exam. Other changes include course archiving, importing of hand-in requirements, and including even more information in the AutoTest Unit Test output.

‍

API Deprecations

• The option to get an entire course when getting an assignment has been deprecated. You should now request the course using the /api/v1/courses/<course_id> route go retrieve this course. The course_id is given when requesting an assignment. If you still use the old behavior you will get a warning, you can already opt-in to the new behavior by providing no_course_in_assignment=true in the request arguments.

• When requesting a plagiarism case we have deprecated getting the two linked assignments within the plagiarism case object. The case will now contain the two linked assignment ids (under the assignment_ids key), and the plagiarism run contains a lookup from assignment id to an assignment like object. If you still use the old behavior you will get a warning, you can already opt-in to the new behavior by providing no_assignment_in_case=true in the request arguments.

• When requesting all courses using the /api/v1/courses/ route getting name of the role that the current user has in this course has been deprecated. Please either use the /api/v1/permissions/

• route to retrieve your own permissions, or the /api/v1/courses/<course_id>/users/ to retrieve the role of a user. If you still use the old behavior you will get a warning, you can already opt-in to the new behavior by providing no_role_name=true in the request arguments.

With CodeGrade Peer Feedback it is now possible for students to review code of other students, allowing them to learn from each other. Furthermore, it is even easier to integrate existing unit tests in AutoTest, by utilizing the new “Unit test” step type.

‍

With our new Feedback Sidebar, you can now view all previous feedback from a student in the course, while grading the current submission. It is now also possible to get submission metadata in AutoTest, for instance to automate late day penalties. Finally, we have upgraded CodeGrade to the newest version of LTI: LTI 1.3 Advantage!

‍

You can now view assignment statistics on the Analytics Dashboard, giving you insight into student performance. Students can now also comment on their own code, and they can reply to comments placed by teachers.

‍

You can now render HTML pages submitted by students right inside CodeGrade, allowing you to preview webpages or test Javascript more easily than ever.

‍

‍

You can now add Continuous Rubric Categories, which can score anywhere on a continuous scale and work great with AutoTest. You can also now set student submission limits and a cool off period.

• Continuous rubric categories: this new type of rubric category can be used to give points anywhere on a scale from 0 to a configurable amount of points. This behavior maps better to certain types of AutoTest categories, such as categories containing only “capture points”

• steps. Rubrics can contain a mix of discrete and continuous categories and both can still be used for AutoTest.

• Make it possible to limit the amount of submissions: the amount of submissions can be limited in two ways:

1. A maximum total amount of submissions for an assignment.
2. A cool-off period: an amount of time a student must wait before they can submit again.

• Separate feedback permissions: the can_see_grade_before_done permission was used for all types of feedback students would get. New can_see_user_feedback_before_done and can_see_linter_feedback_before_done permissions make it possible to show these types of feedback before an assignment is set to done while still hiding others.

‍

• Add warning when creating a wrong external tool link in Canvas: Canvas has multiple ways to integrate external tools, some of which leave CodeGrade unable to communicate correctly with it. This update displays a message when this happens.
• Use the most privileged LTI role available.
• Fix float matching for AutoTest capture points test.

‍

The hand in page for students has been completely redesigned, making it simpler and easier to use. You can now import AutoTest configurations and the ESLint linter is now available.

• Submissions page redesign.
• Add ESLint as a linter option.
• Make it possible to delete assignments.
• Make it possible to copy AT config.
• Add course registration link.

‍

‍

‍

It is now possible to hand in via GitHub or GitLab. You can now also write files back from AutoTest to the Code Viewer to ease manual grading.

‍

‍

AutoTest and Continuous Feedback cooperate even better with this release. Tests are always run immediately after handing in, and even fill in the rubric directly when possible. Teachers can still choose when to make results visible to students.

‍

‍

AutoTest and Continuous Feedback are now more reliable and efficient by fixing many small bugs and tweaks in the back-end. Additionally, a diff-viewer is added to the output of IO tests.

‍

‍

It is now significantly more efficient to run AutoTest or Continuous Feedback by a big improvement in our back-end. Additionally, our latest update adds further improvements to CodeGrade and fixes several minor and rare bugs.

‍

‍

You can now make sure students get near instant automatic feedback using our new extension of AutoTest called Continuous Feedback. To better present this feedback to students, we have redesigned the entire submission page to be more intuitive.

‍

• Add all_whitespace option to IO test: add new option to IO tests to ignore all whitespace when comparing.

• Update stop points to percentages: to better work together with possible disabled tests in Continuous Feedback, all stop or check points now work with percentages instead of points. (not backwards compatible!!)

WARNING: Update is not backwards compatible. Previous stop / check points break if not updated to percentages.

‍

You can now automatically grade code of students using our brand new feature called AutoTest. This enables teachers to easily create test configurations and students to automatically get insightful feedback.

• AutoTest is CodeGrade’s new Automatic Grading Environment: with AutoTest you can automatically grade code of students and provide them with insightful feedback.

‍

‍

You can now set up detailed hand-in requirements for your students, create course snippets and the submission page is easier to and has more information (including the possibility to upload multiple files).

‍

‍

Along with many UI improvements and bug fixes, you can connect grading divisions between assignments and import rubrics from previous assignments.

‍

‍

‍

‍

‍

‍

‍

‍

• Support files encoded as ISO-8859-1 (latin1) in the frontend.

‍

‍

• Add Plagiarism checkers: It is now possible to check for plagiarism in CodeGrade. This enables privacy aware plagiarism checking. It is possible to use check against old CodeGrade assignment and upload base code and old submissions that are not in CodeGrade. For more information see our documentation.

• Make it possible give grades higher than ten: Teachers can now indicate that students can receive a grader higher than 10 for an assignment, making it possible to create assignments with bonus points in CodeGrade. When using within LTI this requires a new LTI parameter.

Minor updates

• Change homepage to login screen:

• The homepage has been improved to show all your courses and assignments at a glance when logged in.

• Use new logos: This updates our logo to the newest and greatest version!

• Allow .tar.xz archives to be uploaded: This further improves the flexibility CodeGrade gives students when handing in submissions.

• Fix infinite loop overview mode: In some combinations of permissions loading the overview mode resulted in an infinite loader.

• Add general feedback tab to overview mode: This further decreases the chance that students will miss any of their feedback.

• Improve speed of diffing by using another library: Viewing the diff between two large files is a lot faster!

• Remove the option to automatically generate keys: It is no longer possible to generate the secret_key or lti_secret_key

• configuration options. Please update your config accordingly.

• Rewrite snippets manager: This rewrite should make creating,

• using, deleting and updating snippets faster and more reliable.

• Drastically improve the experience of CodeGrade on mobile: It is now way easier to use CodeGrade on mobile.

• Filter users in the user selector: When selecting users (when uploading for others, or adding to courses) only show users will be shown that can be selected.

• Improve handling of LTI: A complete rewrite of LTI backend handling. This should improve the stability of passbacks by a lot. This also guarantees that the submission date in Canvas and CodeGrade will match exactly. This also adds a new convenience route /api/v1/lti/?lms=Canvas to get lti config for the given LMS

• (Canvas only supported at the moment).

• Add items to the sidebar conditionally: Depending on what page you are you will get extra items in the sidebar to help quick navigation. Currently plagiarism cases and submissions are added depending on the page.

• Start caching submissions: Submissions are cached in the front-end so changing between the codeviewer and submissions list is now way quicker.

• Ensure all rubric rows have a maximum amount of >= 0 points: It is no longer allowed to have rows in a rubric where the maximum possible score is < 0. If you needed this to create rubrics with bonus categories simply use the ‘Max points’ option in the rubric editor. All existing rubrics are not changed.

‍

‍

‍

‍

‍

‍

• Make sure upload dialog is visible after deadline

‍

• Fix assignment state buttons for LTI assignment

‍

‍

‍

‍

‍

‍

‍

‍

‍

Packages Updates:

• Upgrade bootstrap-vue

‍

‍

‍

Package Updates

• Remove pdfobject and pdf.js dependencies

• Move bootstrap-vue dependency to own org

• Add npm-shrinkwrap.json and delete yarn.lock

• Change to JWT tokens

Initial CodeGrade release

-