Autograding Java code projects on code quality, code style and bad practices using Checkstyle and PMD
Guides
May 10, 2021

Autograding Code Quality and Common Bad Practices in Java Assignments

Java is one of the world’s most popular and most taught programming languages. Most, if not all Computer Science students, will have had a course on Java sometime during their academic career, often even during their introductory courses. Next to using Java as a means to teach Introduction to Programming, Object Oriented Programming, Data Structures or Mobile Development, it is also important to teach good Java practices and code quality. Perhaps especially for such a frequently occurring programming language such as Java, a programming language that is pre-eminently used for larger software projects in which good team work, adherence to a strict style guide and good coding practices are crucial.

In this guide, I will cover ways to autograde Java code quality and eliminate common bad practices. Not only will I cover two very useful tools for this, Checkstyle and PMD, I will also discuss how you can design your own style guides to fit your courses’ learning objectives and how to customize the feedback messages generated by these tools to be more understandable and meaningful for your students. 

Prefer to watch a webinar instead? Most material covered in this guide is discussed in our webinar on Java Autograding, which you can watch on demand here.

Java Code Quality tests in CodeGrade

One of the types of tests offered in CodeGrade AutoTest is the Code Quality step, a way to provide instant feedback on the style, quality and structure of code. This step makes it really easy to set up default quality checks without any configuration, but allows for fully custom tools and style guides too. The feedback generated in this step is not only shown in the AutoTest tab, but also directly on the lines of code, making it very intuitive for students to use and understand their code quality feedback!

Automatic Java code quality feedback to a programming assignment in CodeGrade

You can use any custom tool or linter in a Code Quality step, but also use one that is built-in. For Java, we offer Checkstyle and PMD. These two code analysis tools (or linters) are used for different purposes, but are both very powerful (and even more so when combined). After briefly reviewing both tools, I will show how you can use them in a basic Code Quality test, which we will further customize in the following chapters.

Checkstyle

Checkstyle is a static code analysis tool for Java that is built-in to CodeGrade. It is a traditional linter: it checks if the code style and presentation conform to a style guide, but it does not check or confirm code correctness or completeness. 

Amongst other things, Checkstyle can detect:

  • Use of naming conventions (variables, methods, etc.)
  • Completeness of documentation (think Javadoc)
  • Use of vertical and horizontal whitespace
  • Conventions for parameters, headers and imports

While at first glance it may seem somewhat unnecessary to teach, it is very important for students to learn working with a style guide, as this is very common practice in their later career and in group projects. Furthermore, enforcing a style guide in beginner courses helps students develop with a solid code style following the conventions.

PMD

PMD (probably but unofficially standing for Programming Mistake Detector) is a cross-language source code analyzer that is built-in to CodeGrade. It not only works with Java, but also with JavaScript, XML and PL/SQL. In comparison to Checkstyle, PMD actually detects bad coding practices and potential bugs, but does not assess style.

Amongst other things, PMD can be used to detect:

  • Inefficient code
  • Bad programming habits
  • Overcomplicated expressions that can be simplified
  • Potential bugs
  • Dead code
  • Duplicate code (via the included Copy-Paste-Detector, which also works for Python, C-languages, Ruby, PHP and many other languages)

PMD is often a little bit more advanced than Checkstyle and very useful in more advanced courses where bad practices are harder to spot by eye. Furthermore, it has rules for many very specific bad practices, allowing you to make custom style guides specifically for an OOP course, a multithreading course or another advanced course. Moreover, PMD has an option to create totally custom rules, which is outside the scope of this guide, but can make it very powerful for any advanced Java course.

Start autograding Java Code Quality and common bad practices easily and effectively now!

Setting up a basic Code Quality test

After choosing the Code Quality test in your AutoTest category, you can simply select a linter from the dropdown list. Choose Checkstyle or PMD for your Java assignment. After doing so, you can select your style guide. You can upload and use a custom one, or simply use one of the default style guides available. For Checkstyle, we recommend the very common Google style guide, based on the style guide universally used at Google with hard-and-fast rules, or Sun style guide, a more traditional style guide enforced by Oracle / Sun that is last revised in 1999, which can be downloaded here. For PMD, we recommend Apache Maven’s style guide, which can be downloaded here.

After selecting your style guide, you can optionally provide extra arguments for the selected tool and update the penalties for the different message severity levels. If you wish to just inform your students of style and not grade them, these penalties can all be set to 0%.

Basic Java Code Quality (linter) set up in CodeGrade's autograder

Designing a custom style guide for educational purposes

Autograding code quality can become even more effective when using a custom style guide. Custom style guides are relatively easy but time consuming to make, luckily, they lend themselves for iterative development and can easily evolve with your course. Just some of the benefits you can get with custom style guides for education are:

  • Removing rules that are unnecessary, too advanced or too complicated for students
  • Adding rules that may be unconventional for businesses but useful for education
  • Creating a very small style guide with the most important and basic rules for introductory courses
  • Creating a very specialized style guide for advanced courses (e.g. with common bad practices in multithreading)

In the following two sections, I will go over lists curated by CodeGrade with common rules to use in your style guides for education and teaching Java. These lists are by no means exhaustive, but should rather act as an inspiration and a great starting point for your own adventure in the literally hundreds of available rules and checks.

Checkstyle style guides for educational purposes

You can select the rules in your Checkstyle configuration file (style guide). A list of all available Checkstyle rules can be found on their website here. A Checkstyle configuration file is a simple XML document that specifies the rules to take into account, these rules are childs of the root module called “Checker”. 

-!- CODE language-xml -!-<module name="FileTabCharacter"/>

Checks that no tab characters (`\t`) are used.

-!- CODE language-xml -!-<module name="Indentation">
    <property name="caseIndent" value="0"/>
</module>

Checks if correct indentation is used, with a custom property value as recommended in the style guide of Oracle.

-!- CODE language-xml -!-<module name="ConstantName"/>

Checks the constants naming conventions, e.g.: `FIRST_CONSTANT1`.

-!- CODE language-xml -!-<module name="LocalVariableName"/>
<module name="MemberName"/>
<module name="ParameterName"/>
<module name="MethodName"/>
...

Checks the naming conventions of variables, parameters and methods, these should be lowerCamelCase.

-!- CODE language-xml -!-<module name="LineLength">
    <property name="max" value="80"/>
</module>

Checks the length of the lines, with a configurable maximum value, the style guide of Oracle recommends 80 characters per line.

-!- CODE language-xml -!-<module name="AvoidStarImport"/>

Checks that no star imports (`import java.io.*`) are done.

-!- CODE language-xml -!-<module name="WhitespaceAfter"/>

Checks that tokens are followed by whitespace, not `method(foo,bar)`.

-!- CODE language-xml -!-<module name="NeedBraces"/>

Enforce the use of braces around code blocks.

-!- CODE language-xml -!-<module name="MagicNumber"/>

Checks that no “magic numbers” are used (-1, 0, 1 and 2 are not).

PMD style guides for educational purposes

You can select the rules in your PMD configuration file (style guide). A list of all available PMD rules can be found on their website here. A PMD configuration file is a simple XML document that specifies the rules to take into account, these rule elements all have a `ref` attribute to specify the rule reference.

-!- CODE language-xml -!-<rule ref="category/java/bestpractices.xml/CheckResultSet"/>

Checks if return values of methods like `.next()` are checked.

-!- CODE language-xml -!-<rule ref="category/java/bestpractices.xml/UnusedImports"/>

Checks if none of the imports are unused.

-!- CODE language-xml -!-<rule ref="category/java/bestpractices.xml/UnusedLocalVariable"/>
...
<rule ref="category/java/bestpractices.xml/UnusedPrivateMethod"/>

Checks that there are no variables, private methods, etc. are unused.

-!- CODE language-xml -!-<rule ref="category/java/codestyle.xml/ForLoopShouldBeWhileLoop"/>

Checks if none of the for-loops should be simplified to while loops.

-!- CODE language-xml -!-<rule ref="category/java/design.xml/CollapsibleIfStatements"/>

Checks if two consecutive if-statements can be combined.

-!- CODE language-xml -!-<rule ref="category/java/errorprone.xml/EmptyCatchBlock"/>
...
<rule ref="category/java/errorprone.xml/EmptyWhileStmt"/>

Checks if there is no blocks of code or conditions that are empty, this is likely a bug or a bad coding practice.

-!- CODE language-xml -!-<rule ref="category/java/errorprone.xml/JumbledIncrementer"/>

Checks if multiple variables are referenced in for-loops  (e.g. `for (int k = 0; k < 20; i++)` ).

-!- CODE language-xml -!-<rule ref="category/java/errorprone.xml/UnconditionalIfStatement"/>

Checks if no if-statement is always `true` or always `false`.

-!- CODE language-xml -!-<rule ref="category/java/errorprone.xml/UseEqualsToCompareStrings"/>

Checks that `.equals()` function is used to compare strings, not `==` or `!=`.

Customised messages for meaningful feedback to students

The final step you can take towards the most student-friendly code quality autograding is customizing the messages and severity levels. All rules you can select have default messages, often very useful for experienced programmers, but sometimes complicated to understand for novice programmers. Furthermore, these messages are a great place to point students in the right direction, instead of just pointing out the mistakes. Changing “Avoid unused local variables” to “The variable ‘temp’ you initialized on this line is not used anywhere else in your code, it is better to remove such unused local variables” makes it easier to understand and more actionable for students.

Customizing messages and severity levels is possible for both Checkstyle and PMD and can be simply done by editing some attributes in the XML configuration file.

Customizing rules in Checkstyle

-!- CODE language-xml -!-<module name="RedundantModifier">
    <property name="severity" value="info" />
    <message key="redundantModifier"
        value="The ''{0}'' modifier you used is not needed here."
    />
</module>

In Checkstyle, a custom message can be set using the `<message>` property (a child of the rule module). You have to specify two attributes for this, a key, which can be found for each rule in the Checkstyle documentation here. And the custom message value, which may use placeholders. These placeholders can also be found in the documentation.

Setting a severity level can be done using a property too, named `severity`. The values for this are `info`, `warning` and `error`, corresponding to the severity levels found in CodeGrade AutoTest’s Code Quality step.

Customizing rules in PMD

-!- CODE language-xml -!-<rule ref="category/java/codestyle.xml/UselessParentheses"
    message="The parentheses here are not required and should be removed!"
>
    <priority>3</priority>
</rule>

In PMD, a custom message can be set with the `message` attribute to your rule. More information can be found in their documentation here

Setting a severity level is done using the `<priority>` property as a child of the rule. This is done with a number, corresponding to CodeGrade AutoTest’s Code Quality step as follows: 1 = fatal, 2 = error, 3 = warning, 4 = info.

Powerful Java autograding and plagiarism detection in CodeGrade

This guide has shown you all ins and outs of code quality autograding for Java assignments. But your CodeGrade assignment doesn’t have to stop here! There are many more things you can do to automate your grading workflow, provide your students with better feedback and manage your Java course effectively in CodeGrade.

In our recent webinar on Java autograding, which you can watch on demand here, we also discussed checking the functionality of student Java code, using JUnit unit tests to check individual methods, using JaCoCo to assess unit tests written by your students and how you can set up Hand In Requirements in such a way to enforce correct Java submissions to your Java assignments!

Next to autograding Java assignments, CodeGrade can also help you check for plagiarism in Java code and gives many tools for efficient manual grading

Would you like to learn more about setting up Java automatic grading for your code exercise or discuss certain Checkstyle or PMD rules mentioned in this article? Feel free to email me at support@codegrade.com and I’d be more than happy to help out!

Continue reading

Grow your coding classroom without compromising on quality.