In-class exercise: Code Defenders

High-level goal

The goals of this game-based exercise are to (1) solidify software testing skills and (2) reason about potential sources of program errors.

Set up

  1. Teams are project groups. Unlike previous in-class exercises, this is an individual assignment, but you will be playing with your project group.

  2. Log into the Code Defenders game, hosted on Attu, using the link, username, and password you received in an email.

Overview

You will play a battleground-style game of inserting and detecting defects. The general gameplay is:

  • attackers introduce subtle defects (bugs) into a given class under test and

  • defenders write effective unit tests to detect the bugs.

You will play three games (described below):

  1. ~20 minutes

  2. ~15 minutes

  3. ~15 minutes

Instructions

  1. Join the first game, which is easy and will be the only open game. This game will have both tests and bugs visible. You will be randomly assigned to be either attacker or defender. The goal is to familiarize yourself with the game interface and to freely communicate with opponents.

  2. Join the second game, which is easy. This game will have both tests and bugs visible. To get a feel of playing both sides, you will be assigned the opposite role of what you played in the first game.

  3. For a final challenge, join the last game to play against all students and staff, which is hard. This game will have both tests and bugs invisible. There are no fixed roles, you can attack and defend.

Deliverables

  1. Create subtle defects (bugs) and tests in the game. You do not need to submit anything for this part, as we can collect these directly from the game server.

Frequently Asked Questions

  • In what format can I write tests and assertions?

    You can write JUnit4 tests and assertions.

  • I think one of the mutants (defective programs) is actually correct (it has equivalent behavior to the original, non-buggy program on every possible input), what should I do?

    Defenders can click on the “Claim Equivalent” button for a mutant if they think it is equivalent. This initiates a duel. The attacker either provides a detecting test for that mutant, or surrenders the duel and loses points if they accept that the mutant is indeed equivalent. Note that the attacker has only one chance to provide a detecting test.

  • I’m a defender; where is the “Claim Equivalent” button for a mutant?

    Once you have written tests in an attempt to detect a mutant, then the button should appear to claim equivalence.

  • In a melee game, why can’t my tests detect my own mutants?

    You cannot detect your own mutants, even if you submit a test that would actually detect it. Others are trying to detect your mutants by submitting their tests.

  • What does it mean if a test has a smell?

    You may find more information about test smells here.