Technology

There’s nothing quite like not being able to get something to work the way it should, and implementing a terrible hack instead. It may work for now, but you can only kick that can so far down the road. Recently a coworker discovered one of my terrible hacks, and after months of kicking the can, I finally had to figure it out. The answer involved a long journey, ending in changing a series of hyphens (-) to the in keyword. It wasn’t a bug, just an oddity of the way that the ScalaTest framework works.

The following ScalaTest code passes:

class BeforeAndAfterWorks extends FreeSpec with Matchers with BeforeAndAfterAll {
  var example = false

  override def beforeAll() = {
    example = true
  }

  override def afterAll() = {
    example = false
  }

  "Some Test Set" - {
    "should pass" in {
      example shouldBe true
    }
  }
}

However the following fails on the matcher example shouldBe true:

class BeforeAndAfterDoesNotWork extends FreeSpec with Matchers with BeforeAndAfterAll {
  var example = false

  override def beforeAll() = {
    example = true
  }

  override def afterAll() = {
    example = false
  }

  "Some Test Set" - {
    "should pass"  - {
      example shouldBe true
    }
  }
}

In both cases, I use the BeforeAndAfterAll mixin to correct set the value of the example variable before the test begins. If you’re trying to play spot the differences, you should pay more attention because the answer was in the opening paragraph. It all has to do with replacing the - with in.

Looking through the ScalaTest source code, we find that - is really just a wrapper for registerNestedBranch, while in is a wrapper for registerTestToRun. The hyphen is meant to hold collections of sub-tests in a tree like structure, while in actually registers the following block as a unit test. It’s possible to have nested branches, but you cannot nest in blocks. These in blocks also uniquely identify tests by their string, preventing tests with duplicate names, as shown in the following error output:

[error] Could not run test org.penguindreams.DuplicateTests: org.scalatest.exceptions.DuplicateTestNameException: Duplicate test name: Some Test Set should pass
...
[info] - top in clause *** FAILED ***
[info]   An in clause may not appear inside another in clause. (NestedIn.scala:19)
 (BeforeAndAfterWorks.scala:21)

The issue comes because ScalaTest does allow for matchers/assertions to occur within the tree, but outside of an in block. Code run in this space is not subject to BeforeAndAfterAll or BeforeAndAfterEach traits.

This issue took a while to debug. After I figured it out, and unleashed a storm of rage and profanity, I created a merge request for my co-workers to review. One of the comments added to it, “Too funny - well not really. I ran into this same issue once.”

I haven’t delved into the architecture of ScalaTest yet, but I’m guessing that mitigating this particular situation may not be possible with the ScalaTest architecture. In any case, hopefully this post will help other developers who get stuck in the same situation; attempting to figure out why their pretest requisite functions do not run correctly in their test specifications. The code used in these examples can be found at https://gitlab.com/djsumdog/freespec-beforeandafter-notrunning-example.