Testing tools for Kotlin. Quick recap in 2020.

Posted On October 13, 2020

We can group testing tools into 3 main categories

1. Frameworks
2. Mocks
3. Assertions

1) Frameworks

JUnit4

Just good enough to work, without any cool features. Still default in the Android project because the next version (JUnit5) needs Java8, which still isn’t default for Android, because of compile time.

JUnit5

This is the future. Can work together with old JUnit4 tests. All other frameworks, mentioned here need, this to work. You can achieve all the other behavior with this. All that you know, all new features in one place.
Personally, I don’t need anything more than this.

Top features:

Nested Tests – you can build beautiful hierarchies of tests and group them as you want. For example, you can group them in BDD style like below.

 @Nested
 inner class `GIVEN (...)`{
     @Nested
     inner class `WHEN (...)`{
         @Test
         fun `THEN (...)`() {
         @Test
         fun `THEN (...)`() {

assertAll – assert all assertions before throw error. You don’t have to rerun test to check if other assertions work!

 assertAll(
        { assertTrue(...) },
        { assertFalse(..) }
)

assertThrows – now you can test exceptions like a human being at the end of the tests.

assertThrows<NullPointerException> { 
        nullPointerFunc()
}

@ParameterizedTest – you don’t have to copy&paste test for different scenarios. Feel the power of data class parameters in tests!

@ParameterizedTest
@ValueSource(strings = ["racecar", "radar", "able was I ere I saw elba"])
fun palindromes(candidate: String?) {
        assertTrue(StringUtils.isPalindrome(candidate))
 }

@DisplayName – I know that in Kotlin you can use sentences as names, but for DisplayName you can use emojis.
And yes, probably you will never use it.

 @Test
 @DisplayName("Test name with emoji: ? ? ? ")
 fun testWithDisplayNames() {

Side notes:
1. To use all the above blessings on Android you have to use an external plugin: https://github.com/mannodermaus/android-junit5
2. JUnit5 don’t use @Rules, you need to use extensions but they are very simple:

@ExtendWith(
    MockitoExtension::class, //Mockito have extensions
    InstantExecutorExtension::class 
//Android have only rule, but rewrite it as extension is dead simply
)
//Here you have extension
class InstantExecutorExtension : BeforeEachCallback, AfterEachCallback {
    override fun beforeEach(context: ExtensionContext?) {
        ArchTaskExecutor.getInstance()
            .setDelegate(
                object : TaskExecutor() {
                    override fun executeOnDiskIO(runnable: Runnable) = runnable.run()
                    override fun postToMainThread(runnable: Runnable) = runnable.run()
                    override fun isMainThread(): Boolean = true
                }
            )
    }
    override fun afterEach(context: ExtensionContext?) {
        ArchTaskExecutor.getInstance().setDelegate(null)
    }
}

Spek

The core concept is to write tests as nested lambdas. They force BDD style tests, which is good, but you don’t need them to achieve it.

object CoreConeptTest: Spek({
    group("a group") {
        test("a test") {        }
        group("a nested group") {
            test("another test") {        }
        }
    }
})

object BddStyleTest : Spek({
    Feature("Tested feature") {
        Scenario("Test scenario") {
            Given("...") { }
            When("...") { }
            Then("...") { }

Kotest

Quite similar to above however Kotest gives you many more possibilities.

class SentecesAsTests : StringSpec({
  "Some test nr 1" { }
  "Some test nr 2" { }
})
class BddStyleTest : BehaviorSpec({
    given("...") {
        `when`("...") {
            then("...") { }
        }
        `when`("...") {

List of possible test styles:

Fun Spec	ScalaTest
Describe Spec	Javascript frameworks and RSpec
Should Spec	A Kotest original
String Spec	A Kotest original
Behavior Spec	BDD frameworks
Free Spec	ScalaTest
Word Spec	ScalaTest
Feature Spec	Cucumber
Expect Spec	A Kotest original
Annotation Spec	JUnit

2) Mocks

Mockito

The most popular tool for mocking things. Written in Java, but still a very good tool, even in the Koltin project. For Kotlin you have also a small helper library Mockito-Kotlin, that will allow you to replace your `when` with whenever:
https://github.com/nhaarman/mockito-kotlin
It has everything that you need. Almost. It doesn’t give you full suspend functions support. For example, you will have a problem, when you will try to manipulate the dispatcher’s timeline inside what mock returns.
They have open PR for that supsend answers:
https://github.com/nhaarman/mockito-kotlin/pull/357

For now, when you use coroutines in your project, it would be better to choose MockK.

MockK

Also a very good mocking tool but written purely in Kotlin.
MockK has two main advantages over Mockito:

1. Support suspend functions support
2. Work with Kotlin Multiplatform

Because of above, I choose it as the default mocking tool in new Kotlin projects.

val mock = mockk<ObjectToMock>()
every { mock.function() } returns value //simple mock
coEvery { mock.suspendFunction() } returns value //suspend mock

3) Assertions

List of possible libraries:

Kotest – yes, this one from the framework, have also a separate library for assertions. This is the most popular.

name shouldBe "sam"
user.name shouldNotBe null

Kluent – very similar to above one, second most popular.

"hello" shouldBeEqualTo "hello"
"hello" shouldNotBeEqualTo "world"

Strikt

expectThat(subject)
  .hasLength(35)
  .matches(Regex("[\\w\\s]+"))
  .startsWith("T")

Atrium

expect(x).toBe(9)

HamKrest

assertThat("xyzzy", startsWith("x") and endsWith("y") and !containsSubstring("a"))

Expekt 

23.should.equal(23)
"Kotlin".should.not.contain("Scala")
listOf(1, 2, 3).should.have.size.above(1)

AssertK

assertThat(person.name).isEqualTo("Alice")

Here, I had the biggest problem of what to choose. I still feel that I don’t need anything else than normal JUnit assertions. If I would have to choose I would vote for Kotest because it looks nice, and it’s quite popular. IMHO all those things are a matter of personal taste.

Summary

I hope that I helped you to take a quick look at possible testing tools in Kotlin.
I use JUnit5 with MockK without assertion libraries.
What do you use in your projects or what would you like to try?

See you on The Code Side!
All the best,
Artur