Is there a simpler way to test all permutations with Kotest property-based testing?

General Discussions
1

I’m using Kotlin with Kotest and trying to test all permutations of 2 parameters with forAll and Exhaustive.collection, like this:

"Some test" {
    forAll(4,
        Exhaustive.collection(listOf("a", "b")),
        Exhaustive.collection(listOf("1", "2"))
    ) { begins_with, contains ->
        println("$begins_with:$contains")
        // validation code...
    }
}

I expected it to cover all possible permutations:

a:1
b:2
a:2
b:1

But it repeats in order:

a:1
b:2
a:1
b:2

Using Arbs changes the order but doesn’t guarantee full coverage, and duplicating list elements isn’t sustainable.

Is there a way to generate all exhaustive permutations instead of looping over each list?

2

Kotest provides Exhaustive.permutations to generate all combinations of multiple lists. You can rewrite your test like this:

"Some test" {
    val first = listOf("a", "b")
    val second = listOf("1", "2")

    forAll(Exhaustive.permutations(first, second)) { (begins_with, contains) ->
        println("$begins_with:$contains")
        // validation code...
    }
}

This will give you all 4 permutations (a:1, a:2, b:1, b:2) exactly once. I switched from Exhaustive.collection to Exhaustive.permutations in one of my tests, and it simplified the code a lot.

3

Another approach is Exhaustive.bind, which is useful if you have more than two parameters:

val first = Exhaustive.collection(listOf("a", "b"))
val second = Exhaustive.collection(listOf("1", "2"))

forAll(Exhaustive.bind(first, second)) { begins_with, contains ->
    println("$begins_with:$contains")
}

bind pairs every element of the first collection with every element of the second, producing all combinations.

It’s cleaner than nested loops and works well when you have multiple lists.

4

If you want full control, you can generate the Cartesian product yourself and feed it to forAll:

val first = listOf("a", "b")
val second = listOf("1", "2")
val allCombinations = first.flatMap { f -> second.map { s -> f to s } }

forAll(Exhaustive.collection(allCombinations)) { (begins_with, contains) ->
    println("$begins_with:$contains")
}

This guarantees you test all permutations once, and I’ve used this approach when I needed custom combination logic before.