Why not automate the manual work involved in writing individual test cases? Why not make them more robust by applying a much wider coverage of the input domain compared to picking edge cases by hand? Why not separate input domain definition from validation? Generative testing addresses these problems by testing random_(ish)_ly generated inputs against the expected properties of output instead of expected values. Traditional unit tests are usually written by hand-picking cleverly chosen edge cases and testing them, encapsulating input and validation in (typically) single functions/methods.
An example of traditional unit tests in Clojure:
(defn add [x y] (+ x y))
Generative testing, on the other hand separates the definition of the input domain (possible inputs for the function being tested) with the concept of generators and validators.
Clojure’s test.generative has a nice set of generator functions:
test.check extends this functionality with awesome features:
(require '[clojure.test.check :as tc])
When you have your generators set up, it’s time to declare the properties which is a way to formally define the test:
In the above example we’re stating that for all random vectors of integers (this is what (gen/vector gen/int), our generator gives us) sorting once should result in the same vector as sorting twice. We can quickly check if the test fails or not:
(tc/quick-check 100 sort-idempotent-prop)
Let’s see what we get if our test fails!
Sure, we didn’t specify that the vector should consist of unique integers! (How many of you would have tested for this by hand?) test.check tells us the simplest failing case it found.
It’s also very easy to integrate test.check’s generative tests into clojure.test:
(defspec first-element-is-min-after-sorting ;; the name of the test
Here’s an example of test.check in action:
Here’s a very nice example of why generative testing rocks: Testing a mastermind scoring function
And a quick walkthrough of test.check