Smaller, Clearer Tests

In this article we explore how generating test values can better express intent of input values. The examples are written using Typescript, but the same technique can be used elsewhere.

For JVM based languages I recommend the excellent Arbitrator library from Tyro. Unlike the other languages in this post, the Arbitrator library doesn't required you to write dummy functions manually.

val user = User::class.arbitraryInstance().copy(emailVerified=true)

In Rust I use Derive Macro

And I use template specialization to achieve this in C++

auto randomInt = dummy<int>();
auto randomFloat = dummy<float>();
// random string with length of 10
auto randomString = dummy<std::string>(10);

// vector of 10 random ints
dummyVector<int>(10, dummy<int>);

// lambda passed in to generate the ints
dummyVector<int>(10, [](){
    // random int between 50 and 200
    return dummy<int>(50, 200);
});

// vector of user defined types
dummyVector<MyType>(15, dummy<MyType>);

We work with some large nested data structures. The instantiation of these objects can be a deterrent to writing tests. To create sample data in tests can take tens or even hundreds of lines. Not fun to write, very easy to break, and problematic to maintain if the structure changes.

If writing tests is hard, people write less tests.

Even with smaller objects the need to provide placeholder values for each field can obscure which parts are important to the test, and which are just required to run the test.

Small Example

interface User {
  email: string;
  emailVerified: string;
  firstName: string;
  lastName: string;
  orgKey: number;
}

might be used in a test like this

const users = [
  { "testA@test.com", false, "testAFirst", "testALast", 1},
  { "testB@test.com", true, "testBFirst", "testBLast", 2}
  { "testC@test.com", true, "testCFirst", "testBLast", 1}];

const results: string[] = getVerifiedEmails(users);
expect(results[0]).toEqual("testA@test.com")
expect(results[1]).toEqual("testB@test.com")

Let's look at an alternative

const users = [
  getDummyUser({emailVerified = false}),
  getDummyUser({emailVerified = true}),
  getDummyUser({emailVerified = true}),
];

const results: string[] = getVerifiedEmails(users);
expect(results[0]).toEqual(users[0].email);
expect(results[1]).toEqual(users[1].email);

This second test is much clearer in its intent.

• I need 3 users, the only thing I care about is their emailVerified status
• I need their emails but I don't care what they are (as long as they are unique and that they match the results).

It requires less code, you don't need to make up names for your users and it avoids potential bugs from the first approach.

• Why do users 2, and 3 have the same last name?
• Why is user 2 in a different org?

Are these intentional? Are the problems?

You also have to look out for:

• If I change fields firstName and lastName to first and last I have to update all tests
• If I add a new field I have to update all tests (don't make them optional to get around the problem)

There is a bit of work required to enable this (but it's small, I promise)

export function getDummyUser(partial?: RecursivePartial<User>): User {
  return {
    email = dif(partial?.email, 'example@example.com'),
    emailVerified = dif(partial?.emailVerified, true),
    firstName = dif(partial?.firstName, 'Joe'),
    lastName = dif(partial?.lastName, 'Blogs'),
    orgKey = dif(partial?.orgKey, 1),
  };
}

You will need to implement a getDummy<Type> function (in a <type>.dummy.ts file) that takes an optional Partial and returns a complete version of the type. For each field you check if a value was provided in the partial, and if not put a default value in its place.

dif function allows you to provide a default

/**
 * default if undefined.
 *
 * useful when creating dummy objects, to ensure that you dont override a supplied falsy value with
 * the default value
 **/
export function dif<T>(value: T | undefined, defaultVal: T): T {
  if (value === undefined) {
    return defaultVal;
  }

  return value;
}

Going Bigger

Complex nested structure.

export function getComplexType(
  partial?: RecursivePartial<getComplexType>,
): ComplexType {
  return {
  	identifier: getDummyString(partial?.identifier),
    	nestedObjectA: getDummyNestedObject(partial?.nestedObjectA),
    	nestedObjectB: getDummyNestedObject(partial?.nestedObjectB),
    	nestedObjectC: getDummyNestedObject(partial?.nestedObjectB),
        differentNestedObjectC: getDummyDifferentNestedObject(partial?.nestedObjectB),
        relativeValue: getDummyNumber(partial?.relativeValue),
    	enabled: getBoolean(partial?.enabled),
    	numA: getDummyPercentage(partial?.numA),
    	numB: getDummyPercentage(partial?.numB),
    	numC: getDummyPercentage(partial?.numC),
    	numD: getDummyPercentage(partial?.numD),
    	numE: getDummyPercentage(partial?.numE),
    	numF: getDummyPercentage(partial?.numF),
    	numG: getDummyPercentage(partial?.numG),
        numH: getDummyPercentage(partial?.numH)
  };
}

Then the usage becomes

const complexTypeA = getComplexType({enabled: true, numA: 123});
const complexTypeB = getComplexType({enabled: true, numA: -123});
const complexTypeC = getComplexType({enabled: false});

There's a lot of boiler plate in this dummy implementation, however it pays off as soon as you use it. Because of how nested this object is creating an instance could easily span 50-100 lines. In this test we create 3 of them in 3 lines - all unique, all valid.

It also nests getDummy calls which forward their partials into child Dummy calls.

Any developer adding a new test using a type you created 6 months ago will thank you for providing this. They no longer need to intimately understand what are and are not valid values for each field.

If you write a type, you provide a Dummy. If you modify a type, you modify the Dummy.

Updating the Dummy will update all the Tests automatically. Any problems will be brought to your attention with failing tests and you will only need to tackle the subset that need it.

Behind the scenes

In Typescript we have created RecursivePartial to let us pass through incomplete nested objects.

export type RecursivePartial<T, A = AllowedPrimitives> = {
  [P in keyof T]?: T[P] extends Array<infer U>
    ? Array<Value<U, A>>
    : Value<T[P], A>;
};