A good test

Is it possible to define objective criteria of identifying of testing code as good or bad? Looking at the tests I've written a few years ago, I can definitely say that, at least, the bad tests are identifyiable easily) In this article we are going to look at the 'basis of the basis' - the F.I.R.S.T. principles, going on with my own thoughts about the theme...

Fast

A good test has to be fast. No one needs a hundred of tests, that take a half of a day to execute. The sense of TDD is the idea of constant launching of tests while developing. Such launcings are being fired every a few minutes, literally. And if it takes more that 'a few minutes'... I belive you understand) There are no concrete numbers here, no one, as I know, had serious research of that theme. That is why I would rely on my own experience: ~1000 of tests takes 15-17 seconds normally (but maybe it's not the best idea to launch thousands of tests too often). Let's stay with the idea that tests should be fast enough to make you, as a developer, feel good with them.

Independence

A test must not be dependent on the net connection/a database/state of common variables/other tests or the Moon's phase. Every test lives in it's own Universe, separated from the other world. If the launching process requires concrete order of execution, if tests pass one by one but do not pass while general launch, if tests depend on a database... It is time to admit that the project turned the wrong way and fix things as fast as possible - in a short time such tests are going to become a pain in the... neck and throw a monkey wrench into development process.

Repeatability

The idea is as simple as possible: if launching of a concrete test with identical (depending of the test conditions) parameters show us different results, it would mean that we should say goodbye to it. Developers are tend to trust tests and searching mistakes in tests is the last step after investigation of all other scenarios and possibilities. There is nothing more irritating than spending a lot of time in researching of reasons of a falling test and finding out that the production code is ok and the reason is a bug in the test itself.

Self-sufficiency

A unit test is self-sufficient and able to check does it pass or not. There shouldn't be manual checks of data, investigation of console and etc. We are interested in only one thing - the color of the test (red or green), that shows us one of two possible states. Any try of avoiding this rule would bury the idea of automatic testing in a real software development.

Thoroughness

The common idea here is that the green color of a test should be the sign of real bugless of the system, without any clauses. If a test checks validity of a phone number, all variants of input should be tested (according to the business rules). If the production code shouldn't allow spaces in the incoming string, the test must check all possible positions of a space - in the beginning/in the middle/in the end of the string. Otherwise there would be no profit from all that dozens and hundreds of tests - bugs would still spill from everywhere. End nope, I don't say about the sacred 100% covering of code. The idea that the passed test always should prove working capacity of a method under the test.

Time of writing

The rule that scares many newbie: a good test should be written BEFORE the functional code. Yep, it works as it sounds - you should firstly write a test for non-existing functionality) Actually that rule comes from glorified in legends idea that realization should follow an abstraction. Writing a test before functional code, you create a contract, defining rules for the future functional code. Practice shows that such an approach allows us to increase both the quality and purity of the implementation - developers tend to avoid changes in ready-made code while writing tests for it.

Readability

A test is still just code. You must use right naming for variables and parameters here. A good test should show all the ins and outs of the functional code, without any comments. If a developer have to look at functional code - it's time to think about naming... Also we should try avoiding of 'magic' appearing of data from external classes/static variables and etc. All (as far as possible) the information about a test should be inside the test.

Structure

Preparation of data - calling of checking method - examination of the result. These steps should be separated from each other. I, for example, prefer to separate them with a new comment line '//'. This way we make easier to read a test and apply some kind of standartization - it's always known where to look for concrete infromation.

Naming

Talking about naming of testing methods we should forget about common rules, binding for functional code. It's ok to use three or five or seven words in a name (separated by "_"), the aim here - is to avoid any doubts and misunderstanding of the purpose of the method ufter reading of it's name.

Size does matter

Recall the canonical 10-15 lines for a method from books of Martin and McConnell. And the reason here is not usual readability or avoiding of good structure. A long test means that the test (or functional code under the test) has either too much responsibility, or the test contains logic. Because of such things tests die, quickly and painful... for a developer)

Atomicity

Decomposition - one of the most important aspects of solving of tasks, that helps in better understanding of requirements and build a concrete plan of implementing them. Our task while coding is to make the most thourough decomposion, dividing the whole task into smallest functions; these atomic functions are the aims of our tests. If a method have to validate an input string by three parameters (the lenght, absence of spaces, absence of capital letters), three separate tests should be written, each with a concrete zone of responsibility. It's important because of a few reasons:

  • when a test is broken, understanding of the concrete fragment of code, where the problem is, would take seconds; without rereading and debugging of code or even the test

  • when the functionality changes, we fix/remove a concrete test, not trying to find out logic of a long complicated test, risking by breaking something else

No logic

A test should not have own algorithms, repeating functional code, never, in any circumstances. If you need to sure that a method reverses a string - create two strings (input/expected). Any try of reversing a string inside the test and compare the result with the result of work of functional code would lead you to very big problems:

  • haven't we forgotten some requirements of functional code? Or if we have written so great functionality inside the test, it might be neccessary to write a test to the test?)

  • after changing of functional code logic, we would have to rewrite the test

  • we inevitably would break the rules about readability and conciseness

Minimum assertions

An ideal test has only one assertion. In real life it's not possible to follow that rule and it's commonly accepted that existing of up to three assertions in a test is ok. But these assertions must be linked logically. For example, if we a going to check that some methods returns a collection of strings, containing only one record (let's say "example"), it's right to write something like

    Assert.True(list.Count == 1);
    Assert.True(list.First() == "example");

The assertions here are logically linked and have to cut all incorrect variants. If you have to write more assertions or check a few aspects of work of a concrete method, it would be better to write a few tests. We would prefer to disregard repeating of code (let's forget about such things at all), but not allow increasing of complexity of a test.

P.S.

Violation of one or more rules, described above, leads to a number of code's diseases... Tests are too fragile and break down by any glance onto functional code. Developers will be finally tired of limitless fixing of tests and the idea of unit testing will be derailed. It would be hard to support tests. No one can understand what does a test about and what to do if it was broken. And again it takes our time for reading and fixing and... leads to a bad result for the testing project. Tests that break from time to time - it's like a boy, crying "Wolves! Wolves!". Once everyone would stop take them into account. That's it, the unit testing idea is dead again) Tests that take too much time for executing. Developers will try to launch them as little as possible, the final is a foregone conclusion) It's not a big deal to describe a few more scenarios, but, imho, the idea is understood - violation of these rules is a threat of die of the idea of existing of tests for a project. And if we are already on the glorious way of unit testing, we should do the best to avoid that)