Anna Filina

Many developers claim that to achieve high quality software, developers must create automated tests that ensure that all possible execution routes have been covered. This is also known as full path coverage. I will argue that different types of software require different testing approaches and that full path coverage is impractical in almost every case. Too many tests simply create clutter.

Let’s look at the impracticality first. Writing tests requires skill and effort. Everybody on your team is probably not a testing expert. Not only may it be difficult to get started with testing, but you will generate a great amount of code with its load of maintenance. Tests are not immune to their own errors either. I have uncovered a number of incorrect test cases in every project, including PHP itself.

A full coverage can give a fake sense of security, because you won’t second guess your tests and there will be too many to effectively audit them. That doesn’t mean that more tests is bad. It means that you must strike a good balance between the number of tests and the assurance that you are seeking. Look at it as a car insurance that costs more per year than the price on your car. You want to feel safe with the right insurance, but at some point, it’s more trouble than it’s worth. You can easily end up with more test code than software code, so watch out.

My second point is that different software justifies different tests. Software that is meant to be used by other software — such as a programming language, a framework or an API — would require more in-depth testing. I would recommend writing a few tests per function, to make sure that they return expected values. Testing all paths may still not be necessary, because some paths may prove redundant. Let’s take a date parsing function as an example. If you can parse “2013-05-22 13:00″ and “May 22, 2013 1:00pm” correctly, then perhaps testing the permutation “2013-05-22 1:00pm” may be unnecessary. Be smart about your tests: choose quality over quantity.

For consumer software, you may not need to test everything. I recommend to skip obvious tests. For example, it is obvious that when you first instanciate this class, getTotal() will return zero.

class ShoppingCart {
  protected $total = 0;

  function getTotal() {
    return $this->total;
  }
}

Writing overly obvious test cases is like underlining text that is already highlighted. I usually start testing areas that can have impact the business, such as losing data or selling 0.01 quantity of a product.

In the end, there are no exact rules on how much to test. The most important thing to keep in mind is that writing tests for the sake writing tests is futile and costly. Not only that, but your colleagues won’t bother reading or maintaining tests if they are only there for their own sake and provide too little value. Focus on building great software. Tests are a tool to make it better. Just don’t overdo it.

Many people confuse the three concepts. For a business to grow or for a project to succeed, these need to be well understood. I will use a driving navigation system to explain them.

Before you set out, you need a destination, a goal. A goal needs to be measurable, that is, you need a way to confirm that you have achieved it. When driving, that usually means arriving at the specified coordinates. Additionally, you may want to get there within no more than 40 minutes.

(more…)

A montreal-based newspaper recently released a new mobile product to read news. It’s very nice, but it cost them 40 million and took 3 years to ship, with some 100 people on the team. My company has enough experience with news corporations and other industries to know that there was a lot of waste. My associate stated on social media that he could save them millions on their next project. It’s what we specialize in: generating millions in ROI for our clients.

(more…)

Most see this as a good thing, but I find the whole “we need more women speakers” speech very irritating. Too many PHP conferences are doing it. I don’t like that. I have your attention, but you may have already come up with some questions for me. Are you against diversity? Are you sexist? Do you think that women should stay at home and cook? The simple fact of my disagreeing on the topic almost inevitably lead to such assumptions. I will explain my position through personal stories.

(more…)

I have recently read a blog post claiming that functional tests are not “true” tests. The author also claims that unit testing shows you where the problem is occurring, while functional testing simply identifies that a problem exists. This argument is deceptive and the conclusion dangerous. Different kinds of tests are not mutually exclusive. One is not superior to the other. They have different goals and can happily coexist. Let me explain the kinds of tests so that you could make enlightened decisions.

(more…)