I have been a professional software developer since 1996. Neither in school nor in practice in the early days of my career was I exposed to the concept of writing tests to confirm portions of my code worked as expected. Testing was always done via user testing. You or someone else would use the program to ensure it produced the expected result. That is how I learned to do my job. And I am good at my job.
My initial reaction to unit testing was negative. I know professional developers that sit in cubicles all day long and write code without any knowledge of the overall application. They are asked to write a function that takes a certain input and returns a certain output. They neither have nor desire additional knowledge about the use of their code. I find those people to be very unattractive as coworkers. They are often out of touch with technology. When I told someone in this line of work I used a Mac, he asked "Do they still make software for those?" He was dead serious. This was in 2005. My initial feelings about unit testing reminded me of these people. Just sitting in cubes, all day, writing functions that took X and returned Y. They required no knowledge about where X came from or where Y was going. Furthermore, I felt like people that must have to have that kind of structure in place to do their jobs must not be capable of understanding the big picture. If they did understand the big picture, they would not need to have these things. They should somehow KNOW how the application worked.
The language I use most (and have for the last 17 years) is PHP. PHP development has always had a haphazard culture around it. I don’t see that as a bad thing. Lots of people all doing things their way. That is often a criticism. People in other language communities where things are a bit more dictated and regimented find it disturbing. I find it refreshing. It’s more real world than the rest to me. But, that is a different blog post.
PHP didn’t come with any sort of testing framework out of the box. And none of the core developers put out a way to test your PHP code. The first tests of any kind that I was aware of for PHP were those built by the PHP QA team. It was targeted at testing the PHP engine by writing PHP code that used the engine. That made sense to me since there are so many moving parts in there and you have lots of people all making changes. It makes sense to have some sanity checks.
The first time I heard about testing your own PHP code was PHPUnit. My first impression was that it was an imitation of JUnit which is the de facto standard way to test Java. I don’t really care for Java. I don’t care for the language. I don’t care for the culture. I don’t care for how a java daemon behaves on my server. In general, I don’t care for the platform. It is my personal preference. It is not right or wrong. It is just how I feel. So, when I see something being copied from Java into PHP, I retreat. That probably makes me a bad person. It is a personality flaw. The end result is that I ignored PHPUnit no matter who said they were using it or why. And the only thing I ever heard about when people talked about testing PHP applications was using PHPUnit.
So, how does someone like me start advocating test driven development as a useful tool? It snuck up on me. A few things had to happen.
The first thing that happened was that we started growing our team. In 2006, we were back down to a two person development team. We had been as large as 5 during the dotcom boom. But, we slowly dwindled back down to two people doing development and systems administration every day as their full time jobs. When there are only two of you, you kind of have to know how everything works. If you don’t, you can’t keep running. As we started to add developers, I realized it was increasingly harder for other people to grok the whole application they way I did. And at first, I thought that was just a matter of time. After a year or so they would get it all. So, we hired more people. We are now at six developers, plus myself and one new one starting in two weeks. On top of that we are actively looking to hire two more as soon as possible. Along the way there have been some others come and go as well. Now, to people in the VC bubbles, this may sound boring. I know places where the team grows 20x in 6 months. I can’t imagine. Our application and code base is huge though. Experienced engineers that are now on the team have told me they have never worked with a code base this large. In the last couple of rounds of hiring, I was asked about testing. My standard answer was that if you wanted to write tests for your code and could develop a sane strategy for deploying those tests, I had no problem with it. As a team however, there was no official policy or systems in place for testing. One of the new hires pointed me to a something that peaked my interest. It was doctest for Python. Essentially, you put a bit of Python code in the doc block for the code you are writing. doctest will scan files for these tests and run them. Partly out of curiosity and partly to hopefully satisfy the guys wanting some way to test their code, I wrote a version of doctest for PHP. To prove it worked, I added a test block to all the methods of an array manipulation class I had worked on recently. I was still not convinced that this was a good use of my time however.
The second thing that happened was I met up with Chris Hartjes (@grmpyprogrammer). Chris is an avid supporter of test driven development. I had heard the term test driven development before meeting Chris, but my testing bias had shielded me from really understanding the concept. He and I talked and later I was interviewed by Chris and Ed Finkler (@funkatron) on the /dev/hell podcast. On that podcast we talked about testing as well as other things. Looking back on those two conversations, I now know what Chris did that most changed how I looked at people who are adamant believers in dynamic unit testing. Chris didn’t make it a personal thing. He acknowledged my perspective. We do have a code base and business that has been working for 10+ years. We do have extensive monitoring and graphing in place that tells us when things are going wrong. He told me that if that was working for me, more power to me. Wait, wasn’t he supposed to tell me how I was doing it wrong? Wasn’t he supposed to tell me I was a cowboy and how unprofessional I am? He did none of those things. So, you know what happened? I kept listening to him.
The third thing that happened was also because of Chris (just so you know now, most of this blog post is indirect or direct praising of Chris). Once I started listening, there was something else I was not hearing from him. I was not hearing how I should be using PHPUnit or any particular product for that matter. He only said that code should be written to satisfy a test. He didn’t seem to care how it was tested. He only cared that it was tested. This was again a breathe of fresh air. In this business where every group likes to tell the other group how wrong they are doing things, having someone that didn’t seem to care how you did it, as long as you did it was awesome. So, again, I kept listening to him.
The fourth thing that happened was when Chris made me realize that I already was a test driven developer. I don't think he realized that he did it. He likely does not remember the conversation. And I probably remember it poorly. For all I know, there were beers and/or whiskey involved. It went something like “So, how do you test your code?” said Chris. I replied “Well, if the web page loads, it worked.” To which he asked “So, you don’t test changes as you make them?” I said “Well, yeah, I write some test code and run it till it works while I am making changes. I just don’t save that code. Once its working I don’t need it.” I don’t think it hit me right then. I think it took days or weeks to sink in. Mother F--ker, I already write code in a test driven manner a lot of the time. I have a test.php script in my home directory on my development server. I reuse that thing all the time to test the code I am working on. Instead of assertions, lots of the time I will just print_r() the output and assert with my eyes. But, the end result is the same. So, why not take that extra step? I had the doctest stuff in place. I could just add my test code there and run the test over and over until it worked like I wanted it to. And what do you know, I kind of liked it.
And what I think was the last straw was when I was merging code onto staging and a test failed. Oh my gosh! I made a change to our code base and my all knowing, all seeing eye did not realize that my changes were going to break something. How is this possible? How did I not see this coming? I know everything in the application don’t I? I architected the whole thing. The reality is, this likely (and by likely I mean, it did) happened before. However, sometime very soon, either a monitor or a human would have noticed and a bug report would have been filed. I would have then fixed it. I would have likely justified the bug as the cost of progress. Now I have a way to help prevent these small bugs from rolling out in the first place. And now that I have this tool, there is no excuse. The only reason this should happen now is that we don’t have enough tests.
We are working on improving our test coverage. I am not to the point yet where I require tests for all code. Perhaps we will get to that point. I don’t know. My team is aware of the tool now. And they are aware that I use it and think it is wise to use it. I told them recently that I am not telling them to use it. But, if they roll a bug, and there was not a test, the fix should probably be written using test driven development.
I have intentionally not talked about what we use other than the a fore mentioned doctest. I hope the message that you take away from what I have learned is that you should be using SOMETHING to test code. That is more important than how you test your code. I also have not talked about how you write code that can be tested. That is another hurdle I have had to (still am) get over. I thought a lot of our code was not testable. Because you know, we are solving problems that no one ever in the history of the world has had to solve, like retrieving a web page, hard stuff. There are better resources to learn how to write code to be more testable, especially for PHP. I highly recommend everything Chris Hartjes has ever said or written about the subject. You can find his thoughts on the topic at http://grumpy-learning.com/, http://grumpy-phpunit.com/, and https://leanpub.com/grumpy-testing.
We will be improving doctest.php. We have lots of ideas. It just fits better for us. If PHPUnit works for you, great, use it. Just search for `unit testing PHP`and find something that works for you. In addition, we are starting to work with Selenium for interactive testing of our site and some commercial products that are really good at testing APIs. I am getting a hard time from some guys at work. I have been critical of testing in the past. I just needed to understand that I already worked this way or thought this way.