Pages

Wednesday, December 7, 2011

10 Reasons to Avoid Test Driven Development

no-TDDTest Driven Development (TDD), and all of its derivatives (BDD, ATDD) are, in my opinion great methods to drive a team’s development efforts, and raise the quality of the product. But TDD is not a silver bullet. It doesn’t fit every project. The following post lists the top ten reasons not to write automated tests for your code.

 

If one or more of these conditions apply to you, you should consider forgoing TDD, and in fact any agile techniques, as they will be a waste of effort:

10. There is no client

Sometimes you are developing a product that will not be used by anyone. In this case, any effort expended on raising the quality is a complete waste. Nobody will care.

9. The client is an avid tester

Some people love nothing more than to beta-test new software. The joy of finding new bugs and trying to figure out what went wrong is what they live for. Others are scientists at heart, and love trying to go over stack traces, in order to reverse-engineer the code. If your client happens to be one of those, then writing automated tests will take all the fun out of using your software. Don’t do it!

8. The project is short, simple and straight-forward

If your team can complete the project in a short period of time (no more than a few weeks), and will never, ever have to reopen it for maintenance, then the benefits of maintainability, reusability and extensibility will be lost on you. Spending time and effort on these values is wasteful.

7. Your architecture is perfect

If there is no way to improve your architecture, then there is no need for it to be extensible. TDD, by the nature of its incremental development flow, forces the architecture to be extensible, simply by making you extend it as you go, and this, like lipstick on a pig, is something you just don’t need.

6. Your documentation is perfect

You never miss an API, and any change you make to your software gets documented instantaneously. The tests you create with TDD serve as a form of documentation, an example of how to use the API. Properly named and written tests actually explain the context of the test, so it is easy to find the test that shows you what you need to understand. Your documentation is so complete, that writing tests are a clear violation of the DRY principle, so you should clearly avoid tests.

5. Your team never changes and all members’ memories are perfect

The collective memory never forgets a single line of code it wrote, nor what the context was when writing it. You therefore, do not need the tests to remind you what the code does, why it does it, or how to use it. This also means that your team members never leave, nor are any new members recruited, because if this were to happen, you’d lose memories, or have members who don’t remember the code (having not been there when it was written). If this is the case, don’t bother with tests; they will just interfere with your incredible velocity.

4. “Done” means the code is checked in

Many teams have a definition of done (DoD) that means that the feature is “done” when it is in a state that the end user can receive and run (coded, tested, deployed, documented, etc.). Many others however, your team included, prefer a simpler and more easily achieved definition that accepts “checked in” as “done”. For you it is sufficient that the developer declared that he or she completed his part, and anything else is someone else’s responsibility. If you don’t need the code to be tested for the product owner / manager / user to accept it, then you are better served moving on to the next feature as soon as you can, instead of dragging on your relationship with this feature.

3. You’re paid to code, not test

Ignoring the fact that unit tests are code (sophistry), testing is what we have testers for. Perhaps your team's testers are fast enough that they can run tests on your code and give you feedback within mere moments, pinpointing the areas where you broke the code, so you can fix it while the changes are fresh in your mind, as well as a complete regression suite on the product, in case you broke something in a different component every night (they don’t mind working nights; they love the peaceful quiet). Good for you, cherish those testers, and make sure they have enough work so they won’t get bored and move on to a more challenging company.

2. Debugging doesn’t count, and testing takes too long

Like with any competitive company, your team must deliver on time, which means they must make estimates on the time it will take to deliver. Since your DoD doesn’t include testing, and you probably can’t guess how long it will take to debug the feature, what with all the cycling back and forth from development to QA, you estimate how long it will take to code it. If you want to meet your commitment, you can’t be adding a 20% overhead to your delivery time or you’ll miss the deadline. Worse, if you add 20% to your estimates, your manager might call you out on padding the estimates, which is his job. If that happens, who knows what might happen? Better play it safe.

1. It’s just a theory

Like Evolution (and Gravity), it’s just a theory. Even if all of the above reasons weren’t valid, nobody has ever successfully proven that this product could be completed faster and with better quality using new-age development methodologies like TDD. It’s just a matter of opinion.

Test yourself

Now, to test whether or not you should use test driven development, go over the above list. Count how many reasons apply to you. If you scored ten points, don’t use TDD. In fact, if you scored more than one (reason #8 might actually be legitimate), don’t write any code at all. Perhaps you’d be better served choosing a career that has fewer unknowns and moving parts. Perhaps paving roads?

Disclaimer: This post was written… Aw, just figure it out yourself!

61 comments:

  1. I think reasons 10, 9, and 8 are legitimate. The rest are all impossible!

    ReplyDelete
    Replies
    1. Yeah. Same. I am a heretic with regards to unit tests. I think the religious nature with which people TALK about it is overblown and the reality almost always falls short. The further you get from critical shared code, the less the value the effort delivers (IMHO). This is context dependent, of course.

      10 and 8 actually make sense to me. I started reading this thinking it actually was a list of times when unit testing is overblown. There are times where that is true, religion be damned.

      Delete
    2. This was intended to be a humorous blog post but it didn't come across that way. I think a lot of people are going to miss the very subtle humor here and take you seriously.

      Perhaps if you added a final paragraph to let the less astute readers in on the joke?

      Delete
    3. If there's no client - why write it in the first place?

      Delete
  2. Well, I agree with you on #8. #9 WAS considered a legitimate reason in the late 90's. Customers tend to be a lot more discriminate these days.
    #10 could be legitimate only for throw away code, or exploratory code. You just have to understand that the code isn't production grade.

    ReplyDelete
  3. The only valid point is #8, the rest are NO excuses not to do unit testing. #6: Documents lie on you; I'm not referring to Uncle Bob, but to personal experience. #1: see testing and complexity: http://peripateticaxiom.blogspot.com/search/label/test-first%20complexity.
    etc. etc. Sorry, I consider this blog entry harmful

    ReplyDelete
  4. I have to correct myself after reading the last section: It not considered harmful anymore, but as black humor. But if you are convinced about TDD, you probably won't read to the end.

    ReplyDelete
  5. @Daniel - I thought my sarcasm was obvious from the start, but I'm glad you read till the end. I wouldn't want you to think me a waterfallist :)

    It was kind of an attempt at black humor, but I tried to, in a backhanded way, contradict each reason with the value that TDD gives.

    ReplyDelete
  6. Does anybody think I should have put the disclaimer at the top?

    ReplyDelete
  7. I think a disclaimer at the top would take the fun out of it.

    I perfectly got it at 9 already and I think anyone should immediatelly get it just by looking at the heading for each point...

    ReplyDelete
  8. This comment has been removed by the author.

    ReplyDelete
  9. Maybe it was too early in the morning when I read the blog. Friends of mine laughed on me about my reply ;-)
    I was involved in the past in discussions where people seriously used the points above as excuses not to test. Therefore my rough first reaction.
    I would not add a disclaimer. It would really take the fun of your excellent and funny post. But maybe you can use a different color for the Test yourself section.
    Definitvely going to read your blog more often!

    ReplyDelete
  10. Without using sarcasm, there are some cases where TDD is useless. For instance when using languges that can be calidated from a formal or mathematical point of view, for instance Maude. As well as other paradigms like MDD, model driven development.

    ReplyDelete
  11. Isn't it every new technology is theory atleast practices if going via your opinion ? What motivates is to write better code which can sustain test of time and work on every possible scenario and TDD helps on doing that.

    Javin
    How HashMap works in Java

    ReplyDelete
  12. @Ricardo - I have to admit that I have no idea what "calidating" is (nor does dictionary.com, for that matter :) ), but if I wish to validate that a formula is written properly, and if I wish to give an example of how to use the formula, then [some kind of] TDD will really help.

    Regardless, the post is titled "10 reasons to avoid..." not "THE ONLY 10 reasons..." :)

    ReplyDelete
  13. @Javin - Just to be clear these reasons are NOT my opinion, but rather a parody of commonly held opinions (specifically a paradoy with a nod towards the last Miss Universe pageant).

    As stated at the top of the post, I'm a fervent advocate of TDD.

    ReplyDelete
  14. MDD is just an abstraction layer (generate the code from a model). TDD on models or generated code is not rocket science. For MATLAB see mlunit/slunit. An that's just some free ones...

    ReplyDelete
  15. Hi Assaf,

    You are suggesting that TDD should be avoided in many cases but you're not offering an alternative. What is the alternative in the case where TDD cannot be adopted? (other than paving roads, of course)

    ReplyDelete
  16. @PM Hut - Just in case I wasn't clear, I'm *NOT* suggesting to avoid TDD; Actually, I'm suggesting that you "avoid" TDD only in situations that can't actually exist (perfect architecture, perfect documentation, perfect developers, etc.)

    What I did suggest is that you _can_ forgo TDD if you're working on a very simple and straightforward project with no chance of having to maintain it in the future.

    So you *should* use TDD to drive your development. If it seems like you can't, it is likely that you're doing it wrong.
    (Hint: Separate the easy to test parts from the difficult parts).

    Hope this clears things up,
    Assaf.

    ReplyDelete
  17. Hi Assaf,

    you're suggesting that being stupid on the internet should be avoided, but you're not offering any alternatives. What is the alternative in the case where being clueless cannot be adopted (other than not reading the comments on your blog and still being a dick of course)?

    Best regards,

    Your readers

    ReplyDelete
  18. I agree with you: TDD is not always a solution, it has overheads that may impact the software development, not adding an overall value to the product. You have to weight it before using or not !

    Best regards,

    Wagner

    ReplyDelete
  19. Your post seems valid for unit testing or plain testing, not for TDD itself?

    ReplyDelete
  20. @Wagner - Regarding the overheads, see point #2 (debugging doesn't count and testing takes too long). In my experience TDD doesn't add value only in simple and short term projects. See the rest of the points for the value TDD brings to the table. Remember that if writing the tests is difficult, it is an indicator that there's a flaw in the design! This means TDD helps improve your design.

    The only thing I agree with is that you have to weigh whether TDD brings value to you or not. Don't cargo cult it.

    ReplyDelete
  21. @V. Narayan - while some of the values in my post apply to unit tests, as well as TDD, some of the advantages are gained only when writing the tests to drive the development, rather than just as a regression test suite.

    ReplyDelete
  22. > Actually, I'm suggesting that you "avoid" TDD only in situations that can't actually exist (perfect architecture, perfect documentation, perfect developers, etc.)

    There is no such thing as perfect. Or ... you are kinda cocky, aren't you? :)

    ReplyDelete
  23. @Anonymous - Well, I *will* admit to a certain level of cockiness, but yeah, there _is_ no such thing as perfect. My point exactly!

    Was that too cocky? :)

    ReplyDelete
  24. That's great. Until I got to point 7 I thought it was genuine. When I got to point 5 I knew for sure that it wasn't serious. Nice one

    ReplyDelete
  25. "Writing tests is a clear violation of the DRY principle"

    You are aware that poking fun at an obviously wrong way of doing things is how the waterfall model was born, right? My problem with the above quote is that there is nothing logically incorrect about it. In fact, at face value, it is brilliant. Just like Winston W. Royce's mockery of the waterfall model someone may mistake it for genuine serious reasoning, and bring the fiery, cataclysmic end of modern software development practices, ushering us into a new technological dark age. YOU HAVE DOOMED US ALL!

    ReplyDelete
    Replies
    1. OH MY GOD! I seriously hope you're wrong!!!

      Though I do appreciate the comparison to the late Winston Royce (and am aware of both his son's attempts at rectifying the situation), I do not fear the cataclysmic event you mentioned.

      For one, in the 1970s, the DOD was the single largest customer of software. In those monolithic days, a 4-star general with brass-for-brains could (and did) send the industry into the dark ages. Today, there is no single such entity. Anyone who subscribes to outdated, inefficient methods of development will simply go the way of the dinosaur.

      For another, call me naive, but I'm sad to say that I doubt that anybody dumb enough to think that this post is anything but sarcastic, will find his way towards reading this post.

      I do hope that you at least got a good laugh out of the post; your comment had me rolling for more than a minute.

      Thanks,
      Assaf.

      Delete
    2. All said with tongue in cheek, of course :)

      I'm going to print and hang that quote in my cubicle. Once said, it cannot be unsaid. And as with most quotes, no context will be included. *sinister laugh*

      (Blogger's OpenID integration is throwing a fit)

      Delete
  26. I didn't "hear" the irony until #7. :$

    ReplyDelete
  27. I got a great laugh out of this ... then I cried a little thinking about how many of the points actually describe attitudes of teams and team members I have worked with. Oh how I long to work with a team that takes every bit of your real message to heart. Thanks for the post.

    ReplyDelete
    Replies
    1. Hey, you're welcome to come join my team. I strongly encourage TDD on mine.

      But seriously, if you wish to work on a team that does TDD, start doing it yourself. Pick a project that is:
      1. Either green-field, or an existing one that you need to start a new module (or even just *can* start a new one),
      2. Not likely to be contaminated by people who *don't* write tests.

      Show others the virtues. Help others become as awesome as yourself.

      Oh, and don't get discouraged if it doesn't work smoothly. I had to work rather hard to convince others that it is worth it. Usually takes a month or two until people notice that your life is easier and better.

      Assaf.

      Delete
  28. Calling gravity and evolution 'just a theory' is a big mistake. The amount of proof that exists for both of those dwarfs the religious zealotry that goes on in software development

    ReplyDelete
    Replies
    1. I guess they simply refer to the fact that, so far, nobody has been able to explain what gravity is or where it comes from, just observe its effects. As for evolution, same thing goes for what its mechanisms are... is it really just successful randomness? or is it driven by something else? Ever heard of syntropy? http://www.syntropy.org/
      So yes, they are JUST theories, but I see your point in that they work out pretty well... which might not be the case for all computer scientists and TDD (what? what do you mean "flame-throwers ready, aim,..."?) :D

      Delete
  29. Awesome! Of course, there will be those who will read only the headline, look at the picture, and feel justified. Oh, well. That's approximately where evolution kicks in.

    ReplyDelete
  30. Totally agree... TDD is not a silver bullet, indeed, TDD isn't engineering per se

    ReplyDelete
  31. Great ! At begining I wasnt think that was a joke ;-)

    ReplyDelete
  32. No reason 0? What kind of programmer are you?!
    BTW, Great article! :)

    ReplyDelete
    Replies
    1. How about Reason 0 - Real Programmers never make mistakes?

      Delete
  33. That this post was satirical was completely obvious by the time I got to the second point. No disclaimer required.

    The funniest part about it is all the comments from people who didn't get it...

    ReplyDelete
  34. Awesome!
    http://i.imgur.com/mcmVW.gif

    ReplyDelete
  35. Reason #11: The team has written "automated" tests using QTP or similar tools and they believe their testers are super awesome to do all regression every single time.

    Reason #12: The project sponsor has lot of money to pay for regression testing in every iteration and also doesn't mind spending additional one grand on rework due to defects.

    ReplyDelete
  36. Great post, yet so sad that the subtle sense of humour was lost on so many of the commentors...

    ReplyDelete
  37. Reasons are not strong enough to consider .Even for small project i think automated test is a good choice .

    ReplyDelete
  38. There is nothing in the blog post that indicates to me it is a joke until the disclaimer... I don't know why you did this to me! I was so mad until I read the comments lol

    ReplyDelete
    Replies
    1. @Domenic, thanks for that. It's good to know that I gave you a good laugh. But don't be mad, read the comments - they're even funnier. There are some people here that didn't get the joke even *after* reading the comments... :)

      Delete
  39. As a road paver, I'm outraged that you'd suggest we use Waterfall methods. Regardless of what our clients may tell us about the condition of the ground, it's impossible to predict what you'll find once you start, so the tasks continue to change, messing up estimates and requiring agility.

    ReplyDelete
  40. Test Driven Development - is a fraud! The program is written using TDD contains more test code than production code. But unless someone is testing the test code? Of course not! Thus, under the guise of testing a project goes further untested code! Trying to follow the principles of TDD, you automatically break them! Moreover, if you have 100% test coverage, you have written your program twice! The process of writing code that only satisfies the tests can and should be automated. And then you do not need a production code, you only need a test code. But what is this test code? This is a declarative description of what should or should not do the program in every possible situation, without describing how it should do it!

    Do not give in to provocations, TDD - is simply too complicated and perverse declarative programming!

    ReplyDelete
    Replies
    1. Hi Dmitry,

      Thank you for your comment.I do, of course, completely disagree with you on almost every point you made.

      First, while there *MAY* be more test code than production code, this is, in and of itself, not a bad thing! One of the benefits of TDD is that it can help shape your code into a simpler design - which may mean less code. If TDD helped you reduce the amount of production code you wrote and have to maintain, then its own size is not a problem.

      Second, you state that nobody is testing the test code. While that may be correct in essence, in fact, it is (or rather, should be) of little consequence. BECAUSE there is no coverage for the test code, the test code must be written in a way that is so simple and intuitively understandable that automated testing is not needed here. Reread the third point I made (point #8), which is in my opinion the only valid reason not to test, which is that if the project (the test code) is so simple and straightforward, then it is a waste of time to test.

      Your third point is that trying to follow the principles of TDD you break them. That is simply not true. The principles are that you (1) write test code that proves that you need to change your production code base, then (2) write production code until the test passes, and then (3) refactor the code to improve design and remove redundancies. Rinse and repeat. Nowhere does it say that you need to write test code for every line of code you write, test code included.

      Your next point, that if you have 100% test coverage, then you have written your program twice is wrong on multiple levels: First, you've made a Straw-Man argument - 100% test coverage is neither a viable goal, nor a desirable one. Second, your test code is not a repetition of your production code. Your tests should NEVER repeat production code, but rather VERIFY the results. The simplest example would be that to test a complex algorithm, you need only run it with a known set of arguments and compare the result to a known correct answer. You EXERCISE the production code, you don't repeat it.

      Next, you state that the process of writing code that ONLY SATISFIES (emphasis mine) the tests can and should be automated. Whether or not this is true, I cannot say for sure, but I can say that this is another Straw-Man argument. You shouldn't write code that ONLY SATISFIES the tests. Or if you insist, you should write tests that make it so that ONLY SATISFYING them is the right production code (see this article on property based testing http://fsharpforfunandprofit.com/posts/property-based-testing/). Of course your statement on not needing the production code derives from the previous straw-man, and is therefore false.

      Your last point, that the test code is declarative, is actually the only point I agree with. I completely disagree with your sentiment, because this is a good thing. If it were an implementation spec, then the test would repeat your production code, and/or break every time you change your production code, and THAT would be a bad thing.

      So, yes, do not give in to provocations. Try TDD because it is worth the effort.

      Unless this whole comment was tongue in cheek, and you were simply giving me a treat in my own medicine...

      In that case, well played, sir. Well played.

      Best regards, and happy coding,
      Assaf

      Delete
  41. Test Driven Development - is a fraud! The program is written using TDD contains more test code than production code. But unless someone is testing the test code? Of course not! Thus, under the guise of testing a project goes further untested code! Trying to follow the principles of TDD, you automatically break them! Moreover, if you have 100% test coverage, you have written your program twice! The process of writing code that only satisfies the tests can and should be automated. And then you do not need a production code, you only need a test code. But what is this test code? This is a declarative description of what should or should not do the program in every possible situation, without describing how it should do it!

    Do not give in to provocations, TDD - is simply too complicated and perverse declarative programming!

    ReplyDelete
  42. I'm glad time and adoption has proven this article wrong. WeDoTDD.com

    ReplyDelete
    Replies
    1. Did you even read the post, or just the (admittedly click-baitish) title? Please read the whole thing, before you conclude that the article is wrong. You might be (pleasantly) surprised at how wrong you are...

      Delete
  43. You dont know what you are talking about man.

    ReplyDelete
    Replies
    1. Just out of curiosity, did you read to the end? Did you get that I'm actually in favor of TDD?

      Or do your think that TDD is in fact undesirable, and that the reasons I mentioned are actually valid?

      Delete

Note: Only a member of this blog may post a comment.