A Software Developer’s New Year’s Resolution

Or: 5 Things You Can Do Today to Improve Your Software Product

New Year’s eve is around the corner. It is the end of this year, and there’s a whole new year full of potential to be better, just waiting to begin. So how can you make it better? If you, like me, are a professional developer, then you can improve your life (and your boss’s and your team’s) by reducing software delivery related stress. Here are 5 things you can do, effective immediately, to make your product better:

1. Make a Version Management Plan

If your version control system is a mess, the first thing you should do is straighten it out. If you are not using one, then you should begin doing so immediately. There really is no excuse not to do so. For starters, there are plenty of high quality tools out there that are open source and free (git, mercurial, and subversion are the most common ones), as well as other commercial tools (I happen to use and consult on the use of Microsoft’s TFS). Either download and start using a free one, or get one that offers a trial period.

If you are using a VCS, but it’s a mess, and you find yourself mired deeply in merge-hell every time you want to deliver your software (internally or externally; it doesn’t matter), then you should straighten it out. Unfortunately, if you have a considerably sized source repository, this may seem as an impossible endeavor. Here’s one way to alleviate at least some of the pains of software delivery, in a few easy steps:

1. Create a new root-level folder named sources. This folder will contain your new source code repository.
2. Create a branch from your current repository, and call it main (e.g. /sources/main). This will represent your latest and greatest version, and should be kept as stable as possible, at any moment. No development will be done on it; you will only merge completed changes into it.
3. Create a branch from main, named dev (e.g. /sources/dev). This branch will be the branch that developers work on to create new features, checking code in and out from there. If there are multiple independent teams, you may have multiple dev branches.
4. Create a releases folder, which will contain release branches from main. Every time you wish to release your software (internally or externally), create a release branch from main (e.g. /sources/releases/1.2).

This is just one possible branching plan. It works particularly well for agile development, where effort is made towards developing as many features as possible before a release, merging them into main when ever they are done, and branching when the iteration is over. Note the following division of work:

• New features are developed only in dev. They are merged back into main when completed.
• Fixes are developed on release branches and reverse-integrated into main, and then back to dev, so that they won’t get lost.

This structure allows you to develop new software and stabilize a release at the same time without one effort polluting another. It is also a step towards being able to release a new version anytime.

If your development and release mechanism is well established, but not suited to fit this branch plan, make one that does work. Just make sure to be able to support feature and fix separation as well as the ability release anytime you need.

2. Automate Your Build

Even if all you do is copy sources from the repository to some drop folder, you have some sort of a build process. Automate it. Even just a part of it. Even just the compilation phase. Every step you let a computer do rather than yourself, is one less step to worry about. If you aren’t already using an automated build tool, you should start now. There are many open source tools as well as commercial ones. Most are powerful enough to suit your needs. Remember – If you can run it from command line, you can automate it as well.

Every time you run the build, you will not only create a deployment package of your system, but you will get a good indication about the health of your product (if the product doesn’t build, you will want to know sooner rather than later). Additionally, you will have the start of a platform for testing the system.

For a synergetic benefit with your versioning plan, schedule the following automated builds:

• Build the software every time you check in changes to your code (on dev or a release branch); most build and VCS tools support this – it is called continuous integration.
• Build the software every time you wish to merge dev to main. It is even better if you require a successful build as a prerequisite for the merge. This is called a gated check-in in many tools.
• Schedule a nightly build on main. This will let you know what the state of your system is. Fix any problems here first. Keeping a healthy build will allow you to deliver software at the drop of a hat. Amaze your PM!

3. Add a Health Report

Even a simple email message to the team in case a build fails will greatly improve your ability to fix problems with your software. Shorter feedback cycles means that you can fix problems when their causes are still fresh in your mind! Let your build system email you if it fails. Almost every system supports it. Conversely, you may wish to post successful build reports to some company wiki. Let everyone know of your successes.

4. Add an Automated Test

While the benefits of TDD as a design mechanism are far greater than just having a suite of tests, we should not belittle the benefit of an automated regression test. Even a simple record & replay automated smoke test, with a black-box test tool will greatly improve your system. Start with a single test that runs through as many parts of your system as you can easily manage. (Note – this might mean you need to add a deployment action to your automated build). Report if it fails. Run it every time you build. The more tests you add, the greater confidence you’ll have in your build system! Maintain your tests; fix any tests that fail due to legitimate changes and remove those that are no longer relevant. You don’t want your system to cry wolf!

5. No Broken Windows!

The most important thing you can do is to make a new year’s resolution that you will not suffer any problem with your software delivery lifecycle to remain unfixed! Once you fall off the wagon, your system will start to decay back to uncontrollable chaos. Think of maintaining your ALM like brushing your teeth. You may not like it but you do it every day (more than once), or your teeth will rot out until ultimately you won’t be able to eat!

In conclusion,

My new year’s resolution is to vigilantly keep my teams’ products “on the wagon”.What is yours?

Happy Holidays, and Happy New Year!

See you all next year,

Assaf.

Technorati Tags: Agile,Build,TDD,Test,Report,Version Control

How to Measure Technical Debt

Or: An Executive’s Guide to the Cost of Software Architecture Shortcuts

In my experience, in any software project of sufficient complexity, it is necessary to cut corners in the software architecture. These shortcuts are decision that you make now so that you will be able to release your next version of the software that much earlier; these shortcuts are the result of not having enough time to do things “right”.

This is called your project’s Technical Debt.

Like financial debt, which is the metaphor used to explain it, technical debt happens when you “buy” more software features than you have resources (developers and time) for. Like financial debt, it accrues interest over time. And like financial debt, if you don’t pay it back, you may eventually get foreclosed and go bankrupt. That last part is not a metaphor. Your software project simply ends up costing more to maintain than the value you derive from it, and either the project or the company gets shut down, or both.

There are essentially two ways to deal with debt (technical, like financial):

1. You pay it all at once – This means that you do not “withdraw” any more features, until your debt has been fully paid. It has the benefit of being able to make a fresh start. It has the disadvantage of risking “starvation” while you try to pay the “bank”.
2. You pay it in installments – This means that you pay off your debt, a small part at a time, still “withdrawing” features, but less than you normally would, cutting down to essentials, for as long as you are still paying off the debt. It takes longer to finish, but you can still “eat” while paying.

Of course there’s the third option of continuing to withdraw as before, until you go broke, or luckily win the ‘lottery” (investment, buy-out, etc.)

There is no one clear way to deal with debt (financial or technical). It will depend on the situation, and the players. And most of all it will depend on the amount of debt.

Quantifying the Debt

With financial debt, it is easy to determine how much you owe. There’s the amount of money you borrowed, and the interest rate. Both are usually well known, and it is just a matter of keeping track of it, to make sure you can withstand it.

With technical debt, you still have the amount you borrowed, and the interest rate, but you usually don’t know the numbers.

Let’s start with figuring out the derived value of taking the loan. Whenever you demand that a feature or set of features be completed faster than it would normally take if the team were to maintain an appropriate quality level, you are “borrowing” from the architecture. Figure out how much time you saved, and multiply it by the hourly cost of the developers. That is the amount of money you borrowed from the architecture. Figure out what the reduction in the Time to Market will earn you, and that is the derived value of the loan.

The interest rate is not so easily calculated a-priori, but it can be estimated (with about as much accuracy as the development cost estimates your team makes – which may be good or bad; depends on your experience):

1. Map out the areas impacted by the shortcut
2. Estimate how much it would cost (developers x time) to develop a feature in the impacted areas, before the shortcut(s)
3. Now, estimate the cost to develop the same feature after introducing the technical debt

The difference in the cost is the interest you are paying. Keep track of the interest as it will change with every change you make to the system. A good practice is to calculate the interest every iteration or sprint. You should consider doing this as part of the sprint planning. This will have the added benefit of giving the product owner the opportunity to pay off some of the debt.

Note that like financial debt, the interest grows over time. The rate varies, and it is difficult to guess what it will be, except by projecting based on past experience. As you begin to accumulate this data, you will be able to make educated decisions on when you can afford to go into technical debt, and when you should avoid it. You should reassess your technical debt’s interest rate every sprint planning.

Armed with this knowledge, developers should now be able to collaborate with management in order to decide how to develop and whether or not it is okay to cut a corner on that next feature.

Finally, remember: Your architecture is a cold blooded loan shark, who will just as soon take your installments, as he will bust your project’s kneecaps over a missed payment.

What do you think? Do you have a preferred method of calculating / estimating technical debt, or a way to handle it?

Please share!

Technorati Tags: Agile,Estimation,Scrum,Sprint Planning,Technical Debt

10 Reasons to Avoid Test Driven Development

Test Driven Development (TDD), and all of its derivatives (BDD, ATDD) are, in my opinion great methods to drive a team’s development efforts, and raise the quality of the product. But TDD is not a silver bullet. It doesn’t fit every project. The following post lists the top ten reasons not to write automated tests for your code.

 

If one or more of these conditions apply to you, you should consider forgoing TDD, and in fact any agile techniques, as they will be a waste of effort:

10. There is no client

Sometimes you are developing a product that will not be used by anyone. In this case, any effort expended on raising the quality is a complete waste. Nobody will care.

9. The client is an avid tester

Some people love nothing more than to beta-test new software. The joy of finding new bugs and trying to figure out what went wrong is what they live for. Others are scientists at heart, and love trying to go over stack traces, in order to reverse-engineer the code. If your client happens to be one of those, then writing automated tests will take all the fun out of using your software. Don’t do it!

8. The project is short, simple and straight-forward

If your team can complete the project in a short period of time (no more than a few weeks), and will never, ever have to reopen it for maintenance, then the benefits of maintainability, reusability and extensibility will be lost on you. Spending time and effort on these values is wasteful.

7. Your architecture is perfect

If there is no way to improve your architecture, then there is no need for it to be extensible. TDD, by the nature of its incremental development flow, forces the architecture to be extensible, simply by making you extend it as you go, and this, like lipstick on a pig, is something you just don’t need.

6. Your documentation is perfect

You never miss an API, and any change you make to your software gets documented instantaneously. The tests you create with TDD serve as a form of documentation, an example of how to use the API. Properly named and written tests actually explain the context of the test, so it is easy to find the test that shows you what you need to understand. Your documentation is so complete, that writing tests are a clear violation of the DRY principle, so you should clearly avoid tests.

5. Your team never changes and all members’ memories are perfect

The collective memory never forgets a single line of code it wrote, nor what the context was when writing it. You therefore, do not need the tests to remind you what the code does, why it does it, or how to use it. This also means that your team members never leave, nor are any new members recruited, because if this were to happen, you’d lose memories, or have members who don’t remember the code (having not been there when it was written). If this is the case, don’t bother with tests; they will just interfere with your incredible velocity.

4. “Done” means the code is checked in

Many teams have a definition of done (DoD) that means that the feature is “done” when it is in a state that the end user can receive and run (coded, tested, deployed, documented, etc.). Many others however, your team included, prefer a simpler and more easily achieved definition that accepts “checked in” as “done”. For you it is sufficient that the developer declared that he or she completed his part, and anything else is someone else’s responsibility. If you don’t need the code to be tested for the product owner / manager / user to accept it, then you are better served moving on to the next feature as soon as you can, instead of dragging on your relationship with this feature.

3. You’re paid to code, not test

Ignoring the fact that unit tests are code (sophistry), testing is what we have testers for. Perhaps your team's testers are fast enough that they can run tests on your code and give you feedback within mere moments, pinpointing the areas where you broke the code, so you can fix it while the changes are fresh in your mind, as well as a complete regression suite on the product, in case you broke something in a different component every night (they don’t mind working nights; they love the peaceful quiet). Good for you, cherish those testers, and make sure they have enough work so they won’t get bored and move on to a more challenging company.

2. Debugging doesn’t count, and testing takes too long

Like with any competitive company, your team must deliver on time, which means they must make estimates on the time it will take to deliver. Since your DoD doesn’t include testing, and you probably can’t guess how long it will take to debug the feature, what with all the cycling back and forth from development to QA, you estimate how long it will take to code it. If you want to meet your commitment, you can’t be adding a 20% overhead to your delivery time or you’ll miss the deadline. Worse, if you add 20% to your estimates, your manager might call you out on padding the estimates, which is his job. If that happens, who knows what might happen? Better play it safe.

1. It’s just a theory

Like Evolution (and Gravity), it’s just a theory. Even if all of the above reasons weren’t valid, nobody has ever successfully proven that this product could be completed faster and with better quality using new-age development methodologies like TDD. It’s just a matter of opinion.

Test yourself

Now, to test whether or not you should use test driven development, go over the above list. Count how many reasons apply to you. If you scored ten points, don’t use TDD. In fact, if you scored more than one (reason #8 might actually be legitimate), don’t write any code at all. Perhaps you’d be better served choosing a career that has fewer unknowns and moving parts. Perhaps paving roads?

Disclaimer: This post was written… Aw, just figure it out yourself!

Technorati Tags: Agile,Scrum,TDD,Estimation,Unit Tests,Definition of Done