Yesterday during the monthly Agile Belgium Drinkup (which is an informal user group meeting), the never-ending discussion “Is Quality Negotiable?” led to some interesting thoughts.
Most people in software development have experienced the moment in a project where the quality level starts to be discussed.
In agile projects this happens on 2 levels:
- Do we keep a zero defect policy?
- Does all code need to be covered by tests?
1 . Do we keep a zero defect policy?
Agile software development is centered around the idea of delivering a potentially shippable product on a regular basis. In practice this means that at the end of an iteration, your application needs to be in a healthy state, ready to deploy to production. This can be interpreted in different ways. Some claim this means you can’t have any open bugs anymore, while others categorize bugs and agree on which categories should be fixed immediately. The first group doesn’t negotiate on quality, while the second group does. I personally think that there is no right answer because it often depends on context.
If your project has a clear focus on one of the angles of the iron triangle (I hope we’re past the phase where we thought all 3 could be fixed), this can strongly influence the discussion. For instance:
- Create a release before a certain date
Some projects need to be released before a certain date. Because your product is targeting Christmas shoppers, for instance. Or your company will have to pay a huge fine if it crosses a date.
Since the schedule is that important, stakeholders will focus on continuously monitoring progress to quickly adapt when the outlook on making the date is starting to look bad. One option to adapts is to work on the team’s capacity, by helping them to improve their process or getting extra help. Another option is to work on the scope, by making it more simple, reducing it, or negotiate on quality.
This last one is a tricky thing to do. Sure, deciding to only fix critical bugs, will make the team go faster, in the short run! The problem is that bugs start to pile up, causing more and more issues in the process. It will decrease the stability of the system, increase confusion while testing and make your process less lean because you regularly need to review the open bugs and categorize them. Even the release will get more difficult, because training and support need to be aware of the bugs. Risk increases that after the release you will get a lot of bug fix requests, causing an extra investment on top of support.
So is it worthwhile, then? To some degree, in my opinion. Many bugs, whether critical or low, don’t take that much time to fix if you do it on a continuous basis. Many projects have bugs of a generic type, who take more effort to fix because they are linked to the design of the application. This is where you can gain time if you want to make your deadline. But make sure that the number of the bugs you won’t fix stays extremely low and get them high up the list of the second release. Otherwise you’ll soon be in the mess I’ve been describing.
- Get as much value for a fixed budget
It’s healthy to set a target budget when you’re still figuring out the functionality of your system. This gives you a baseline that helps to focus on not overdoing it and making a proper business case.
When budget is your main constraint, of course your stakeholders want as much value for it, as possible. So naturally, at some point, quality will be traded in order to have some extra features.
Again this is a tricky thing to do. For a while, the illusion exists that you got extra features and therefore made a nice deal. But soon things start to backfire in the sense of increased overhead, slower delivery and most importantly an increased risk on a budget overrun by extra support and bug fixing afterwards.
Good idea? In the short run, maybe. In the long run, certainly not.
- Release a certain scope set
If scope is your only constraint, it’s much easier to strive for a zero defect policy. Since scope is much more important than time or budget, stakeholders will generally have a good idea about what the functionality needs to be. Keeping the product high quality will guarantee that you’re living up to expectations. The features are the main focus of everybody involved, so they better work!
Continuously attacking all bugs will make the release much more comfortable and the risk on having to spend an extra budget on patches afterwards will be dramatically reduced. So from an investor viewpoint, this increases reliability, which is always interesting if you’re in that business.
It seems like I always come to the same conclusion. Quality can be negotiated, but to a very small degree. And even then it will generate a cost that you will have to pay back eventually.
2. Does all code need to be covered by tests?
Since I have limited technical knowledge, I will not even try to say something intelligent about this and leave it up to Marc Lainez to write a blog post about it 🙂
If you want to talk about this kind of stuff, or just want to tell me in person how wrong I am, come to the next Agile Belgium Drinkup.
It may be just an informal gathering, but I’ve gotten new insights each and every time!