Tag Archives: code quality

Agile Metrics

Posted on by
Reply

In my last post, Is Working Software Enough, I lamented about the desire of teams to optimize velocity and talked about how that can lead to sub-optimal performance. It has the tendency to push design off the team and onto a separate team which is usually not the best solution (I won’t say never because there are always exceptions). As I mentioned in that post, we would be better off measuring and optimizing value delivered…but how do we do that? Over the past week we have been talking about metrics here at the office as we have been starting to think about changes to our company and department goals for next year. So I thought I would bring up some of the thoughts that have come up that seem like better things to measure and work on optimizing than velocity.

Customer Satisfaction/Delight
If we are delivering valuable software to our customers frequently, they should be satisfied, correct? Given there are often many other aspects that go into customer satisfaction like sales, support, and training just to name a few, this could be a stretch. However, it is still something that most software product companies already measure that is a good indicator of the overall value delivered to customers. Also, there are often some lower level questions in these surveys that can tie back more closely to the software specifically. For example, one thing we have added to our satisfaction survey is a question about perceived quality of the software. Obviously this is a lagging indicator, but depending on how frequently you measure this it may not be lagging by much. Some people avoid lagging indicators, however I think they are important because in reality, there is no way to determine whether what you did was successful or not until after it is done. We can always come up with leading indicators that have historically predicted success, but that doesn’t mean they will always predict success.

Usage
An obvious measure of value could be revenue, or even better, profit. However, calculating this can often get very complicated especially if you are doing incremental enhancements. So if you are having trouble measuring revenue, another measure could simply be usage. If people are using the software, that is a good indicator it is providing them value and this can be much easier to track than revenue in many cases. If you are working on delivered software that would require users to upgrade, looking at the number of clients that choose to upgrade would potentially be another good indicator of value delivered with that release.

Focus Factor
So we’ve discussed two good lagging indicators. Now we get into some leading indicators. One thing that companies usually want to measure is efficiency. Focus factor has been around as a measure for some time, though the standard definition has some issues with it such as it was usually only done at an individual level and assumed that all time spent on a task was productive. Scott Downey at RapidScrum has re-defined this measurement so it does a much better job at measuring efficiency on a scrum team. Using Scott’s new definition, Focus Factor is a measure of how much effort during a sprint resulted in requested, completed, accepted working software. If you’re familiar with Scott, you will know he never uses hours. You always estimate in story points and you also report work in story points. So you end up with some number of story points reported during the sprint (work capacity) and velocity (the sum of the original story point estimate of completed accepted tickets). So a team’s Focus Factor would be their velocity divided by work capacity. An added benefit of this measure, since it is a percentage, it can be used across teams even if they have different story point scales.

What is a desirable focus factor? Well, explaining that isn’t really the focus of this post, so for now I’ll just tell you that 80% is the target for focus factor (this number comes from W. Edwards Deming). So anything too far away from 80% (lets say lower than 70% or higher than 90%) would be an indicator that there may be an issue. The team may be committing to too much or too little, or a number of other things could be going on.

Found Work
In Mike Cottmeyer’s recent post on The Real Reason We Estimate, he points out that many times, an estimating problem really isn’t an estimating problem at all, it’s a shared understanding problem. So if we buy into that (which I do), what is a metric we can look at as an indicator that there is a shared understanding problem? Scott again provides a key measurement here he calls Found Work. Basically, it is a measure of how much unexpected complexity a team discovered mid-sprint on work they had already committed to. So how well did the team understand the work going into the sprint. Scott measures this as a percentage of the original commitment, so he takes the work reported on a card minus the original estimate on the card (again, both estimates and reported work are in story points in this case) and divides that by the original commitment for the sprint (in story points). So this would give us a percentage and the closer to zero the number is the better the shared understanding.

Conclusion
So there we have four metrics, two lagging and two leading, which can help provide good indicators on how our scrum team is performing. All of these measures also encompass the dynamic between the product owner and the team. Both parties need to be doing a good job for these numbers to be good. If the team isn’t building valuable stuff then customer satisfaction and usage would be low. If the team is not working efficiently, then the focus factor will be out of the target zone. Found work being high indicates a low level of shared understanding either between the team and the product owner or among the team about the technology. Does anyone else have other metrics they like to track on their agile teams?

UPDATE: While I was working on this post, Mike Cottmeyer posted an article on his blog, Gaming the Numbers, that talks about this subject as well and there are some good thoughts and discussion in the comments thread I would encourage you to check out.

Does Your Code Tell a Story

Posted on by
Reply

I recently attended a presentation given by Alan Stevens entitled Does Your Code Tell a Story. There are many metaphors that are commonly used in software development (construction, manufacturing, etc) but in this talk, Alan discussed a much less commonly used metaphor, comparing software development to writing. It was a great presentation full of lots of good points and ideas, but there were a couple that stuck out in my mind that I would like to focus and expand on here.

Read a Lot, Write a Lot
When asked how to become a better writer, most writers will suggest one of the most useful things to do is to read a lot and write a lot. I think most developers do a fairly good job at writing a good amount of code, but the general population does very little in the way of reading a lot. The most obvious way to increase the amount of code you read would be to do peer review of code other members of your team have written. Peer code review on development teams is starting to become a much more common practice, but still has a decent way to go. One common pitfall to avoid is treating this as solely a quality control practice. Peer code review should not just be a bug hunt, but should also be treated as a learning experience. You should obviously review code for areas of the code base you are familiar and definitely look for bugs, but also review code for areas of the code base you are less familiar with. You will learn more about your code base and often be much more open to new ideas when reviewing changes to code over which you don’t necessarily feel any personal ownership.

Even if you are doing peer code review or after you have it implemented, if you really want to take your game to the next level, you need to read and write even more code. The obvious place to make this commitment would be on an open source project. There are tons of projects out there you can contribute to, likely one you’ve probably used before and have some ideas on what sort of enhancements could make it better. As your working on writing those changes you will be forced to do some level of code review to understand the code base and how to make your change, but you shouldn’t stop there. Try to understand the code base as a whole, see how they solved certain problems. Maybe you use some third-party components in the software you work on. If so, you likely have the source code and that is another great candidate of code to review. Find out how it works, see how that developer solved certain problems.

The First Draft
A writer would never release his or her first draft, however, this is very common in software development. We have a problem, we write some code to fix it, we do some quick tests, and we push it out the door. Writers send their text through a rigorous review process and developers should do the same thing with their code. Perform your own review several times and then have as many people as possible review it for you. We wouldn’t think twice if a writer said they rewrote entire sections or maybe even scraped an entire chapter or more and started over, yet this practice is heavily questioned in software development. We don’t always know the correct way to solve the problem just as a writer doesn’t always know the correct way to tell the story. It may take a couple tries, it may take some tweaking and refactoring, it may take a complete rewrite. Obviously, this has to be balanced with making forward progress to a degree, but it should not be completely frowned upon, writers have to meet deadlines as well.

As Alan pointed out, writers often refer to the first draft as the “down” draft. The main focus here is to get something down. It may not be perfect, but that’s okay, they know they will revise it and perfect it. Having something to work with and revise is much better than having nothing to work with. The one thing I will point out here is that this would only be acceptable if you really are planning to revise and perfect the code later. If you’re planning to write and ship your first draft, then your first draft needs to be a little more polished. However, I think you will definitely find that you end up with much better code when you focus on getting the problem solved in the simplest way possible at first, then revisiting and revising it later.

Conclusion
Comparing software development to writing brings up some good practices that are definitely worth considering and using. It is definitely a metaphor that would be useful in supporting the practices of peer code review and refactoring. The next time someone asks why you had to spend time refactoring a piece of code, ask them if they usually turned in term papers in school after one draft? If they happen to be someone who has published anything or written larger, more important papers in graduate school, they will definitely get the picture. Of course, like any metaphor, it is only useful to a point. By definition, a metaphor compares two unlike things that actually have something in common, so remember, it also means those two things have many characteristics that are not common as well, so don’t try to take the metaphor too far.