Jeff McKenna's Agile Development Blog

Thoughts on Individual Measurement in Agile Development

Jeff: We talked in an earlier post, The People Part of People, Process and Tools in Agile Development, about the difficulties of measuring people. This is a real issue as Agile scales up and becomes more mainstream. HR concerns, such as individual career paths and growth in capacity in Agile development projects, have becomes more prevalent. Agile really started with small teams on small, pilot projects. This is a new problem in Agile.

Bill: Is this problem because Agile was initially done for small projects and now it is being used for much more. Or is the problem that is especially acute because Agile focuses more on people issues than traditional software development methods.

Jeff: Both. I think the people issues are more acute in Agile. Before Agile, we tended over the years to do things like rank ordering all the people in the organization through an evaluation process. We have known companies that have literally fired the bottom ten percent of people once a year.

Bill: Of course that encourages everyone to stab their neighbor in the back.

Jeff: Right. It has particularly bad consequences on Agile because Agile is a team sport. If you are always throwing out a person on the team there will be jockeying around this. So if Agile is a team sport how do we measure and help people with evaluation and growth?

To address this concern, we put together some guidelines on this for Agile that people have used. I do not claim to be an expert in this area. There are a lot of trials going on.

Here are the guidelines I recommend. For an individual I think about half of their evaluation should be based on how the team is doing. So we need some metrics on how the team is doing in terms of being an Agile team. We will talk about this in the another post but let’s assume that it is possible to implement.

Bill: So half of an individual’s performance measurement is based on the measurement of their team?

Jeff: Right. Now if you have lot of people in an organization that are not part of a team but just floating around, of course this does not work so well. We prefer when doing Agile to get people on teams and leave them there. The team dynamics are the difficult problem. The difficulty is not the coding, it is not the work people are doing. It is the team dynamics. In Agile we are suggesting that the team is the right unit to think about planning and execution, not the individual. Hence a team evaluation is an important component of an individual evaluation.

So let’s assume that you can evaluate teams. This leaves the other 50%. I think half of it (one quarter of the evaluation) should be on how that person is perceived by the team. So the team gets some say on how you are doing. Sometimes we see a situation where a team does not like working with a particular person. We have been in places where a team says we don’t want this person on our team. If we find a better place for that person a win-win situation occurs.

This indicates a level of team maturity that does not always exist. Teams need to know themselves and be able to talk openly to make such a conclusion. And, of course, it has to be handled well. But it is possible. Once, we took a team member off a team and its productivity instantly went up by 12% because this person was slowing the team down. His work was fine so I want to be clear that this was an issue of team dynamics not individual capacity. It does mean that you want to fire the person but they need to be on a different team.

The analogy I think you need to look at is the sports team and watching how people move around. Look at how some people work well in one team and not in another. This is a people issue.

Bill: I was about to mention sports earlier because in many sports situations, more so than business, the individual team member’s evaluation and their monetary rewards are based on team performance. Do they make the playoffs, etc. Someone is not considered a real MVP unless the team does well. We certainly experienced this in Boston when the Red Sox got rid of Manny Ramirez, one of the best hitters in baseball. The team did better because everyone on the team, except one, voted that he should go. Maybe they were being like a mature Agile team.

Jeff: Exactly. The team is trying to figure out how to be a better team. Team balance is very important. Sometimes the team needs a star to disrupt the balance and inspire them to do different things. Teams tend to know this more than we give them credit for. Managers think they know what is going on but the team often has better knowledge about this. Of course, good managers understand this.

So half of an individual’s evaluation is how the team does. A quarter of the evaluation is based on how the person is viewed by the team. 360˚ evaluations are a pretty classic way to do this.

This leaves the remaining 25 percent. Team member usually have an area of technical or functional focus. They might be in documentation or development. So we expect that this last 25 percent should be based on their growth in their functional areas. You can set goals for each person, such as learning a new computer language or a new technology. Some companies get pretty formal about this with classes and credit. Thus, the last 25 percent of an individual’s evaluation should be about their functional skills and if they get better.

Bill: I think one of the keys to success relates to something you said earlier. These measures cannot be comparative. You should not rank people based on the individual measures so it is not competitive. It takes out the competition. It has to be normative rather than competitive. Everyone can get an A if they deserve it.

Jeff: Exactly. Otherwise you cannot get a high performance team to engage in making the team succeed.

In looking at team dynamics, a lot of people have been exposed to Myers Briggs personality profiles. A lot of companies use this and many people know their ‘type’. Myers Briggs is very popular and the most widely known of these methods. Myers Briggs asks questions about how you are, how you function. If you are an introvert you tend to think before you speak. If you are an extrovert you tend to speak before you think. Teams that learn profiling such as Myers Briggs learn that different people function in different ways. Team members need to know this. As an example, if you have an introvert on a team you learn you must give them a chance to speak. If you make decisions at a team meeting too quickly it can inhibit the introverts from speaking.

Bill: There is the method of not letting any one speak twice until everyone speaks once.

Jeff: Yes, that does it.

In another method Kathy Kolbe is asking different questions. The main question she is asking is ‘what makes you want to do something?’ What motivates you to come to work? It refers to the will to act. Her classification includes things like starting new projects or being a follow through person. Do you like to complete things? Do like to be a fact finder? Do you like to have all the facts before you move on? Or do you like to make things beautiful?

Alaska Airlines uses the Kolbe method for their employees. On each employees badge there is a graph of their Kolbe metric. People use this. Kolbe suggests that better teams are balanced. We need some quick starts to ge going but if the team does not have any people who finish things the team will have difficulty completing anything. I have found Kolbe much more useful in making sure teams are balanced with the right mix of people to start and finish things.

The work Kathy Kolbe did came out of the skunk works projects from several years ago. The skunk works were a team concept that was developed at Lockheed as a better way to make planes. The U-2 and Sr-71 were both developed in this way. (They were very Agile!!!) Soon Skunk Works became very. In practice some worked and some did not. Kolbe looked at this. She discovered that a lot of these teams would get started quickly as they attracted a lot of quick starts but then no one on the team wanted to finish things. And they did not finish things. We would prefer our teams to finish things.

To summarize:
50% Team performance
25% Team evaluation
25% Individual growth in functional area
_________________________________________________
100% Individual evaluation

Comparing Extreme Programming, Scrum, and Lean Software Development in Agile

I have been talking to a number of people, including some journal editors, who seem to be concerned or even confused about the differences between Extreme Programming (XP), Scrum, and Lean Software Development in Agile. So I want to explain the differences in this post. Bill and I talked it over as I explained these concepts.

Bill: So what are the differences between these three approaches and why is this an issue for some people? What are some of the questions you are hearing?

Jeff: Well, for one, some people see these as contrasting flavors of Agile. They ask which one should we use, for example. These are different aspects of Agile but they do not operate at quite the same level. Asking which one to use is not the really right question.

My view is that these three all relate to Agile but represent different focus points. At the most granular level, Extreme Programming or XP is the one that caught attention first. It is targeted toward individual team practices. It is mostly about what individuals do to improve their development practices such as Test Driven Development, Refactoring, Small Releases, and Simple Design. XP also describes other practices that the team can use to improve its function such as the Planning Game, Daily Stand Ups and Collective Code Ownership.

Since Agile mostly started with individual teams, Extreme Programming was very valuable to them. It taught people a lot of things that are now not seen as radical as they were when they first appeared in the early 90s when XP became visible. So things like test driven development, continuous integration, and refactoring are not seen as very radical … now.

Many people use these three techniques whether they are doing Agile or not since they promote good engineering practice. One of the ways we have seen Agile fail is when it is implemented on top of poor engineering practice. This was not the only thing XP did but it was its major focus.

Scrum predated Extreme Programming in creation but not in terms of popularity. Scrum is agnostic about team development practices and assumes that you have good ones. It is more concerned with work management and team dynamics. XP does have things to say about team dynamics but Scrum is a little stronger on them.

Bill: How is Scrum stronger?

Jeff: It is stronger as it puts in place some behaviors that were not explicitly called out. For example, in retrospection, that we discussed before, the roles are more clearly defined in Scrum than XP.

Bill: Is Scrum a more articulated and defined Extreme Programming? Is it taking it in new directions?

Jeff: No. Scrum, as defined, actually doesn’t say anything about software. Scrum is about work management and team dynamics that can be used in non-software projects. Most people use it in software development and books refer to it in this way but Scrum does not call out any specific software practices. XP, in contrast, is very specifically for software development.

Scrum did not take any of the Extreme Programming practices except the daily standard meeting. It is targeted at a different place, for the management of work and team dynamics. Because of this it has things to say about scaling to larger teams while XP is quiet about this.

When you get even larger in the enterprise then Lean starts to come into play. Lean is a lot like some of the Six Sigma work in that Lean is looking at reducing waste and looking at ways to eliminate unnecessary work, value chain analysis. Scrum teams often use Lean thinking to improve the management their work and that has some interesting implications.

Bill: So if I understand you, it is really a layered situation.

Jeff: That is how I think about it and yet there are things that cross continuously. Lean concepts are useful on any Agile project. So, in fact, I see Agile projects which use all three.

Bill: Extreme Programming started down in the trenches. Scrum is there at the next level at the team leader or supervisor level. Then Lean goes up another level. It goes at the cross team, enterprise level. So all of the higher levels ones can flow back down to the layers underneath.

Jeff: Yes, but the thing I want to emphasize is that those are not cut and dry levels. There is lots of overlap. When XP talks about the practice of pair programming, which is one of the more radical concepts, there is no counterpart in Lean. But the Lean concept of taking smaller bits flows down to XP.

Bill: So it seems clear to me as to why they operate at different levels but you say there is confusion. What is the cause of this confusion? Why do they think they are mutually exclusive?

Jeff: When people hear there are different flavors of Agile, they think they are distinct and one has to choose one or the other. Some of the advocates for the different approaches present their way as the better way over the other approaches. This is what I see going on and it creates unnecessary confusion.

The People Part of People, Process and Tools in Agile Development

Jeff: In this post I want to talk a bit about the people side of Agile. The first item in the Agile Manifesto states, “We value individuals and interactions over processes and tools.” When you ask Agile practitioners what this means in a more informal sense and they will say that this means we talk to each other. We really communicate a lot more than we are used to in teams.

Now everyone agrees that we should talk more but there is more to it in my experience. Kent Beck in Extreme Programming Explained talks about values. He talks about communication, honesty, and feedback. These are things that you have to do in interactions with people. Let’s take honesty. Honesty is not, in my opinion, that there is some absolute truth for all to know. Honesty is about saying what is true for you in the specific moment.

Feedback is being able to say to someone when they have done great work and also being able to say when it isn’t working so well. There are always both of these sides to it. It is not all one way. I sometimes see executives talk about communication and they simply sit and talk to you about communication. But communication is a two way street and it requires a level of trust and safety in the group.

Now there are some mechanics in communication theory. For example, Virginia Satir’s communication model makes the mechanics very clear. But if you are not comfortable with saying something to someone then it is not going to happen no matter how good your model of communication. If you do not feel safe in saying something in the team, it will not happen. Good friends build up trust, a sense of safety, with each other over a long period of time. Over time they can say things to each other with a deeper sense of honesty.

I am not suggesting that members of teams should immediately turn into truth-telling machines that are always telling everyone when they are ticked off. I am not talking about that kind of response. But I am talking about teams getting to the point where unpleasant things can be talked about openly.

We discuss in the retrospective process about talking about things that did not go so well. This is one reason why I suggest that retrospectives be kept within the team. The team itself conducts retrospectives and managers and others not on the team should not attend. This gives the team a chance to build a level of trust to gain a deeper level of honesty.

Bill: To pick up on this point. I think that truth and honesty are relative things. For what you are saying I feel you would agree with this. For example, I know this one guy who feels he always needs to say the “truth.” In the process he alienates and hurts a lot of people. He will say things that are his opinion but are also destructive in the context in which they are said. He defends this by saying he just speaks the truth. I think instead, you need to speak an intelligent truth to say things that are constructive and move people forward.

Jeff: I agree. Truth is a subtle thing. It has to do with appropriateness and context. One way to talk about it is to consider the two people talking and include the context so there are three entities involved. In the case of this blog there is you, me, and the context. The context is a blogging mechanism and our readers.

Bill: There is also the issue of establishing the right, credibility, or authority to speak the truth. You need to establish you have been there to really have people accept what you are saying. It is much easier for a member of a group to express concerns about the group than someone from the outside. This gives you some legitimacy and reduces the chance that you will seem simply offensive.

Jeff: Right. So all these things matter in communication. Better teams have higher levels of trust. This enables more profound conversations. They tend to be more efficient. High performance teams know each other better. They trust each other and they need fewer words to communicate. When I work with a team these are the kinds of things I am listening for. I look at the interactions. If you have manager who does not listen, people will not speak. The Meyers Briggs introvert and extrovert aspects come into play here. The high performing team will always give the introvert space to talk. Sometimes with introverts you have to be comfortable not saying anything for a while to give them space.

I wanted to talk about this because we discuss the importance of people in the Agile Manifesto. But often when we think of people we say: did we provide enough food at the right time; are they paid enough; how do they like their office. But there is also the people aspect of team dynamics. These things involve softer values. We know that co-location can be good but you can have a co-located team and still not have trust in that team.

Bill: I am a big believer in this also, both being trained as a psychologist and being part of a lot of IT implementations. I was on a panel at the 2007 Enterprise 2.0 conference that we called, “90% People and 10% Technology. We did this because people always say, well of course it is really 90% people and 10% technology in implementations but in fact they invest funding and focus in the exact opposite direction. At the same time it has been my experience that there is a very positive correlation between the amount of time spent on people issues and the success of the project. But people not get this 90 percent of the time. Why do you think this is?

Jeff: There are several reasons. In the computer software many people got into the field because they were not comfortable with people issues. In some ways software development was an escape for them from interactions with people. I also think it is much harder to measure and show results when you are dealing with the people issues.

Bill: I think this measurement thing is a critical point, especially in the US. We are a measurement culture. We look at wines in terms of Robert Parker’s numbering system that many Europeans laugh at. The numbers can be meaningless but they give us comfort. The people side is hard to measure. In think that what you said about people involved with computers can be true but it also applies to many people involved in business management. They get an MBA and learn to run the numbers. But they not learn how to deal with people. In many situations, it is the business manager who is less oriented to people issues than the IT people.

Jeff: I agree with this. I remember years ago I knew people in the economics program at Berkeley and it was all about the measurement. But perhaps they were not measuring the right things.

Bill: Yes, the balanced scorecard is an attempt to look at a broader range of issues.

Jeff; We have used the balanced score card and 360 reviews with success in Agile teams. Perhaps this measurement should be the topic for another conversation.

The State of Certification within the Scrum Alliance

Here is a conversation that Bill and I had following my attendance at the Scrum Gathering a few weeks ago. There was a good deal of discussion about the certifications that are available from the Scrum Alliance. There are now five certifications available: Certified Scrum Master (CSM), Certified Scrum Product Owner (CSPO), Certified Scrum Practitioner (CSP), Certified Scrum Trainer (CST), and Certified Scrum Coach (CSC). TLA (Three Letter Acronym) heaven.

The oldest one is the Certified Scrum Master that was started by Ken Schwaber a number of years ago. I first got mine in 2003 and he had been doing them for a little while before then. It basically indicated that you had taken a two day class where the principles and practices of Scrum were taught. It was actually a bit of a joke in the community because it did not really certify anything. It just said you took the class.

But it did work and it got things off the ground. It got people’s attention. I believe the next one to come into existance was the Certified Scrum Trainer. These were people who could give the CSM class – they could certify people. The CST has more meat behind it. You must really ‘prove’ that you know your stuff.

Certified Scrum Practitioner was put together to indicate the step beyond just taking the class. Are you actually practicing Scrum? Successful applicants must be able to document their experience.

More recently, I worked to help put together the Certified Scrum Coach designation. That one was the first certification that was serious about whether you knew what you were doing. Some people have said that it has taken as long as 30 hours to complete the application. You had to have 1500 hours coaching experience and it is pretty serious. Not everyone gets through it.

The bar for these certifications has been going up. At the recent Scrum Gathering the Scrum Alliance announced that they will be giving a test for CSM to determine if they are certified. It appears that this test will be in place in the third quarter of this year (2009). True certifications can only last a certain amount of time before they need to be renewed, usually through a test. Over 65,000 people have become CSMs and they will have two years to take the test or lose their certification.

This is quite a change. All the certifications have gotten more serious. I think this is a good thing. It is part of the maturing of Agile. People are starting to request these certifications and the certifications are starting to have value. It is interesting to watch how this has changed over the years.

Bill: Let me get some more context here. Who administers these certifications?

Jeff: Up until this point for the CSM whenever a Certified Scrum Trainer gave a class he would submit the names of the people who were there to the Scrum Alliance who would issue the certification. The Scrum Alliance administers the certification and they will now administer the test. So as things go forward the certifications will expire and you will have to take another test and demonstrate that you have been practicing Scrum.

Bill: What is the Scrum Alliance? Who are they? Who supports them financially? How is it run?

Jeff: The Scrum Alliance is group of people who are practicing and interested in Scrum. You are invited to join when you become a CSM. It‘s mission to help promote Scrum, handle certifications, and do publicity. They hold the Scrum Gatherings twice a year, one in Europe and one in the North America. This year there is an additional Scrum Gathering in Brazil. They are typically three-day events where people interested in Scrum get together and talk about their experiences.

They have a great web site, http://www.ScrumAlliance.org. You can go there and see a lot of papers and other useful information.

Bill: Is it run as an academic or professional organization? I am curious because in my experience with knowledge management, there were a number of certification efforts that were really designed to be money making efforts for the person running them and very self-promoting. They were trying to create a business for themselves out of this movement or approach. Now this is not necessarily a bad thing if it is appropriate. In the case of knowledge management some times it was not or it was a gray area. So is this a volunteer organization or a business in itself? How does all this come to play?

Jeff: Well, originally it was just a group of people and then there were some money making aspects. The actual formation of the Scrum Alliance as a legal entity only occurred about three years ago even though certifications have been given out by Ken Schwaber for quite a while. It is now a legal non-profit organization. It is not designed to make money. It is not designed to support particular people. It has an independent Board of Directors from various companies. It does have a director and some staff to manage the web site, the events, and the certification process.

Ken Schwaber was the main early advocate of Scrum. He wrote some books and kept pushing it while the rest of us went off and did things. He was involved with Jeff Sutherland very early in the process.

These recent moves have made the Scrum certifications more substantial. There was talk before that it only meant you attended a class. Now the requirements are more involved. The Alliance made a decision not to change the name of the certification (CSM) because of all the people who already had it. They just added the testing requirement.

Bill: Is the Scrum Alliance sponsored by a software firm or firms, or is it independent?

Jeff: There are a number of firms who help sponsor the events but it is independent.

Bill: So it is an organization run by the evangelists rather than sponsored by several software firms.

Jeff: Yes. The Agile Alliance is a similar type of organization. It was formed later. They put on the biggest Agile conference.

Bill: The state of the nation of the Agile Alliance sounds like a good topic for another post.

Does Agile Development Require UML?

I am often asked about UML or Unified Modeling Language. A lot of young developers don't seem to be using UML or any kind of modeling language when doing development.
I think this is a big mistake.

When we go back to the roots of Agile, which I consider to be extreme programming, we see the issue of quality elevated to the highest level of importance. Quality is very important and low quality can affect new feature development, team morale, and drown an organization in bug triage, reporting, fixing, regressing, documenting and forward fixing.

We know that unit testing and even better, test driven development, can drive the quality higher. But what does that have to do with UML?

The best way to have much better software is to have better design. Having a better design is the product of communication with your other team members about design choices. A great language for that communication is UML.

Text or just word-based communication about design is slow and subject to interpretation errors. Most folks find communication about patterns, which are really just examples of good designs for common problems, much easier with UML. (This is why we use UML in my Emergent Design Class.)

When I talk about UML, I do not mean UML models that are so specified that code can be
generated directly from them. I mean 'blackboard UML'. Usually this is the class diagram, the sequence diagram and the interaction diagram - just three of the possible diagrams in UML. Other diagrams can be used in specific situations. State and component diagrams are seen as well. I would say that if you only use class diagrams, you have achieved a significant percentage of the value. For a simple introduction to the class diagram go here. The book on UML I like best is UML Distilled by Martin Fowler.

High performing Agile teams tend to have visual things around their work areas or on their wikis. In particular I see UML on white boards or on wiki pages to show high-level designs of areas of their work. People have good discussions around these. And make changes as the project unfolds. Product Owners use UML to capture Domain Models.

I believe that every person (or at least every developer) on an agile team should be able to use simple (blackboard) UML to talk about their work. And everyone should be able to diagram the high level abstractions in the team’s code - the architecture as it were. What do you think?