Jeff McKenna's Agile Development Blog

Agile Myth Busters at Serena Tag 2008

I blogged much of the Serena Tag 2008 conference. This was post eight about Serena’s moves into Agile led by Jeff McKenna and the Agilistas. While waiting for the start, I listened to the Stones mashed up with Queen. This mashing up of music greats was constant theme of the conference music. Nice job. Now there is a video of some local high school kids creating graffiti style backgrounds for the event. Serena also gave us cow bells with the More Cowbell! label. Rene Bonvanie kicked off the program designed to squash some Agile myths.

Jeff McKenna (see above) came on to the sound of cow bells to deal with these myths. He introduced the Agilistas from Valtech and Serena. Myth one: Agile means you never have to write documentation. Both sides were briefly covered. The audience was asked and 79% said it is false. This is also the position of Serena and Valtech. However, documentation can be produced at the right time when the system is ready. There is less editing and revisions.

Myth Two: Agile is more disciplined than other methods. Again, both sides were briefly covered. The audience was asked. 43% - false and€“ 41% true. The panel believes that Agile is more disciplined but it is bottom up discipline rather than imposed discipline. There is more feedback and engagement. This is a common, and counter intuitive, aspect of enterprise 2.0. With transparency you get accountability and more discipline. It results in greater productivity. I have seen many examples of the switch to more transparent project management leading to significant increases in productivity. Below is a panel debate.

Myth Three: Agile means I can change my mind whenever I want to. Both sides were briefly covered by the panel. The audience was asked and 79% said it is false. This is also the position of Serena and Valtech. There needs to be some stability. Within the sprints in development, change needs to be on hold. Then there can be times for change.

Myth Four: Agile works on all sizes of projects. Both sides were briefly covered. The audience was asked and 59% said true. This is the position Serena and Valtech but you need to recognize that there are greater needs for large projects for them to succeed. The key differential is that you can keep Agile teams small and link together these teams to handle size. One large project had 27 teams successfully linked together.

Myth Five: Agile means teams cannot be controlled by management. Both sides were briefly covered. The audience was asked and 72% said false. Serena and Valtech agrees with this. Agile is about control through planning, monitoring, and adapting. Management is about getting more done by removing obstacles. There is a different style of management.

Myth Six: Agile requires detailed architecture and design. Again, both sides were briefly covered by the panel. The audience was asked. 55% said true but Serena and Valtech disagree with this. There is less architecture and design. You need architecture but it is not the driver. Architecture comes out in dealing with the problem and should not be a guiding factor. Let architecture emerge and be open to change.

Myth Seven: Agile is just the latest hype. Both sides were briefly covered. The audience was asked. 62% said false and Serena and Valtech agrees with this. It has been around for a while. There are demonstrated successes. Most employees like it and there are productivity improvements.

Myth Eight: Agile works on complex projects. Both sides were briefly covered. The audience was asked and 71% said true. Serena and Valtech agree with this. However, it does require more management.

Myth Nine: Agile teams do not work hard, they just play foosball. Both sides were briefly covered. The audience was asked and 75% said false. Serena and Valtech agree with this. There is a balance of work and fun required. Focus is key.

Myth Ten: Agile is only used for mission critical projects. Both sides were briefly covered. The audience was asked and 95% said false. Serena and Valtech agree with this.

Comments on Amr Elssamadisy's new book Agile Adoption Patterns

I just took a detailed look at Amr Elssamadisy's new book Agile Adoption Patterns - A Roadmap to Organizational Success.

I like it. It brings together a wide range of agile practices and gives good analysis of each. As with good pattern books, it shows why, how, why not, and such for each pattern. Since there are 38 (!) practices, finding the one that matters for you is important. They have been clustered a bit and are still hard to find.

I personally find patterns a useful way to capture and present what is needed and at the same time difficult to understand. Since I already understand these patterns I found them useful and occasionally found a new way to think about a particular one. If I did not already know the pattern, I am not so sure how useful the book would be. I will defer on this to others in that position.

And one comment early in the patterns did catch my attention and has, I hope, influenced my thinking: Amr comments on the importance of learning. Much of the agile feedback mechanisms can be viewed as opportunities for the team to learn.

The importance of the learning cannot be dismissed. Getting better in every aspect of creation - planning, design, measuring, coding, testing, communication, delivery - is critical to create a high performance team. Such teams look at every change, comment, bug, and hick-up as an opportunity to improve.

I use the repetition of activities as an easy way to identify an area for improvement. If the team is spending time on activity over and over again then an opportunity exists for improvement. In particular these opportunities often are automated to eliminate the human time/errors that are present.
So, Amr, thanks for the stimulation.

Conducting Release Retrospectives in Agile Development

In this post Bill and I discuss the concept of release retrospectives that I briefly touched on in the blog overview.

Bill: You mentioned how you turned a software development retrospective on its head and discussed what went right rather than what went wrong. This is a nice twist on lessons learned and should encourage participation and engagement. After all what you want to repeat is what went right. So let’s have the details.

Jeff: This project had been going for six months. I talked with management about having a release retrospective. Most people are familiar with having a retrospective after each iteration or spread. This usually occurs about every four weeks for an hour or two with each team doing this independently. But we also recommend that after longer periods, say after a major milestone is reached, two practices. First, the teams take a week off and stop the sprinting and breathe for a bit. And second, the teams have a more relaxed release retrospective.

In the book Agile Retrospective, Esther Derby and Diana Larsen talk about the need for the shorter sprint retrospectives while you are still in the fray. These typically last one hour or so and involve one team. In contrast, the release retrospective involves the whole group of teams and takes a longer look.

Bill: What was the problem that set up this release retrospective? Why did you do it?

Jeff: The problem is a generic one. People need time if they are going to learn. One of the Agile tenets is that teams need to be able to learn and reflect. The release retrospective is looking at it from a longer time perspective and from the perspective of all the teams on the project. Typically problems are solved at the individual team level. So as we move to the multiple team level I am a little less concerned with specific problems and more concerned with attitude and making sure that people feel they are on a larger project where teams are working together.

The heads down sprint-to-sprint activity in a team can be very focused and sometimes isolating. It can be harder to come up and breathe. So in the release retrospective there are discussions at a higher level about such things as the vision for the next six months or big changes that are being considered. People can reestablish the larger context that they are working in.

There is a type of inquiry called appreciative inquiry that is a technique for exploring issues that allows people to focus on solutions and positive aspects of their experiences. It is referred to as the discipline of positive change. There is some data in research that indicates that larger groups of people who focus on things that work and spend less time on problems such as we typically do in our industry, people feel better, are more productive, and their customers are happier. I was exposed to this research and I thought it might be interesting to bring this approach into the retrospective.

Here is what we did.

First we had everyone introduce themselves and provide a fact about themselves that they felt no one else knew. This activity helped establish a relaxed atmosphere.

We we used a time line. We had put a 20 foot roll of brown paper on the wall and listed key events during the six months (e.g., the big snow storm, the hiring of key people). Then in the retrospective individuals put on the key events that mattered to them. We ended up with this long time line with all these sticky notes that documented the things that mattered to individuals.

As each person put up a note they told the group about the event. This generated lots of laughing and feeling that they were together on this thing. It was fun, interesting and informative.

Next we added a temperature reading. People went along and added below the time line their memory of their general feeling at that time.. This is a technique where people say things like I was feeling really good about things here and not so good here and all that. All thirty five people did this. In this particular time line case was only one situation were there was an alignment of feelings. Everyone felt bad when we slipped a date by three weeks. Other than that some people were up and others were down at any particular time. People began to realize to that people were different. People enjoyed this.

Next, we had people tell stories about when they worked on teams that went really well and they loved the experience. This is appreciative inquiry technique.

Bill: Do you mean they told stories about events not just on this project but in general?

Jeff: Yes, in general. Some people called out things on the project. Some people called out things from the other parts of their life. This was fascinating as well. We told stories in small groups of five. Each group picked the best story to present to the larger group. As the stories were told we abstracted out the commonalties. We recorded what were the things that mattered.

A this point in time we were at five hours since we had had breaks and lunch. People were feeling jazzed. They felt they had heard what was important from each other and had expressed what was important to them. So we quit early which always makes people feel good. Hopefully, I will have a chance to go back to the same group after another major cycle and continue this.

People came up to me afterwards and said they had been very worried that we were going to spend six hours talking about what went wrong in the prior six months. There was even one guy who did not come into work that day and started the session on the phone. He got so interested that he drove in to participate live and be more involved. The most senior person there was excited as he valued building a cohesive team. Everyone was pleased with the reverse in expectations.

Report on the Scrum Gathering

I am in Orlando at the biannual Scrum Gathering put on by the Scrum Alliance. The gathering alternates between the US and outside the US and is attended by Scrum Masters, Practitioners, Trainers and Coaches. There are some 260+ folks attending this one.

Jim Cundiff has been running the Scrum Alliance for a bit over a year now and I am glad to report that I am seeing considerable improvement in the professionalism, focus and activity level of the Alliance. The program for this meeting has brought in a number of folks from outside the immediate Scrum community. Ron Jeffries and Chet Hendrickson from XP, Dr. Alistair Cockburn of Crystal, Dr. Mark Paulk of CMM, and Gregory Balestrero from PMI and Jim Coplien of Patterns are all here and adding to a more inclusive feel of the gathering. In particular Mark is beginning some academic research on Scrum and the PMI is 'embracing' Scrum with a large number of PMP folks attending.

The Alliance is showing off the redesigned Scrum Alliance website. I suggest you take a look. It appears to be much more comprehensive and there is a lot of content now easily accessible.

Last night I met with 8 other Certified Scrum Coaches as we considering our next moves. I was involved with the initial definition, construction and roll-out of the Scrum Coach certification program. It is a rigorous certification with a non-trivial application and thoughtful evaluation. One really has to be a Scrum Coach to get certified. Initially there were six of us certified. Now the number is up to 12 and increasing each month. The coaches I met with last night were from Australia, Germany, Denmark, South Africa and the US. I am proud of the work we have done on this program and we are hoping that it grows and becomes recognized as a guide level certification.

With Alliance support we are seeing more and more Scrum User Groups getting started. Now over 50 world wide. And also growing. Some folks have wondered if at over 50,000 certified Scrum Masters if the 'market' was becoming saturated. It appears not to be so. Altho training rates have slowed somewhat, the demand is still reasonably strong. The requests for coaching are increasing as well.

In short, Scrum is alive and well and in fact, thriving.

Achieving Immediate and Continuous Bug Fixes in Agile Development

As I mentioned in our blog overview, most development shops spend noticeable time prioritizing and dealing with bugs. However, that effort only manages the symptoms. It can be more useful in the long run to look at the operational side of development. An interesting issue is the time that elapses from when a bug enters the system and when it is detected. I have been in situations were this time was measured in minutes. Then developers could quickly learn from their mistakes, make immediate corrections, and move on. They learned and over time produced fewer and fewer bugs. Then we did not spend time prioritizing and dealing with long lists of bugs.

Bill and I had a conversation going into more depth on how this might work.

Bill: How do you get the bug reporting time down to minutes? I am used to seeing many bugs getting all the way out to users as an accepted byproduct of software development. Some vendors are famous for this.

Jeff: The first thing you need to do is to establish the practice of continuous build, or better continuous integration. Most successful Agile development
shops have this practice in place. This means that every time a developer turns in their code, the application is rebuilt and tested with the new code. So you are able to quickly see if the new code brings in any unwanted behavior, aka bugs. This integration often takes a few minutes to build a typical change. The book Continuous Integration by Paul Duvall, Steve Matyas, and Andrew Glover is a good resource on this topic.

So the first part is having the ability to quickly turnover the integration of new and old code. The second step is to have good unit tests. These tests should be part of the automatic build. If a unit test fails, the ‘build’ fails. If you have unit tests in the area being changed, the effects of the bad code will show up right away.

Bill: I have often heard of unit tests but usually just assume I know what they are. I think I need to make sure I know what you mean here.

Jeff: Unit test are the lowest level of tests. They test classes and methods and small assemblies. They look at code function to see if it does what the developer wants it to. Functional tests on the other hand look at code to see if it performs what the customer or end-user wants. Unit test are designed to run very fast and look at all the code on a specific section of the application. Test coverage measurements should be made on unit tests.

If you use a technique call “test first,” then no line of code is written except in response to a failing test. That means that there is not a lot of code in the system that was produced without a test informing the code’s development. This gets interesting because what this means that every line of code has a test associated with it. So if something changes the code later the unit test can see if anything breaks.

Because the unit tests are so fine grained you will know right away what the problem is without having to debug. Debugging is figuring out what is wrong and with unit tests you know what is wrong. So if you make a code change and it breaks right away you know right away the cause. You still need to understand it but you do not need to spend time finding it.

So the combination of continuous build and unit test allow you to get to the condition of bugs being found right away. In the particular case I was thinking of we did not need a bug tracking system. In this effort we had twelve developers over eighteen months and only had about five bugs at any one time. There was no need to prioritize bugs. We said all bugs are important and need to be fixed right away.

Bill: So this makes total sense to me. Why doesn’t everyone do it?

Jeff: There are a couple of reasons. One reason is that historically we have considered testing not part of development. We have considered testing as something done by QA, often by a different department. Code got thrown over the wall for someone else to deal with and quite a bit of development was done before testing occurred. Responsibility was passed on and not shared.

The other reason is that we do not tend to measure debugging effort so the cost is not apparent. It is almost always much more than people think. When we do not measure fixing bugs there is not any drive to improve the process. Also, folks just assume there will be lots of bugs. ‘That is just how software is.’ Well, it does not have to be that way and I think it is time to change our attitude.

Bill: So is the concept of immediate fix one of the cores of Agile development?

Jeff: Let’s say that most successful Agile teams just do it. One of the practices in extreme programming (XP) is the concept of collective code ownership. In this case an individual does not have ownership of code. So no one can say, “well I cannot fix this code because it is someone else’s code.” So when in Agile we all own the code so if I see a bug, I just fix it.

Lightning Interview by Ward Cunningham

I thought it might be fun to show you an interview that Ward Cunningham did with me late last year. For those of you who do not know of him, Ward is a brilliant developer. He has many claims to fame including the development of CRC (Class Responsibility Collaboration) cards as a design tool and invention of the wiki.

I had the pleasure of working with Ward in the late 1980's at Tektronix in Oregon. We were working on Smalltalk and working on Patterns and the beginnings of the development style now known as Agile. Other interesting folks were at Tektronix at that time including Kent Beck, Rebecca Wirfs-Brock and Norm Kerth. It was a heady time and place with lots of passion and lots of great software.

Ward and I have kept in touch and this interview was an early one in his lightning series.

Welcome to Jeff McKenna’s Agile Development Blog

This blog will share my thoughts on Agile development and broader issues of team dynamics that go beyond Agile and software. I started this blog for several reasons. I have been coaching software development teams for some time (see Author Overview). Now I am focusing on the transition from traditional software development to Agile. In these efforts, I notice a lot of things and wanted a place to write my ideas down, share them, and start conversations. I also want a platform for my general musings on Agile development. I am partnering with other people including Bill Ives and David Socha in order to provide other perspectives (see Author Overview for their backgrounds and how we will work together). We plan to experiment with a number of different blog formats and welcome your input.

While I will reflect on the tools a bit, tool technology is not my main focus. I am more interested in the team and process issues since it is people, in the end, that have to get these tools to work. I have studied family and team dynamics and like to apply principles of complex adaptive systems. I feel that when things break down, or even better before breakdown occurs, we need to move beyond managing symptoms and treat root causes.

Let me offer an example, in most development shops noticeable time is spent prioritizing and dealing with bugs. That effort only manages the symptoms. It can be more useful in the long run to look at the operational side of development. An interesting issue is the time that elapses from when a bug, an unwanted change, enters the system and when it is detected. I have been in situations were that time was measured in minutes. Then developers could quickly learn from their mistakes, make immediate corrections, and move on. They learned and produced fewer and fewer bugs. Then we did not have to spend time prioritizing and dealing with long lists of bugs.

I will get into much more detail on many stories such as this one in the blog.

Sometimes it helps to turn things on their head. I was once asked to faciltate a release retrospective for a group of teams. I asked for six hours of their time. In these sessions in the past they would engage in examining what went wrong. The groans were loud and some team members decided to only dial in. Instead of looking at the mistakes, we talked about the times when they were most thrilled with a team’s performance. We never once discussed what went wrong, the positive energy level rose, and some of those on dial in came into the office for the remainder of the session. The team got back on a more positive track. Of course, I do not always get it right and I will share these stories also.

This blog is sponsored by Serena Software, a provider of application development and business software for companies of all shapes and sizes. I work for Serena. Some Serena’s tools and services support Agile development. We might talk about Serena on occasion, but this conversation is not about Serena, it is about the issues that everyone faces in Agile development.

Bill, David and I encourage you to browse our categories and not just look at our latest content. The categories contain the archives of our past posts. Many of these posts should remain useful as we will discuss issues in depth and not simply provide current news. Also use our search field. Often good blog content gets buried as it moves off the front page.

If you like what you find here please join the conversation through our comment fields. Our blog will allow us to learn of your comments regardless of when the original post was published. We look forward to hearing from your perspectives. We want to learn from you and we look forward to your participation.

a

Our Agile Development Blog Team and Roles

Welcome to this blog on Agile development. I am Jeff McKenna, the Chief Agile Evangelist at Serena Software, I will be leading this blogging effort and creating most of the content (see Blog Overview for our plans). In this effort, I hope to capture some of my random and perhaps not so random thoughts, musings, teachings and rants as they occur to me.

Here is a bit about my background. I have been involved in software development for over 45 years and have been actively participating in the creation and delivery of agile software development since 1987. I have been involved in all areas of software product development from programming, system design and architecture to project management, testing, sales and marketing. Since the late 1980s I have focused on the people/process side of development. Jeff Sutherland, John Scumniotales, and I created the Scrum methodology of agile software development in 1993 at Easel Corporation and started the very first Scrum team. (Little did we know what we were creating!)

I have worked extensively with object-oriented programming systems, languages and applications and was the Chair of OOPSLA in 1994 that helped to incubate and increase adoption of several related disciplines including agile software development. Working with IBM, Cigna, ObjectShare, Easel Corp, Millennium Pharmaceutical and the US Air Force, I provided expertise in architecture, design and implementation for their first Smalltalk based object-oriented projects. 

As an Architect at Rational Software (now IBM), I made various contributions to Rational’s industry leading Rose product. These included extensions to the Unified Modeling Language (UML) to support a UML model of testing.

I also worked with industry pioneers, including Grady Booch, to extend the Rational Rose product to natively support Design Patterns.
I am a Certified Scrum Coach, Certified Scrum Trainer and Scrum Master, I have taught, coached and mentored agile teams using Scrum and Extreme Programming practices for many companies, including Oracle, PayPal, Borland, Vanguard, Microsoft, Google, Lockheed Martin, Tumbleweed Communications, V-Mark and Net Objectives.

Philosophically I am a Buddhist and I love good food and race cars. My wife of more than 27 years is Cory.

Bill Ives

I have asked Bill Ives to join me in this effort. Bill is veteran blogger who writes for two group blogs on enterprise 2.0 (the AppGap and FastForward), as well as his own Portals and KM. Bill has helped Serena with its blogging efforts before (see Blogging Serena TAG), including covering my Serena TAG session on Agile Myth Busters.

Bill has served for over 25 years in leadership positions as a consultant in knowledge management, community building, and other business applications of emerging technologies. He has worked with large and small companies in a variety of industries, and is currently focused, as a consultant and writer, on what the new Web brings to business. Bill also participates in several search-based start-ups including iQuest, and Wikigazette. Prior to his present roles, Bill led the Knowledge Management - Portals Practice at a major consulting firm. Bill has a Ph.D. in educational psychology from the University of Toronto and conducted post-doc research on how media affects cognition at Harvard.

Bill’s oldest daughter spent two years in a Buddhist country, Mongolia, and he also likes their views, especially respect for women. He is a painter when not blogging.

Bill is engaged to provide a partner and outside perspective to the discussions. Since this blog is about conversations, we will be starting many of these conversations between ourselves. Operating under the model of two heads being better than one, at times Bill may interview me, challenge what I present, or act as a sounding board on discussions. In some cases he may contribute posts to offer another perspective. However, we will be clear on the authorship on all posts and list whether it is post by myself, Bill, or a combined effort. You see the author or authors as a category at the bottom of each post and browse the blogs in each category in our right side bar. We also do conversations between the two of us and they are categorized as Interviews. We see this as an opportunity to experiment with different models of conversation and blog collaboration in a transparent manner. We plan to have fun and hope it is contagious. We welcome your input.

David Socha

David Socha is another colleague who will be co-blogging with me. David is a long-term student of the human side of software development. He holds a PhD in Computer Science and Engineering from the University of Washington, and has spent 18 years as a leader and programmer in a variety of software organizations.

I introduced David to agile practices back in 1999 when we both worked at Rational. As David recalls it, one day I said to him "Let's go to the bookstore. There is an interesting new book that I want to look at." That was Kent Beck's book Extreme Programming Explained: Embrace Change. According to David, software development has not been the same since for him. It made software development much more engaging. Ever since, he has been using agile practices and his people skills to help organizations improve their practices. He is a Certified ScrumMaster and Certified Product Owner, and has a particular fondness for biomimicry.