Agile Software Development

Daniel Robert Friday, December 12, 2008

Overview
Agile1 is a mechanism for running software projects. From its beginning in 2001 with The Agile Manifesto2, Agile differs from other project management practices in several important ways. At its base level, Agile focuses on short iterations of stable, tested releases of the product. It emphasizes frequent, personal interaction between the client and the team. Agile translates better to the real world of software than traditional methods because the client is empowered to continually refine the nature of the product. Why the frequency? Problems and bugs are cheaper to fix the earlier they are identified and the product can never go more than two weeks off-course.

The Team
An Agile team is comprised of seven (7) people, give or take two. The small number was arrived at by noting the difficulty in producing results in meetings where too many people are involved. This is similar to the Law of Diminishing Returns; after a certain number of people are brought together, there is more noise than critical dialogue. the project is on-track. Larger teams are split into separate groups and their respective headers are brought together in meta meetings to ensure

Throughout this document, Agile software development will be denoted by a capital A. The adjective agile will be left lower-case. 2 http://agilemanifesto.org/

Agile hinges on the entire team being in frequent interaction. Standard software practices involve many hand-offs from one group to another. Anyone that has played the childrens game telephone can see how this often goes wrong in the real world. With a close-working team, the project can handle frequent changes to requirements or business needs. Turnaround times improve and the difficulty in keeping everyone on the same page drops dramatically.

The Roles
The Product Owner is the client. While in the real world there any many stakeholders, Agile involves a sole Product Owner. If such a person is not immediately identifiable or available, a proxy can be used in their place. In this organization, at least at first, I do not believe this role is realistic. I think its important to have a small team (1-3 people) as the Product Owners and they will be somewhat responsible for talking with their team and ensuring they understand the product. This is an important safeguard against frequent scope creep3. Every software project involves developers. In Agile, the 7-person team will likely involve three to five developers. Different Agile methodologies have different approaches to how the developers interact. Extreme Programming (XP)4 enforces pair programming practices. That is at any given time, two programmers are working on the same component at the same time. The aim is to both reduce the number of errors made and to ensure at least two developers are equally competent with any given part of the system. The SCRUM Alliance 5 does not have such a requirement, but does stipulate that the team can self-manage in terms of which member works on which components. This allows the team to quickly rebalance according to expertise or as critical external tasks occur (such as pages or trouble tickets).

I will address the term scope creep later on. In essence, scope creep can be thought of as scope expansion. While a changing scope is encouraged in Agile, scope creep is not healthy and will require additional resources, be it bodies, time, or money (or some combination of all three). 4 http://www.extremeprogramming.org/ 5 http://www.scrumalliance.org/

The leader of the group is the Scrum Master (SM). The Scrum Master is the closest member of the team to the PMP role of Project Manager6. The Scrum Master facilitates communication between members of the team and helps resolve personnel conflicts. In many Agile arrangements the Scrum Master is responsible for tracking progress, including assuring that the team is tracking their time and accomplishments diligently. Progress reports, charts/graphs, etc. originate from the Scrum Master. The Scrum Master is also responsible for making sure meetings, such as the less-than-fifteen-minute daily Scrum, remain on-track and within the allotted time frames. Other members of the team vary by project. Many projects will include a Business Analyst, a Q.A. member, a systems architect, and perhaps others. Often a subject-matter expert (SME) is needed for some percentage of the time. Architects and SMEs are often mandatory in the project, but do not necessarily constituted a full member of the team. That is, their time is frequently split between several projects to they will be brought into meetings and planning sessions as needed.

The Process
The Agile process involves an up-front discovery period like any other software methodology. The difference is that Agile recognizes that identifying all the details of a project at the beginning is not a realistic goal. The Product Owner will invariably change their mind on already-proposed features. They may think of more features as the project continues. Business priorities may change due to changes in the market during the lifetime of the projectmaybe selling houses isnt the most important part of the business anymoreetc. Agiles ability to adapt to changing scope is apparent in its process.

Discovery
6

The two roles, SM and PM, are not at odds. Agile groups certainly can, and often do, contain both members.

The discovery period of an Agile project is initially similar to other methodologies. The team is brought into a room with notecards and all ideas are written down on said cards. This is essentially a big brain-storming session. Some ideas will be vague, others will be somewhat well-defined. The degree of precision is irrelevant at this point, its most important to define the vision of the product and as many use cases as possible in the allotted time. There will be many more opportunities later to refine these. At this point, the ideas are re-written as stories. A story is a semantic construct including a user (be it human or machine), an action, and a purposea who, a what, and a why. The team then works to assign business values to each of these stories. In the class, business values where ranked from 1-7 (or 10-70). A one would indicate a perk or a nice-to-have and a seven would indicate a mission-critical piece of functionality. The development team will spend the next week or so identifying any potential red flags. This will most likely involve preliminary testing of some software solutions to be able to estimate complexities. Once the stories are assigned business value and the team has a general sense of what is required to implement each, the developers assign relative complexities to each story. The complexities are on a numeric scale, much like the business values. These values are referred to as story points. The relative complexities are the key to tracking progress. Stories are only evaluated relative to each other and duration is not factored in; complexity is a measure of effort on an independent scale. The amount of work the team promises to deliver in each iteration is determined via the number of story points, and its worth is determined in business value. All stories from this point forward are listed in a project backlog which is used to define the project.

Iterations

Each iteration is a two-week7 slice of time at the end of which the team delivers a fully tested, functional product. An iteration begins with the full team spending about a day on an Iteration Planning Meeting. In the meeting, the team identifies the top 20% (or so) of the stories found in discovery and drills down into requirements for these. These are broken down into specific stories as needed until there are sets of stories than can be accomplished in a time span no longer than half an iteration (for any individual story). These stories are assigned business values and story points. The teamwhich includes the Product Owneridentifies which stories will be completed within this iteration, and includes a few just in case theres time stories as well. The number of story points the team decides can be done in this span defines the project velocity. This is important as it demonstrates that estimates are done with complexities rather than time estimates. The stories chosen for a given iteration are not always the highest business value stories. It is a factor of both business value and complexity. There are many strategies available for analyzing what should be done in any given iteration. Between the Product Owner, the team, and the business analyst(s), a good balance will be determined. The team then starts development of what they promised to deliver. Every day, a meeting is held for less than fifteen minutes called a Scrum. During this meeting, the team briefly informs everyone else what they accomplished the day before, what they plan to accomplish that day, and any obstacles they have in achieving their goals. Product Owners and other team members are welcome to attend, but are not allowed to speak at this meeting. Agile is designed to maximize time efficiently. Halfway through the iteration, the team re-convenes and analyzes progress. If the team is ahead of schedule, some of the just in case stories can be added in to the iteration. If the team is behind, this is another opportunity for the Product Owner to adjust priorities. At the end of the iteration, the team reconvenes for a working demonstration of the product so far. The Product Owner will give feedback, which will be captured as stories and added to the product backlog. At the start of the next week, a new Iteration Planning Meeting is held and these comments are
7

XP (extreme programming) now dictates one-week cycles, SCRUM allows up to four. The right duration depends on the team and the project. Most Agile documents recommend two weeks to get started, with an emphasis on the shorter the better, assuming it is practical to do so.

factored in as more stories to be given story points and business value and prioritized for future iterations.

Benefits of Iterations
A common criticism of Agile at a cursory glance is that is sounds meeting-heavy and time-intensive for the client, but this warrants a deeper look. The meetings are specifically designed to keep the team focused on maximizing business value in the shortest amount of time. Clients that are paying for this service are often happy to devote more time to working with the team if it means saving money overall and getting exactly the product theyre looking for. We have already seen this with the My Open Doors team headed by Ethel Baker who informed us verbatim that the time would be absolutely worth it. The principal of the short iteration has a simple analogy. Across a great distance, small differences matter. If one aims for the moon but is initially off by a single degree, they will miss big. If the trajectory is frequently adjusted, it is not very critical how off the initial launch was as it will be corrected over time. The agile approach is to illuminate problems quickly before they have taken a large amount of time and/or money. The software equivalent is referred to as fail-fast although the term fail has unfair connotations in this scenario. Agile will demonstrate quickly how effectively the team is working and will show early on if the project has insurmountable failures. The project cannot become more than an interations duration behind as the project is reevaluated at the start of every iteration. The other big advantage of Agile is that the software is built in vertical slices. Just like up-front requirements gathering, building whole subsystems at a time is flawed in its difficulty to adapt to changing requirements. Agile focuses on functional development that delivers real, measurable business value in every iteration. This develops a system designed to work together as it expands rather than having to attempt to glue components developed in silos together which often results in expensive modifications that deliver no new value to the client.

Conclusion
Agiles goal is to maximize the teams time spent producing software that delivers realworld, measurable business value to the Product Owner. The ability of the Product Owner to adjust their needs and priorities is a vital part of the process. It focuses the team of real value and it focuses on the business to be able to define its needs. Agile requires substantial face time between members of the whole team, which prevents miscommunications and the telephone effect. In many situations, Agile is the most effective development framework for maximizing business value from development effort.

The Class
The class was put on by a company named Big Visible with primary instructor Alex Singh and secondary Brian Bozzuto. They offer public classes on Agile software development, such as this class, as well as on-site coaching and a few other on-site offerings. I was impressed by the depth of knowledge of both Alex and Brian in the subject. Rather than simply reading from slides, Alex was able to quote many authors on similar subjects, both in favor of and opposed to. The class was not evangelical; they freely discussed benefits and shortcomings of several methodologies and even related Agile to several physical production methods from Toyota and other firms. Due to the small size of the class, we were all given ample room for questions and answers and I think it really gave us a chance to understand how Agile would fit in our organization. I believe it was invaluable to have so much of the team together in the same training. It is difficult to get a group of people on-board with new ideas when only one member has been exposed to it. A new learner is not always capable of immediately playing the role of educator. I did get the opportunity to ask about the PMIs adoption of Agile as one of their certifications and best practices. While this is not yet one of their offerings, the PMI has set up a special interest group for Agile and is working closely with the Agile communities to understand how to best produce an offering for Agile position similar to PMP.

Simple Agile Model Example

By Scott Sehlhorst | December 3rd, 2008
0tweetsretweet

A picture is worth a thousand words. Agile values working software over comprehensive documentation, and it values customer collaboration over contract negotiation. With that in mind, how much is a picture of a model worth? Check out a simple example, how it helped, and what we didnt do.

A Complex Conversation
I was working with a client on an eCommerce website, where one of the things that was important to the client was having and managing an affiliate network that refers visitors (and ultimately customers) to the website. The conversations around exactly what the stakeholders of the eCommerce site wanted were pretty convoluted, and different stakeholders described the requirements differently. Affiliate Network Background An affiliate network can work in the following way: 1. When someone visits a website, and does so by directly clicking on a link on an affiliate partners page, the website is able to identify the affiliate partner who sent the visitor to the website. 2. The website keeps track of who sent the visitor. 3. If the visitor becomes a customer, the affiliate partner who sent the visitor is compensated. One way to compensate that affiliate partner is by giving them a percentage of the revenue that the customer generates for the website. This can be a lucrative model for both the eCommerce site and the affiliate partner. The partner gets paid for funneling traffic to the eCommerce site, but not necessarily doing any other work (beyond what it takes to encourage people to go to the eCommerce site). The eCommerce site gets increased traffic, at a manageable cost proportional to the revenue generated by the incremental visitors (who would probably not otherwise visit the site). Each business gets to focus on its core competency (sending traffic, for the affiliate partner; converting that traffic into revenue, for the eCommerce site). The eCommerce site invests the majority of the time and money (probably orders of magnitude more investment), and the affiliate partner shoulders all of the risk, but can do this for almost no cost (if they want), in exchange for an uncertain revenue stream. This blog is currently trying out an affiliate relationship with Carbonite because I love the product (and therefore think people who visit Tyner Blain will too), and because the revenue generated, if any, will help defray the costs of writing and maintaining the blog. Specific Complexity of an Affiliate Model

The complexity of this simple model becomes apparent when you ask a few clarifying questions.

Will there be multiple affiliate partners? (Yes) If someone is referred by multiple affiliate partners (on separate occasions), who gets credit? (The last affiliate partner who sends the customer to the site) Does the visitor have to purchase during the visit that was made from the affiliate partners site? (No) Is there an expiration date, after which an affiliate will not get credit for customer purchases? (Yes, N days). [Note: I am obscuring the actual value for this article, but N is a real number.] If a customer who was referred by an affiliate makes multiple purchases, for how many of them is the affiliate partner credited? (One)

Asking these questions in different ways generated some different, and potentially conflicting answers. The list above is the after we put it all together answer.

A Simple Model
We solved this problem by grabbing the immediately available stakeholders and pulling them into an office with a whiteboard for about 3 minutes. [Ed: Our first article about using objectoriented analysis as part of requirements gathering is from three[!] years ago, this is not a new idea, and wasnt then, either.] I drew the following state transition diagram on the whiteboard, using an active listening technique, to confirm that I had consolidated the inputs into the right set of business requirements. Documenting business rules with a state transition diagram was very effective at highlighting them.

Most of the explanations from stakeholders were in more of a use-case format, but a statetransition diagram was more effective at validating the rules and requirements in this situation. The diagram above uses the following approach to describing the problem:

Some customers are considered to be actively referred by an affiliate partner and when that is true, an affiliate receives compensation for the order placed by the customer. Other customers are not considered to be actively referred and no compensation is due to the affiliate partner. Customers can be temporarily actively referred and events modify that state.

What the diagram shows, given that context is the following business rules:

When a customer is referred to the site, the to be compensated affiliate partner is identified, and associated with the customer. When the same customer is referred to the site by a different affiliate partner, the new affiliate replaces the old one. If N days pass after a customer was referred by an affiliate partner, that affiliation expires, and the affiliate partner is not compensated for future purchases. An affiliate partner is only compensated for the first purchase made by the referred customer.

The stakeholders that were present confirmed that this perfectly represents their business requirements. Even cooler a couple days later, the business person most-directly-responsible for implementing the programs came by, looked at the whiteboard, and signed her name on it to record her approval. We also have a camera-phone snapshot of that.

What We Didnt Do
I could have titled this article Mini-Model and Requirements, but Agile seemed more appropriate, because we didnt go over the top with it (and it is a rapid-development, incremental project). The diagram above also represents the minimum deployable version of the affiliate program for a given release. We stopped short of creating the following diagram:

This diagram is more of a classic artifact that you would see in a BUFR (big, up-front requirements) project. Our team collaborates regularly, and the affiliate program details were now relevant/timely. We had the conversations, drew the diagram, confirmed (in about 10 minutes), took a picture, and shared it with the rest of the team. There would have been marginal incremental value in making a pretty version of the diagram, so we didnt do it. Our goal is working software, not comprehensive documentation.

Conclusion
Just do what you need to do, then move on. For our team, clarifying the intent was important. Creating a formal, pretty picture added no extra value. So we didnt do it.