Agile Risk Management

Alan MoranSpringerBriefs in Computer ScienceAgile Risk Management201410.1007/978-3-319-05008-9_1

© The Author(s) 2014

Agile Software Development

Alan Moran¹  

(1)

Zurich, Switzerland

Alan Moran

Email: [email protected]

Abstract

Looking back at the origins of agile software development we provide an overview of its unique blend of supple practices and vivid cultural facets that contrast so sharply with the plan-driven world of Waterfall methodologies. We discuss the iterative and incremental nature of agility and introduce a tool, agile charting, that can be used to facilitate communication within agile teams. Against this backdrop we introduce and compare the three methodologies (i.e., XP, Scrum and DSDM) used throughout this book to illustrate our application of agile risk management. We conclude with a glimpse at the state of affairs of agility today and at the management perspective on agile project management thereby setting the tone for the remainder of the book.

Agile Defined

Agility, as a concept, came to prominence in the late 1990s through the endeavours to address perceived difficulties with existing software development processes that were rooted in, and owed their rigidity to, plan-driven practices. Advocates of agility promoted the notion that project uncertainties should be embraced and sought to balance planning and control with execution and feedback. Accordingly, agile projects exhibit features of open communication amongst heterogeneous stakeholders, emergent behaviour within self-organizing teams and a culture of openness and learning (Cockburn 2007). Central to agile software development are the notions of iterative development and incremental delivery based on shared values enshrined in the agile manifesto reproduced here for convenience.

We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value:

Individuals and interactions over processes and tools

Working software over comprehensive documentation

Customer collaboration over contract negotiation

Responding to change over following a plan

That is, while there is value in the items on the right, we value the items on the left more.

(Beek et al. 2001a).

This manifesto expresses the beliefs that interactions amongst project team members and their customers should underpin efforts to create working software in a flexible manner. Equally important are the twelve principles (Beck et al. 2001b) that emphasize continuous delivery, an attitude of embracing change, frequent delivery of functional components, daily interaction with business stakeholders, empowerment through trust and support, direct communication, measurement of progress through functional software, sustainability, continuous excellence of design, simplicity through minimalism, self-organization and team reflection. Agility is as much a cultural stance on the process of software development, as it is a set of practices and values. The discipline and focus needed to practice agility is frequently underestimated (Boehm and Turner 2009) and has even been famously described as hard and disruptive (Schwaber 2006) by one of its leading proponents.

Agility embodies a rich texture of humanist (e.g., cognitive, social and interpersonal), organizational (e.g., managerial and cultural) and technological (e.g., practical and technical aspects) traditions (Hazzan and Dublinsky 2008) and there already exists a mature body of literature concerning both its culture (Cockburn 2007) and its practices (Coplien and Harrison 2005; Cohn 2010; Derby and Larsen 2009). There are numerous comparative surveys of agile methodologies (Larman 2003; Boehm and Turner 2009; Cockburn 2007) that highlight both the commonality rooted in the manifesto (and its principles) together with the uniqueness of focus and purpose with which each approach is endowed. It is not our purpose to explore the fine detail of all these methodologies, but rather we limit ourselves to a detailed discussion of three popular approaches (XP, Scrum and DSDM¹) and to the techniques they employ. For convenience, the techniques are described in Appendix A and we encourage the reader to refer to this list when we mention a specific technique (e.g., Kanban² boards, burndown charts). Throughout this book we use the term methodology to refer a system of methods and principles used in a particular discipline (McKeown and Holmes 2009) consistent with the notion that it embodies not only practices but also principles, roles, artefacts and phases. We refer the interested reader to (Cockburn 2007) for a detailed discussion of this topic.

The origins of agility are rooted in part in the Japanese manufacturing and industrial sectors to which many an agile concept owes its heritage. These include the visual control concept found in the Toyota Production System that later anticipated agile information radiators, the Kanban charts used in agile task assignment and tracking, and the continual influence of lean thinking on agility today (Poppendieck and Poppendieck 2003). The synthesis of Eastern and Western thinking so persuasively laid out in (Takeuchi and Nonaka 2013), from the authors that introduced the term Scrum, reflects the spirit in which the agile manifesto itself was conceived. The timeline depicted in Fig. 1 marks out some of the key elements of relevance to this book³ and is adapted and extended from (Agile Alliance 2013).

Fig. 1

Agile Timeline. Adapted from (Agile Alliance 2013). Published with kind permission of © Alan Moran 2013. All Rights Reserved

Iterations and Increments

From its earliest inception, agility was understood as more of an evolution rather than a revolution. Its present day form is heavily influenced by the school of iterative development and incremental delivery that was by the 1980s already well established. Indeed movements such as Rapid Application Development and the IBM Rational Unified Process gave birth to their modern day agile counterparts in the form of Dynamic Systems Development Method and Agile Unified Process respectively. Embedded in all of these is the notion of the Software Development Lifecycle⁴ (SDLC) which refers to a generic model of software development comprising of phases broadly divided into the soliciting and management of requirements, technical analysis and design, implementation of software solutions, validation and verification of software artefacts and concludes with the deployment and maintenance of the software solution. Implementations of the SDLC vary from the Waterfall model (Benington 1983; Royce 1970), that advocates transition through the model in terms of a strictly linear process with each phase being commenced only when the previous phase is complete, to the agile approach that promotes rapid and repeated transitions through all phases (some of which could even be said to be merged) resulting in the iterative derivation of a solution. It is interesting to note that the Waterfall method, which was adapted from the manufacturing industry, brought with it the assumption that changes late in the development process were prohibitively costly. This has been rigorously challenged by agilists who argue that postponement of design decisions until sufficient information is available is possible and endorse practices such as refactoring⁵ to cope with change. Between these the two viewpoints of Waterfall and agility, which were conceived approximately three decades apart, can be found a plethora of variants comprising of both plan-driven and humanistic influences. Indeed agility finds itself today in a continual state of evolution with latter day influences emerging once again from industry in the form of lean development practices as described in (Poppendieck and Poppendieck 2003). The iterative and incremental practices, atop of which agility is built, laid the foundations for feedback mechanisms used to cope with change that have become the hallmark of agility.

For the purposes of this book, iterative development refers to the traversal of the entire software development lifecycle with the aim of producing a self-contained, tested and partially functional product within a fixed timeframe. During successive iterations the product is further refined thereby enabling lessons learned from earlier iterations to be fed back into the process. Incremental delivery refers to the packaging and deployment of a product artefact that can be meaningfully used by the customer. Typically an increment will require several iterations to complete and several increments are necessary before the final product is delivered. Borrowing from a metaphor appearing in (Patton 2008) Fig. 2 describes the iterative manner in which an artefact evolves. Though the general idea is clear from the outset, details emerge gradually over time allowing participants to incorporate feedback back into the production process. At each stage there is a gaining of understanding and an adding of detail which can be clearly demonstrated and delivered to the customer.

Fig. 2

Metaphor for the Iterative Evolution of a Solution. Published with kind permission of © Alan Moran 2013. All Rights Reserved

Some methodologies, notably Scrum, merge these two concepts into one referring to the process of iterative and incremental development (Schwaber 2004). Thus consistent with our view of iterative development, Jeff Sutherland, a leading advocate of Scrum, describes iteration as the act of traversal of the whole during each pass of which the product gradually comes into focus. For him increments are concerned with the notion that incremental development is iterating on the whole thing and that each iteration should conclude with a minimal useable feature set that is potentially shippable indicating that this implies that code must satisfy the following definition of done for the increment

thoroughly tested, well-structured and well-written code that has been built into an executable and that the user operation of the functionality is documented, either in Help files or in user documentation (Schwaber 2004).

though he later concedes that he could have been clearer on what ‘potential shippable software’ means (Sutherland 2010). There is thus a suggestion that incremental activity embodies the characteristics of a deployable artefact though Scrum leans towards the use of this term in the substantive rather than adjectival form e.g., each iteration delivers a fully functional increment (Sutherland 2010). In essence our use of the term, increment, reflects a conceptual designation that implies the potential release of a deployment artefact. In reality there is little consensus in the agile community concerning the precise definition of an increment or indeed where the boundaries of agility lie. For example, owing to the structure of most organizations, it ought not be assumed that a product development team has full control over the means of deployment or of the dissemination of documentation (e.g., corporate design, accessibility). Moreover, it is ordinarily the case that some deployment activities (e.g., training of users and service desk staff) are assumed by people outside of the project team. Whilst product focused methodologies (e.g., XP and Scrum) rarely consider what happens to the deliverable, other methodologies (e.g., DSDM, DAD, SAFe) consider it within their remit to manage the entire delivery process from inception to transition. Accordingly, we allow ourselves to make this important conceptual distinction between these two related but different activities but accept that the terms iteration and increment will have their own connotations within specific methodologies.

Agility in Practice

Agile teams tend to be small⁶ comprising of heterogeneous generalizing specialists (Ambler 2003) capable of engaging in several distinct types of work (e.g., analysis, development, testing). Customer representatives are expected to be highly engaged, attend planning and demonstration events and be available on short notice should the solution team require their input. In this context the acronym CRACK,⁷ first coined in (Boehm and Turner 2009), sets the tone for what is expected of such representatives. This is in stark contrast to pre-agile approaches where business and development teams tended to be separated with relatively little contact beyond exchange of requirements and specifications. Agile methodologies employ a wide range of techniques and there is considerable overlap amongst the common methodologies both in their interpretation and their application of them. Some of these practices work well in concert (e.g., refactoring and continuous integration) whilst others might be considered complementary approaches to tackling a problem (e.g., modelling and prototyping). There does appear, however, to be a core set of practices that are used by most agile teams (which we see later comprises of daily stand-ups, iteration and release planning, unit testing, retrospectives, continuous integration, automated builds and burndown charting) and enjoy a common interpretation, though the precise wording may vary by methodology. Thus whilst it is true that all methodologies bear their own interpretation of the agile manifesto, it is equally fair to say that they borrow heavily from each other.

Generally speaking a team must balance the need for adaptation (e.g., innovation) against the necessity of optimization. This suggests that lightweight methodologies service high adaptation and low optimization environments better, whereas heavier methodologies are to be found in low adaptation and high optimization contexts (Cockburn 2007). It ought to be noted, however, that an enterprise typically requires both. Indeed it has been argued that the stability provided by a mature process