Overview of Software Engineering

CS 330
Spring 2007

Key Ingredients in successful organizations

People

Process

Technology

A better view
Process and Technology supporting people

People

Processes

Technology

Pyramids are stable. Wedges are not!

What is software?
Computer programs and associated documentation

Software products may be developed for a particular customer or may be developed for a general market Software products may be
Generic/COTS - developed to be sold to a range of different customers Custom- developed for a customer according to their specification

Engineering
Engineering is
The application of scientific principles and methods to the construction of useful structures & machines

Examples
Mechanical engineering Computer engineering Civil engineering Chemical engineering Electrical engineering Nuclear engineering Aeronautical engineering

Software Engineering
The term is 35 years old: NATO Conferences
Garmisch, Germany, October 7-11, 1968 Rome, Italy, October 27-31, 1969

The reality is it is finally beginning to arrive

Computer science one the scientific basis
Years of studies/experience/statistics provide basis too

Many aspects have been made systematic

Methods/methodologies/techniques Languages Tools Processes

Why Engineer Software ?

The problem is complexity Many sources, but size is a key:
Mozilla contains 3 Million lines of code UNIX contains 4 million lines of code Windows 2000 contains 108 lines of code

Second is role and combinatorics of state Third is uncertainty of inputs and their timing Fourth is the continuing changing environment and demands.

Software engineering is about managing all the sources of complexity to produce effective software.

Software Engineering in a Nutshell

Development of software systems whose size/complexity warrants team(s) of engineers
multi-person construction of multi-version software [Parnas 1987]

Scope
study of software process, development/management principles, techniques, tools and notations

Goal
production of quality software, delivered on time, within budget, satisfying customers requirements and users needs

What does a software engineer do?

Software engineers should
adopt a systematic and organised approach to all aspects of software development. use appropriate tools and techniques depending on
the problem to be solved, the development constraints and the resources available

Understand and communicate processes for improved software development within their organization Be effective team members and/or leaders. Can be very technical or more managerial depending on organizational need.

What is the difference between software engineering and computer science?

Computer Science
theory fundamentals

Software Engineering
the practicalities of developing delivering useful software

is concerned with

Computer science theories are currently insufficient to act as a complete underpinning for software engineering, BUT it is a foundation for practical aspects of software engineering

What is the difference between software engineering and system engineering?

Software engineering is part of System engineering System engineering is concerned with all aspects of computer-based systems development including
hardware, software and process engineering

System engineers are involved in system specification, architectural design, integration and deployment

Difficulties?
SE is a unique brand of engineering
Software is malleable Software construction is human-intensive Software is intangible and generally invisible Software problems are unprecedentedly complex Software directly depends upon the hardware
It is at the top of the system engineering food chain

Software solutions require unusual rigor Software state means behaviors can depend on history. Software has discontinuous operational nature

Software Engineering Software Programming

Software programming
Single developer Toy applications Short lifespan Single or few stakeholders
Architect = Developer = Manager = Tester = Customer = User

One-of-a-kind systems Built from scratch Minimal maintenance

Software Engineering Software Programming

Software engineering
Teams of developers with multiple roles Complex systems Indefinite lifespan Numerous stakeholders
Architect Developer Manager Tester Customer User

System families Reuse to amortize costs Maintenance accounts for 60%-80% of overall development costs

Economic and Management Aspects of SE

Software Engineering is about improved ROI (can be Capital and/or Social ROI) Software production = development + maintenance Maintenance costs 60%-80% of all (successful) development costs
20% corrective (12%-16% total costs) 30% adaptive (18%-24% total costs) 50% perfective (30-40% total costs)

Quicker development is not always preferable

higher up-front costs may defray downstream costs poorly designed/implemented software is a critical cost factor in system cost and delays

Relative Costs of Fixing Software Faults

200

30 10 1
Requirements

2
Specification

3
Planning

4
Design Implementation Integration Maintenance

Mythical Man-Month by Fred Brooks

Published in 1975, republished in 1995
Experience managing development of OS/360 in 1964-65

Central argument
Large projects suffer management problems different in kind than small ones, due to division in labor Critical need is the preservation of the conceptual integrity of the product itself

Central conclusions
Conceptual integrity achieved through chief architect Implementation achieved through well-managed effort software developers are not interchangeable work units.

Brooks Law
Adding personnel to a late project makes it later

Software Engineering: From Principles to Tools

TOOLS

METHODOLOGIES

METHODS AND TECHNIQUES

PRINCIPLES

Software Qualities
Qualities are goals in the practice of software engineering, and directly relate to many of the guiding principles.

External vs. Internal qualities Product vs. Process qualities

Software Qualities
Critical Quality Attributes
Correctness Maintainability Dependability Usability Reliability

Other Attributes
Completeness Compatibility Portability Internationalization Understandability Scalability Robustness Testability Reusability Customizability Efficiency

External vs. Internal Qualities

External qualities are visible to the user
reliability, usability, efficiency (maybe), robustness, scalability

Internal qualities are the concern of developers

they help developers achieve external qualities verifiability, maintainability, extensibility, evolvability, adaptability, portability, testability, reusability

Product vs. Process Qualities

Product qualities concern the developed artifacts
maintainability, performance, understandability,

Process qualities deal with the development activity

products are developed through process maintainability, productivity, predictability

Some Software Qualities

Correctness
ideal quality established w.r.t. the requirements/specification absolute

Reliability
statistical property probability that software will operate as expected over a given period of time/inputs relative

Some Software Qualities (cont.)

Robustness
reasonable behavior in unforeseen circumstances subjective a specified requirement is an issue of correctness; an unspecified requirement is an issue of robustness

Usability
ability of end-users to easily use software extremely subjective

Some Software Qualities (cont.)

Understandability
ability of developers to easily understand produced artifacts internal product quality subjective

Verifiability
ease of establishing desired properties performed by formal analysis or testing internal quality

Some Software Qualities (cont.)

Performance
equated with efficiency assessable by measurement, analysis, and simulation

Evolvability
ability to add or modify functionality addresses adaptive and perfective maintenance problem: evolution of implementation is too easy evolution should start at requirements or design

Some Software Qualities (cont.)

Reusability
ability to construct new software from existing pieces must be planned for occurs at all levels: from people to process, from requirements to code

Interoperability
ability of software (sub)systems to cooperate with others easily integratable into larger systems common techniques include APIs, distributed programming interfaces (CORBA, DCOM), plug-in protocols, etc.

Some Software Qualities (cont.)

Scalability
ability of a software system to grow in size while maintaining its properties and qualities assumes maintainability and evolvability goal of component-based development

Process Principles
Prescribes all major activities Uses resources, within a set of constraints, to produce intermediate and final products May be composed of sub-processes Each activity has entry and exit criteria Activities are organized in a sequence Has a set of guiding principles to explain goals Constraints may apply to activity, resource or product

Software Development Stages

Requirements Analysis & Specification
Conceptual/System/Architectural Design Detailed/Program Design Implementation/Coding Unit & Integration Testing System Testing/Validation System Delivery/Deployment Maintenance
Note there are many variations on the names. You are responsible for the main categories above (an on the next pages)..

Software Lifecycle Models

Waterfall Model V Model Phased Development Model
Incremental Model

Prototyping Model Spiral Model

Software Development Lifecycle Waterfall Model

Requirements

Plan/Schedule Replan/Reschedule

Design

Implementation Integration Validation Deployment

V Model
REQUIREMENTS ANALYSIS Validate requirements OPERATION & MAINTENANCE

ACCEPTANCE TESTING SYSTEM DESIGN SYSTEM TESTING

Verify design

PROGRAM DESIGN

UNIT & INTEGRATION TESTING

[Pfleeger 98]

CODING

Phased Development Model

DEVELOPERS
Development systems

Build Release 1

Build Release 2

Build Release 3

Time

USERS

Use Release 1

Use Release 2

Use Release 3

Production systems

[Pfleeger 98]

Software Development Lifecycle Incremental Model

Requirements Version 1: Design Complete General Design Implementation Integration Validation Deployment
Requirements Design Implementation Version 2: Integration Design/Implement first set Validation of planned new features. Deployment Note overlap with V1 schedule Version 3: Design/Implement second set of planned new features

Requirements Design Implementation Integration Validation Deployment

Prototyping Model

Listen to Customer

Build/Revise Mock-Up

Customer Test-drives Mock-up

[Pressman 97]

Prototyping Model
LIST OF REVISIONS LIST OF REVISIONS LIST OF REVISIONS

revise prototype

user/ customer review

PROTOTYPE REQUIREMENTS

PROTOTYPE DESIGN

PROTOTYPE SYSTEM

TEST

SYSTEM REQUIREMENTS (sometimes informal or incomplete)

DELIVERED SYSTEM

[Pfleeger 98]

Spiral development
Process is represented as a spiral rather than as a sequence of activities with backtracking. Each loop in the spiral represents a phase in the process. No fixed phases such as specification or design loops in the spiral are chosen depending on what is required. Risks are explicitly assessed and resolved throughout the process.

Spiral model of the software process

Spiral model sectors

Objective setting
Specific objectives for the phase are identified.

Risk assessment and reduction

Risks are assessed and activities put in place to reduce the key risks.

Development and validation

A development model for the system is chosen which can be any of the generic models.

Planning
The project is reviewed and the next phase of the spiral is planned.

Evolutionary development
Exploratory development
Objective is to work with customers and to evolve a final system from an initial outline specification. Should start with well-understood requirements and add new features as proposed by the customer.

Throw-away prototyping
Objective is to understand the system requirements. Should start with poorly understood requirements to clarify what is really needed.

Evolutionary development

Evolutionary development
Problems
Lack of process visibility; Systems are often poorly structured; Special skills (e.g. in languages for rapid prototyping) may be required.

Applicability
For small or medium-size interactive systems; For parts of large systems (e.g. the user interface); For short-lifetime systems.

Component-based software engineering

Based on systematic reuse where systems are integrated from existing components or COTS (Commercial-off-the-shelf) systems. Process stages
Component analysis; Requirements modification; System design with reuse; Development and integration.

This approach is becoming increasingly used as component standards have emerged.

Reuse-oriented development

Component-Based Development
Develop generally applicable components of a reasonable size and reuse them across systems Make sure they are adaptable to varying contexts Extend the idea beyond code to other development artifacts Question: what comes first?
Integration, then deployment Deployment, then integration

Different Flavors of Components

Third-party software pieces Plug-ins / add-ins Applets Frameworks Open Systems Distributed object infrastructures Compound documents Legacy systems

Process iteration
System requirements ALWAYS evolve in the course of a project so process iteration where earlier stages are reworked is always part of the process for large systems. Iteration can be applied to any of the generic process models. Two (related) approaches
Incremental delivery; Spiral development.

Incremental delivery
Rather than deliver the system as a single delivery, the development and delivery is broken down into increments with each increment delivering part of the required functionality. User requirements are prioritised and the highest priority requirements are included in early increments. Once the development of an increment is started, the requirements are frozen though requirements for later increments can continue to evolve.

Incremental development

Incremental development advantages

Customer value can be delivered with each increment so system functionality is available earlier. Early increments act as a prototype to help elicit requirements for later increments. Lower risk of overall project failure. The highest priority system services tend to receive the most testing.

Extreme programming
An approach to development based on the development and delivery of very small increments of functionality. Relies on constant code improvement, user involvement in the development team and pairwise programming. Covered in Chapter 17

Software Development Lifecycle Waterfall Model

Requirements

Plan/Schedule Replan/Reschedule

Design

Implementation Integration Validation Deployment

Software specification
The process of establishing what services are required and the constraints on the systems operation and development. Requirements engineering process
Feasibility study; Requirements elicitation and analysis; Requirements specification; Requirements validation.

Requirements
Problem Definition Requirements/Specification
determine exactly what the customer and user need (maybe want) Requirements develop a contract with the customer Specification say what the software product is to do

Difficulties
client is computer/software illiterate (no idea what is doable) client asks for wrong product (want vs need) client is computer/software literate (specifies solution not need) specifications are ambiguous, inconsistent, incomplete

Studies have shown that the percentage of defects originating during requirements engineering is estimated at more than 50 percent. The total percentage of project budget due to requirements defects is 25 to 40 percent.

The requirements engineering process

Software design and implementation

The process of converting the system specification into an executable system. Software design
Design a software structure that realises the specification;

Implementation
Translate this structure into an executable program;

The activities of design and implementation are closely related and may be inter-leaved.

Design process activities

Architectural design Abstract specification Interface design Component design Data structure design Algorithm design

The software design process

Structured methods
Systematic approaches to developing a software design. The design is usually documented as a set of graphical models. Possible models
Object model; Sequence model; State transition model; Structural model; Data-flow model.

Architecture vs. Design

[Perry & Wolf 1992]

Architecture is concerned with the selection of architectural elements, their interactions, and the constraints on those elements and their interactions necessary to provide a framework in which to satisfy the requirements and serve as a basis for the design. Design is concerned with the modularization and detailed interfaces of the design elements, their algorithms and procedures, and the data types needed to support the architecture and to satisfy the requirements.

Architecture/Design
Requirements/Specification Architecture/Design
architecture: decompose software into modules/objects/components with interfaces design: develop module/object/component specifications (algorithms, data types) and communication details maintain a record of design decisions and traceability specifies how the software product is to do its tasks

Difficulties
miscommunication between module designers design may be inconsistent, incomplete, ambiguous How to achieve a requirement may be unknown

Planning/Scheduling
Before undertaking cost of development, need to estimate the costs/sizes of various steps
Estimate Code size Estimate tools needed Estimate personnel

Often Done after Architecture and before rest of design, but revised again after full design. Develop schedule for aspects of project lifecycle If doing predictive/quantitative SE, build on past experience, considering how to improve process.

Implementation & Integration

Design Implementation
implement modules; verify that they meet their specifications combine modules according to the design specifies how the software design is realized

Difficulties
module interaction errors order of integration may influence quality and productivity

Programming and debugging

Translating a design into a program and removing errors from that program. Programming is a personal activity - there is no generic programming process. Programmers carry out some program testing to discover faults in the program and remove these faults in the debugging process.

The debugging process

Software validation
Verification and validation (V & V) is intended to show that a system conforms to its specification and meets the requirements of the system customer. Involves checking and review processes and system testing. System testing involves executing the system with test cases that are derived from the specification of the real data to be processed by the system.

Verification and Validation

Analysis
Static Science Formal verification Informal reviews and walkthroughs Dynamic Engineering White box vs. black box Structural vs. behavioral Issues of test adequacy

Testing

The testing process

Testing stages
Component or unit testing
Individual components are tested independently; Components may be functions or objects or coherent groupings of these entities.

System testing
Testing of the system as a whole. Testing of emergent properties is particularly important.
Testing with customer data to check that the system meets the customers needs.

Acceptance testing

Testing phases

Quality Assurance
Done as part of each step Reduce costs by catching errors early. Help determine ambiguities/inconsistencies Help ensure quality product. 200

30
Requirements

Specification

3
Planning

4
Design

10
Implementation Integration Maintenance

Deployment
Completed End-User Documentation
Separate from Developer documentation

Installation Process(es) Customer test procedures Support Processes (help desk, etc) Trouble Tracking Repair/rework to address bugs Regression testing (as bugs are fixed)

Maintenance & Evolution

Operation Change
maintain software during/after user operation determine whether the product still functions correctly

Difficulties
Rigid or fragile designs lack of documentation personnel turnover

Software evolution
Software is inherently flexible and can change. As requirements change through changing business circumstances, the software that supports the business must also evolve and change. Although there has been a demarcation between development and evolution (maintenance) this is increasingly irrelevant as fewer and fewer systems are completely new.

System evolution

Why I include CASE Tools

Computer Aides Software Engineering tools support good SE processes (e.g. UML) Some tools absolute requirement for scaling e.g. build and configuration management. Integrated CASE (ICASE) tools embody good processes and improve productivity (E.g. Rational tool set) Some tools (e.g. debuggers, Purify) do almost impossible for humans.

But.. Tools change

No SE tools from my first 3 jobs exist (except Fortran/C languages) I use regularly use 3 SE tools from my next set of jobs. Other tools I learned have been replaced with similar but expanded concepts.. Understanding today;s tools gives a basis for learning future ones.

ICASE Design Tools

Rational Rose and Rational Unified Development. From UML drawing to code and back. Generates stubs and eventually testing code. Supports multiple languages

Car
public class Car { public Driver theDriver; /** * @roseuid 3EAFF17E035B */ public Car() { Associations are } }

Driver
public class Driver { /** * @roseuid 3EAFF53F02FD */ public Driver() {

implemented as reference attributes. No explicit role name defined so, Rose adds automatically a role name to the code: theDriver

} }

Templates for the default constructors are provided. (Similar for methods/members when given in the class diagram.)

Configuration Management
CM is a discipline whose goal is to control changes to large software through the functions of
Component identification Change tracking Version selection and baselining Managing simultaneous updates (team work) Build processes with automated regression testing Software manufacture

CM in Action
1.0 1.1

1.2
1.3 1.4 1.5

2.0 2.1

4.0

2.2

3.0 3.1

Build Tools
Necessary for large projects. Keep track of what depends upon on what, and what needs recompiled or regenerated when things change. Important even for small 1-person projects as soon as you have multiple files. Can do much more than just compile, can generate document (if using code-based docs), generate manufactured code (e.g. SOAP interfaces), even send emails or suggest alternatives.
E.g. in our IUE project, edit some files compile was one in seconds, edit another and a rebuild taking days would be needed. If more than 30 files impacted, our make process recommend a new branch to avoid conflicts!

Debugging Tools
How do you see what the code is really doing (not what it seems it should do)? How to you see what happened to code during compiler optimization? How do you find/track down the cause of Segfault/GFP in code youve never seen before? How can you test various possibilities without generating special code or recompiling. How do you track down a memory leak?

Tools, workbenches, environments

CASE techn olo g y T ls oo Wor kb en ch es Envir ments on

Ed ito rs

Co mp ilers

File comp ar ators

Integ rated en vir ments on

Process -cen tr ed en vir ments on

An alys is and des ig n

Pro gramming

T estin g

Mu lti-metho d workb en ch es

Sin gle-meth od workb en ch es

General-pu rp os e workb en ch es

Lang uage-sp ecific workb en ch es

The Rational Unified Process

A modern process model derived from the work on the UML and associated process. Normally described from 3 perspectives
A dynamic perspective that shows phases over time; A static perspective that shows process activities; A practive perspective that suggests good practice.

RUP phase model

Phase it erat ion

Incept ion

Elaborat ion

Const ruct ion

Transit ion

RUP phases
Inception
Establish the business case for the system.

Elaboration
Develop an understanding of the problem domain and the system architecture.

Construction
System design, programming and testing.

Transition
Deploy the system in its operating environment.

RUP good practice

Develop software iteratively Manage requirements Use component-based architectures Visually model software Verify software quality Control changes to software

Static workflows
Work flow Business modelli ng Requirements Analysis and design Implementation Description The business processes are modelled using business use cases. Actors who interact with the system are identified and use cases are developed to model the system requirements. A design model is created and documented using architectural models, component models, object models and sequence models. The components in the system are implemented and structured into implementation sub-systems. Automatic code generation from design models helps accelerate this process. Testing is an iterative process that is carried out in conjunction with implementation. System testing follows the completion of the implementation. A product release is created, distributed to users and installed in their workplace. This supporting workflow managed changes to the system (see Chapter 29). This supporting workflow manages the system development (see Chapter 5). This workflow is concerned with making appropriate software tools available to the software development team.

Test

Deployment Configuration and change management Project management Environment

Computer-aided software engineering

Computer-aided software engineering (CASE) is software to support software development and evolution processes. Activity automation
Graphical editors for system model development; Data dictionary to manage design entities; Graphical UI builder for user interface construction; Debuggers to support program fault finding; Automated translators to generate new versions of a program.

Case technology
Case technology has led to significant improvements in the software process. However, these are not the order of magnitude improvements that were once predicted
Software engineering requires creative thought - this is not readily automated; Software engineering is a team activity and, for large projects, much time is spent in team interactions. CASE technology does not really support these.

CASE classification
Classification helps us understand the different types of CASE tools and their support for process activities. Functional perspective
Tools are classified according to their specific function.

Process perspective
Tools are classified according to process activities that are supported.

Integration perspective
Tools are classified according to their organisation into integrated units.

Functional tool classification

Tool type Planning tools Editing tools Change management tools Configuration management tools Prototyping tools Method-support tools Language-processing tools Program analysis tools T esting tools Debugging tools Documentation tools Re-engineering tools Examples PERT tools, estimation tools, spreadsheets T ext editors, diagram editors, word processors Requirements traceability tools, change control systems Version management systems, system building tools Very high-level languages, user interface generators Design editors, data dictionaries, code generators Compilers, interpreters Cross reference generators, static analysers, dynamic analysers T est data generators, file comparators Interactive debugging systems Page layout programs, image editors Cross-reference systems, program re-structuring systems

Activity-based tool classification

Re-en g ineering tools Testin g too ls Debu gg ing too ls Prog ram analy sis to ols Lang uage-p ro ces sing too ls Meth od s up po r t to ols Proto ty ping too ls Co nfiguration management to ols Ch an ge man ag emen t too ls Do cu men tatio n too ls Ed iting too ls Planning to ols

Sp ecif icatio n

Design

Implemen tatio n

V erification and V alidatio n

CASE integration
Tools
Support individual process tasks such as design consistency checking, text editing, etc.

Workbenches
Support a process phase such as specification or design, Normally include a number of integrated tools.

Environments
Support all or a substantial part of an entire software process. Normally include several integrated workbenches.

Boults view of SE
SE must balance risks in software development process:
Risks of error in
requirements specification, design, implementation, and integration

Risks of exceeding available resources Risks of being late on delivery or missing the market

Dont let push for formality dominate your process. Dont let push for expedience destroy your process.

Software Process Qualities

Process is reliable if it consistently leads to highquality products Process is robust if it can accommodate unanticipated changes in tools and environments Process performance is productivity Process is evolvable if it can accommodate new management and organizational techniques Process is reusable if it can be applied across projects and organizations

Assessing Software Qualities

Qualities must be measurable/quantifiable Measurement requires that qualities be precisely defined Improvement requires accurate and consistent measurements For most SD groups, qualities are informally defined and are difficult to assess

Software Engineering Axioms

Adding developers to a project will likely result in further delays and accumulated costs The longer a fault exists in software
the more costly it is to detect and correct the less likely it is to be properly corrected

Up to 70% of all faults detected in large-scale software projects are introduced in requirements and design
detecting the causes of those faults early may reduce their resulting costs by a factor of 200 or more

Basic tension of software engineering

better, cheaper, faster pick any two! functionality, scalability, performance pick any two!

Want/Need Managements buy in to formal SE process. If you dont document your process, you dont have one!

Boehms Spiral Model

DETERMINE GOALS, ALTERNATIVES, CONSTRAINTS EVALUATE ALTERNATIVES AND RISKS
Risk analysis
4

Risk analysis

Risk analysis Risk analysis Budget

4

1 1

Budget

Budget

Budget

Prototype
1

Proto type 2

Proto type 3

Proto type 4 Detailed design

start

Requirements, life-cycle plan

Concept of operation

Code Unit test Implementation plan Acceptance test System test

PLAN

DEVELOP AND TEST

Key points
Software processes are the activities involved in producing and evolving a software system. Software process models are abstract representations of these processes. General activities are specification, design and implementation, validation and evolution. Generic process models describe the organisation of software processes. Examples include the waterfall model, evolutionary development and componentbased software engineering. Iterative process models describe the software process as a cycle of activities.

Key points
Requirements engineering is the process of developing a software specification. Design and implementation processes transform the specification to an executable program. Validation involves checking that the system meets to its specification and user needs. Evolution is concerned with modifying the system after it is in use. The Rational Unified Process is a generic process model that separates activities from phases. CASE technology supports software process activities.