Software Reengineering

Dr. Isma Ul Hassan

Office: C503-A
Email: [email protected]
Textbooks and Reading Material
Object-Oriented Reengineering Patterns
Serge Demeyer, Stephane Ducasse, and Oscar Nierstrasz
free copy from: http://scg.unibe.ch/download/oorp/

Working Effectively with Legacy Code

Michael Feathers, Prentice Hall, 1 edition, 2004
Software Engineering vs.
Software Reengineering
Greenfield vs. Brownfield development
How often did you ...
... encounter brownfield software development?

Challenges faced?
Brownfield Software Development
Advantages:

•Involves less work as the environment is already pre-structured and there is a

baggage of existing code bases available. The team just has to put in some
enhancements.

•If it is a long-term project, the existing team has to repeat the process, utilizing
previous experience. The basis of a project is well understood, so those who
have worked on it from the very beginning can easily work on incremental
changes and improvements continuously.

•Existing code can be reused with a set of new features on top, saving much time
and effort. Less coding means a product can be launched faster.
Brownfield Software Development
Disadvantages:

•The ability to work on something new, to develop a new code, is limited. The
chances of innovation are low.

•The approach cannot be implemented without a good understanding and

thorough knowledge of the existing system and data on which a new one is.

•The presence of legacy code in a project can slow down the overall development
process or even increase costs.

•New business requirements and outdated environments cannot coexist without

considerable re-engineering.
 Software Reengineering

Software development is rarely a “green fields” activity.

•Software Reengineering is the examination and alteration of a
system to reconstitute it in a new form.

•Improving the software to make it more efficient and effective.

General Idea
The goal of reengineering is to:

• understand the existing software system artifacts, namely, specification, design,

implementation, and documentation, and

• improve the functionality and quality attributes of the system.

Software systems are reengineered by keeping one or more of the following four
general objectives in mind:
• Improving maintainability.
• Migrating to a new technology.
• Improving quality.
• Preparing for functional enhancement.
Software Reengineering vs.

Software Maintenance vs.

Software Evolution
Software Maintenance:
Is the modification of a software product after delivery to correct faults,
to improve performance or other attributes, or to adapt the product to
a changed environment.

Software Evolution:
A continuous change from a lesser, simpler or worse state to a higher or
better state.

Software Maintenance vs. Software Reengineering:

Different, but boundaries are not well-defined
Software Maintenance and Reengineering
•Software maintenance is the stage of software development that requires
more effort and resources.
• Maintenance effort is between 70 and 80% while only the 20-30% of time
is spent on other development stages.

• Software maintenance is a key activity.

• Extends the lifespan of software systems, which implies a higher return of

investment.

• One of the most widespread software maintenance techniques is software

reengineering.
The Reengineering Spectrum
Reengineering covers a very wide spectrum of activities
including

•database reengineering,

• program transformation,

•design refactoring,

•user interface reengineering / Web-enabling

•Supporting activities like

• risk management
• costing and guidelines for reengineering projects
Software Reengineering
Reengineering process consists of three stages:

(i) Reverse Engineering: which analyzes existing software and identifies the
different components and their interrelationships to build one representations
of the system at a higher degree of abstraction;

(ii) Restructuring: transforms it into an enhanced representation of the system at

the same abstraction level by preserving the external behavior; and

(iii) Forward Engineering: generates physical implementations of the target

system at a low abstraction level from the restructured system.
In this course, you will learn and apply
•Practices to analyze and understand software systems (i.e., reverse
engineering)

•Heuristics and tools to detect shortcomings in the design and

implementation of software systems.

•Testing and re-factoring techniques to systematically resolve these

shortcomings
How will you be assessed?
1. Class Project/Assignments : 4 (20%)

2. Quiz: 4 (10%)

3. Your contribution to the lecture (+/- 5%)

Semester Project/Assignments
•Pick an unfamiliar C/C++/Java/Python system, then you will have to perform the
following tasks on it:

1. Reverse engineering: comprehension and architecture recovery of the subject

system using the tools and manual analysis.
2. Producing technical documentation for the system.
3. Performing the following reengineering task on the system:
➢ Refactoring
➢ Transforming to another language
➢ Web enabling
 Questions?

Your Expectations from this course

•Reverse engineering and reengineering are essential activities in the
lifecycle of any successful software system. (And especially OO ones!)

•There is a large set of lightweight tools and techniques to help you with
reengineering.

•Despite these tools and techniques, people must do job and they
represent the most valuable resource.
 Lecture-1: Introduction
Object-Oriented Reengineering
Patterns
Chapter #1
 What is a Legacy System ?
“legacy”
A sum of money, or a specified article, given to another by
will; anything handed down by an ancestor or predecessor.
— Oxford English Dictionary

Typical problems with legacy systems:

A legacy system is a piece of • original developers not available
software that: • outdated development methods used
• you have inherited, and • extensive patches and modifications
have been made
• is valuable to you.
• missing or outdated documentation

 so, further evolution and development may be prohibitively expensive

Software Maintenance - Cost Relative Cost
Relative Maintenance Effort
of Fixing Mistakes x 200
Between 50% and 75% of
global effort is spent on
“maintenance” !
x 20
x 10
Solution ?
• Better requirements engineering?
x5
• Better software methods & tools
x1

requirement coding delivery

design testing
Continuous Development
17.4% Corrective data from [Lien78a]
(fixing reported errors)

60.3% Perfective
(new functionality)
18.2% Adaptive
(new platforms or OS)

4.1% Other

The bulk of the maintenance cost is due to new functionality

 even with better requirements, it is hard to predict new functions
Modern Methods & Tools ?
[Glas98a] quoting empirical study from Sasa Dekleva (1992)
Modern methods(*) lead to more reliable software
Modern methods lead to less frequent software repair
and ...
Modern methods lead to more total maintenance time
Contradiction ? No!
• modern methods make it easier to change
... this capacity is used to enhance functionality!
Lehman's Laws
A classic study by Lehman and Belady [Lehm85a] identified several “laws” of system
change.

Continuing change
A program that is used in a real-world environment must change, or become progressively
less useful in that environment.

Increasing complexity
As a program evolves, it becomes more complex, and extra resources are needed to
preserve and simplify its structure.

Those laws are still applicable…

What about Objects ?
Object-oriented legacy systems
= successful OO systems whose architecture and design no longer responds to changing
requirements

Compared to traditional legacy systems

The symptoms and the source of the problems are the same
The technical details and solutions may differ

OO techniques promise better

flexibility,  they do not come for free
reusability,
maintainability
…
What about Components ?

Components are very delicate …

After a while one inevitably resorts to glue :)
How to deal with Legacy ?
New or changing requirements will gradually degrade original design
… unless extra development effort is spent to adapt the structure

New Functionality

Hack it in ?

• duplicated code First …

• complex conditionals • refactor
• abusive inheritance • restructure
• large classes/methods • reengineer

Take a loan on your software Investment for the future

 pay back via reengineering  paid back during maintenance
Case Studies
Domain LOC Reengineering Goal
pipeline planning 55,000 extract design

user interface 60,000 increase flexibility

embedded switching 180,000 improve modularity

mail sorting 350,000 portability & scalability

network management 2,000,000 unbundle application

space mission 2,500,000 identify components

Different reengineering goals … but common themes and problems !

Common Symptoms
Lack of Knowledge Process symptoms
• obsolete or no documentation • too long to turn things over to
• departure of the original production; simple changes
developers or users take too long
• disappearance of inside • need for constant bug fixes
knowledge about the system • maintenance dependencies
• limited understanding of • difficulties separating
entire system products
• missing tests

Code symptoms
• big build times
• duplicated code
• code smells
Common Problems
Architectural Problems
•insufficient documentation
= non-existent or out-of-date
•improper layering
= too few are too many layers
•lack of modularity
= strong coupling
•duplicated code
= copy, paste & edit code
•duplicated functionality
= similar functionality
by separate teams
Goals/Need of Reengineering
Unbundling
split a monolithic system into parts that can be separately marketed
Performance
“first do it, then do it right, then do it fast” — experience shows this is the right sequence!
Port to other Platform
the architecture must distinguish the platform dependent modules
Design extraction
to improve maintainability, portability, etc.
Exploitation of New Technology
i.e., new language features, standards, libraries, etc.
Reduce human dependencies
Documenting knowledge about the system and making it easier to maintain.
Goals of Reverse Engineering
Cope with complexity
need techniques to understand large, complex systems
Generate alternative views
automatically generate different ways to view systems
Recover lost information
extract what changes have been made and why
Detect side effects
help understand ramifications of changes
Produce higher abstractions
identify latent abstractions in software
Facilitate reuse
detect candidate reusable artifacts and components
]
The Reengineering Life-Cycle
(0) requirement
Requirements analysis

(2) problem
(3) problem
detection
resolution

Designs

• people centric
(1) model
• lightweight
capture

Code

(4) program transformation

 Course Outline
Software Aging: How and why software systems age.
Legacy Systems: Their issues and challenges.
Software Evolution, Maintenance and Reengineering.
Reverse Engineering: Program analysis methods, software complexity
and maintenance metrics, program visualization.
Forward Engineering: Refactoring, code transformation, Web enabling.
Software Reengineering Strategies and Management
Terminology and the processes pertaining to
software Evolution
Software Aging: How and why software systems age
Software has one big advantage
over hardware: it does not age
Software Aging
•Programs, like people, get old

•We can’t prevent aging,

➢ understand its causes,

➢ take steps to limit its effects,
➢ temporarily reverse some of the damage it has caused,
➢ prepare for the day when the software is no longer viable
Software “Aging”?
•“It does not make sense to talk about software aging!”

– Software is a mathematical product; mathematics does not decay with

time.
– If a theorem was correct 200 years ago, it will be correct tomorrow.
– If a program is correct today, it will be correct 100 years from now.
– If a program is wrong 100 years from now, it must have been wrong
when it was written.

• All of the above statements are true, but not really relevant.
Software Does Age
Software aging is gaining in significance
because:

– of the growing economic importance of software,

– software is the “capital” of many high-tech firms.

Software Does Age
• The authors and owners of new software products often look at aging
software with
disregard.

• “If only the software had been designed using today’s languages and
techniques …”

• Like a young jogger teasing an 86 year old man (ex-champion swimmer)

and saying that he should have exercised morein his youth!
The Causes of Software Aging
•There are two types of software aging:

– Lack of Movement: Aging caused by the failure of the product’s

owners to modify it to meet changing needs.

– Ignorant Surgery: Aging caused as a result of changes that are

made.

• This “one-two punch” can lead to rapid decline in the value of a

software product.
 Lack of Movement
•Unless software is frequently updated, its user’s will become dissatisfied and
change to a new product.

• Excellent software developed in the 60’s would work perfectly well today, but
nobody would use it.

•That software has aged even though nobody has touched it.

•Actually, it has aged because nobody bothered to touch it.

Ignorant Surgery
•One must upgrade software to prevent aging.

•Changing software can cause aging too.

•Changes are made by people who do not understand the

software.

• Hence, software structure degrades.

 Ignorant Surgery (Cont’d)
• After many such changes nobody understands the software:

– the original designers no longer understand the modified software,

– those who made the modification still do not understand the software.

• Changes take longer and introduce new bugs.

• Inconsistent and inaccurate documentation makes changing the software

harder to do.
The Cost of Software Failure

• Inability to keep up,

• reduced performance,
• decreasing reliability
Inability To Keep Up
As software ages, it grows bigger.

• “Weight gain” is a result of the fact that the easiest way to add a feature is to add new
code.

• Changes become more difficult as the size of the software increases because:
– There is more code to change,
– it is more difficult to find the routines that must be changed.

• Result: Customers switch to a “younger” product to get the new features.

Reduced Performance
As the size of the program grows, it places more demands on the computer
memory.

• Customers must upgrade their computers to get acceptable response.

• Performance decreases because of poor design that has resulted from

long-term ad-hoc maintenance.

• A “younger” product will run faster and use less memory because it was
designed to support the new features.
Decreasing Reliability
As the software is maintained, errors are introduced.

• Many studies have shown that each time an attempt is made

to decrease the failure rate of a system, the failure rate got
worse!

• That means that, on average, more than one error is

introduced for every repaired error.
Decreasing Reliability (Cont’d)
• Often the choice is to either:

– abandon the project

– stop fixing bugs

• For a commercial product, the author was once told that the
list of known unrepaired bugs exceeded 2,000.
Reducing the Cost of S/W Aging
We should be looking far beyond the first release to the time
when the product is old.

• Inexperienced programmers get a “rush” after the first

successful compile ordemonstration.

• Experienced programmers realize that this is only the

beginning ...
Reducing the Cost of S/W Aging
(Cont’d)
• Responsible, professional, organizations realize that more work is
invested between the time after the first successful run and the first
release than is required to get the first successful run.

• Extensive testing and rigorous reviews are necessary.

Preventive Medicine
• Design for success
• Keep records (documentation)
• Seek second opinions (reviews)
Design for Success
• Design for change.

• This principle is known by various names:

– information hiding
– abstraction
– separation of concerns
– data hiding
– object-orientation
Design for Change
• To apply this principle one begins by trying to characterize the changes
that are likely to occur over the “lifetime” of a product.

• Since actual changes cannot be predicted, predictions will be about

classes of changes:

– changes in the UI
– change to a new windowing system
– changes to data representation
– porting to a new operating system ...
Design for Change (Cont’d)
• Since it is impossible to make everything equally easy to
change, it is important to:

– estimate the probabilities of each type of change

– organize the software so that the items that are most likely
to change are “confined” to a small amount of code
Why is Design for Change
Ignored?
• Textbooks fail to discuss the process of estimating the probability of
change for various classes of changes.

• Programmers are impatient because they are too eager to get the first
version working.

• Designs that result from this principle are different from the “natural”
designs of the programmer’s intuition.
Why is Design for Change
Ignored? (Cont’d)
• Few good examples of the application of the principle. Designers tend
to mimic other designs they have seen.

• Programmers tend to confuse design principles with languages.

• Many practitioners lack training in software development.

Keeping Records
(Documentation)
• Even when software is well designed, it is often not documented.

• When documentation is present it is often:

– poorly organized
– inconsistent
– incomplete
– written by people who do not understand the system
Documentation
• Hence, documentation is ignored by maintainers.

• Worse, documentation is ignored by managers because it does not

speed up the initial release.
Second Opinions (Reviews)
• In engineering, as in medicine, the need for reviews by other
professionals is never questioned.

• In designing a building, ship, aircraft, there is always a series of design

documents that are carefully reviewed by others.
Reviews
• This is not true in the software industry:

– Many programmers have no professional training in software at all.

– Difficult to find people who can serve as quality reviewers; no money to
hire outsiders.
– Time pressure misleads designers into thinking that they have no time for
proper reviews.
– Many programmers resent the idea of being reviewed.
Why is Software Aging
Inevitable?
• Our ability to design for change depends on our ability to predict the future.

• We can do so only approximately and imperfectly.

• Over a period of years:

– changes that violate original assumptions will be made

– documentation will never be perfect
– reviewers are bound to miss flaws ...
Software Geriatrics
• Retroactive Documentation:

– A major step in slowing the age of older software, and often renewing it, is to
upgrade the quality of the documentation.

• Retroactive Modularization:

– Change structure so that each module hides a design decision that is likely to
change.
Software Geriatrics (Cont’d)
• Amputation:

– A section of code has been modified so often, and so thoughtlessly,

that it is not worth saving.

• Major Surgery (Restructuring):

– Identify and eliminate redundant components and dependencies.

Reading Task
Fast Software Aging
Slow Software Aging