Operational Profile

SR SE-308

By Priya Singh (Assistant Professor, Dept of SE, DTU)

Background
• A software-based product's reliability depends on just how a customer will use it.
• Making a good reliability estimate depends on testing the product as if it were in the
field.
• Role of Operational Profile (OP)-
i. a quantitative characterization of how a system will be used, is thus essential in
software reliability engineering (SRE).
ii. can be applied to hardware as well as software- complete systems.
iii. shows you how to increase productivity and reliability and speed development by
allocating development and test resources to functions on the basis of use.
iv. helps you plan test activities, generate test cases, and select test runs.
v. guides system testing ensures that if testing is terminated and the software is shipped
because of imperative schedule constraints, the most used operations will have received
the most testing, and the reliability level will be the maximum that is practically
achievable for the given test time.

By Priya Singh (Assistant Professor, Dept of SE, DTU)

Examples-
1. Consider the scenario of developing and applying the operational profile by AT&T's
International Definity project (a PBX switching system). This project combined the
operational profile with other quality-improvement techniques to reduce customer-
reported problems and maintenance costs by a factor of 10, system-test interval by a
factor of 2, and product-introduction interval by 30 percent. The system experienced no
serious service outages in the first two years of deployment; customer satisfaction
improved significantly. The marked quality improvement and a strong sales effort
resulted in an increase in sales by a factor of 10.
2. In a similar quality improvement program, Hewlett-Packard applied software reliability
engineering and the operational profile to reorganize their system-test process for a
multiprocessor operating system. With automated test and failure recording and using
the operational profile to guide testing, they reduced system-test time and cost by at
least 50 percent

By Priya Singh (Assistant Professor, Dept of SE, DTU)

Cost of developing OP-
• The cost of developing an operational profile varies.
• Some researchers indicate that the effort to construct the operational profile for an
average project-about 10 developers, 100,000 source lines, and a development interval of
18 months—is about one staff month.
• Large projects can cost more, but the increase is clearly less than linear with project size.
• Experience to date indicates that operational profiles are beneficial even when very
simple and approximate. For example, one project used an operational profile defined on
only five operations. However, defining operational profiles in the range of 50 to 200
operations has usually been definitely worth the extra effort.

By Priya Singh (Assistant Professor, Dept of SE, DTU)

Concepts
• Profile- a set of disjoint (only one can occur at a time) alternatives called elements, each with the probability
that it will occur. If A occurs 60 percent of the time and B 40 percent, for example, the profile is A, 0.6 and B,
0.4.
• In order to select and define the terms we will use in analyzing a work process and developing the
operational profile for the system that will implement the activities of the work process, we must recognize
two practical needs-
i. Need to distinguish between the view of the system taken in specifying its requirements (function) and the
view of the system as it is built (operation).
ii. Need to talk about tasks that represent the smallest divisions of work that can be initiated by external
intervention, either human or by a system external to the one being analyzed. Although work can always be
divided into smaller packages, there is no useful purpose and only greater cost and reliability risk involved
in allowing initiation access to these packages if the same sequence of work packages will always be
followed.
Runs- smallest initiable divisions. E.g. in a switching system, a run might be a telephone call, in
interactive systems, it might be a command input by a user, in transaction-based systems, it might be a
transaction, in an aircraft control system it might be a maneuver.

By Priya Singh (Assistant Professor, Dept of SE, DTU)

• Input states- Runs are associated with input states.
The input state is the complete set of input variables of the system,
• Input variable- any data elements that exist external to the system and influence it.
E.g. externally initiated interrupts, such as interrupts generated by the system clock, by operator actions, and by
other components of the system outside the program, are input variables.
Intermediate data computed by the program during a run and not existing external to the program are not input
variables. Hence, interrupts generated directly by a run or interrupts that are determined from other input
variables (for example, overflow and underflow) are not input variables.
• Run types- Runs that have identical input states are said to belong to the same run type.
E.g. Airline-reservation transactions that are exact duplicates have the same run type. However, reservations
made for different people, even on the same flight, are different run types. If you test all run types, you have
exhaustively tested the system.
You do not need to execute repeated runs or instances of the same run type.

By Priya Singh (Assistant Professor, Dept of SE, DTU)

• Function- A function is a set or grouping of run types, as conceived at the requirements stage.
• Operation- An operation is a set or grouping of run types for the system as built.

• Why group run types?

We need a much smaller number of elements (generally not more than hundreds) on which we will collect usage
data. Functions and operations are defined and used for that purpose.
The functional profile is a profile of functions; an operational profile, of operations.
A run (and hence function or operation) is a logical rather than physical concept. It can extend over multiple
machines (for example, in distributed systems).
It can execute in segments of time rather than continuously.

By Priya Singh (Assistant Professor, Dept of SE, DTU)

Development Procedure
• Stakeholders involved in developing OP-
1. systems engineers,
2. high-level designers,
3. test planners,
4. product planning,
5. marketing professionals, and
6. customers.
• Basic Idea-
To determine an operational profile, you look at use from a progressively narrowing perspective—from
customer down to operation and, at each step, you generate an intermediate profile by specifying the probability
that each of the elements in that step will be used.
This procedure represents the best current approach, based on experience with a variety of projects.

By Priya Singh (Assistant Professor, Dept of SE, DTU)

• In many cases, usage information is available or can be estimated, most easily in terms of rates like
transactions per hour. But these data are not true profiles until you convert them to probabilities by dividing
by the total transactions per hour. Converting to probabilities is helpful because you can make a quick
completeness check by seeing if the probabilities add to 1. On the other hand, the raw data are useful to
recreate actual traffic levels in test.
• Need to establish the granularity (number of elements for which you collect occurrence probabilities) and
accuracy (error in estimated occurrence probabilities with respect to those actually experienced in the
field) needed for the operational profile.
1. You should base these decisions on the net economic gain resulting from the better decisions that result from
more fine-grained and accurate data, minus the extra cost of gathering and analyzing these data being
considered.
2. In many cases, however, you will use informed engineering judgment rather than a formal economic
analysis.
3. The engineering judgment will be based on such factors as product complexity, product maturity, product
cost, and schedules.
4. In practice, you must limit profiles to several hundred (several thousand, at most) elements, because the cost
of developing them increases approximately in proportion to the number of elements. Reliability
measurements are generally robust with respect to inaccuracies in determining the operational profile.

By Priya Singh (Assistant Professor, Dept of SE, DTU)

Procedure for developing an OP
• Developing an operational profile to guide testing involves as many as five steps:
1. Developing a customer type list
2. Developing a user type list
3. Listing system modes
4. Developing a functional profile
5. Converting the functional profile to an operational profile

• In the first four steps, you progressively break down system use into more detail.
• Customer types (large retail stores) break down into user types (sales clerks and information
system specialists).
• A single user type may invoke several system modes (information system specialists perform both
database cleanup and generate reports).
• In turn, each system mode has several functions (the generate-reports mode has several types of
reports).
• In the fifth step, functions evolve into operations as the system is implemented.

By Priya Singh (Assistant Professor, Dept of SE, DTU)

• Some steps may not be necessary for a particular application.
1. A customer list is unnecessary if you have only one customer or if all customers use the system the same
way.
2. Sometimes you can skip the functional profile, especially when the requirements are so detailed they specify
how users will execute the system to accomplish their tasks.
3. At some other times the information you need to develop the operational profile is not available until the
design or even a substantial part of the implementation is complete, so you must develop the functional
profile if you want to have guidance in allocating development resources and determining priorities during
the design and implementation phases.
• Even if you have all the information you need to develop the operational profile before design commences,
you may prefer to generate the functional profile first.
1. Because it generally has fewer elements,
2. it is easier to develop and
3. It is easier to use for guiding pretest development than an operational profile.
Should you decide to develop the operational profile directly, you should consider the tasks outlined in the
functional profile step in combination with those in the operational-profile step.

By Priya Singh (Assistant Professor, Dept of SE, DTU)

• The procedure can conceivably be iterative. If you have an existing system, you may determine the current
operational profile, then analyze your current and prospective future customer groups, and then proceed
through the steps to obtain a modified operational profile.
• At each step, you must determine the level of detail you need. Whether you distinguish and treat an item
differently at a given step will depend on the net economic gain for doing so. E.g. in developing a customer
list broken down by industry, it may be cost-effective to distinguish certain important customers with the
intention of performing special testing on the product supplied to them. The degree of detail does not have to
be uniform across the system. For example, some important system modes may require greater levels of
detail.

By Priya Singh (Assistant Professor, Dept of SE, DTU)

• Decide upon the number of OPs-
1. When attaining average reliability over all of a system's applications is acceptable, there may be no need for
more than one operational profile. You must still separately identify different customers, users, and system
modes so that you can weigh the contribution each makes to the operational profile.
2. If it is important to assure particular reliability for a particular use (even if all reliability objectives are the
same), then you must determine multiple operational profiles.
3. You may also need multiple operational profiles if the system will operate with different hardware
configurations.
4. Finally, lab and test resource limits may force you to divide testing, using different operational profiles.

By Priya Singh (Assistant Professor, Dept of SE, DTU)

• Supersystem profile-
Sometimes a software product is part of a network of systems. In that case, it may be useful to develop an
operational profile for the entire network before developing the operational profile for the product. This
supersystem profile can be very useful to determine which systems in the network are most important and
should receive the most attention. It is also possible to decompose a system into subsystems and to develop an
operational profile for each.

NOTE-
• All profiles that are developed should be baselined and placed under change control, with appropriate
traceability requirements.
• Operational profile is independent of design methodology-its determination will not be affected by an object-
oriented approach, for example. The one exception might be a case in which functions designed with one
methodology map to a considerably different set of operations when designed with another.

By Priya Singh (Assistant Professor, Dept of SE, DTU)

Step-1 Create Customer type list
• Customer- the person, group, or institution that is acquiring the system.
• Customer type- a set of customers who will use the system in the same way. E.g. Large pharmacies use a
switching system very much like other large retailers and thus could be grouped with them, even if they are
not in the same market segment.
• Customer type list- complete set of customer types.
• You obtain information on potential customers from marketing data for related systems, modified by
marketing estimates that take into account the new system's appeal.
• The business case developed for a proposed product usually includes the expected customer base.
• It is a valuable source for developing operational profiles, analyzing performance, and reviewing
requirements.

As an example, consider a hypothetical PBX that is sold to institutions for internal use and external
connections.
Assume there are two customer types, large retail stores and hospitals.

By Priya Singh (Assistant Professor, Dept of SE, DTU)

 Step-2 Create User type list
• A system's users are not necessarily identical to its customers. For PBX, user types in each
customer type include-
• User- a person, group, or institution that employs, not acquires, the system. 1. telecommunications users
• User type- set of users who will employ the system in the same way. By (people making calls and
identifying different user types, you can divide the task of developing the sending data),
operational profile among different analysts, each an expert on their user type. 2. attendants (internal operators
• User type list- set of user types. who answer the main
number),
• Sometimes the users are customers of your customers; sometimes they are
internal to your customer's institution. In any case, different users may employ 3. a system administrator (who
the system differently. manages the system and adds,
deletes, and relocates users),
• The differences may be the result of job roles--an entry clerk will view an and
insurance company’s claim processing system differently than an actuary.
4. maintenance personnel (who
• You derive the user type list from the customer type list by refinement: looking test the system periodically
at each customer type and determining which user types exist. and diagnose and correct
• If you find similar user types among different customer types, you should problems).
combine them. Each of these user types employs
the system differently.
By Priya Singh (Assistant Professor, Dept of SE, DTU)
Create System mode list
• System mode- set of functions or operations that you group for convenience in analyzing execution behavior.
• A system can switch among modes so that only one is in effect at a time, or it can allow several modes to exist
simultaneously, sharing the same resources.
• For each system mode, you must determine an operational (and perhaps functional) profile.
• Thus multiple system modes means multiple operational and perhaps functional profiles.
• The same function or operation can occur in different system modes.
• There are no technical limits on how many system modes you can establish.
• You must simply balance the effort and cost to determine and test their associated operational profiles against
the value of more specialized information and organizational convenience they provide.

By Priya Singh (Assistant Professor, Dept of SE, DTU)

• Basis for characterizing system modes-
1. Relatedness of functions/operations to larger task (System administration, data entry, customer
representative queries, transaction processing, report generation.)
2. Significant environmental conditions (Overload versus normal traffic; initialization (start-up or reboot for
failure recovery) versus continuous operation (includes warm start after an interruption); system location;
time.)
3. Operational architectural structure (Online retail sales mode versus after-hours billing mode.)
4. Criticality (Shutdown mode for a nuclear power plant in trouble.)
5. Customer or user (Customer group or user group requiring special functions/operations)
6. User experience (Novice versus expert mode.)

By Priya Singh (Assistant Professor, Dept of SE, DTU)

 Step-2 Create User type list
For PBX, five system modes are-
• A system's users are not necessarily identical to its customers. 1. Telecommunications business use
2. Telecommunications personal use
• User- a person, group, or institution that employs, not acquires, the
system. 3. Attendant use
4. Administration
• User type- set of users who will employ the system in the same way. By 5. Maintenance
identifying different user types, you can divide the task of developing the
operational profile among different analysts, each an expert on their user
type. Comment- The last three system
• User type list- set of user types. modes represent user types (disjoint in
that they do not share functions or
• Sometimes the users are customers of your customers; sometimes they
are internal to your customer's institution. In any case, different users operations).
may employ the system differently. The first two modes share most
functions or operations and could be
• The differences may be the result of job roles--an entry clerk will view an
insurance company’s claim processing system differently than an actuary. combined. However, both the
functional and operational profiles for
• You derive the user type list from the customer type list by refinement:
looking at each customer type and determining which user types exist. the two modes are expected to be very
different.
• If you find similar user types among different customer types, you should
combine them. Note- different modes can't execute
simultaneously

By Priya Singh (Assistant Professor, Dept of SE, DTU)

• When a system has different priorities for its tasks, it is particularly important that system modes be defined
for both normal traffic and heavy traffic conditions, because their operational profiles will differ.
• Defining different system modes is a convenient way of accommodating changes in the operational profile as
users become more experienced. In practice, you can capture the variations in experience with two extremes,
novice and expert, mixed in different proportions.
• Some systems control the operations they will accept on the basis of environmental variables, such as traffic
level and system-capability status, in order to reject noncritical, non functioning operations and dedicate
capacity to more critical ones.
• If a system must function in these conditions, each of these situations should be established as a system mode
and tested with the guidance of separate operational profiles.
• It is most convenient to group critical operations into one or more system modes, where each system mode
incorporates operations of the same criticality.
• Critical system modes will then receive accelerated or increased testing. The factor of acceleration or increase
is usually selected to yield enough execution time for the critical functions to assure achieving a desired level
of failure intensity with acceptable confidence.

By Priya Singh (Assistant Professor, Dept of SE, DTU)

Step 4 Create Functional Profile
• The next step is to break each system mode down into the functions it needs creating a function list—and
determine each function's occurrence probability.
• Functions, as noted, are defined from the user's perspective; they do not necessarily consider architectural or
design factors.
• They are established during the requirements phase and are closely related with requirements.
• In general, developing a functional profile is considered part of the job of developing the requirements.
• The functional profile is used in the management of the architecture and design phases and in the design of
the architecture itself.
• The functional profile is baselined and placed under change control, with appropriate traceability
requirements, just as the requirements are.
• Because you determine the functional profile before design begins, it can help guide the allocation of
resources during design, coding, unit test, and possibly subsystem test. To allocate resources and set priorities,
you must consider other factors as well, such as risk and developer expertise.

By Priya Singh (Assistant Professor, Dept of SE, DTU)

• Number of functions
1. A functional profile does not have a set number of functions, but it typically involves between 20 and several
hundred.
2. Factors affecting the number of functions-
• project size,
• number of system modes,
• number of major environmental conditions, and
• function breadth-the extent to which a function accommodates task variations.
3. How to define good functions-
Functions should be defined such that each represents a substantially different task, in the sense
that we are likely to assign a different priority and allocate different resources to the development of, or design a
different architecture for, that part of the system that supports that task.
The task can be substantially different either as a result of work accomplished (most common) or the
environment encountered.
4. Need for different functions-
If tasks differ considerably in criticality; the run types you group in a function should have approximately the
same criticality, because they will be treated as if they did.

By Priya Singh (Assistant Professor, Dept of SE, DTU)

5. Explicit versus implicit-
• Functional, and operational profiles can all be expressed in two forms, explicit and implicit, although it is
usually easiest to express the latter two profiles in implicit form.
• Explicit and implicit refer to different ways of specifying functions and operations and hence selecting them
for execution. At this point you must choose between an explicit or implicit operational profile or some
combination of the two because that determines if you should develop an explicit or implicit functional
profile.
Key input variable- An input variable that is common to the input states of two or more operations,
and whose value is needed to differentiate among some of these operations.
• In many cases, the values of a key input variable that differentiate operations are actually ranges, which are
called levels.
• E.g. the name of the operation is a key input variable.
• A parameter may be a key input variable if two or more operations have the same name and the value of the
parameter is the only way of distinguishing between them.

By Priya Singh (Assistant Professor, Dept of SE, DTU)

• Explicit profile- A profile is explicit if each element is
designated by simultaneously specifying the values of Sample Implicit Operational Profile
all the key input variables necessary to identify it.
• Implicit profile- A profile is implicit if it is expressed
in terms of sequences of subprofiles, each subprofile
representing the possible values of one key input
variable and their conditional probabilities of
occurrence, given the values specified for the previous
key input variables in the sequence.

• Example-
Suppose you have two key input variables, C and D, each
with three values. For simplicity, assume that the
variables are independent (if not, the subprofiles become
more complex since all the preconditions must be stated).
We can define nine operations based on the values of
these key input variables. Example implicit and explicit
operational profiles for these operations are given in Table
below-

By Priya Singh (Assistant Professor, Dept of SE, DTU)

Representation of an implicit profile-
• An implicit profile is most conveniently expressed as
a directed graph or a tree with the nodes representing
key input variables and the branches, their values, and
the associated conditional probabilities.
• For example, figure along shows sample "call trees"
used by International Definity.
• It represents an implicit operational profile.
• Instead of selecting test cases from a complete list of
all possible paths with associated probabilities
(explicit profile), you select from each set of branch
alternates with their associated branch probabilities.
• An explicit profile can always be determined from an
implicit profile by tracing all paths through the
directed graph or tree, multiplying the conditional
probabilities of the branches together. Note how a
test call is generated by pairing the call trees for a
calling and a receiving party.

By Priya Singh (Assistant Professor, Dept of SE, DTU)

Advantages of Implicit profile-
• It usually requires you to specify fewer elements—as few as the sum of the number of leyels
of the key input variables, depending on the amount of independence among these variables.
Using an explicit profile, the number of elements you must specify can be as high as the
product of the number of levels of the key input variables.
• The implicit profile can be used in cases where the number of elements of an explicit profile
would be too large. Alternatively, use of the implicit profile may give you finer granularity in
measuring usage for the same effort you would use for an explicit profile.
• The implicit approach is suited to transaction-based systems, in which processing depends
primarily on transaction attributes that have known occurrence probabilities. A system to
generate personalized direct mail, for example, depends on customer attributes such as
location, income, and home ownership.

By Priya Singh (Assistant Professor, Dept of SE, DTU)

Initial function list-
• Need-
1. The initial function list focuses on features, which are functional capabilities of interest and value to users.
2. It consists of one list if you are developing an explicit profile, and one list for each key input variable if you
are developing an implicit profile.
3. System requirements are usually the best source of information on features. If you have trouble identifying
the function, it is often because the requirements are incomplete or fuzzy.

In the PBX example, we will generate an explicit profile for the administration system mode functions.
The initial function list has four elements because the system-administration mode has four principal
functions:
1. adding a new telephone to the exchange,
2. removing a telephone,
3. relocating a telephone or changing the service grade provided, and
4. updating the online directory

By Priya Singh (Assistant Professor, Dept of SE, DTU)

Environmental variables-
• Identify the environmental input variables and their values or value ranges that will require separate
development efforts, such as substantial new modules.
• Environmental variables describe the conditions that affect the way the program runs (the control paths it
takes and the data it accesses), but do not relate directly to features.
• Examples- Hardware configuration and traffic load
• How to finalize environment variables-
Best approach is to have several experienced designers brainstorm a list of environmental variables that might
cause the program to respond in different ways, and then decide which of these would likely have a major effect
on the program.

In the PBX example, telephone type is an environmental variable that has a major effect on processing.
Although telephone type can have several values, here only analog (A) and digital (D) telephones have
substantially different effects on processing. So there are two levels for the environmental input variable, A
and D.
When environmental variables and their values are associated with their occurrence probabilities, you have
an environmental profile.

By Priya Singh (Assistant Professor, Dept of SE, DTU)

Final function list- Before you create the final function list, you should examine dependencies among the key
environmental and feature variables.
If one variable is totally or almost totally dependent on another, you can eliminate it from the final function list.
If one variable is partially dependent on another, you must list all the possible combinations of the levels of both
variables, along with all the independent variables. You will have one final function list for an explicit profile.
For an implicit profile, the number of function lists will equal the number of feature and environmental key
input variables.
• The number of functions in the final function list is the product of the number of functions in the initial list
and the number of environmental variable values, minus the combinations of initial functions and
environmental variable values that do not occur.

The final function list for the PBX, is developed from the initial function list enumerated above and the
telephone type environmental variable. It has seven elements: the three initial functions with two
environmental variable values, plus the initial function "online-directory updating," which is not affected by
telephone type.

By Priya Singh (Assistant Professor, Dept of SE, DTU)

Occurrence probabilities-
• Sources- The best source of data to determine occurrence probabilities is usage measurements taken on the
latest release, a similar system, or the manual function that is being automated.
• Advantage of selecting functions-
The interaction with the customer that it requires highlights the functions' relative value.
It may be that some functions should be dropped and others emphasized, resulting in a more competitive
product.
Reducing the number of little used functions increases reliability, speeds delivery, and lowers cost.

Final Function list

By Priya Singh (Assistant Professor, Dept of SE, DTU)
• In the sample PBX, there are 80 telephone additions, 70 removals, and 800 relocations or changes per month.
• Online-directory updating represents 5 percent of the total use in the system-administration mode.
• We will assume that the occurrence probability for the system-administration mode is 0.02.
• Thus the overall occurrence probability for each of these functions, without consideration of environmental factors, is the product
of their occurrence probability and the system-administration mode's occurrence probability in the overall system.
• To take into account environmental factors, assume that 80 percent of the telephones are analog and 20 percent are digital.
• The environmental profile is shown. Also assume that the occurrence probabilities of the first three functions and telephone type
are independent.

By Priya Singh (Assistant Professor, Dept of SE, DTU)

Functional and operational profiles with correlated elements
• What’s desired?
1. It is desirable to define functions and operations such that their probabilities of selection are independent of
previous functions/operations selected.
2. Then you only need to determine occurrence probabilities for the functions/operations in your list. The
foregoing implies that if you have a sequence of functions/operations that are highly correlated, you should
try to bunch them into one function/operation.
• Why?
1. You can assume that functions/operations with little correlation are independent, but you must recognize the
fact of your approximation.
2. The risk here is that independent random selection will distort the true occurrence probabilities to some
extent.
3. Positively correlated sequences will occur less frequently than they do in reality, increasing the risk that
failures occurring in them may be missed.
4. Negatively correlated sequences will occur more frequently than they do in reality, wasting test and
debugging resources by overemphasizing failures associated with them.

By Priya Singh (Assistant Professor, Dept of SE, DTU)

Operational profile
• The functional profile is a user-oriented profile of functions, not the operations that actually implement
them.
• But it is operations, not functions, that you test.
• To allocate testing effort, select tests, and determine the order in which tests should be run, the operational
profile must be available when you start test planning.
• Functions evolve into operations as the operational architecture of the system is developed.
• A function may evolve into one or more operations, or a set of functions may be restructured into a different
set (and different number) of operations.
• Thus the mapping from functions to operations is not necessarily straightforward. E.g., an administrative
function in a switching system might be to relocate a telephone. This single-function may be implemented by
two operations, removal and installation, because these tasks may be assigned to different workgroups.
• Generally, there are more operations than functions, and operations tend to be more refined.
• The principal steps in determining the operational profile are to list the operations and determine the
occurrence probabilities.

By Priya Singh (Assistant Professor, Dept of SE, DTU)

By Priya Singh (Assistant Professor, Dept of SE, DTU)
Listing operations
• Use the functional profile to develop the list of operations, mapping functions to operations by following the
operational architecture of the system (the way the operations combine to accomplish the functions).
• Since functions differ from one another by virtue of implementing substantially different tasks, operations
will generally be distinguished from one another by substantially different processing.
• Different operations will usually execute substantially different code paths. Thus the definition of operation
can be affected by program structure.
• You will usually want to select at least one test case for each operation, as each has a substantial risk of
experiencing a failure that is not experienced by other operations.
• As with functions, operations may differ as the result of work accomplished or the environment encountered.
• If possible, it is desirable to define functions such that functions and operations will map one-to-one (greatly
simplify deriving the operational profile from the functional profile.)
• Creating and examining a list of those events that initiate program execution (such as commands or
transactions) may help you create the operations list.
• If the events have parameters, consider the effects of different parameter values. If a value of a parameter
causes significantly different processing to occur, you may want to define a new operation. If there are N
significantly different kinds of processing associated with an event as a result of different parameter values,
then the original function associated with the event should be replaced by N operations.
• In creating the list, you must also consider environmental variables.

By Priya Singh (Assistant Professor, Dept of SE, DTU)

• The definition of a run should imply an execution time that is short enough so that sufficient executions exist
during:
i. Field operation to permit satisfactory accuracy in characterizing usage
ii. Test to permit satisfactory accuracy in reproducing usage
• Also, the run should be short enough so that the input state needed to characterize its interaction with the
environment is not excessively long. For example, a complete flight of a space vehicle or aircraft might be
defined as a run, but the execution time would be too long to meet the constraints just noted.
• Avoid excessive interaction between operations.
• Now you need to verify that the list of operations is as complete as you can reasonably make it.
i. First, you develop a list of input variables and their ranges of values that is practically complete. A
"practically complete" list identifies all input variables except those that take on one value with very high
probability. You are ignoring and thus won't be testing the alternate values. This is acceptable because they
occur so rarely that they have little effect on reliability even if they fail. The degree to which you can do this
decreases for systems requiring higher reliability.
ii. You then create a representation of the input states, also making it practically complete. Because the
identification process will never be perfect, you should employ other strategies, such as reducing the
number of input states and using indirect input variables to ensure that you handle hidden input variables
properly. The amount of effort you put into this should be based on reliability requirements, the cost of the
extra effort, and any information you have on the probability that these interactions will occur.

By Priya Singh (Assistant Professor, Dept of SE, DTU)

Reducing number of operations
• Ways to reduce #operations:
i. Reduce the number of run types.
ii. Increase the number of run types grouped per operation.
iii. Ignore the remaining set of run types expected to have total occurrence probability appreciably less than the
failure intensity objective of the system.

• Some ways to reduce the number of input variables are:

i. Reduce functionality
ii. Reduce the number of possible hardware configurations
iii. Restrict the environment the program must operate in.
iv. Reduce the number of types of faults (hardware, human, software) the system must tolerate.
v. Reduce unnecessary interaction between successive runs.

• All these approaches have costs, in addition to the costs of analysis and redesign.

By Priya Singh (Assistant Professor, Dept of SE, DTU)

Occurrence probabilities
• Since different system modes have different operational profiles but may share common operations, you may
have to determine multiple occurrence probabilities for the same operation.
• In some cases, there may be field data already existing on the frequency of events the system must respond to.
• You should expend some effort searching for such data, because it may provide the most cost effective
approach to determining occurrence probabilities. This is especially true if the alternative is building special
measurement and recording tools.
• There are two general ways to determine occurrence probabilities for operations:
i. Count the occurrence of operations in the field.
ii. Rely on estimates derived by refining the functional profile.
• The first is more accurate, but can be done only if a previous release exists. The operational profile for the old
system can be measured; for the new operations, estimated. The two parts are joined by weighting each one's
occurrence probabilities by the proportion of total system usage that part represents.
• If operations are independent and do not depend on the history of preceding operations, you need only to
count the execution of each operation.
• If they are not independent, you must record the sequence of operations.
• You can later process the sequence to determine conditional probabilities. An operational profile can be
recorded in either explicit or implicit form.

By Priya Singh (Assistant Professor, Dept of SE, DTU)

• The recording process usually adds some overhead to the application.
• As long as this overhead is not excessive, it may be feasible to collect data from the entire user community.
• However, if the overhead is large, you will probably have to employ a user survey instead of recording.
• In sampling users, the same guidelines and statistical theory used for polling and market surveys apply; a
sample of 30 or even fewer users may suffice to generate an operational profile with acceptable accuracy.
• If the costs of obtaining occurrence probabilities are an issue, you may make measurements of moderate
granularity and accuracy for the first version of a software product, refining them for later versions only if you
discover in the field that the reliability predictions from test are in error.
• If you will be using usage data for testing only, it is acceptable to directly drive testing by recording complete
input states in the field.
• You need not determine the operational profile or create test cases.
• This saves time and effort, but these savings will be reduced by the extent to which you add new operations.
For the new operations, you must estimate occurrence probabilities and develop test cases.
• In order to estimate by refining the functional profile, determine the function to operation mapping. Then
allocate the function occurrence probabilities to operations. In doing this, you may need to obtain occur rence
probabilities of environmental variables whose values distinguish different operations.

By Priya Singh (Assistant Professor, Dept of SE, DTU)

• Let's examine one common type of systems, command-driven for PBX.

• Update for operation ‘add’ as it depends upon service grade and not on location.

By Priya Singh (Assistant Professor, Dept of SE, DTU)

• Operational Profile Segment on features (without environmental
variable)-

By Priya Singh (Assistant Professor, Dept of SE, DTU)

• Operational Profile Segment on features (with environmental
variable)-
•

By Priya Singh (Assistant Professor, Dept of SE, DTU)

• Incorporating environment variable into operational profile

By Priya Singh (Assistant Professor, Dept of SE, DTU)

• Operational Profile (for Administration module of PBX system)

By Priya Singh (Assistant Professor, Dept of SE, DTU)