4.

1 POLICY IMPLEMENTATION: AGENCIES AND PROBLEMS

Objectives
 Introduction
 Stages of Policy Implementation
 Implementation Perspectives
 Aspects of Policy Design for Implementation
 Modes of Policy Delivery and Implementers
 Basic Functions for Public Administrators
 Voluntary Sector and Pressure Groups
 Let Us Sum UP
 Exercise

OBJECTIVES
This lesson explains you some of the problems associated at policy implantation level.
 After going through this lesson, you will be able to understand:
 problems involved in aspects of policy implementation design;
 problems involved in delivery level to the implementers;
 the role of voluntary sector and pressure groups at implementation level.

4.1.0 INTRODUCTION
Human life is pack of action. Every action has chain reactions which are potentially
equal and opposite in nature and degree. The preceding chapter dealt with the group
efforts in which governmental and nongovernmental actors in India interacted directly and
indirectly to formulate their national policies. When policies are enacted, the next immediate
stage is execution of public policy. This is the most vital phase and a highly important
venture in the life of the people and the nation.

Public policies carry the best of the nation’s goals, aims, intentions and the means for their
realization, which aims at the well-being of the people. The happiness of the greatest
number of masses and the peace of the nation are attainable through positive realization
of public policy. Both the making and the execution of public policy are the functional roles
of all the governmental and non-governmental actors and factors.

Implementation is an action-oriented process which translates into physical reality the

1
plans, goals, projects and programmes declared by competent authority. Policy – making
takes into consideration various external factors, like the constitutional guidelines and
directives to operate within the legal framework of the laws of the land, the customs, the
traditions and conventions. This avoids conflicts and duplication of efforts in decision-
making. Furthermore, the international regard is also kept in mind in devising public
policies. The execution of public policy cannot ignore these sensitive socio-economic and
political areas of human endeavours. There is no denying the fact that the successful
achievements of public policy depend to a great extent on the success of its
implementation. The execution of public policy is a sacrificial function of different persons
within a given political set-up, playing assigned roles order to attain set national goals.

According to Webster’s dictionary, ‘programme ’is a brief outline of the order to be pursued
or the subject to be embraced. This involves obtaining a complied data or information and
utilizing these in consonance with the available resources of the nation to provide the
required goods and services to the people. Pressman (1973) observes that the cards in this
world are stacked against things happening as much effort is required to make them move.
The remarkable thing is that new programme works at all. Policy implementation can be
viewed in a systematized stage as enumerated henceforth.

STAGES OF POLICY IMPLEMENTATION

Policy Initiation
This involves identifying the appropriate policy to be executed and obtaining the approval of
the computer policy which touches on several aspects of people’s life, broadly seen in the
socio-economic and political spheres.
Organization
With the knowledge of goals to attain, manpower, the necessary finance, the equipment,
machines and other relevant materials are mobilized and assembled into workable units and
organizations for the realization of set objectives. The motivation techniques and
compensations are harmoniously applied to boost the morale of the work force towards
greater achievements.
Operational Strategies
This is the action stage when the known principles and practices of communication,
information, utilization, data analysis, coordination’s supervision, resources management and
public relations are skilfully managed to accomplish predestined goals.
Feedback Mechanism
The knowledge of what has happened is the function of feedback mechanism. This makes a
2
speedy evaluative analysis of the policy implementation activities and reports observations
positive or negative to the appropriate quarters. The feedback mechanism is a two-way
traffic which communicates not only the roles of the action side of policy execution, but
also the effects of reactions.
Every nation has its own kinds of people, its peculiar political system, its unique environment
and its style of managing its own affairs.
IMPLEMENTATION PERSPECTIVES
It is often said that the policies formulated are good enough but don’t succeed because
of a faculty and defective implementation process. Commenting on the vast criticism of
the policy execution process, Louis, W. Koening says, ‘Nonetheless, in the hubbub of
criticism, the solid long-running accomplishments of implementation should not be
overlooked. Men are put on the moon, wars are fought and won. Day after day, mail is
delivered, police and fire protection are provided, taxes collected, social security
benefits disbursed, pollution reduced, the nation’s health improved. Some of the
society’s ablest individuals work for the government and compile enviable records as
achieving implementators. The bureaucracies of which they and others are members are
highly diverse and are neither equally efficient nor inefficient. Similarly, there are poor
as well as able bureaucrats”.
Implementation, as a process, in a number of respects is a non-transparent phenomenon.
It is difficult to easily lay hands on such of its characteristics which has far reaching and
significant consequences. It has the quality of being pliant as a result of which it undergoes
tremendous and rapid transformation. There are several perspectives on the process of
execution. Some of these are as follows:
4.1.2.1 Implementation as Evolution

A policy which is formulated has a set of objectives, goals and ideas which are loosely
connected with each other. Acasual chain between the multiple goals and objectives of
the policy is forged by the execution process. Changing environmental considerations,
vested interests of the political parties and influential groups at times compel the framing
of such policies which have rather numerous, contradictory and equivocal goals. At the
time of framing of such policies, the formulators are not in a position to anticipate the
possible problems and the constraints to be faced at the time of implementation.
Execution is shaped by the original policy from which it springs. Such problems and
constraints, in fact, should be thought of and worked upon accordingly at the time when
policy is framed. Maybe it is impossible to do so at that stage and so these appear on the
scene when the policy is put in to execution. Consequently, the executor addresses
himself to number of points, viz. which of the different objectives of the policy are to be
implemented? How it is to be done? Which priority order is to be made? What resources
are to be put to the said task? Etc. besides, a number of other problems, like support or
opposition of the target group; contradiction to some other related policies already in
existence; and the feedback about the positive and negative points of the policy are also
noticed at the implementation stage. 3

It goes without saying that execution is a critical juncture at which such essentials are
included in the policy which at the time of its original manifestations could not be done. As
the policy moves across the execution spectrum new circumstances and problems are
realized, and the policy continues to evolve. These must be duly coped with so that the
required potentials and facets are incorporated into original policy idea.
Implementation of Work Mechanism
Execution as a process is translated into implementation through institutions and agencies
which are assigned the said task. Besides planning, the hierarchical levels and elements
of control are essential ingredients of the work mechanism of the execution. Alack of these
would make it difficult for the superiors to keep a check on the performance of the
subordinates and also the output of the task assigned cannot be measured. The significant
imperatives are, rather, a must to be satisfied by the execution process. First execution
must, through process and action, submit courteously to the legal contents included in the
policy documents. Unless, it is insured on the part of the implementation agency, policy
execution in the desired way cannot be looked for. Second, to translate policy intent into
virtual action coupled with instrumental rationality, the implementing officials and
functionaries’ commitment to the task has to be activated and assured. Third, concerned
effort have to be made for a consensus on working regarding implementation not only
within the said given implementing agency, but a cordiality of relationship between the
internal system and external system, that is micro-system (the implementing agency) and
macro system (the political agency) is essential. Stress is laid on a hierarchical approach
in order to control the discretion used by the subordinates, thus affecting the
implementation process. In the words of Koeing, “the hierarchical approach uses
organization control processes of policy into actual accomplishment. Implementation,
according to this perspective, also entails administrative learning, anoutgrowth of the
difficultyof implementing policies through multiple institutional structures with their distinctive
interests, fragmented power, and narrow skills that order their efforts and that make
coordination both a necessity and elusive function”.

4.1.3.2 Implementation as Politics

The process of implementation, which is followed by the governmental agencies, has elements
like bargaining, gaming, accommodating, compromising, etc., thus making it a kind of
political issue. In the words of M. Helen and A. Dean, “implementation is conducted
largely through accommodation and bargaining,4 with relentless give and take, winning
and losing, chronic disjointedness, and pervasive disarray. The policy to be implemented is
simply a point of departure for bargaining among the implementers, and what materializes
as implemented policy is essentially determined by specific local characteristics, which are
unique to each implementation context and which shape the contours of the eventual
outcomes” (Helen and Dean, 1980). In simple words, it means that the inclination of the
implementers and the importance of the local areas, that is, the target area and group play
an important role in implementing the policy. It is upon these variables that the success or
failure of the policy being implemented depends.
Implementation as Problem Tracing
It has been also stated earlier that a number of problems which could not be appreciated
or anticipated at the stage of policy formulation appear at the implementation stage. The
implementation process is, in fact, a problem tracing process. The problems/constraints,
the viewpoints of the affected groups, the requirements on count of infrastructural and
financial resources are the ones which add to the process of improving with policy further.
Sabatier and Mazmanian, have stated that, “implementation proceeds through several
stages, commencing with policy outputs or decisions of the implementing agencies, which
include the translation of statutory objectives into substantive regulations and standards
operating procedures. Asecond stage is the target group’s compliance with policy outputs,
which is influenced by such factors as the relative costs and benefits of compliance, the
probability that non-compliance will be detected and penalized, and its members’ attitudes
towards the legitimacy of the implementing rules”.
Implementation is a process which makes the implementers performs the role of linkers
between the policy formulators and the target groups. Though, mostly, the permanent
executive has the major role in implementing the policy, yet the role performed by other
governmental institutions cannot be minimized or ignored.
Implementation as Approaches
There are three types of implementation approaches, they are

1) Top-Down Approach

2) Bottom-Up Approach

3) Other Approaches

a) TOPDOWN APPROACH:
In top-down approach consisting of Goals are clearly defined Policies are confirmed at a
coherent single domain. Policy consists on authorities’ statement. Policy Designer allocates
the resources in organized way at each implementation level. A common cause of interest
exists at all levels. A well- established structure of command control practical example is
from top to bottom which require strict adherence to compliance and regulation procedure.

b) BOTTOM UPAPPROACH 5
In bottom-up approach consisting Goals are loosely defined Policies are independent
domains computing for resources. Resources are expertises are not allocated properly.
Implementation through communication is compromised. Street bureaucrat has latitude in
enforcement. Accommodate local norms incentives to find common grand in procedural
implementation.
 c) OTHER APPROACHES
In addition to these two approaches, other social scientists have contributed a lot to facilitate
the understanding of implementation process and tried to correct or explain the gaps in
these two above-mentioned approaches.
Structural Approach
In this case, from policy implementation point of view, policy design and organizational
structure should be studied together. The complicity of any policy extends itself beyond
the organizational boundaries, in terms of control, pace and expertise. Implementation is
perceived as a technical or managerial practice. Our earlier argument suggested us that
policy making process, right from the beginning is iterative rather than a liner process. A
continue feedback and back and forth interaction between policy design and
implementation is a common phenomenon.
The structural approach is a relatively hierarchical, representing bureaucratic organization.
For its external operation or working environment, the processes are identical to vertical
organization. However, when it reaches to policy implementation, more “organic” features
are required; less emphasis on hierarchy, more flexibility and adoption to changing
environments.
To process more information and take action simultaneously demands more organic or
less vertical organizational behaviour. A possible compromise between bureaucracy and
“adhocracy” is a cross section of “horizontal” and “vertical” authorities.To change into
traditional organizational structures, in any country, in itself is a Pandora Box. We may not
comprehend the issues aroused out of this attempt, and it might be beyond the scope of
this book.
Procedural Approach
Appropriate procedure and processes are more important in an organizational structure
than anything else. Implementation process is heavily dependent on managerialand technical
skill. Procedures are introduced to control, set pace, coordinate, scheduling timeline,
monitoring progress and management. Procedures define managerial boundaries, control,
logical sequences, and allocation of resources. These procedures define enhance the
probability to move in the right direction, timeline management and anticipate risks.
Behavioural Approach
Keep in mind that all these organizational structures, policy processes, techniques and
managerial tools are as good as underline human beings who are executing them. The
competency of human resources is as crucial as the strength of a chain that is used to pull
the weight. The chain is as strong and reliable as its weakest link is.
Human behaviour and attitudes, individual or collectively as a nation, has great impact on
implementation process. For any changes, moving away from status quo, the human
6
reactions varies; active resistance, indifference, passive acceptance. Fear of change
represents uncertainty, ambiguity and people reflect low tolerance for both. For more
direct concerns as personal benefits, economic condition, privileges, social status, security
and career prospect, reaction is more obvious and certain.
In design phase and before implementation process an extensive consultancy and
communication should occur among all the policy actors and target population to avoid
any possible resistance. A full disclosure of information should be provided as early as
possible, including concerns, difference of opinions, objectives and logistics. Seek to
convince might be more effective approach than command. The war on terror might have
positive result if U.S. administration and coalition partners try to win “hearts and minds”
instead on their weaponry power.
Political Approach
Political approach is not limited to party politics. In terms of policy implementation, pattern
of power use within organization and its influence ‘over flow’ on other organization and
policies need a careful examination. If the policy is not crafted according to the political
authority of relevant organization, the probability of success in implementation phase can be
drastically reduced. The dominance of a “political will” is “a must” requirement for
successful policy implementation. This dominance always prevails regardless it is through
coordination, coalition, by partisan, mutual understanding or through decisive command
and control.
The success of policy is very much correlated with coherent willingness of dominant group;
an ability of pursuit by coalition partners, within organization or with outside agencies. In
any arrangement, participating organizations assert their political jurisdiction and
authority, via constitutional discretion or through bargaining in policy process.
The inter-governmental coordination, coalition and subordination, and exercise of political
will and jurisdiction might bring repercussions and complications and worth consideration at
before implementation process. Some policies are completely dependent on political
strategy for their successful execution.

ASPECTS OF POLICY DESIGN FOR IMPLEMENTATION

Dryzek defines policy design as “the process of inventing, developing and line-tuning a course of
action with the amelioration of some problem (in mind).” The design perspective, in particular, calls
attention to matching content of a given policy to the political context in which the policy is
formulated and implemented. The crafting of policies typically entails a long process of analysis of
problems and options, politically acceptable courses of action, and an n authoritative decision to
enact a policy. Here it should he understood that the political forces that shape policy
design also greatly influence policy implementation.
Further, in implementing a policy design, a great deal of attention has to be given to such
aspects as allocating tasks to personnel, financial resources, scheduling dependencies and
making decisions which arise in the course of using resources and generating outputs.
7
Implementing with a Network
Implementation of policy is an important phase in the policy cycle. It has the objective of
anticipating deviations from planned performance and making proper adjustments. An
important task of the policy implementing director/manager is to keep track of the
implementing tasks, their functional relationship and their scheduling dependencies. The
director has to be capable of resolving conflicts and making decisions which arise in the
course of mobilizing resources and generating policy outputs.
For the implementation of a given policy, it is important for the policy implementation
director to construct a policy implementing network which can help himensure that
policy tasks occur in proper sequence and on time. It should help him to identify which
tasks in the policy are most critical to overall policy performance. The construction of the
network requires composing an Activities Duration List. This list should project tasks for
each activity and the time required for each activity. For scheduling policyimplementation
activities, the Network helps the director in depicting which activities are more important for
economical and efficient management of the policy programme.
Allocating Tasks to Personnel
Most policies do not succeed because of poorly qualified staff. Implementation of a given
policy requires the manager to see that the activities happen on time and within the given
budget. But it requires, first, implementing personnel. If policies are to be carried out, there
has to be allocation of appropriate tasks to the personnel resources. The manager has to
ensure that the most qualified people are involved in the appropriate task. Regardless of
their status, specialized knowledge, experience and qualifications, all personnel need to
work as a cohesive team for the purpose of achieving desired targets.
For an effective policy implementation, a manpower plan is needed. This plan helps the
policy director to assign the staff the task of policy implementation. Further, for the purpose
of assigning role and responsibilities to a staff member, it is necessary to develop position
classification like a job description. The position classification may include: (a) status, role,
and duties of the staff, (b) reporting relationships, and (c) accountability criteria with a
view to evaluate the incumbent‘s performance in the position. It is important to organize
the stafffor securing technical performance. This mayinvolve such principles of organization
as unity of command, span of control, division and integration of efforts, and hierarchy of
authority and responsibility.
Making Decisions
In the implementation of a policy programme, the manager has to be adept at solving
problems and making decisions which arise in the course of using resources and seeing
what happens to the target group. Making decisions is the most difficult job for a policy
implementing director. Even with the best planning, there will always be a need to make
good decisions in the face of unanticipated events during the stage of policyimplementation.
For major decisions which impact resource requirements and technical outputs, this is a
major activity because such a decision requires full support of several constituents
(beneficiaries, sponsors, politicians, planning agencies, government). In this context of
decision making, three mechanisms have been suggested.
Exception principle: The exception principle states that difficult decisions (other than
8
routine decisions) involving unusual or unprecedented problems that have broader
implications for the whole policy implementation, should be reserved for senior staff at
higher levels in the organization.
Delegation of authority: The exception principle does not operate unless there exists some
degree of delegation of authority. The authority is needed at each level to enable the
implementing staff to perform their duties and tasks.
Consensus building: Consensus in this context refers to an agreement to support a
particular decision. Consensus building in a participative management strategy ensures that no
good ideas are ignored. It also builds a strong group among all those programme
constituents involved in the implementation process.
For effective implementation of a policy programme, there is the need for forming a steering
committee. The purpose of the committee is to ensure that a programme is being
implemented within the budget and on time.
MODES OF POLICY DELIVERYAND IMPLEMENTERS
Policy delivery in terms of provision of public goods and services involves a diverse set of
institutions and agencies-government, market and voluntary organizations. Modes of policy
delivery or systems of policy delivery have drawn the attention of the policy analysts.
These deliverysystems, in terms of the way in which public goods and services are provided
through a network of public and private institutions assume considerable importance.
This fragmentation creates new problems for control and accountability in a democratic
country like India. People now face an often bewildering array of agencies responsible
that the provision of public services. Hierarchies in the administrative system have given
way to policy delivery systems which use a mix of partnerships between the public and
private sectors, market mechanisms and new roles are being defined for the voluntary
sector and the community. The resultant mix of policy delivery agents includes a mix or
blend of bureaucratic, market and community agencies. It may be mentioned here that
thinking in terms of clearly defined sectors or modes of coordination is not, in practice,
helpful, since in the real world there is considerable ambiguity and overlap between them.
In practice, the mix of market, bureaucratic and community forms overlaps and combines
to provide ‘hybrids’ or plural forms.
Administrative Agencies
According to the Wilsonian viewpoint, the main function of (public) administration is the
implementation of (public) policies. In other words the administration has concentrated
on the machinery for the implementation of policies, as given, rather than on making them.
The job of the administrator is to carry out policies formulated by decision/policy-makers
(politics), and the role of the service provider is to carry out the policies administered by
the bureaucrat (bureaucracy).
The (public) bureaucracy is an important agency of the government. It is an
administrative organization consisting of a legal body of non-elected employed officials
organized hierarchically into departments in accordance with the rules governing the
conditions of their service. This is an important institution which performs most of the
9
day-to-day work of the government. Also, it is the bureaucracy, which controls the
personnel, the financial and the legal powers of the government, and it is this institution
that receives most of the implementation directives from the executive, legislature and
judiciary.
Civil servants are recruited, in theory, to serve political masters by carrying out their decisions.
Ministers decide on policies, and civil servants take the necessary executive actions to
implement them. The importance of the senior administrator‘s role in policy implementation
arises because he is concerned with ends and not merely the means. He is exclusively
concerned with the implementation of policy decisions made by the politicians. More
important is the work of the senior administrator on the development of major policies in
line with government commitments. He has a constitutional responsibility to advise on
the financial and administrative implications of different policy options, thus helping
ministers to find ways of achieving their political objectives.
Policy programme implementation by the administrator feeds back into policy formation so
that he can advise authoritatively from experience on the practicability of different policy
options. A. great deal of legislation and policy builds on the past administrative practice
and accumulated experience. Further, the knowledge derived from direct experience of
policy implementation gives the senior administrator a near monopoly ofknowledge relevant to
policy-making. New policy emerges, as senior administrators bring their experience and
ideas to bear on problems which political masters wish to solve. As repositories of
knowledge and experience, senior administrators are able to give instructions and advice to
the lower staff as to how to implement policydecisions. Theycan foresee the administrative and
political difficulties likely to be encountered from the interests more affected. They are
able to argue from positions of great strength about new methods of dealing with policy
implementation problems for which no satisfactory solution has been found. In this way the
administrators role inpolicyimplementation is of considerable importance. The administrator
then, is a catalyst who seeks to bring out what is required in those whose support is
required.
But bureaucratic organization has been subjected to a body of criticisms. It is said to be
afflicted with excesses of red tapisrn, stringent rules and an attitude of unresponsiveness.
Despite its maladies, it holds importance because implementation is the continuation of
policy- making through other means. Legislation is never self-implementing but always
requires delegation to appropriate organizations and personnel. Placing a programme in its
perspective is the first task of implementation and administering the day-to-day work of
an established programme is the second. It is because delegation and discretion permeate
bureaucratic implementation that it plays a crucial role in the power structure of policy-
making and policy action. Technically, the task of all public organizations and personnel is
to implement, execute and enforce laws and policies. In doing so, most personnel do use
bureaucratic discretion. It is pointed out that legislation does not minimize discretion and
more details may even increase personnel discretion. As Davis remarks: “Perhaps nine-
tenths of the injustice in our legal system flows from discretion and perhaps only ten per
cent from rules”
Despite the fact that the bureaucracy has been criticized, the government increasingly
approaches public organizations for pursuing their policies. Since so much power and
control over implementation is held by the 10administrative organizations and personnel,
chief executives, must put in efforts to control their discretion. This rests on many strategies.
First, if the public agencies do not implement a law to the satisfaction of the legislature, the
policy can be legislatively changed. The executive may also overrule routine bureaucratic
interpretation of legislation. Secondly, from time to time, most of the problems associated
with administration could be solved either by transferring responsibility to a friendly agency
or by replacing a recalcitrant agency head, or by paring the agency budget. Thirdly, the
legislature has an important role to play in curbing the bureaucratic discretion by making
legislation more detailed and clear. The bureaucracy can also be pressured through public
hearings, the media and other forums. When all else fail, the legislature can take the
concerned public agency to court for malfeasance in implementing policy.

BASIC FUNCTIONS FOR PUBLIC ADMINISTRATORS

On the assumption that the public administrators or government servants do accept the
basic values and the general direction of the political system, the public administrators, in
their dual capacity as those who give expert advice on policy issues and those who
implement policies have some functions to perform in connection with policy management,
which can be discussed here.
Policy implementation is not easy. Without determined political support and without willing
cooperation of many top administrators, little can be achieved. In policy implementation,
administrators, especially senior executives, should have the following functions and
role to perform.
In the first place, administrators must clearly understand the nature and significance of
policies which the political masters have set. They are responsible for advising in the
formulation ofpolicies designed to achieve goals and also mobilizing, organizing and managing
the resources necessary to carry through these policies. Second, they should assist policy
makers to avoid ambiguities. They should advise them on the importance of adopting
policies which can be implemented. Third, they should be able to translate the general
policies and their objectives into operational targets. This function should also include
analysis of probable costs and benefits of each for achieving the operational targets. As far
as possible they should adopt a rational approach and use management techniques to
implement policies. Finally, they should be able to pay special attention to the question of
coordination of policies and policy instruments. They should analyze the policy in question
in relation to other policies to see if any inconsistencies exist and examine whether it
complements or supplements other policies to produce better results,

VOLUNTARY SECTOR AND PRESSURE GROUPS

New roles are being defined for the voluntary sector for the provision of public services.
The voluntary sector seeking to do good works for the poor and needy has been well
recognized in the history of welfare state. Religious organizations and charitable institutions in
particular in India, have served as principal providers of many social services. The growth
of welfare state and mass education made the existence of the voluntary sector apparently
less necessary in the 20th century. However, in the recent years, the role of the voluntary
sector has received wide recognition and impetus as the State no longer desires to provide
the range of public services that was once 11expected of it. The voluntary organizations
(non - governmental organizations) have made an enormous contributions to the
implementation of public policies, notably the forest policy, population policy, health and
education policies. Their role has been explained more extensivelyand more accurately in
terms of being private agent of public policy.
It has been already discussed that group action is considered a more effective method than
individual action for implementation of public policies. NGOs and civil society group are
important means of enhancing the effect of public opinion. They can communicate more
effectively than individual citizens with public officials on implementation of public policies.
LET US SUM UP
It may be observed that although some studies have been made in public policy on the
political economy processes, the cybernetics theory of Deutsch has not been able to
create much influence. There is no doubt that Deutsch has made outstanding contribution
to the study of public policy through his political economic theory, but it is hardly anything
more than suggestive. Through his models, he has raised a number of questions regarding
the performance of governments, but these models are of little use in answering the
questions.
In public policy approach is particularly usefulfor an analysis of the processes of bargaining,
conflict resolution, decision making, and evaluation of policies, estimating the impact of
publicity and propaganda as well as for understanding the dynamics of public policies.
However, this approach is hardly useful in the analysis of power structure of society,
ideologies, allocation of resources, or in explaining the phenomena of violence and
revolution, etc.., communication theory can, therefore, be commended for application
only in limited sphere of public policy.
EXERCISE
1) Identify the policy implementation stages and explain with suitable examples.
2) What are the implementation approaches and identify the important approaches?
3) Discuss the policy design for implementation.

12
4.1 PUBLIC POLICY: IMPACT, COMPLIANCE AND NON- COMPLIANCE

4.1.0 Y. Pardhasaradhi
STRUCTURE
4.1.1 Objectives

4.1.2 Introduction

4.1.3 Conceptualising Policy Impact

4.1.4 When to Conduct Evaluation
4.1.5 Types of Impact Evaluation

4.1.6 Methods or Approaches for Impact Evaluation

4.1.7 Problems in Evaluation the Impact of Public Policy

4.1.8 Policy Compliance and Non-Compliance

• Causes of Compliance

• Causes of Non-Compliance
4.1.9 Exercise

• OBJECTIVES

In this lesson you will understand the ways and means in which we understand the
impact of a particular policy. You also get some idea about why there is compliance
13
or non-
compliance for a particular policy. After going through this lesson, you will be able
to know:
5 what is policy impact and how important it is;
6 how to conduct policy impact evaluation and what are the basic types of involved in
it;
7 why some policies achieve higher compliance and why not others.

• INTRODUCTION
In a democracy, the political executives and administrative machinery formulates
various types of policies for the socio-economic development of different sections of
the society. After formulation of the policy the government invests huge resources,
material and intellectual, in the implementation of the policy. It is the responsibility of
the policy makers to ensure whether the intended or pre-designed objectives are
achieved or not and what extent the benefits of government to policies are reaching
to the stakeholders. Therefore, there is a need for government to evaluate the impact
of public policies directly or indirectly. At present different departments adopt
different methods and procedures for measuring the impact of public policies on the
stakeholders.

• CONCEPTUALISING POLICY IMPACT

From a conceptual point of view there is no single or general definition of the
policy impact. The specialized studies use to refer to this in a variety of expressions
like “estimating the impact” the “impact analysis” the “impact study” or the “impact
evaluation” or the “impact assessment” and later “the impact assessment of
regulations”. According to one definition the impact evaluation process represents an
analytical approach based on the information in order to evaluate the possible costs,
consequences and effects of a planned instrument of public policies. The impact
evaluation has been also defined as a set of logical steps to be followed when one
prepares a policy proposal. It is a process that prepares evidences for political
decision makers on the advantages and disadvantages of public policy options by
assessing their potential impacts (economic, social and environmental). According to
the erstwhile Planning Commission of India, impact evaluation is a systematic,
186
mandatory and consistent evaluation of social, economical or environmental aspects or
impacts (such as benefits and/or costs), of the affecting the external interests of

187
governments, of the regulation proposals and ofany other kinds ofpublic policy
instruments to:

1. subsume the public policy decisions before adopting;

2. assess external impact, of regulatory and administrative practices;

3. assess the accuracy of an earlier assessment.

At a first glance, the role of impact assessment seems straightforward: “to inform
the decision-makers about the potential consequences of their policies”. In essence,
the role of the impact assessment is more complex; it aims to improve decision-
making processes’ by systematically collecting information about the likely impacts
of a planned policy and thereby providing the basis for deciding “the best policy”.
In other words, the impact assessment is an aid to decision-making, not a substitute
for it, contributing to a coherent justification of a government intervention, as wellas
improving the capacityof public policy- making. In many countries, this is strongly
related to a “better regulation agenda” that aims to improve the quality of regulation
and to reduce the administrative burden. In a synthetic form, the overall aim of impact
evolution is to assist the Governments to make their policies more efficient, and is an
important factor in responding to the impact determined by international markets
and budgetary constraints on modern economies and the consequences of
competing policy demands.

From the aspects mentioned above, we can outline the dual nature of impact
evaluation, each with its own methodological approaches:

1. As a technical tool for analysis of the consequences of planned governmental

interventions, providing information to stakeholders and decision-makers;
2. As a legal and institutional procedure linked to the decision-making process of
public interventions.

Additionally to the meanings already mentioned, the impact evaluation is considered

as a valuable communication tool. The process of consulting the stakeholders often,
creates useful debates, bringing valuable information and analyses. Moreover, refining
the meanings given by different studies it can stress that the research of impact

188
evaluation and implementation is growing and this shows that learning process of
using impact assessment

189
is a cumulative one. Information generated by impact evaluations reforms decisions
on whether to expand, modify, or eliminate a particular policy or program and can be
used in prioritizing public actions. In addition, impact evaluations contribute to
improve the effectiveness of policies and programs by addressing the following
questions:
8 Does the program achieve the intended goal?
9 Should this pilot program be scaled up? Should this large scale program be
continued?
10 Can the changes in outcomes be explained by the program, or are they the
result of some other factors occurring simultaneously?
11 Do program impacts vary across different groups of intended beneficiaries
(males, females, and indigenous people), regions, and over time?
12 Are there any unintended effects of the program, either positive or negative?
13 How effective is the program in comparison with alternative interventions?
14 Is the program worth the resources it costs?

• WHEN TO CONDUCT EVALUATION

Impact evaluations demand a substantial amount of information, time .and
resources. Therefore, it is important to select carefully the interventions that will be
evaluated. One of the important considerations that could govern the selection of
interventions (whether they, be projects, programs or policies) for impact evaluation is
the potential of evaluation results for learning. Four questions can help guide the
decision of when to conduct an impact evaluation.

1. Is the policy or program considered to be of strategic relevance for poverty

reduction?

The decision of what to evaluate depends on what are the most critical public
actions to reduce poverty. Interventions that are expected to have the highest poverty
impacts may be evaluated to ensure that poverty reduction efforts are on the right
track and allow for any necessary corrections.

190
2. Is the intervention testing an innovative approach to poverty reduction?

Impact evaluation can help to test pioneering approaches and decide whether they should

191
be expanded and pursued at a larger scale. Hence, the innovative character of
policies or programs also provides a strong reason to evaluate. This can be built into
project design where; before committing large amounts of resources, multiple-
variations of the intervention are tested against each other.

3. Is there sufficient evidence that this type of intervention works well in a number
of different contexts?

If the answer of this question is yes, then the scarce resources may best be devoted
to helping adapt this intervention to local conditions and paying close attention to
monitoring and supervision. If however, there are significant differences in local
conditions and the target population that cast doubt on the applicability of results
from elsewhere, and then an evaluation may be worth considering. (See our
database of completed evaluations to compare results.).

4. When do we expect outcomes to show an effect?

Certain outcomes or impacts take time to materialize. In some cases this may mean
that it is better to delay the final stage of the evaluation until these will show an
effect. In other cases it may be better to choose approximate set of indicators which
are casually linked to the ultimate outcomes and are likely to show an effect
earlier. Of course, the most comprehensive strategy is to combine both of these
types of indicators.

• TYPES OF IMPACT EVALUATION

Taken into consideration the “analysis field” criterion we identify social,
Economical, environmental impact evaluation. The social impact evaluation is an
analysis for the distribution of public policy impacts, on target group welfare and
consists on “the process of analyzing, monitoring and managing the intended and
unintended social consequences, both positive and negative, of the planned
interventions policies, programs, plans)”. While the social impact assessment focuses
on the distribution of social impacts, it also addresses issues of sustainability and risks
analysis of policy alternatives. The internationalAssociation for Impact evaluation
defines the social impact as a change in the following fields:
15 people’s way of life — how they live, work and interact in their day-to-day life;
192
16 their culture shared beliefs, values, language’s or dialects;

193
 17 their community — its cohesion, stability, character, services and facilities;
18 Their political systems — the way people participation the decision-making
process that affects their life and the level of democratization.
19 Their environment and health — the health is a state of ‘complete physical,
mental, social and spiritual well being.
20 Their personal and property rights — especially when people are economically
affected or in the case when their civil liberties are violated;
21 Their aspirations and fears — the perception about their safety, their fears about
the future of community. The social impact assessment must not be understood
only as the prescriptive activity of the social impacts from an integrated impact,
but also as a methodology or an independent tool.

The economic impact evaluation involves aspects of micro and macro economic impact
regarding the selected option, such as changes regarding the compliance costs, the
implementation costs that public authorities have to make and the impact on
prospective innovation and technological development. Akey element of the impact
assessment is the appraisal of potential economical impacts of public policy
proposals. The necessity for analyzing the economic impact, determined by a policy
proposed on society is even more important since estimating the net cost of this
proposal is an issue with special significance on the study of impact. The institution
that is responsible for the proposal has to take into consideration all the costs with an
impact on public budget (not only the cost from the own budget or from the
subordinated agencies) because the public policies in general brings an impact on the
overall economy.

The environmental impact assessment consists on the analysis \c)f the implications and
levels that a policy alternative can have on the environment. The quick
developments occurring in the present century have intensified the concern of the
authorities for analyzing the impacts of public interventions on the environment and
human health. The roots of environmental impact assessment can be traced far early
back in the history of decision- making theory and according to some scholars,
“there is nothing new about the idea of incorporating information regarding the

194
environment into planning and design decision, being possible to find examples of
using analytical prediction tools in XVI century.

195
The European Commission collected these types of impact evaluation into one,
namely “the integrated impact evaluation”. At a first glance, the new model of
integrated impact assessment seems to be designed by taking into account, the
lessons from international practice and from a theoretically point of view it is more
complete and effective.

Depending on the stage of policy-making process in which the impact evaluation

is completed we can distinguish between
22 Ex-ante impact evaluation of public policies — this is an activity conducted at
the beginning of policy-making process, when .public policy experts and these involve
in public policies planning using qualitative and quantitative research methods,
trying to make predictions regarding the impact that could be exercised on the society
as a result of public policy implementation.
23 Ex-post impact evaluation — this is an activity conducted during or after the
implementation of public policies. It measures the achieved results and identifies the
real deviations from the planned objective, extra time and the Additional costs
related to resources and .other factors. The aim of ex-post evaluation is not only to
identify the mistakes, but also to make prescriptions concerning the appropriate
solutions for the future activities.

• METHODS ORAPPROACHES FOR IMPACT EVALUATION

Several approaches and methods been suggested for evaluating the impact of
policies. A few notable methods can now be discussed in detail.

Comparing Sets of Statistics: A simple method to ascertain policy impact is

through a comparative study. It is concerned with comparing a problem or a
situation with what it was like before the intervention. In addition, certain goals or
targets of a policy or a programme could be specified, and compared with what has
already happened in that particular field of policy. This method is concerned with
comparing the findings with specified or fixed standards usually expressed as a
desirable level of output.

Experimental Approach: Experimental approach attempts to quantify success in,

achieving initial policy objectives, based on some form of ‘before’ and ‘after’ study
196
of those involved in the experiment in comparison with a control group. In this
approach,

197
experiments are conducted to test the impact of a programme on a group or an
area- against what has happened to a group or an area which has not been the
target of’ intervention. An experiment may seek to study a problem before and after
the, intervention.

Cost-Benefit Analysis Method: Cost-benefit analysis approach involves the use of

economic concepts to identify and measure the costs and benefits of policies and
programmes. The cost-benefit analysis approach focuses on measuring costs against
the benefits which have taken place as a result of the policy intervention. The
quantitative outcomes of cost-benefit calculations show whether the gains exceed
the total costs or not. This approach is also useful in estimating the impact of
existing programmes and comparing them with that of the proposed programmes,
and ranking their effectiveness.

Performance Measurements Approach: Another ‘method to ascertain policy

impacts is through using performance measures. This method is applied to assess if
goals or targets have been met. Implementing agencies may be assigned the task of
assessing as to achievement of the intended goals. A performance measure may be
expressed as how efficient the use of given resources has been how much should
have been achieved, and how much was actually achieved.

Rapid Assessment Method: Ong and Humphris are associated with the Rapid
Appraisal Methodology, and they argue that the citizens should be made an integral
part of the valuation process. Under this approach, impact assessment is undertaken
by a number of professional and non-professional persons. Broader judgments and
conclusions are then arrived at through communication between team members. The
techniques associated with this method are interactional, such as, direct observation,
informal communication with key informants, and group discussions. These
judgmental approaches help evaluate success or failure of policy and programmes.

Using-Citizen’s Response: An old and popular method to assess policy impact is

through citizen’s response to provision of social and economic services. It is seen
that a large number of social programmes concerning health, education and social
security are given a wide publicity to the weaker sections of the community in
developing countries. The implementing agencies are expected to follow the

198
procedures the rules as propagated by the authorities. In case of any kind of
diversions by the administrative agencies, the

199
prospective beneficiaries may complain to the appropriate authorities or they may air
their grievances through the media. Though somewhat crude, this method is simple
for getting some kind of feedback. If the number of complaints or grievances is few
and negligible, it would indicate the positive impact or good work of the
implementing agencies. However, the method is not without its shortcomings. Not all
citizens voluntarily submit complaints against governmental programmes.

• PROBLEMS IN EVALUATING THE IMPACT OF

PUBLIC POLICY
Evaluating the impacts of programs or policies is very difficult, even under the best
of conditions. Hog wood and Gunn identify several factors that pose severe
problems for evaluating public policies or programs. These problems include:

1. Objectives: Nothing illustrates more clearly the problems of doing policy

evaluation than the way in which policy objectives shape evaluation. If the policy
objectives are unclear, or they are not specified in any ‘measurable form, then the
criteria for a policy’s success are unclear. However, vagueness in goals can often be a
consequence of differences in points of view about policy objectives. Even when
there is a clear statement of goals, problems remain. For example, how important are
goals relative to each other when more than one is specified.”

2. Defining the Criteria for Success: Even when objectives are clearly stated,
there is the question of how the success of the objective will be measured. For
example, suppose the objective is to produce an improvement in student
performance in math using microcomputers. Even this specific objective is clouded
by whether one wants to assess the student’s increase in enjoyment of math using
computers, or an improvement in understanding math itself, or an ability to apply
this improvement to other areas, or a combination of all these.

3. Side effects: Sometimes impacts from other policies or programs affect the policy
or program under evaluation. Difficulties can be presented when one tries to
identify and measure side effects and separate these side effects from the policy or
program being evaluated. Thus, there is the problem of how other factors (both
adverse and beneficial) should be brought into the evaluation, and how much these

200
factors should how weighted relative to the central objectives. For example, in
attempting to evaluate the effects of

201
the55 mile-per-hour speed limit on reducing traffic fatalities, there is the complicating factor of
seat-belt use. To what extent is the reduced speed a factor relative to the use of seat
belts?

4. Data problems: Quite often the information necessary to assess the impact of a
policy or program may not be available or may be available in an unsuitable form.
For example, if one were interested in evaluating the impacts of. President Reagan’s
“new federalism” on state environmental protection, he or she would want to have
data on the extent to which the individual states replaced the federal budget cuts with
their own-source funds. However, not all 50 states (or even very few of them) have
kept careful records of the extent to which they replaced federal funds with state
funds for environmental protection, nor how much the states provided in this
category of expenditure that was unique to the state as opposed to local
governments.

5. Methodological Problems: It is also common for a single problem or single

group of the population, to be the target of several programs with the same of
related objectives. For example, several policies are directed to the problem of
poverty. In such cases, assessing the impact of a single program is difficult. For
example, if crime rates go down, is this due to better policing, better education,
welfare assistance, or employment opportunities?

Big problems tend to have a lot of programs directed toward them, which makes it difficult to
assess which, if any, are producing an effect.

6. Political problems: Evaluation is very threatening to some people. The success

or failure of a policy or program to which politicians or bureaucrats have committed
their personal reputations and careers, and from which clientele groups are receiving
benefits, is being evaluated. Evaluation may be seen as a threat to the continuation
of a policy or program in which a number of people have an important stake. These
considerations will obviously affect both how evaluation results are utilized and the
ease with which the evaluation can be conducted, as the cooperation of public
officials and clients of often required in the evaluation.

202
7. Cost: It is not uncommon for a program’s evaluation to cost as much as one
percent of the total program cost. This is especially the case when more
sophisticated methods are

203
used in evaluation studies, such as experimental designs. Such costs are a diversion from the
delivery of the policy or program. There are just a few of the difficulties posed in
the evaluation of public policies and programs. Simply because policy analysts have
these problems in the process ofevaluation, theyshould not be undulydiscouraged
fromconducting evaluation activities. Rather than seeing these difficulties as
insurmountable obstacles, they should see them as challenges for designing effective
evaluations. In the following case study, we discuss an example of evaluation
research-the impact of school spending on student performance

An impact evaluation assesses changes in the well-being of individuals; households,

communities or firms that can be attributed to a particular project, program or policy.
The central impact evaluation question is what would have happened to those
receiving the intervention if they had, not ill fact received the program. Impact
evaluation is aimed at providing feedback to help improve the design of programs
and policies. In addition to providing for improved accountability, impact evaluations
are a tool for dynamic learning, allowing policy makers to improve ongoing programs
and ultimately better allocate funds across programs. There are various types of
impact evaluations viz. social, economic and environmental. Cost benefit analysis,
performance measurements approaches are being, used to evaluate the impact of
public policies. There are also certainly problems or difficulties encountered in the
impact evaluation process. Rather they seeing these difficulties as insurmountable
obstacles, they should see them as challenges for designing effective evaluations.

• POLICY COMPLIANCE AND NON-COMPLIANCE

All public policies are intended to influence or control human behaviour in some way, to
induce people to act in accordance with governmentally prescribed rules or goals,
whether reference to policy or such diverse matters patent or copyrights, open
housing interest rates, night time burglary, agricultural production, or Military
Recruitment. If compliance with policy is not achieved, if people continue to act in
undesired ways, if they do not take desired action, or if they cease doing what is
desired, to that extent policy become ineffective or, at the extreme, a nullity Foreign
policy also depend for its effectiveness on compliance by the affected foreign
countries and their officials. To make consideration of this problem

204
more manageable, we will focus primarily, but not exclusively, on compliance with
domestic economic policies.

• Causes of Compliance
24 Psychological makeup: substantial respect for authority exists in our society.
Respect for, & deference to, authority is built in our psychological make up by
the process of socialization. Most of us are taught from birth to respect the
authority of parents, knowledge, status, & govt. officials. Consequently we grow
up generally believing it to be morally right and proper to obey the law. So this
becomes the part of habit of the individual to obey the authority of law.
25 Reasoned & conscious acceptance: compliance with policy may also be based
on some form of reasoned, conscious acceptance. Even some whose immediate
self interest conflict with a particular policy may be convinced that it is reasonable
and just. Most people undoubtedly would rather not pay taxes but when people
believe that tax laws are reasonable and just, & necessary to provide
governmental services, such belief in all likely hood contribute to compliance
with policy.
26 Legitimate character of governmental policies: another possible cause of
compliance is the belief that a a governmental policy should be obeyed because
it is legitimate, in the sense that it is constitutional, or was made by the officials
with proper authority to act, or that correct procedure was followed.
27 Self interest: self interest is the important factor of compliance. Individuals and
groups may directly benefit from acceptance of policy norms and standards.
Thus farmers have complied with production limitations in the form ofacreage
allotments and marketing quotas in order to qualify for price support and benefit
payments.
28 Threat of punishment: the possibility of punishment in the form of fines,
imprisonment and other sanctions also works to secure compliance. In many
instances, sanctions are effective more because people desire to avoid being
stigmatized as law breaker then because they fear the penalties involved.

205
29 Acceptance with Time: Acceptance of most policies seems to increase with
the length of time they are in effect. As time passes and it always does a once
controversial

206
 policy becomes more familiar a part of accepted state of things, one of the conditions of
doing business.

• Causes of Non-Compliance
It will be readily apparent even to the most casual observer that all person are affected by
public policies comply with them. Statistical information on reported violations is
readily obtainable as in the Federal Bureau of Investigation’s Uniform Crime Reports.
In addition, a lot of violation goes undetected or unreported. Non compliance may
result when laws conflict too sharply with the prevailing values, mores, and belief of
people generally or of particular groups. The extensive violation of national
prohibition and war time price and rationing control can be attributed in considerable
measure to this cause.
30 Conflict between laws and morality: it is not very useful, however, to
ascribe noncompliance to a broad conflict between law and morality. Failure to
compliance occurs when a particular law comes in conflict with particular
values or beliefs in a particular time and situation. For example the beef ban
was not complied by the particular community because it came in conflict with
its value system.
31 Selective disobedience of law: some laws are felt to be less binding than others
on the individual. Those who strongly obey what are ordinarily designated as
criminal laws sometimes have more relaxed or permissive attitude towards
economic legislation and regulation or laws regulating the behaviour of the public
officials. Many people apparently believe that laws relating to banking
operations, trade practices taxation, environmental pollution control are not as
mandatory for individuals as laws prohibiting robbery, burglary, and
embezzlement.
32 Group membership: one’s associate or group memberships may contribute to
non compliance. Association with person who hold ideas disrespectful of law
and government, justify or rationalize law violation, or who violate the law may
cause the individual to acquire deviant norms and values that dispose him to
noncompliance.

207
33 Desire to make a fast buck: the desire to make a fast buck or something akin
thereto, is often stated as a cause of noncompliance. This would certainly seem
the case in many instances of fraud and misrepresentation, scandals and scams.

208
34 Lack of clarity: noncompliance may also result from such factors as ambiguity
in the law, lack of clarity, or conflicting policy standards. Income tax violation
often stem from the ambiguity or complexity of provisions of Internal Revenue
Code, which some one described as a “sustained essay in obscurity”. Violation
may also result from difficulties in complying with the law, even when its meaning
is understood. For example insufficient time is allowed for filling complicated forms
or for making required changes in the existing patterns of action, such as in the
installation of pollution control devices.
35 Ignorance of law: sheer ignorance of law/policy or rules regulating conduct
cannot be discounted as a cause of noncompliance. While ignorance of law may
be no excuse, it often does account for violations.

In sum, non compliance may stem from structural defects in the law and its
administration, and from ignorance and lack of understanding of law, as well as
from behaviour that is more consciously or deliberately deviant. The burden of
securing compliance with public policies rests primarily with the administrative
agencies.

• EXERCISE
1. Discuss various types of impact evaluation.

2. Define impact evaluation and discuss different methods for impact evaluation.

3. Write a note on compliance and non-compliance.

209
 M.A. Political Science, Semester IV, Course No. 405, Public Policy Making
&Analysis Unit – IV: Policy Implementation and Evaluation

4.2 POLICY EVALUATION: MONITORING TECHNIQUES, COST-

BENEFIT ANALYSIS AND PROBLEMS

4.2.0 Y. Pardhasaradhi
STRUCTURE
4.3.0 Objectives
4.3.1 Introduction
4.3.2 Types of Public Policy Evaluation
4.3.2.1 Summative Evaluation
4.3.2.2 Formative Evaluation
4.3.2.3 Theory-based Evaluation
4.3.2.4 Goals-based Evaluation
4.3.2.5 Experimental and Qasi-Experimental Evaluation
4.3.2.6 Qualitative Evaluation
4.3.3 Economic Appraisal and Evaluation
4.3.4 How Does Policy Evaluation Relate to Policy Management?
4.3.5 Monitoring
4.3.5.1 Four Major Functions of the Monitoring
4.3.5.2 Monitoring Techniques
4.3.6 Cost Benefit Analysis
4.3.6.1 Cost Benefit Analysis: Evaluation
4.3.7 Problems of Policy Evaluation
4.3.8 Let us Sum Up
4.3.9 Exercise

210
4.3.0 OBJECTIVES

In this lesson you will understand importance of evaluating a particular policy from
its intended initiation. While explaining issues involved evaluation, the lesson
makes you understand the techniques of monitoring, particularly cost benefit
analysis. After going through this lesson, you will be able to know:
36 policy evaluation and multiple ways in which policy can be evaluated;
37 how monitoring constitute important element of policy evaluation and the
techniques involved in it;
38 the importance of Cost Benefit Analysis in the policy evaluation; and
39 problems involved in policy evaluation.

4.3.1 INTRODUCTION

The difficulties of public policymaking in the complex milieu of modern society can
hardly be overstated. The consequent need for coherent analytical frameworks or
paradigms to guide policymakers through the myriad of factors surrounding particular
policy issues will probably never be fully satisfied. Accordingly, new perspectives on
social and economic policymaking require ongoing scrutiny by public policy
analysts in order to determine whether theyassist in reducing the manifold
complexities of policy formulation. The policy makers follow different types in policy
evaluation.

4.3.2 TYPES OFPUBLIC POLICY EVALUATION

Asimple assumption generally acceptable is that most citizens think and behave
rationally. Hence, they would like to see that the public moneys are spent by their
democratically elected government effectively and efficiently in whatever policies the
government adopts and finances. Put it differently, citizens would like to have an
account of these policy expenditures. If a policy proves inefficient, the citizens
would want it altered so it can become efficient; or maybe they would rather see
those funds shifted to other policies; or they would even like some of those funds to
be returned indirectly to them through lower taxes.

211
To have this accountability substantiated, the policymaker must conduct some evaluation,

212
s/ he must research systematically, find out what has happened and then pass
some judgement on the policy in question.

Assuming as stated earlier that the policy-maker thinks and behavesrationally, these
decisions and judgements are optimised only if the policy-maker possesses valid,
reliable and comprehensive information on how the policy has fared; that is whether
it did well or not, what were its weaknesses, what were its strengths, and so forth.
How is this reliable information produced? It is produced through sound methods
with which the policy- maker can gather data and methods with which s/he can
analyse the data gathered.

Evaluations however are not conducted for the sake of accountability only. For
example, the organisations and their public officials that are involved in the planning and
implementation of policies, whether acting as Principals or Agents, would also want
some feedback which would assist them in improving their ongoing policy operations
or the planned operations for the future.

Different countries use different terms to describe evaluations. This includes the
commonly used terms of summative and formative evaluations.

4.3.2.1 Summative Evaluation

Which is sometimes referred to as impact evaluation, asks questions such as: What
impact, if any, does a policy, programme or some other types of government
intervention have in terms of specific outcomes for different groups of people? It
seeks to provide estimates of the effects of a policy either in terms of what was
expected of it at the outset, or compared with some other intervention or with doing
nothing at all (i.e. the counterfactually)

4.3.2.2 Formative evaluation:

Which is sometimes referred to as process evaluation, asks how, why, and under
what conditions does a policy intervention (or a programme, or a project) work, or
fail to work? These questions are important in determining the effective
development (i.e. formation), implementation and delivery of policies, programmes
or projects. Formative evaluation typically seeks information on the contextual factors,
mechanism and processes underlying a policy’s success or failure. This often involves
213
addressing questions such as for whom a policy has worked or not worked, and
why.

214
This distinction between summative and formative evaluations is not always as rigid
as the above characterization might suggest. Proponents of the Theories of Change
approach to evaluation would argue that determine whether or not a policy has
worked, or has been effective, necessarily involves asking questions about how it has
worked, for whom, why, and under what conditions it has worked or not worked.
Nonetheless, the contrast between evaluating whether a policy intervention has been
effective (summative evaluation), and why it has done so (formative evaluation), is
one that is conventionally made in the policy evaluation literature.

Based on the different objects purpose and questions, a set of standard types of
evaluation is used across governments. This set of types of evaluation is based on
the base model which links inputs to activities, outputs, outcomes and impacts,
which is also used in the framework for Managing programme Performance
Information. This will help to develop a common language, and establish standard
procedures. Following table summarizes three types and their uses.

Table: Summary of types of evaluation across government

215
Types of Covers Timing
evaluation

Diagnostic This is preparatory research (often called ex-ante evaluation) to ascertain At key stages
evaluation the current situation prior to an intervention and to inform intervention prior to design or
design. It identified what is already known about the issues at hand, the planning.
problems and opportunities to be addressed. Causes and consequence,
including those that the intervention is unlikely to deliver and the likely
effectiveness of different policy options. This enables the drawing up of
the theory of change before the intervention is designed.

Design Used to analyse the theory of change, inner logic and consistency of the After an
evaluation programme, either before a programme starts or during implementation to intervention has
see whether the theory of change appears to be working. This is quick to been designed, in
do and uses only secondary information and should be used for all new first year and
programmes. It also assesses the quality of the indicators and the possibly later
assumptions.

Implementati Aims to evaluate whether an intervention’s operational mechanisms Once or several

on evaluation support achievement of the objectives or not and understand why. Looks times during the
at activities, output, and outcomes, use of resources and the causal links. intervention.
It builds on existing monitoring systems and is applied during programme
operation to improve the efficiency and efficacy of operational processes.
It also assesses the quality of the indicators and assumptions. This can be
rapid, primarily using secondary data or in- depth with extensive field
work.
Impact Seeks to measure changes in outcomes (and the well- being of the target Designed early
evaluation population) that are attributable to a specific intervention. Its purpose is to on baseline,
inform high-level officials on the extent to which an intervention should implemented
be continued or not and if there are any potential modifications needed. early, impact
This kind of evaluation is implemented on a case- by- case basis. checked at key
stages e.g. 3/5
years

Economic Economic evaluation considers whether the coasts of a policy or At any stage
evaluation programme have been overweighed by the benefits. Types of economic
evaluation include:
Cost-effectiveness analysis, which values the coasts of implementing and
delivering the policy and relates this amount to the total quantity of
outcome generated to produce a “cost per unit of outcome” (e.g. cost per
additional individual laced in employment); and
Cost benefit analysis (CBA), which goes further in placing a monetary
value on the changes in outcomes as well (e.g. the value of placing an
additional individual in employment)
Evaluation Synthesizing the results of a range of evaluations to generalize finding After a number of
synthesis across government e.g. a function such as supply chain management, a evaluations are
sector, or a cross-cutting issue such as capacity. DPME will undertake completed.
evaluation synthesis based on the evaluations in the national evaluation
plan and do an annual report on evaluation.

216
4.3.2.3 Theory-Based Evaluation Approaches

Theory-Based approaches to evaluation, which include the Theories of Change

approach mentioned above, as well as programme theory evolution (Rogers et al,
2000) and some aspects of Realistic Evaluation (Pawson and Tilley, 1997), focus on
unpacking the theoretical or logical sequence by which a policy intervention is
expected to bring about desired effects. Theory-Based approaches attempt to identify
the mechanisms by which policies and or programmes might produce their effects.
For instance, the common underlying theory of the juvenile awareness programme
for preventing juvenile delinquency (such as the ‘Scared Straight) programmes in the
United States, (Petrosino, Turpic-Petrossino, and Buehler, 2002) suggest the

Visit to a Prison First – hand Exposure to Prison

Experience of Life and Prisoners
Prison Life as negative
following sequential steps:
An alternative possible sequence of outcomes, which can be tested empirically, might
be as follows

Reduces
Crime and
Offending

Visit to a First – hand Experience

Exposure to Prison
Prison by of Prison Life
positive Role models
Juveniles

Stimulates or Increases Crime

Attracts juveniles and offending
Towards

Failure to be clear about the causal sequence by which a policy is expected to

work can result in well intentioned policies being misplaced, and outcomes that are

217
contrary to those that were anticipated. Theory-based evaluation provides a number
ofways of carrying

218
out an analysis of the logical or theoretical consequences of a policy, and can
increase the likelihood of the desired outcome being achieved.

4.3.2.4 Goals-Based Evaluation

This is one of the most frequently asked questions in policy evaluation, and is
sometimes referred to as Goal-Based evaluation. In the American evaluation literature
it is sometimes referred to as ‘legislative monitoring’, because it monitors whether the
outcomes that were expected from some government policy initiative have been
achieved. In the United Kingdom, the achievement of targets that have been set by
Public Service Agreements and Service DeliveryAgreements are evaluated using
Goal-Based methods of evaluation.

An example in the UK context might be whether or not the goals and targets of
the National Literary Strategy (i.e. increasing the reading, writing and comprehension
abilities of children and adults) have been achieved. Another example might be
whether the goals of the hospital waiting lists and or the time they had to wait for
treatment have been achieved. Such outcomes may, or may not, be made explicit in
policy statements and documents.

Goals Based evaluation make no assumptions about whether or not the chosen
goals or targets are valid or appropriate measures of effectiveness. It may indeed by
the case that waiting no more than four hours for hospital treatment is less valid to
patients and their careers than waiting for two hours or less. Or it may waiting times
for treatment are less valid than making sure that the most effective and evidence-
based evaluations simply measure whether some goals or targets set by policy
makers have been achieved.

Even when goals of a policy, programme or project have been achieved, however,
this does not necessarilymean that the policyin question has been responsible for this
outcome. Other factors, including other policy initiatives, may have been
responsible. In order to know whether the policy in questions has been responsible
for an anticipated outcome, some happened anyway, or because of other
interventions. Randomized control trial methods are generally considered to be the
most appropriate way of determining the counterfactual of a policy, programme or

219
project, though carefully controlled matched comparisons studies and some forms of
statistical modelling also provide estimates of the counterfactual.

220
 Policy makers and evaluators are often interested in the unintended consequences
or outcomes of a policy, programme or project. These unintended outcomes may be
beneficial or harmful. Goals-free evaluation does this focusing on the actual effects
or outcomes of some policy, programme or project, without necessarily knowing what
the intended goals might be. This type of policy evaluation is more commonly
undertaken by evaluators who are independent of government and who are more
interested in the range of consequences of a policy, programme or project than in the
anticipated outcomes alone. Goals-free policy evaluation however should be of
interest to government social researchers and policy analysts because of the
importance of establishing the balance between the positive and negative
consequences of policies. Such balanced evaluation is important in order to establish
the coat-benefit and coat-utility of a policy or programme interventions.

4.3.2.5 Experimental and Quasi-Experimental Evaluation

Experimental and quasi-experimentalresearch method, provide valid and reliable

evidence about the relative effectiveness of a policy intervention compared with
other policy interventions, or doing nothing at all (sometimes called the
counterfactual). They provide appropriate evidence about questions such as whether
a personal adviser service is more or less effective in terms of advancing low paid
people in the labour market than for example providing skill training or doing
nothing at all.

The purest form of experimental method is the randomized controlled trial.

Randomized control trials deal with the problem of other possible factors
influencing an outcome by exposing an experimental group of people, and a non-
experimental group of people to exactly the same factors except the policy,
programme or project under investigation. The allocation of people to the
experimental policy intervention, or to the control situation, is done not to guarantee
that the experimental and control groups will be identical, but it reduces the
influence of extraneous factors by ensuring that the only differences between the two
groups will be those that arise by chance.

221
Randomization may be by individuals or by units, clusters or whole areas. Some
welfare- to-work initiatives have allocated units such as schools hospitals, housing
estates or entire neighbourhoods, to experimental or control groups.

Quasi-experimental methods refer to those research designs that compare the

outcomes

222
of experimental and control groups by methods other than randomization. These
include:
40 Controlled before and after involving pre-test and post-test comparisons using a
single group of people
41 Controlled before and after designs in which pre-test and post-test comparisons
are made between two or more groups of people.
42 Interrupted time series studies based on repeated observation over time of valid
and reliable standardized measures of outcome.
43 Various types of matching designs using matched comparisons of individuals or
units before and after an intervention.
44 Regression discontinuity designs.

4.3.2.6 Qualitative Evaluation

Qualitative evaluation is designed to “permit the evaluator to study selected issues in

depth and detail”. Such depth and detail is usually required to determine the
appropriate questions to ask in an evaluation, and to identify the situational and
contextual conditions under which a policy, programme or project works are fails to
work.

Qualitative methods of evaluation are particularly important for formative

evaluation which, as Patton again suggests, “Is limited entirely to a focus on a
specific context”, Patton goes on to argue that: “Formative evaluation services the
purpose of improving a specific program, policy, group of staff (in a personnel
evaluation), or product. Formative evaluations aim at ‘forming’ the thing being
studied….There is no attempt in formative evaluation to generalize findings beyond
the setting in which open is working. The purpose of the research is to improve
effectiveness within that setting”.

Qualitative evaluationuses a range ofmethods including in-depth interview, case

studies, consultative methods, focus groups, ethnography, observational and
participant observational studies, and conversation and discourse analysis.

223
4.3.3 ECONOMICAPPRAISALAND EVALUATION

Politics, programme and projects involve the allocation of scare and finite resources
to competing demands and interests. Consequently, it is necessary to understand
economic

224
appraisal at the outset (i.e. ex ante) of different policy options and likely outcomes
(both positive and negative) that will be achieved by them, and of the costs involved
in achieving these outcomes. It is also necessary to undertake an economic
evaluation after (i.e. post hoc) a chosen policy, programme and project has been
running for some time in order to determine whether or not the anticipated outcomes
(or other outcomes) have been achieved.

There are different types of economic appraisal and evaluation. The simplest
type is cost appraisaland evaluation, which simplycompares the costs ofdifferent
initiatives without considering the outcomes to be achieved or that have been
achieved. The limitations of such appraisals and evaluations are fairly obvious they
tell us very little about the relative effectiveness or benefits of different interventions
and are of little value alone in policy evaluation.

Other types of economic appraisal and evaluation, which are more analytically
powerful and useful to policy making, include coast-effectiveness and cost- benefit
analyses. The former compares the differential costs involves a consideration of
alternative uses of a given resources. Cost benefit analysis involves a consideration
of alternative uses of a given resources, or the opportunitycost ofdoing something
compared with doing something else. Another type of economic appraisal is cost
utility analysis, which evaluates the utility of different outcomes for different users to
consumers of a policy or service. Cost utility analysis typically involves subjective
appraisals and evaluations ofoutcomes using qualitative and quantitative data. Economic
appraisal and evaluation uses a variety of tools to estimate the costs and benefits of
policy initiatives over time, such as the discount rate for adjusting the value of
outcomes that will occur in the future.

HOW DOES POLICY EVALUATION RELATE TO POLICY

4.3.4
MANAGEMENT?
Policy evaluation and analysis requires a structured and organized approach to
defining an answerable question, summoning appropriate and relevant evidence,
critically appraising and analysing that evidence, identifying the risks and opportunities
of a policy, programme or project, and determining the likely effects (positive and
negative) of a project at hand. Project and programme management has emerged in
225
recent years as a structured and organized way of planning, implementing and
concluding projects and programmes. The congruity of interest between policy
evaluation and project management is clear.

226
4.3.5 MONITORING

Policy monitoring and evaluation form part of integral stages in public policy cycle.
Policy monitoring comprises a range of activities describing and analyzing the
development and implementation of policies, identifying potential gaps, outlining areas
for improvement, and holding policy implementers accountable for their activities.
Monitoring is defined as an analytic procedure that produces information on cause and
consequences of public policies. It is the process of observing policy implementation
progress, and resource utilization, and anticipating deviations from expected policy
outcomes.
Monitoring in essence plays a methodological role in policy analysis. Information
about a policy is transformed from mere information to policy outcomes. The resultant
is production of problem situations this is what is picked when structuring policy
problems. Monitoring and control thus forms the heart of policy management.
4.3.5.1 Four Major Functions of the Monitoring

There are four major functions for monitoring. These are:

Compliance: Monitoring verifies whether the program administrators, staff and all
stakeholders are in compliance with the standards and procedures put in place by
the legislatures, regulatory bodies and other professional bodies.
Auditing: Monitoring helps to determine whether the target groups and beneficiaries
for e.g, families, states, municipalities or regions, have been reached by the allocated
resources and services. Here the essence is to check out if; for example county
resources have reached to the grassroots.
Accounting: Information is produced and enhances accounting for social and
economic changes that follow the implementation of broad sets of public policies and
programs over time. An example can be analyzing changes in qualityof life that
requires looking at indicators such as average education, percentage of population
below poverty line and type of housing lived in.
Explanation: Monitoring helps to determine why the outcomes of public policies
and programs differ. For example social experiments in criminal justice, education

227
and social welfare help us to find out what policies and programs work best, how
they work and why.

228
4.3.5.2 Monitoring Techniques

Mainly there are two techniques of policy evaluation that is PERT and CPM which
stands for Programme Evaluation Review Technique and Critical Path Management
respectively. They were developed in the 1950s to control large defence projects, and
have been used routinely since then. As the Critical Path Method (CPM) helps you
to plan all tasks that must be completed as part of a project.

They act as the basis both for preparation of a schedule, and of resource planning.
During management of a project, they allow you to monitor achievement of project
goals. They help you to see where remedial action needs to be taken to get a project
back on course.
Within a project it is likely that you will display your final project plan as a Gantt
Chart (using Microsoft Project or other software for projects of medium complexity
or an excel spreadsheet for projects of low complexity).The benefit of using CPA
within the planning process is to help you develop and test your plan to ensure that it
is robust. Critical Path Analysis formally identifies tasks which must be completed on
time for the whole project to be completed on time. It also identifies which tasks can
be delayed if resource needs to be reallocated to catch up on missed or overrunning
tasks. The disadvantage of CPA, if you use it as the technique by which your project
plans are communicated and managed against, is that the relation of tasks to time is
not as immediately obvious as with Gantt Charts. This can make them more difficult
to understand.

Afurther benefit of Critical PathAnalysis is that it helps you to identify the minimum
length of time needed to complete a project. Where you need to run an accelerated
project, it helps you to identify which project steps you should accelerate to
229
complete the project within the available time. Graphically it can be represented in
the following way:

230
A network of events and activities is presented pictorially in CPM with the help of
several circles and arrows. Each circle represents an event and each arrow represents
an activity. An event involves a specific point of time at which a part of programme
is completed. An event signifies the beginning of one activity and the end of another
activity. Events are assigned serial numbers for expressing their sequence and separate
identification, an activity, on the other hand, implies time consuming efforts or
actions required for achieving an event. The flow or sequential activity is indicated
by the arrow head and such flow calls for estimating time in number of days or
weeks in respect of each activity between any two events. That is, there must be an
elapsed time for each activity between events.

After a network of events and activities have been shown, the critical path is
computed with reference to those strategic events and activities which take the longest
time to complete the whole project and which thereby leave the least slack time. Slack
time is the difference between the target time and project completion time. In other
words, the critical path involves the minimum expected time in which the project as
a whole can be completed. Although attention is focused on one critical path in
network system, there exist several critical paths in order of importance. The above
figure illustrates a simple network with its critical path.

PERT stands for Program Evaluation and Review Technique. APERT chart is a
graphic representation of a project schedule, showing the sequence of tasks, which
tasks can be performed simultaneously, and the critical path of tasks that must be
completed on time in order for the project to meet its completion deadline. The chart
can be constructed with a variety of attributes, such as earliest and latest start dates
for each task, earliest and latest finish dates for each task, and slack time between
tasks. A PERT chart can document an entire project or a key phase of a project.
The chart allows a team to avoid unrealistic timetables and schedule expectations, to
help identify and shorten tasks that are bottlenecks, and to focus attention on most
critical tasks.

The following advantages are derived from the PERT:

231
1. It compels managers to plan their projects critically and analyze all factors affecting
the progress of the plan. The process of the network analysis requires that the
project planning be conducted on considerable detail from the start to the finish.

232
 2. It provides the management a tool for forecasting the impact of schedule changes
and be prepared to correct such situations. The likely trouble spots are located
early enough so as to apply some preventive measures or corrective actions.

3. Alot of data can be presented in a highly ordered fashion. The task relationships
are graphically represented for easier evaluation and individuals in different
locations can easily determine their role in the total task requirements.

4. The PERT time is based upon estimate and hence is the most objective time in the
light of uncertainties and results in greater degree of accuracy in time
forecasting.

5. It results in improved communication; the network provides a common ground

for various parties such as designers, contractors, project managers etc. and they
must all understand each other’s role and contributions.

The network willhighlight areas that require attention of higher priority so that concentration
can be applied to the key jobs without ignoring the lower priority tasks. This gives
the management an opportunity to shift attention to any critical task so that the entire
project is completed in time.

4.3.6 COST-BENEFITANALYSIS

Cost–benefit analysis is often used bygovernments and other organizations, such as private
sector businesses, to appraise the desirability of a given policy. It is an analysis of
the expected balance of benefits and costs, including an account of foregone
alternatives and the status quo. CBA helps predict whether the benefits of a policy
outweigh its costs, and by how much relative to other alternatives (i.e. one can rank
alternate policies in terms of the cost–benefit ratio). Generally, accurate cost–benefit
analysis identifies choices that increase welfare from a utilitarian perspective.
Assuming an accurate CBA, changing the status quo by implementing the alternative
with the lowest cost–benefit ratio can improve Pareto efficiency. An analyst using
CBA should recognize that perfect appraisal of all present and future costs and
benefits is difficult, and while CBA can offer a well-educated estimate of the best
alternative, perfection in terms of economic efficiencyand social welfare are not
guaranteed.

233
4.3.6.1 CBA: Evaluation

Cost Benefit Analysis attempts to measure the positive or negative consequences of

a project, which may include:
45 Effects on users or participants
46 Effects on non-users or non-participants
47 Externality effects
48 Option value or other social benefits.

A similar breakdown is employed in environmental analysis of total economic value.

Both costs and benefits can be diverse. Financial costs tend to be most thoroughly
represented in cost-benefit analyses due to relatively abundant market data. The net
benefits of a project may incorporate cost savings or public willingness to pay
compensation (implying the public has no legal right to the benefits of the policy) or
willingness to accept compensation (implying the public has a right to the benefits of the
policy) for the welfare change resulting from the policy. The guiding principle of
evaluating benefits is to list all (categories of) parties affected byan intervention and
add the (positive or negative) value, usually monetary, that they ascribe to its effect on
their welfare.

The actual compensation an individual would require to have their welfare unchanged
by a policy is inexact at best. Surveys (stated preference techniques) or market behavior
(revealed preference techniques) are often used to estimate the compensation
associated with a policy; however, survey respondents often have strong incentives
to misreport their true preferences and market behavior does not provide any
information about important non- market welfare impacts.

One controversy is valuing a human life, e.g. when assessing road safety measures or
life- saving medicines. However, this can sometimes be avoided by using the related
technique of cost-utility analysis, in which benefits are expressed in non-monetary
units such as quality-adjusted life years. For example, road safety can be measured in
terms of cost per life saved, without formallyplacing a financial value on the life.
However, such non-monetary metrics have limited usefulness for evaluating policies

234
with substantially different outcomes. Additionally, many other benefits may accrue
from the policy, and metrics such as ‘cost

235
per life saved’ may lead to a substantially different ranking of alternatives than
traditional cost–benefit analysis.

Another controversy is valuing the environment, which in the 21st century is

typically assessed by valuing ecosystem services to humans, such as air and water
quality and pollution. Monetary values may also be assigned to other intangible effects
such as business reputation, market penetration, or long-term enterprise strategy
alignment.

The value of a cost–benefit analysis depends on the accuracy of the individual cost
and benefit estimates. Comparative studies indicate that such estimates are often
flawed, preventing improvements in efficiency. Causes of these inaccuracies include: a)
overreliance on data from past policies/programmes (often differing markedly in
function or size and the skill levels of the team members); use of subjective
impressions by assessment team members; c) inappropriate use of heuristics to derive
money cost of the intangible elements;
d) confirmation bias among project supporters (looking for reasons to proceed).

To conclude, whenever people decide whether the advantages of a particular action

are likely to outweigh its drawbacks, they engage in a form of cost-benefit analysis.
In the public arena, formal CBA is a sometimes controversial technique for
thoroughly and consistently evaluating the pros and cons associated with
prospective policy changes. Specifically, it is an attempt to identify and express in
material terms all of the effects of proposed government policies or projects. While
not intended to be the only basis for decision making, CBA can be a valuable aid to
policymakers.

4.3.7 PROBLEMS OF POLICY EVALUATION

Evaluating a public programme is an important exercise in the policy process.

However the evaluation of a policy is confounded by many factors. Some of these
are as follows:

Ambiguity and vagueness in policy goals: ambiguous and diffused policy goals
make the policy evaluation task more difficult. This is because if the goals of the
policy are diffused it becomes difficult to evaluate the effects of the policy. If an

236
evaluator could not understand the goals of a particular policy it becomes difficult
to determine whether a particular target is achieved or not. In fact the unclear policy
goals render the entire policy process out of gear.

237
Non availability of information: the second problem which is encountered by the
evaluator in the evaluation of policy is the non-availability of information about the
public policy. Generally, there is a lack of required information which causes faulty
evaluation of policies.

Scattered policy impacts: generally the impact of the public policy results on both
the targeted and non- targeted groups of the society are scattered and thus it becomes
difficult to measure their impact on the population. For example, the social welfare
policy not only affects the disadvantaged section of the society but also which
occupy the upper strata in the socio-economic structure.

Problem of time and finance: lack of a time and finance also create a hurdle in the
policy evaluation especially if the evaluator is a research scholar because they have
to complete the policy evaluation in a stipulated time frame. Not only time, lack of
finance also create equal problem in the policy evaluation process.

Lack of support from the masses: generally it is found that the target population
do not cooperate with the evaluator in a required manner which makes it difficult
for him to evaluate the policy in a clear and efficient way.
The problems discussed above that there are several problems faced by the
evaluators in the countries, in general, and in the developing countries like India in
particular. Policy evaluation involves specifying the goals of the programme,
measuring the degree to which these goals have been achieved, and suggesting the
reasons behind non-achievement of targets so the required improvements can be
made in the next incremental steps.

4.3.8 LET US SUM UP

Policy evaluation is a family of research methods that are used to systematically

investigate the effectiveness of policies, programmes, projects and other types of
social intervention, with the aim of achieving improvement in the social, economic
and everyday conditions of people’s life. Different methods of policy evaluation are
used to answer different questions.

238
 4.3.9 EXERCISE

1. Describe various types of Evaluation

2. Discuss the theory based approaches of Public Policy.

3. Comment on Quantitative and Economic Appraisal evaluation methods.

4. Write a note on Monitoring and main techniques involved in it.

5. Briefly state the advantages of Cost Benefit Analysis.

6. Critically analyse problems associated with policy evaluation

239