Champions of Change in Aspirational Districts: Operational Manual of The Real-Time Outcome Monitoring System
Champions of Change in Aspirational Districts: Operational Manual of The Real-Time Outcome Monitoring System
in Aspirational Districts
Operational Manual of the
Real-time Outcome Monitoring System
A Overview
B About
C Data Entry
D Dashboard
E Reports
F Indicators
Champions of Change – Operational manual of the Real-time Outcome Monitoring System A-1
A- Overview
One of the proven strategies to transform rapidly is to institute a robust monitoring mechanism.
If progress in any sector is measured on a real-time basis, it is a sure recipe of success.
The choice of ideal set of Key Performance Indicators (KPIs) that captures efforts, commitments
and progress made by all stakeholders in transforming district can be district specific, given wide
variation in opportunities and challenges faced by a district. However, there are sectors which
are important to all and if key indicators in these sectors are monitored, they present a holistic
picture of development. These sectors are: Health and Nutrition; Education; Basic Infrastructure;
Financial Inclusion and Skill Development, Agriculture and Water Resources including Irrigation.
The KPIs are designed keeping in mind a Strategy map for each theme right from the goals and
objectives under the theme to the input/activities involved in meeting these goals.
Once the data repository for the key indicators pertaining to each sector in evaluating the Out-
comes and the underlying input/activity for each outcome is created, this becomes a strong
foundation for laying down transformational strategy in achieving the aspirational goals set for
each district.
Government of India (GoI) has formulated a programme to monitor and improve the perfor-
mance of aspirational districts in the country across the above selected key areas. NITI Aayog
is delegated with the responsibility to anchor the programme in consultation with the Central
Ministries concerned and take all measures which improves the efficiency and impact of the
Scheme, in partnership with the State Governments.
It is envisioned that NITI Aayog in consultation with concerned Ministries identify 25 districts for
providing most focused attention so as to improve their performance on various socio-economic
key indicators and thus achieve quick turnaround resulting in effective transformation of these
districts. The Central Ministries have also identified another 43 districts. In addition, Ministry of
Home Affairs has identified 33 Left-Wing Extremism (LWE) districts for similar focused attention.
In all, the envisaged dashboard would cover 101 districts across the country.
The Andhra Pradesh State Development Planning Society (APSDPS), under Planning Depart-
ment of Government of Andhra Pradesh, has established a similar digital monitoring system
named Real Time Outcome Monitoring System (ROMS) to monitor progress and performance
of key indicators from its Outcome Budget, and Vision 2029 and SDG 2020 targets. The digital
platform has been designed, developed and deployed for GoAP by APSDPS. On the basis of
effectiveness of AP-ROMS, NITI Aayog chose to develop a similar digital system for monitoring
its program / schemes for development of the 101 aspirational districts in the country.
Champions of Change – Operational manual of the Real-time Outcome Monitoring System A-2
Indicator Analysis
Indicator Metadata
Item Description
Indicator name Brief name of the indicator
Indicator definition Description of the indicator for clarity
Unit Unit of measurement of the indicator
Category Whether the indicator belongs to NITI Aayog, one of the Ministries, or LWE districts
Periodicity Frequency in which the indicator data is to be reported
Baseline value Value of the indicator at the beginning of monitoring exercise
Baseline year Year for which baseline value of the indicator is recorded
Benchmark value Value of the best performing district in the Country to set a target reference.
Annual Target Annual targets for the district to achieve
Aggregation logic Whether the indicator is to be aggregated over time or geography or any other metric
Validation logic To avoid errors in data entry necessary validations are embedded in the system
Source of indicator value Whether entered manually or captured using web services
Weightage Weightage for the indicator in the Composite index
Change in Rank District ranking and the change in rank over the last period.
Champions of Change – Operational manual of the Real-time Outcome Monitoring System A-4
1
Features
1 of the Monitoring
Flat, not System
Data capture only at one level- either through data entry or direct capture using web
1 Flat, not
hierarchical services and APIs
Data capture only at one level- either through data entry or direct capture using web
1 Flat, not
hierarchical services and APIs
Data capture only at one level- either through data entry or direct capture using web
Flat, not
hierarchical services and APIs
Data capture only at one level- either through data entry or direct capture using web
2 hierarchical services and APIs
2 Modular Easy to add, delete, or modify features/ modules
2 Modular Easy to add, delete, or modify features/ modules
2 Modular Easy to add, delete, or modify features/ modules
3 Modular Easy to add, delete, or modify features/ modules
360 degree NITI Aayog | Ministry view | LWE view
3
360view
degree State view
NITI Aayog | District
Ministryview
view | LWE view
3
360view
degree State view
NITI Aayog |
| District view
Ministry view | LWE view
3
360view
degree State view
NITI Aayog || District
Ministryview
view | LWE view
45 State view | District view
45 Web
view
Visualization
and app Ranking | Change in Rank | Distance from target | Percentage of achievement
Visualization
4 Web
and and app
analysis
The system
Global
Ranking
can be accessed
and| national
Changebenchmarking
using web
in Rank | Distance
browser
| Change and app on Android and iOS devices
from over
targettime
| Percentage of achievement
compatible
5 Visualization The system can be accessed using web browser and app on Android and iOS devices
and analysis
4 compatible Global
Rankingand| national
Changebenchmarking
in Rank | Distance| Change
from over
targettime
| Percentage of achievement
5 Web and app
Visualization Global and national benchmarking | Change over time
and
Web analysis
and app
The system
Ranking | can be accessed
Change in Rank using web browser
| Distance and app
from target on Androidof
| Percentage and iOS devices
achievement
compatible
6 and analysis Global
The and national
system can be benchmarking
accessed using | Change
web over
browser andtime
app on Android and iOS devices
SMS and email alerts can be generated for data entry, missing data, data reporting
Alerts
6 compatible etc.
SMS and email alerts can be generated for data entry, missing data, data reporting
Alerts etc.
6
SMS and email alerts can be generated for data entry, missing data, data reporting
6 Alerts etc.
SMS and email alerts can be generated for data entry, missing data, data reporting
Alerts
7 Perception
An
etc.optional perception-capturing module can be built into the system. People’s
7 Perception perception,
capture An optional feedback, and grievances
perception-capturing can can
module be recorded.
be built into the system. People’s
capture
7 Perception perception, feedback, and grievances can be recorded.
An optional perception-capturing module can be built into the system. People’s
7 Perception perception,
8 capture An optional feedback, and grievances
perception-capturing can can
module be recorded.
be built into the system. People’s
8 SDG
capture perception,
The goals, feedback,
targets, andand grievances
indicators can can
be be recorded.
mapped to SDGs to track the progress of
SDG
mapping the districts towards achieving SDGs
The goals, targets, and indicators can be mapped to SDGs to track the progress of
8
mapping the districts towards achieving SDGs
8 SDG The goals, targets, and indicators can be mapped to SDGs to track the progress of
SDG
mapping the
Thedistricts towards
goals, targets, achieving
and SDGs
indicators can be mapped to SDGs to track the progress of
the districts
mappingArchitecture towards achieving SDGs
Application
The process flow explains the data entry including validations before data is stored. Aggrega-
tion and Ranking engine is built for displaying tabular and analytical reports and further visual-
ization including change in rank based on district performance. Dashboard provides common
entry point for customized views various stakeholders.
Survey data
The Government of Andhra Pradesh believes that what gets measured gets done. To achieve
its Vision 2029 and strategic development goals, the Government of Andhra Pradesh has cre-
ated a comprehensive performance management system. The system, called the Real-time
Outcome Monitoring System (ROMS) manages the performance of all 13 districts and 33 sec-
retariat departments of the State Government.
ROMS is driven by an indicator framework, and on online and app-based IT system, developed
after due consultations with all State Government departments. The Chief Minister and the
Chief Secretary frequently use the System for performance reviews. ROMS developed and
managed by the Planning Department of the State Government under the overall guidance of
the Chief Secretary.
Components of ROMS
Champions of Change – Operational manual of the Real-time Outcome Monitoring System A-6
Analysis
Benchmarking
Benchmarking selected key indicators under the Women and Child Development De-
WOMEN partment.
WOMEN
AND CHILD
AND CHILD
DEVELOPMENT:
DEVELOPMENT:BENCHMARKING
BENCHMARKINGAND GAP
AND ANALYSIS
GAP ANALYSIS
OF KEY
OF INDICATORS
KEY INDICATORS
Performance of the districts identified by NITI Aayog, Home Ministry, and other ministries can
be viewed separately.
District performance review report, provides a one-page report on the key indicators where
the district performance is poor (red) and good (green) department-wise.
BASIC INFRASTRUCTURE
Rural Development 10% achievement for construction of check dam (32 out of targeted 135)
ODF GPs - Target Achieved. 100% of the GPs in the district have been de-
Swachh Andhra
clared ODF
Houses under NTR Rural 16-17 & 17-18 - 2017-18 target achieved. 83% of the
target has been constructed.
Houses under NTR gramin (16-17) - The District has achieved 19 of the target
Housing
set for the year which is the lowest performance in the state
Beneficiary Led Construction (BLC) and Hud Hud Housing - The district has
achieved 50% of target set for 2017-18
Champions of Change – Operational manual of the Real-time Outcome Monitoring System A-7
Seed supplied 12% 17% 6% 29% 53% 27% 23% 62% 23% 67% 39% 50% 44% 39% 8 2
Chemical Fertilizers
supplied
16% 49% 28% 47% 44% 31% 61% 26% 45% 42% 85% 37% 66% 41% 3 2
Area Sown - Kharif 38% 83% 62% 102% 82% 97% 89% 92% 51% 72% 90% 93% 93% 82% 2 2
Disbursement against
Agriculture credit 29% 49% 46% 55% 40% 52% 52% 42% 43% 78% 49% 54% 49% 50% 1 3
outlay
Very Poor 7 5 4 4 4 3 3 3 3 2 2 2 1
Poor 2 0 2 0 1 2 1 2 2 1 2 3 3
Change Report
Farm ponds constructed under Pantasanjeevani – achieved only 14,000 out of 60,000 targeted.
Construction of check dams (PMKSY) - very slow progress in the last 3 months, only 20% of the target
achieved.
No. of GPs having Solid Waste Management system - only 3 achieved out of 939 targeted.
Percentages of ITIs with own building with functional toilets internet and drinking water infrastructure –
1 XXXXX
progress is only 33% with no improvement during this year.
Livelihood activities for ST beneficiaries – amount disbursed is only 23% till January.
Energy - Revenue per unit electricity has come down from Rs.6.1 to Rs.5.06 (Target Rs. 5.54).
Champions of Change – Operational manual of the Real-time Outcome Monitoring System A-8
TIMELINES SECOND
EVERY MONTH X 10th 15th 16th FORTNIGHT
B- About
The districts are ranked using an index based ranking system developed by NITI Aayog mea-
suring the status and change in these KPIs.
1. Data Capture
The performance or achievement, and change, of each district on each indicator is cap-
tured based on the periodicity (monthly/quarterly/annual) decided for the indicator. The
data entry is made by the stakeholder responsible for data collection:
a. Benchmark (or Aspirational Target/ Best in India) is prepared and entered by NITI. It
consists of a single value for each indicator across all districts.
b. Baseline, with status data as on 31st March (or nearest date) of previous financial year,
is prepared and entered either by District Magistrates or Agencies, as the case may be,
depending on their responsibilities.
c. Annual Target is prepared and entered by the District Magistrates, with values specific
to each district and each indicator.
d. Progress, single source, manual entry, is prepared and entered by District Magis-
trates or Agencies, as the case may be, depending on their responsibilities.
e. Progress, multiple sources/ audited data, manual entry, is prepared and entered
by Agencies. System provides an option for multiple parties to enter the value on the
Champions of Change – Operational manual of the Real-time Outcome Monitoring System B-2
same indicator in case of audited data. Same indicator, if audited, will be entered by the
District Magistrate with its administrative value, and by the audit agency with its audited
value.
f. Progress, from existing web sites, automatic capture, will be integrated automatical-
ly in the system once the linking process through a web service can be established.
For effective monitoring and evaluation, it has been decided that the periodicity of cap-
ture should be made as frequent as possible (monthly/quarterly) with the ultimate aim of
achieving real time capture and evaluation.
2. Rank Calculation
The system calculates the rank of each district on the 6 themes as well as a composite
rank based on the weights assigned to each indicator and the performance of the district
with reference to other districts. The ranking methodology and weights are provided in
the later parts of the manual.
3. Data Visualization
Dashboard provides customized view for various themes and for various stakeholders -
NITI Aayog, Ministry of Home Affairs, Pool of Ministries.
Each indicator’s progress/status can be viewed across time for all aspirational districts
along with the ranking and change in rank with the previous period.
Champions of Change – Operational manual of the Real-time Outcome Monitoring System C-1
C- Data Entry
C1
• Login for data entry
• District Collectors
• Agencies
C2 C3
Baseline &
Annual Target
Progress
Entry
Entry
By default your State and District names are displayed, as well as year,
if you have the authorisation of a District Magistrate.
a) Select Theme
from the list by clicking on the arrow.
Data Entry
3
1 1. Type Numerator
2 2. Type Denominator
3. Indicator is calculated
automatically
Data Validation
Data is validated before saving.
There are 2 types of login: one for District Collectors / District Magistrates (DC/DM) and one for Agencies (A).
Target Entry is authorised only for DC/DM. If you have logged in as a DC/DM, you can choose between Prog-
ress Entry and Target Entry. Click on Target Entry.
By default your State and District names are displayed, as well as year.
You have to select Theme from the list by clicking on the arrow.
Data Entry
Type the value in the cell
Champions of Change – Operational manual of the Real-time Outcome Monitoring System C-6
Data Validation
Data is validated before saving.
d) Confirm
and Ranks
and Ranks
E- Reports
Report 1: All indicators for 1 district
Report 1
All indicators for 1 district
Click on Theme
Champions of Change – Operational manual of the Real-time Outcome Monitoring System E-2
E- Reports
Report 1: All indicators for 1 district
Select Theme
from the list by clicking
on the arrow.
Headers
E- Reports
Report 2: All districts for 1 indicator
Report 2
All districts for 1 indicator
Click on User
Champions of Change – Operational manual of the Real-time Outcome Monitoring System E-4
Percentage of ANC registered within the first trimes- HMIS, MoHFW Yes
1.2 3 0.9
ter Monthly Quarterly
Project monthly
Percentage of pregnant women taking Supplementa- progress re- Yes
2 3 0.9 port-ICDS
ry Nutrition under the ICDS programme regularly Quarterly
Monthly
Percentage of High Risk Pregnant women who had HMIS, MoHFW Yes
3.1 5 1.5
institutional delivery Monthly Quarterly
HMIS, MoHFW Yes
3.2 Percentage of Pregnant Women tested for Anemia 4 1.2
Monthly Quarterly
HMIS, MoHFW
4.1 Sex Ratio at birth 3 0.9
Monthly
Percentage of low birth weight babies (Less than HMIS, MoHFW Yes
6.2 3 0.9
2500 gms) Monthly Quarterly
HMIS, MoHFW Yes
6.3 Proportion of live babies weighed at birth 3 0.9
Monthly Quarterly
Project monthly Yes
7 Percentage of underweight children under 5 years 7 2.1 progress report-
ICDS Monthly Quarterly
Survey
8.1 Percentage of stunted children under 5 years 2 0.6
Quarterly
Percentage of children with Diarrhoea treated with Survey
8.2 2 0.6
ORS Quarterly
Percentage of children with Diarrhoea treated with Survey
8.3 2 0.6
Zinc Quarterly
Project monthly
progress re-
13f Proportion of Anganwadis with own buildings 2 0.6 port-ICDS
Quarterly
Education
Weight in
Weight in Source / Validated
Sl. Overall
Indicators Education by Survey /
No. Composite Periodicity
Index Periodicity
index
Ministry of Hu-
man Resource
1 Transition Rate (a) Primary to Upper Primary level 14 4.2 Development
(MHRD) - UDISE
/ Annual
MHRD - UDISE /
(b) Upper Primary to Secondary level 6 1.8
Annual
Concerned Offi-
Yes
2 Toilet access: % schools with functional girls’ toilets 5 1.5 cer-DC/ DM
Quarterly
Monthly
Concerned Officer,
Number of Transactions in District mandi linked to DC/DM
4 10 2
electronic market
Monthly
Concerned Officer,
Percentage share of high value crops to total sown DC/DM
6 2.5 0.5
area in district
Half Yearly
Concerned Officer,
7 Agricultural productivity of 2 major crops 5 1 DC/DM
Half Yearly
Concerned Offi- Yes
8 Percentage of animals vaccinated 7.5 1.5 cer-DC/DM
Monthly Quarterly
Concerned Offi-
Number of Soil Health Cards distributed in II Cycle cer-DC/DM
10 2.5 0.5
as compared to I cycle
Monthly
Total 100% 20%
Champions of Change – Operational manual of the Real-time Outcome Monitoring System F-5
Weight in Weight in
Source / Validated
Sl. Financial Overall
Indicators by Survey /
No. Inclusion Composite Periodicity Periodicity
Index index
Department of
Total Disbursement of Mudra loan (in rupees) per 1 Financial Services
1 16.67 0.83 (DFS) /
Lakh population
Monthly
Skill Development
Ministry of Skill
Development and
No of youth certified in short term and long term Entrepreneurship
1 training# schemes / no of youth in district in age 25 1.25 (MSDE) & Line Yes
group 15-29* Ministry /
Monthly
MSDE & Line Min-
No of certified youth employed# /No of youth istry /
2 15 0.75 Yes
trained under short term and long term training
Monthly
Directorate General
Number of Apprentices completing/Total number of Training (DGT) /
3 25 1.25 Yes
of trainees registered on the portal
Monthly
MSDE & National
Skill Develop-
No of people certified under Recognition of Prior ment Corporation
4 25 1.25 Yes
Learning# / Non formally skilled workforce** (NSDC) /
Monthly
Total 100% 5%
#
Only data from PMKVY (2016-20) has been referred here
*The data of ‘number of youth in district in age group 15-29’ to be used from the same source by
NITI Aayog for calculation as used for calculation of other parameters.
** The data for non formally skilled workforce has been estimated from NSSO 2011-12 EUS Survey
and Census 2011
Champions of Change – Operational manual of the Real-time Outcome Monitoring System F-7
Basic Infrastructure
Concerned Officer-
Percentage of Gram panchayat with internet DC/DM
2 5 0.5
connection
Monthly
Concerned Officer-
(a) Percentage of habitations with access to DC/DM
3(a) 7.5 0.75
all weather roads under PMGSY
Monthly
(b) Cumulative number of kilometres of Concerned Officer-
all-weather road work completed as a DC/DM
3(b) 7.5 0.75
percentage of total sanctioned kilome-
tres in the district under PMGSY Monthly
Concerned Officer-
Coverage/ establishment of Common Service DC/DM
6 5 0.5
Centres at Gram Panchayat level
Monthly